Shulou(Shulou.com)06/03 Report--

Before learning any spark technology, please correctly understand spark, you can refer to: correct understanding of spark

The following is a spark RDD java api implementation to read data from a relational database. Here, a local derby database is used. Of course, it can be a relational database such as mysql or oracle:

Package com.twq.javaapi.java7;import org.apache.spark.api.java.JavaRDD;import org.apache.spark.api.java.JavaSparkContext;import org.apache.spark.api.java.function.Function;import org.apache.spark.rdd.JdbcRDD;import java.io.Serializable;import java.sql.* Public class JavaJdbcRDDSuite implements Serializable {public static void prepareData () throws ClassNotFoundException, SQLException {/ / use the local database derby, of course, you can use the relational database Class.forName ("org.apache.derby.jdbc.EmbeddedDriver") such as mysql; Connection connection = DriverManager.getConnection ("jdbc:derby:target/JavaJdbcRDDSuiteDb;create=true") Try {/ / create a table FOO,ID is a self-incrementing primary key and DATA is an INTEGER column Statement create = connection.createStatement () Create.execute ("CREATE TABLE FOO (" + "ID INTEGER NOT NULL GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1)," + "DATA INTEGER"); create.close () / / insert data PreparedStatement insert = connection.prepareStatement ("INSERT INTO FOO (DATA) VALUES (?)"); for (int I = 1; I

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.