Spark association

It is found that writing tables in spark join is much easier than flink, at least schema can be saved. Here is an example.

Public static void main (String [] args) {SparkSession s = SparkSession.builder () .appName ("rec") .getOrCreate () Dataset user=s.read () .format ("jdbc") .option ("driver", "com.mysql.jdbc.Driver") .option ("url", "jdbc:mysql://*") .option ("dbtable", "user") .option ("user", "1") .option ("password", "1") .load () Dataset house=s.read () .format ("jdbc") .option ("driver", "com.mysql.jdbc.Driver") .option ("url", "jdbc:mysql://") .option ("dbtable", "house") .option ("user", "1") .option ("password") "1") .load () User.cache (); house.cache (); user.createOrReplaceTempView ("user"); house.createOrReplaceTempView ("house"); Dataset temp= s.sql ("select user.user_name, house.house_name from user inner join house where user.uid=house.uid"); temp.write (). Csv ("/ home/ziroom/house-user");}