How to configure spark sql query hive
This article introduces the relevant knowledge of "how to configure spark sql query hive". In the actual case operation process, many people will encounter such difficulties. Next, let Xiaobian lead you to learn how to deal with these situations! I hope you can read carefully and learn something!
Configure idea spark sql connection hive
1. First copy hive-site.xml of/etc/hive/conf/to resources of the project
2. To add maven dependencies to the pom.xml file, you need to add spark-sql, spark-hive, spark version 2.2.0, scala version 2.10
org.apache.spark
spark-sql_2.10
2.2.0
org.apache.spark
spark-hive_2.10
2.2.0
3. Set it in the program
val spark = SparkSession
.builder()
.appName("test_note")
.master("local")
.enableHiveSupport()//Set support hive
.getOrCreate()
To this place, you can directly query the hive data of the server
Configure CDH Cluster
Cluster configuration only needs to copy hive-site.xml from/etc/hive/conf/directory to/etc/spark2/conf/directory to spark-submit.
"How to configure Spark SQL query hive" content is introduced here, thank you for reading. If you want to know more about industry-related knowledge, you can pay attention to the website. Xiaobian will output more high-quality practical articles for everyone!