Install sqoop1.99.7
There are big differences between sqoop2 and sqoop1. The whole ETL process is divided into two parts: server and client. Sqoop2 is an interactive operation.
Decompression and installation
[hadoop@fa01] $tar-zxvf sqoop-1.99.7-bin-hadoop200.tar.gz-C. /
Rename
[hadoop@fa01 ~] $mv sqoop-1.99.7-bin-hadoop200 sqoop-1.99.7
At the same time, create a new extra file and place the driver file of mysql (the file name is arbitrary, and the configuration file corresponds to the configuration file)
[hadoop@fa01 sqoop-1.99.7] $mkdir extra
[hadoop@fa01 ~] $mv mysql-connector-java-5.1.5-bin.jar / home/hadoop/sqoop-1.99.7/extra/
Configure environment variables. Configure sqoop third-party jar package reference variables
[hadoop@fa01 ~] $vim .bash _ profile
Export SQOOP_HOME=/home/hadoop/sqoopsqoop-1.99.7
Export SQOOP_SERVER_EXTRA_LIB=/home/hadoop/sqoop-1.99.7/extra
Modify the core-site.xml file of hadoop, where hadoop is the user name
[hadoop@fa01 ~] $cd hadoop-2.6.0-cdh6.5.2/etc/hadoop
[hadoop@fa01 hadoop] $vim core-site.xml
Hadoop.proxyuser.hadoop.hosts
*
Hadoop.proxyuser.hadoop.groups
*
Configure # Hadoop configuration directory in the conf file in sqoop
[hadoop@fa01 ~] $cd sqoop-1.99.7/conf/
[hadoop@fa01 conf] $vim sqoop.properties
Org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/home/hadoop/hadoop-2.6.0-cdh6.5.2/etc/hadoop
Check whether sqoop is configured successfully
[hadoop@fa01 bin] $. / sqoop2-tool verify
Setting conf dir: / home/hadoop/sqoop-1.99.7/bin/../conf
Sqoop home directory: / home/hadoop/sqoop-1.99.7
Sqoop tool executor:
Version: 1.99.7
Revision:435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb
Compiled on Tue Jul 19 16:08:27 PDT 2016 by abefine
Running tool: class org.apache.sqoop.tools.tool.VerifyTool
2017-06-21 01 INFO 31 core.SqoopServer 48489 [main] core.SqoopServer (SqoopServer.java:initialize (55))-Initializing Sqoop server.
2017-06-21 01 core.PropertiesConfigurationProvider 31 core.PropertiesConfigurationProvider 48539 INFO [main] core.PropertiesConfigurationProvider (PropertiesConfigurationProvider.java:initialize (99))-Starting config file poller thread
Verification was successful.
Tool class org.apache.sqoop.tools.tool.VerifyTool has finished correctly.
Start sqoop
[hadoop@fa01 bin] $. / sqoop2-server start
Setting conf dir: / home/hadoop/sqoop-1.99.7/bin/../conf
Sqoop home directory: / home/hadoop/sqoop-1.99.7
Starting the Sqoop2 server...
2017-06-21 01 INFO [main] core.SqoopServer (SqoopServer.java:initialize (55))-Initializing Sqoop server.
2017-06-21 01 core.PropertiesConfigurationProvider 32 INFO [main] core.PropertiesConfigurationProvider (PropertiesConfigurationProvider.java:initialize (99))-Starting config file poller thread
Sqoop2 server started.