Get the App
SLTechnology News&Howtos  ›  Internet Technology  › 

Install sqoop1.99.7

Shulou Source: shulou.com Published: 2022-06-03 08:14:53 11月23日 Update

There are big differences between sqoop2 and sqoop1. The whole ETL process is divided into two parts: server and client. Sqoop2 is an interactive operation.

Decompression and installation

[hadoop@fa01] $tar-zxvf sqoop-1.99.7-bin-hadoop200.tar.gz-C. /

Rename

[hadoop@fa01 ~] $mv sqoop-1.99.7-bin-hadoop200 sqoop-1.99.7

At the same time, create a new extra file and place the driver file of mysql (the file name is arbitrary, and the configuration file corresponds to the configuration file)

[hadoop@fa01 sqoop-1.99.7] $mkdir extra

[hadoop@fa01 ~] $mv mysql-connector-java-5.1.5-bin.jar / home/hadoop/sqoop-1.99.7/extra/

Configure environment variables. Configure sqoop third-party jar package reference variables

[hadoop@fa01 ~] $vim .bash _ profile

Export SQOOP_HOME=/home/hadoop/sqoopsqoop-1.99.7

Export SQOOP_SERVER_EXTRA_LIB=/home/hadoop/sqoop-1.99.7/extra

Modify the core-site.xml file of hadoop, where hadoop is the user name

[hadoop@fa01 ~] $cd hadoop-2.6.0-cdh6.5.2/etc/hadoop

[hadoop@fa01 hadoop] $vim core-site.xml

Hadoop.proxyuser.hadoop.hosts

*

Hadoop.proxyuser.hadoop.groups

*

Configure # Hadoop configuration directory in the conf file in sqoop

[hadoop@fa01 ~] $cd sqoop-1.99.7/conf/

[hadoop@fa01 conf] $vim sqoop.properties

Org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/home/hadoop/hadoop-2.6.0-cdh6.5.2/etc/hadoop

Check whether sqoop is configured successfully

[hadoop@fa01 bin] $. / sqoop2-tool verify

Setting conf dir: / home/hadoop/sqoop-1.99.7/bin/../conf

Sqoop home directory: / home/hadoop/sqoop-1.99.7

Sqoop tool executor:

Version: 1.99.7

Revision:435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb

Compiled on Tue Jul 19 16:08:27 PDT 2016 by abefine

Running tool: class org.apache.sqoop.tools.tool.VerifyTool

2017-06-21 01 INFO 31 core.SqoopServer 48489 [main] core.SqoopServer (SqoopServer.java:initialize (55))-Initializing Sqoop server.

2017-06-21 01 core.PropertiesConfigurationProvider 31 core.PropertiesConfigurationProvider 48539 INFO [main] core.PropertiesConfigurationProvider (PropertiesConfigurationProvider.java:initialize (99))-Starting config file poller thread

Verification was successful.

Tool class org.apache.sqoop.tools.tool.VerifyTool has finished correctly.

Start sqoop

[hadoop@fa01 bin] $. / sqoop2-server start

Setting conf dir: / home/hadoop/sqoop-1.99.7/bin/../conf

Sqoop home directory: / home/hadoop/sqoop-1.99.7

Starting the Sqoop2 server...

2017-06-21 01 INFO [main] core.SqoopServer (SqoopServer.java:initialize (55))-Initializing Sqoop server.

2017-06-21 01 core.PropertiesConfigurationProvider 32 INFO [main] core.PropertiesConfigurationProvider (PropertiesConfigurationProvider.java:initialize (99))-Starting config file poller thread

Sqoop2 server started.

Tags: File configuration variable large successful interactive at the same time overall file name environment user user name third party process part check driver Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno MySQL NVidia OPPO Reno Linux Docker