Sqoop2 installation startup
(1) Installation environment
Operating system: Linux (centos 6.5)
JDK version: 1.7.0_45
Hadoop version: hadoop 2.6.0
Sqoop2 Version: sqoop-1.99.4-bin-hadoop200
hadoop installation directory: /usr/local/hadoop/hadoop-2.2.0
Sqoop2 installation directory: /usr/local/sqoop/sqoop-1.99.4-bin-hadoop200
Hadoop and Sqoop are both the same user hadoop Below, hadoop user's home directory: /home/hadoop
(2) Modify Sqoop2 configuration file
1. First modify the configuration file/home/hadoop/sqoop-1.99.4-bin-hadoop200/server/conf/sqoop.properties Specify the path to which the hadoop configuration file belongs.
Replace the original configuration with the following:
# Hadoop configuration directory
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/etc/hadoop/conf/
Amend to read:
# Hadoop configuration directory
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/hadoop/hadoop-2.6.0/etc/hadoop/
2. Modify the configuration file/home/hadoop/sqoop-1.99.4-bin-hadoop200/server/conf/catalina.properties.
Here immediately add all *.jar packages under/home/hadoop/hadoop-2.2.0/share/hadoop to the classpath of sqoop2.
Replace the original configuration with the following:
common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../ lib/*.jar,/usr/lib/hadoop/*.jar,/usr/lib/hadoop/lib/*.jar,/usr/lib/hadoop-hdfs/*.jar,/usr/lib/hadoop-hdfs/lib/*.jar,/usr/lib/hadoop-mapreduce/*.jar,/usr/lib/hadoop-mapreduce/lib/*.jar,/usr/lib/hadoop-yarn/*.jar,/usr/lib/hadoop-yarn/lib/*.jar
Amend to read:
common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../ lib/*.jar,/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/*.jar,/usr/local/hadoop/hadoop-2.6.0/share/hadoop/common/lib/*.jar,/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/*.jar,/usr/local/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/*.jar,/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/*.jar,/usr/local/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/*.jar,/usr/local/hadoop/hadoop-2.6.0/share/hadoop/tools/*.jar,/usr/local/hadoop/hadoop-2.6.0/share/hadoop/tools/lib/*.jar,/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/*.jar,/usr/local/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/*.jar,/usr/local/hadoop/hadoop-2.6.0/share/hadoop/httpfs/tomcat/lib/*.jar
(3) Modify environmental variables
Modify ~/.bashrc directly and add the following at the end of the file:
export SQOOP_HOME=/usr/local/sqoop/sqoop-1.99.4-bin-hadoop200
export PATH=$SQOOP_HOME/bin:$PATH
export CATALINA_HOME=$SQOOP_HOME/server
export LOGDIR=$SQOOP_HOME/logs
Of course, after completing the above modifications, you need to execute the command under the hadoop user to make the configuration take effect:
Start Stop sqoop-server:
$SQOOP2_HOME/bin/sqoop.sh server start$SQOOP2_HOME/bin/sqoop.sh server stop or $SQOOP2_HOME/bin/sqoop2-server start$SQOOP2_HOME/bin/sqoop2-server stop
Verify successful startup:
Method 1: jps View Progress: Bootstrap
Method 2: http://hadoop000:12000/sqoop/version
wget -qO - hadoop000:12000/sqoop/version
Start SQOOP client:
$SQOOP2_HOME/bin/sqoop.sh client or $SQOOP2_HOME/bin/sqoop2-shell
Configure the server for clients:
sqoop:000> set server --host hadoop000 --port 12000 --webapp sqoop
View server-side information:
sqoop:000> show server --all