How to deploy Hadoop stand-alone mode
This article mainly shows you "Hadoop stand-alone mode how to deploy", the content is easy to understand, clear, hope to help you solve your doubts, the following let the editor lead you to study and learn "Hadoop stand-alone mode how to deploy" this article.
I. Hadoop deployment model
Stand-alone mode: runs as a separate Java process on a separate machine by default, mainly for debugging environments
Pseudo-distributed mode: simulated as a distributed multi-node environment on a single machine, with each Hadoop daemon running as a separate Java process
Fully distributed model: real production environment, built in a fully distributed cluster environment
Add users and groups
$sudo adduser hadoop # # create user hadoop$ sudo usermod-G sudo hadoop # # add hadoop user to sudo user group
Third, install related software
$sudo apt-get update$ sudo apt-get-y install openssh-server rsync openjdk-7-jdk$ sudo service ssh restart$ java-version
Configure SSH secret-free login
$su-l hadoop # # switch to hadoop user $ssh-keygen-t rsa-P "" # # configure SSH secret-free login $cat ~ / .ssh/id_rsa.pub > > ~ / .ssh/authorized_keys # # append the public key to the authorized_keys $ssh localhost # # authentication
V. Hadoop configuration
① download and install Hadoop
$su-l hadoop$ wget http://apache.fayea.com/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz $tar-zxvf hadoop-2.7.1.tar.gz$ sudo mv hadoop-2.7.1 / usr/local/hadoop
② configuration Hadoop
a. Add environment variabl
$vi / home/hadoop/.bashrc#HADOOP STARTexport JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64export HADOOP_INSTALL=/usr/local/hadoop/export PATH=$PATH:$HADOOP_INSTALL/binexport PATH=$PATH:$HADOOP_INSTALL/sbinexport HADOOP_MAPRED_HOME=$HADOOP_INSTALLexport HADOOP_COMMON_HOME=$HADOOP_INSTALLexport HADOOP_HDFS_HOME=$HADOOP_INSTALLexport YARN_HOME=$HADOOP_INSTALLexport HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/nativeexport HADOOP_OPTS= "- Djava.library.path=$HADOOP_INSTALL/lib" # HADOOP END$. / home/hadoop/.bashrc # # overload to make it effective
VI. Testing
① create input data
$su-l hadoop$ cd / usr/local/hadoop$ sudo mkdir input$ sudo cp / etc/passwd input/
② executes Hadoop WordCount applications (word frequency statistics)
$bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.7.1-sources.jar org.apache.hadoop.examples.WordCount input output
③ views generated word statistics
$hadoop dfs-ls output-rw-r--r-- 1 hadoop hadoop 0 2015-12-18 13:18 output/_SUCCESS-rw-r--r-- 1 hadoop hadoop 2128 2015-12-18 13:18 output/part-r-00000$ hadoop dfs-cat output/*
These are all the contents of the article "how to deploy Hadoop stand-alone mode". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!
