I. hadoop pseudo-distributed deployment

= I. Hadoop pseudo-distributed = pseudo-distributed environment configuration requires installation of openjdk and openjdk-devel packages (version 1.8.0) rpm-ql Java-openjdk-devel-1.8.0 | grep'/ bin/javac'vim / etc/profile#Javaexport JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.151-1.b12.el6_9.x86_64CLASSPATH=.:$JAVA_HOME/jre/lib/rt .jar: $JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jarexport PATH=$PATH:$JAVA_HOME/bin#hiveexport HIVE_HOME=/data/hadoop/hiveexport PATH=$HIVE_HOME/bin:$PATH#hadoopexport HADOOP_HOME=/data/hadoop/hadoop-2.7.4export PATH=$HADOOP_HOME/bin:$PATHsource / etc/profile java-version add hadoop user useradd-m hadoop- s / bin/bash gives sudo permission to edit the configuration file on line 98 of visudo Add the hadoop user root ALL= (ALL) ALLhadoop ALL= (ALL) ALL under the root line to switch to the hadoop user and set the remote login generation key pair ssh-keygen-t rsa cat id_rsa.pub > > authorized_keys chmod 600. / authorized_keys (pay attention to permission settings) = two. Install hadoop=wget http://114.242.101.2:808/hadoop/hadoop-2.7.4.tar.gztar xf hadoop-2.7.4.tar.gz-C / data/hadoopchown-R hadoop.hadoop / data/hadoop/hadoop-2.7.4= III. Configuration file = / data/hadoop/hadoop-2.7.4/etc/hadoop/ path hdfs-site.xmlcore-site.xmlmapred-site.xml.templatehdfs-site.xml: what is defined here is the number of data nodes in block file size core-site.xml: the header machine mapred-site.xml.template of hadoop defined: defines the scheduling method for storing hadoop files for pseudo-distributed three configuration files as follows (empty by default) The core-site.xml configuration file is as follows: hadoop.tmp.dir file:/data/hadoop/hadoop-2.7.4/tmp Abase for other temporary directories. The fs.defaultFS hdfs://localhost:9000 hdfs-site.xml configuration file is as follows: dfs.replication 1 dfs.namenode.name.dir file:/data/hadoop/hadoop-2.7.4/tmp/dfs/name dfs.datanode.data.dir file:/data/hadoop/hadoop-2.7.4/tmp/dfs/data The mapred-site.xml.template configuration file is as follows: mapreduce.framework.name yarn cd / data/hadoop/hadoop-2.7.4/ execution. / bin/hdfs namenode-format for formatting. / after sbin/start-dfs.sh startup is complete You can use the command jps to determine whether the startup is successful or not. If started successfully, the following processes are listed: "NameNode", "DataNode" and SecondaryNameNodevim ~ / .bashrc add export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.151-1.b12.el6_9.x86_64source ~ / .bashrc and restart the scaffolding to complete access: http://192.168.0.12:50070