One of big data: Hadoop installation
First, install Hadoop1. First extract the hadoop to the relevant folder
two。 After entering the extracted folder, modify the relevant files
3. Configure hadoop-env.shvi hadoop-env.sh:export= JAVA_HOME=/opt/inst/jdk181
4. Configure core-site.xmlvi core-site.xml fs.defaultFS hdfs://bigdata:9000 hadoop.tmp.dir / opt/hadoopdata hadoop.proxyuser.root.users * hadoop .proxyuser.root.groups *
5. Configure hdfs-site.xmlvi hdfs-site.xml dfs.replication 1
6. Configure cp mapred-site.xml.template mapred-site.xmlcp mapred-site.xml.template mapred-site.xml
7.vi mapred-site.xml mapreduce.framework.name yarn
8.vi yarn-site.xmlvi
9.vi / etc/profileexport HADOOP_HOME=/opt/bigdata/hadoop260export HADOOP_MAPRED_HOME=$HADOOP_HOMEexport HADOOP_COMMON_HOME=$HADOOP_HOMEexport HADOOP_HDFS_HOME=$HADOOP_HOMEexport YARN_HOME=$HADOOP_HOMEexport HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/nativeexport PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/binexport HADOOP_INSTALL=$HADOOP_HOME
10. Update and format source / etc/profilehdfs namenode-format
11. Start start-all.sh
Jps# View process
twelve。 Import linux into hadoop hdfs dfs-put / opt/a.txt / cm/hdfs dfs-ls / cm
Hadoop understands the three core 1.HDFS: Hadoop Distribute File System distributed file system
NameNode: master node, directory
DataNode: slave nodes, data
SecondaryNameNode: backup of the primary node
2.YARN:Yet Another Resource Negotiator Resource Management and scheduling system
dispatches memory resources and CPU computing power.
is dispatched through ResourceManager (only one)
The main functions of ResourceManager are:
1. Processing client request
2. Monitoring NodeManager
3. Start or monitor ApplicationMaster ()
4. Allocation or scheduling of resources
NodeManager (multiple)
The main functions of NodeManager are:
1. Manage resources on a single node
2. Processing commands from ResourceManager
3. Processing commands from ApplicationMaster
3.MapReduce: distributed Computing Framework
operation
