Shulou(Shulou.com)06/03 Report--

First, install Hadoop1. First extract the hadoop to the relevant folder

two。 After entering the extracted folder, modify the relevant files

3. Configure hadoop-env.shvi hadoop-env.sh:export= JAVA_HOME=/opt/inst/jdk181

4. Configure core-site.xmlvi core-site.xml fs.defaultFS hdfs://bigdata:9000 hadoop.tmp.dir / opt/hadoopdata hadoop.proxyuser.root.users * hadoop .proxyuser.root.groups *

5. Configure hdfs-site.xmlvi hdfs-site.xml dfs.replication 1

6. Configure cp mapred-site.xml.template mapred-site.xmlcp mapred-site.xml.template mapred-site.xml

7.vi mapred-site.xml mapreduce.framework.name yarn

8.vi yarn-site.xmlvi

9.vi / etc/profileexport HADOOP_HOME=/opt/bigdata/hadoop260export HADOOP_MAPRED_HOME=$HADOOP_HOMEexport HADOOP_COMMON_HOME=$HADOOP_HOMEexport HADOOP_HDFS_HOME=$HADOOP_HOMEexport YARN_HOME=$HADOOP_HOMEexport HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/nativeexport PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/binexport HADOOP_INSTALL=$HADOOP_HOME

10. Update and format source / etc/profilehdfs namenode-format

11. Start start-all.sh

Jps# View process

twelve。 Import linux into hadoop hdfs dfs-put / opt/a.txt / cm/hdfs dfs-ls / cm

Hadoop understands the three core 1.HDFS: Hadoop Distribute File System distributed file system

​ NameNode: master node, directory

​ DataNode: slave nodes, data

​ SecondaryNameNode: backup of the primary node

2.YARN:Yet Another Resource Negotiator Resource Management and scheduling system

​ dispatches memory resources and CPU computing power.

​ is dispatched through ResourceManager (only one)

The main functions of ​ ResourceManager are:

​ 1. Processing client request

​ 2. Monitoring NodeManager

​ 3. Start or monitor ApplicationMaster ()

​ 4. Allocation or scheduling of resources

​ NodeManager (multiple)

The main functions of ​ NodeManager are:

​ 1. Manage resources on a single node

​ 2. Processing commands from ResourceManager

​ 3. Processing commands from ApplicationMaster

3.MapReduce: distributed Computing Framework

​ operation

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.