Hadoop stand-alone installation
Experimental environment:
Operating system: Ubuntu 16.04LTS
Hadoop version: Hadoop 2.7.1
First, install JAVA environment
Dblab@dblab-VirtualBox:/$ sudo apt-get install default-jre default-jdk
Dblab@dblab-VirtualBox:/$ vim / .bashrc
Export JAVA_HOME=/usr/lib/jvm/default-java
Dblab@dblab-VirtualBox:/$ source ~ / .bashrc # enables variable settings to take effect
Dblab@dblab-VirtualBox:/$ echo $JAVA_HOME
/ usr/lib/jvm/default-java
Dblab@dblab-VirtualBox:/$ java-version
Openjdk version "1.8.0,131"
OpenJDK Runtime Environment (build 1.8.0mm 131-8u131-b11-2ubuntu1.16.04.3-b11)
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)
Second, install Hadoop
Dblab@dblab-VirtualBox:/$ cd ~ / download
Dblab@dblab-VirtualBox:~/ download $wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.6.5/hadoop-2.6.5.tar.gz
Download $sudo tar-zxvf hadoop-2.6.5.tar.gz-C / usr/local from dblab@dblab-VirtualBox:~/
Dblab@dblab-VirtualBox:/usr/local$ sudo mv. / hadoop-2.6.5/. / hadoop # change the folder name
Dblab@dblab-VirtualBox:/usr/local$ sudo chown-R hadoop. / hadoop/ # modify folder permissions
Dblab@dblab-VirtualBox:/usr/local/hadoop$. / bin/hadoop version # displays version information
Hadoop 2.7.1
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git-r 15ecc87ccf4a0228f35af08fc56de536e6ce657a
Compiled by jenkins on 2015-06-29T06:04Z
Compiled with protoc 2.5.0
From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a
This command was run using / usr/local/hadoop/share/hadoop/common/hadoop-common-2.7.1.jar
III. Hadoop stand-alone configuration
The default mode of Hadoop is non-distributed mode (local mode) and can be run without additional configuration.
You can execute the example to feel the operation of Hadoop. Hadoop comes with a wealth of examples
Dblab@dblab-VirtualBox:/usr/local/hadoop$. / bin/hadoop jar. / share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar
An example program must be given as the first argument.
Valid program names are:
Aggregatewordcount: An Aggregate based map/reduce program that counts the words in the input files.
Aggregatewordhist: An Aggregate based map/reduce program that computes the histogram of the words in the input files.
Bbp: A map/reduce program that uses Bailey-Borwein-Plouffe to compute exact digits of Pi.
Dbcount: An example job that count the pageview counts from a database.
Distbbp: A map/reduce program that uses a BBP-type formula to compute exact bits of Pi.
Grep: A map/reduce program that counts the matches of a regex in the input.
Join: A job that effects a join over sorted, equally partitioned datasets
Multifilewc: A job that counts words from several files.
Pentomino: A map/reduce tile laying program to find solutions to pentomino problems.
Pi: A map/reduce program that estimates Pi using a quasi-Monte Carlo method.
Randomtextwriter: A map/reduce program that writes 10GB of random textual data per node.
Randomwriter: A map/reduce program that writes 10GB of random data per node.
Secondarysort: An example defining a secondarysort to the reduce.
Sort: A map/reduce program that sorts the data written by the random writer.
Sudoku: A sudoku solver.
Teragen: Generate data for the terasort
Terasort: Run the terasort
Teravalidate: Checking results of terasort
Wordcount: A map/reduce program that counts the words in the input files.
Wordmean: A map/reduce program that counts the average length of the words in the input files.
Wordmedian: A map/reduce program that counts the median length of the words in the input files.
Wordstandarddeviation: A map/reduce program that counts the standarddeviation of the length of the words in the input files.
$cd / usr/local/hadoop
$sudo mkdir. / input
$sudo cp. / etc/hadoop/*.xml. / input # use the configuration file as the input file
$. / bin/hadoop jar. / share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar grep. / input. / output 'dfs [a Murz.] +'
$cat. / output/* # View the running result
Dblab@dblab-VirtualBox:/usr/local/hadoop$ cat. / output/*
1 dfsadmin
