Shulou(Shulou.com)06/03 Report--

MyEclipse under Win7 remotely connects to Hadoop clusters in Mac/Linux

(Also available at tn.51cto.com/article/562)

Software required:

(1) Download Hadoop 2.5.1 to Win7 and decompress

Hadoop 2.5.1 Download Address: Indexof/dist/hadoop/core/hadoop-2.5.1

http://archive.apache.org/dist/hadoop/core/hadoop-2.5.1/

Note:Win7 download is Hadoop 2.5.1, Mac version of Hadoop is Hadoop 2.5; version is inconsistent, but when testing, the results are not wrong, but some people test that the version is the best, here you can pay attention to the next.

(2) Hadoop runs under Linux system, run hadoop program in eclipse under windows, first install hadoop2. 2-common-bin.zip, download address is as follows: github.com/srccodes/hadoop-common-2.2.0-bin

Note: hadoop2.2-common-bin.zip is used here, which is inconsistent with Hadoop version, but the result is no error and no exception is reported. Running Hadoop without configuring the plugin will report an exception:

Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

(3) Configure hadoop environment variables (similar to Java environment configuration)

Windows environment variable configuration: Right click "My Computer" -> Properties-> Advanced environment variable configuration-> Advanced tab-> Environment variables-> Click New HADOOP_HOME-> Configure download Hadoop path.

(4) Download hadoop-eclipse-plugin-2.2.0.jar, plug-ins needed to develop Hadoop in MyEclipse/Eclipse. Download and copy it to the MyEclipse installation path at the following location;

(5) Restart MyEclipse or Eclipse will appear on the right "blue elephant" icon, indicating that the plug-in hadoop plug-in installation is successful.

(6)Eclipse environment configuration: Click "Menu Window"-->Preference-->Hadoop Map/Reduce to configure, as shown in the following figure:

(7)The view shown in the figure below appears, indicating successful configuration.

(8)Switch to profile and modify

① cd hadoop/hadoop2.5/etc/hadoop

② cat core-site.xml

When configuring hadoop pseudo-distributions, localhost is generally used as the identifier of namenode, as shown in the figure.

This needs to be modified to Linux hostname or IP address, otherwise Hadoop programs under Windows cannot connect to Linux or Hadoop on Mac to run, and errors are reported.

As shown in the figure, it indicates that the connection failed:

Note: If you are connecting using a hostname, you need to configure the hostname to IP mapping.

As shown in the figure:

(9) Start hadoop on Mac or Linux and use jps to see if it runs successfully.

(10) Configure hdfs, create a New Hadoop location or Edit Hadoop location

Specific configuration description:

(11) Test: Check whether Hdfs can be accessed in Mac or Linux. As shown in the following figure:

The image above shows a file in hdfs.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.