Get the App
SLTechnology News&Howtos  ›  Internet Technology  › 

The functions of three core components of hadoop

Shulou Source: shulou.com Published: 2022-06-03 05:56:10 11月23日 Update

The three major component functions of hadoop:

Common:

Tools, foundations, for services

MapReduce:

The processing of massive data

Distributed system

Thought

divide and rule

Big data set is divided into small data sets.

Each data set, dedicated logic business processing map

Merge statistics results reduce

HDFS:

Store huge amounts of data

Distributed system

Security.

Copy data

Data is stored in the form of block

NameNode

Memory

Local disk

Fsp_w_picpath: mirror fil

Edites: Editing log

YARN:

Distributed resource management framework

Manage the resources of the entire cluster (memory, cpu cores)

Allocate the resources of the scheduling cluster

Tags: Data distributed resource memory mass cluster processing storage management three function component security divide and conquer business replica foundation security tools ideas Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno NVidia Shulou Information Linux Redmi Shulou Tech Info