Get the App
SLTechnology News&Howtos  ›  Internet Technology  › 

What are the three components of hadoop?

Shulou Source: shulou.com Published: 2022-06-02 13:03:33 11月17日 Update

This article mainly introduces what are the three major components of hadoop. It is very detailed and has a certain reference value. Friends who are interested must read it!

Hadoop three major components: 1, HDFS, a highly reliable, high-throughput distributed file system; 2, MapReduce, a distributed offline parallel computing framework; 3, Yarn, distributed resource management framework.

The three major components of hadoop:

1 、 HDFS

A distributed file system with high reliability and high throughput

Store huge amounts of data

Distributed system

Security.

Copy data

The data is stored in block, 128m

For example: 200M-128M 72m

2 、 MapReduce

A distributed off-line parallel computing framework

The processing of massive data

Distributed system

Thoughts:

divide and rule

Big data set is divided into small data sets.

Logical business processing (map) for each dataset

Merge statistics results (reduce)

3 、 Yarn

Distributed resource management framework

Manage the resources of the entire cluster (memory, CPU cores)

Allocate the resources of the scheduling cluster

These are all the contents of the article "what are the three components of hadoop?" Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Tags: Distributed data three components framework resources management content throughput files mass articles systems resource management clustering throughput processing storage security divide and conquer Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno Apple Shulou Technology MySQL Microsoft vpn