Shulou(Shulou.com)06/02 Report--

This article mainly introduces what are the three major components of hadoop. It is very detailed and has a certain reference value. Friends who are interested must read it!

Hadoop three major components: 1, HDFS, a highly reliable, high-throughput distributed file system; 2, MapReduce, a distributed offline parallel computing framework; 3, Yarn, distributed resource management framework.

The three major components of hadoop:

1 、 HDFS

A distributed file system with high reliability and high throughput

Store huge amounts of data

Distributed system

Security.

Copy data

The data is stored in block, 128m

For example: 200M-128M 72m

2 、 MapReduce

A distributed off-line parallel computing framework

The processing of massive data

Distributed system

Thoughts:

divide and rule

Big data set is divided into small data sets.

Logical business processing (map) for each dataset

Merge statistics results (reduce)

3 、 Yarn

Distributed resource management framework

Manage the resources of the entire cluster (memory, CPU cores)

Allocate the resources of the scheduling cluster

These are all the contents of the article "what are the three components of hadoop?" Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.