How to treat Hadoop
How to treat Hadoop, I believe that many inexperienced people are at a loss about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.
1. 2013 global data total 4.4 ZB estimates 2020 44ZB 1ZB=1000EB=1000 PB 1PB=1024TB2. The data generated by individuals is growing the data generated by the Internet of things 3. Big data is better than a good algorithm
For some applications (such as recommending movies and music based on past preferences), no matter how good the algorithm is, the recommendation effect based on small data is often not as good as that of general algorithms based on a large amount of available data.
Problems encountered in reading multiple hard drives
Hardware failure
How to ensure the correctness of the data
MapReduce
Each query needs to process the vast majority of the entire dataset or at least one dataset
In essence, a batch processing system is not suitable for interactive analysis.
Hard disk development trend: the improvement of addressing time is far less than the improvement of transmission rate. MapReduce is suitable for write-once and multiple-read applications, while relational databases are more suitable for continuously updated data sets. Hadoop is very effective grid computing for unstructured or semi-structured data.
Hadoop tries to store data on computing nodes to achieve fast local access to the data. Data localization is the core of Hadoop data processing, and good performance is obtained as a result.
Three Design goals of Voluntary Computing hadoop
Provide services for assignments that can be completed in just a few minutes or hours
Runs in the same data center with a high-speed network connection
The computers in the data center are all reliable and specialized hardware
After reading the above, have you mastered the way to view Hadoop? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!
