What does big data mainly study?
The basis of learning big data
1. Java SE, EE (SSM)
90% of big data's frames are written by Java.
2 、 MySQL
SQL on Hadoop
3 、 Linux
Big data's framework is installed on the Linux operating system
What do you need to learn
. In the process of getting started learning big data, I have encountered learning, industry, lack of systematic learning route, systematic learning planning, welcome you to join my big data learning communication skirt: 251956502, skirt files have my big data learning manual, development tools, PDF documents and books, you can download them by yourself.
Big data offline analysis
General processing of Thum1 data (T: maybe 1 day, 1 week, 1 month, 1 year)
A, Hadoop: generally do not choose the latest version, stepping on the pit is difficult to solve
(common, HDES, MapReduce, YARN)
The idea of building environment and processing data
B, Hive: big data's data warehouse
Manipulate data by writing SQL, similar to sql in MySQL database
C, HBase: NOSQL database based on HDFS
Column-oriented storage
D. Collaboration framework:
Sqoop (Bridge: HDFS "=" RDBMS)
Flume: collect information from log files
E. Scheduling framework
Anzkaban
Understand: crotab (included with Linux)
Zeus (Alibaba)
Oozie (cloudera)
F, Frontier Framework extension:
Kylin, impala, ElasticSearch (ES)
Big data real-time analysis
Mainly based on spark framework
Scala:OOP (object-oriented programming) + FP (function is programming)
SparkCore: analogical MapReduce
SparkSQL: analogical hive
SparkStreaming: real-time data processing
Kafka: message queuing
Frontier Framework extension: flink
Alibaba: blink
Big data machine learning
Spark MLlib: machine Learning Library
Pyspark programming: the combination of Python and spark
Recommendation system
Python data analysis
Python machine learning