What is the essence of Spark Streaming

This article introduces the knowledge of "what is the essence of Spark Streaming". In the operation of actual cases, many people will encounter such a dilemma. Then let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

An understanding of Spark Streaming

Spark Streaming is streaming computing, and the current era is an era of streaming processing. All data is invalid if it is not streamed or has nothing to do with streaming.

Streaming is our first impression of big data, not batch processing and data mining. Of course, the strength of Spark is that his streaming processing can directly use the results of machine learning, graph computing, SparkSQL and Spark R online.

Spark Streaming differs from other subframeworks in that Spark Streaming is much like an application based on Spark Core.

two。 How to start? To see the inside story through the observation of the inflow, outflow and processing of Spark Streaming data! Observe the operation of the Batch Interval by making it large enough, such as once a minute, or once every 5 minutes to observe the spark UI. four。 Experiment

1 launch nc-lk 9999, publish the application to run on the Spark cluster, and send the following data in nc

2016-05-01 mahout2016-05-01 scala2016-05-01 hadoo2016-05-01 spark

2 after the application receives the data, it will have the following output

2016-05-01 scala2016-05-01 spark

3 from the history server of Spark, we observe that the following jobs is running

This is the end of "what is the essence of Spark Streaming"? thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!