How to implement Spark API map and filter in local mode in Spark API programming

This article is to share with you about how to use local mode to carry out Spark API map and filter in Spark API programming. The editor thinks it is very practical, so I share it with you. I hope you can get something after reading this article.

First test spark API in spark's native mode and run spark-shell as local:

Let's take a look at the filter operation:

We use the most authentic style of scala functional programming:

From the results, we can see that the result is the same as the previous step-by-step approach.

But in this way, it is not only a compound scala style writing, but also in line with the spark application style writing. In spark programming, most functions can be realized with only one line of code.

The above is how to use local mode to carry out Spark API map and filter in Spark API programming. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.