Shulou(Shulou.com)06/01 Report--

This article mainly introduces "the method of submitting tasks to the cluster by spark". In the daily operation, I believe that many people have doubts about the method of submitting tasks to the cluster by spark. The editor has consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the doubts of "how to submit tasks to the cluster by spark". Next, please follow the editor to study!

# Run on a Spark standalone cluster in client deploy mode (running cluster mode, client publishing mode)

Spark-submit\

-- class org.apache.spark.examples.SparkPi\

-- master spark://207.184.161.138:7077\

-- executor-memory 20g\

-total-executor-cores 100\

/ path/to/examples.jar\

one thousand

Such as: Run on a Spark standalone cluster in cluster deploy mode with supervise (release mode is cluster, then jar package, every worker must have this jar package)

At this point, the study on "the method of submitting tasks to the cluster by spark" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.