Get the App
SLTechnology News&Howtos  ›  Internet Technology  › 

Spark-submit test task submission

Shulou Source: shulou.com Published: 2022-06-03 04:52:17 10月04日 Update

Spark-submit test task submission

./ spark-submit --keytab /usr/local/noah/basp-dataprocess-log/conf/noah.keytabs --principal noah@BJ.CTC --class org.apache.spark.examples.SparkPi --master local[*] ../ lib/spark-examples-1.6.3.2.6.0.3-8-hadoop2.7.3.2.6.0.3-8.jar 100./ spark-submit --class org.apache.spark.examples.SparkPi --master local[*] /app/hdp/2.6.0.3-8/spark2/examples/jars/spark-examples_2.11-2.1.0.2.6.0.3-8.jar 100 /usr/hdp/2.6.0.3-8/spark2/bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode cluster /app/hdp/2.6.0.3-8/spark2/examples/jars/spark-examples_2.11-2.1.0.2.6.0.3-8.jar 100./ spark-submit --class org.apache.spark.examples.SparkPi --master yarn-client /app/hdp/2.6.0.3-8/spark2/examples/jars/spark-examples_2.11-2.1.0.2.6.0.3-8.jar 100

Tags: Tasks testing Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno Shulou Information MySQL Shulou Tech Info Xiaomi Docker