Shulou(Shulou.com)06/03 Report--

The international version of Azure can support Spark applications normally, but when we are using the Chinese version of Azure, we find an introduction to the Spark service on the official website of the Chinese version of Azure, but when we actually use it, we find that it cannot be created. Only Storm,HBase and Hadoop. As shown in the figure:

So the question is, does HDInsight of the Chinese version of Azure support Spark?

Of course the answer is yes, but you need to customize it:

First, we need to create a PAAS layer SQL Database to store the metadata of Hive and Oozie.

Create a HADOOP cluster based on the PAAS tier:

HDInsight 3.1 clusters will install Spark 1.0.2

HDInsight 3.2 clusters will install Spark 1.2.0

Configure cluster nodes and virtual machines:

Configure access users, and check configure Oozie Cloud Storage:

Configure Oozie to store the SQL Database database:

Configure storage:

Use a script to install Spark:

Insert Spark configuration steps:

Https://hdiconfigactions.blob.core.windows.net/sparkconfigactionv03/spark-installer-v03.ps1 . Select "header node only" for node type

Finally, click the "create" button.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.