What are the characteristics of BigDL

This article introduces the relevant knowledge of "what are the characteristics of BigDL". In the operation of practical cases, many people will encounter such a dilemma. Then let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Properties:

Rich deep learning support. BigDL mimics Torch by providing comprehensive support for deep learning, including numerical computation (through Tensor) and high-level neural networks. In addition, users can use BigDL to load pre-trained Caffe or Torch models into Spark programs.

Extremely high performance. To achieve high performance, BigDL uses Intel MKL and multithreaded programming in each Spark task. Therefore, it is several orders of magnitude faster than out-of-the-box Caffe, Torch, or TensorFlow on a single-node Xeon.

Scale out effectively. Through the use of Apache Spark and efficient implementation of synchronous SGD, BigDL can comprehensively reduce communication on Spark, effectively scale out, and perform data analysis on the "big data scale".

Use the scene:

You want to analyze huge amounts of data (stored on HDFS, HBase, Hive) in big data Cloud (Hadoop/Spark).

You want to add deep learning (training or prediction) to your big data (Spark) program and / or workflow.

You want to use existing Hadoop/Spark clusters to run deep learning programs, which can then be dynamically shared with other workloads (e.g. ETL, data warehouse, functional engine, classic machine learning, image analysis, etc.).

This is the end of the content of "what are the characteristics of BigDL"? thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!