A distributed deep learning library for Apache Spark with high performance and efficiently scale out
BigDL is a distributed deep learning framework organically built for Apache Spark. It makes easier to develop deep learning applications as standard Spark programs using Scala or Python and then run those applications on existing Spark or Hadoop clusters without expensive, specialized hardware. Apache Spark enables applications to seamlessly scale across Intel® Xeon® processor-based nodes on a massive scale and BigDL delivers extraordinary performance across the Intel® Xeon® processor-based infrastructure by leveraging the highly-optimized Intel® Math Kernel Library. Plus, BigDL provides support for rich deep learning, and delivers feature parity with popular open source deep learning frameworks such as Caffee, Torch, Tensorflow, and others. BigDL is supported and complemented by a high-level end-to-end pipelines Analytics Zoo platform (https://azuremarketplace.microsoft.com/en-us/marketplace/apps/intel-bigdl.analytics-zoo) to further facilitate building Apache Spark-based AI applications.