HDInsight Spark solution provides Log Analytics, Monitoring and Alerting capabilities for HDInsight Spark.
With this solution, you can:
Monitor multiple cluster(s) in one or multiple subscriptions
Troubleshoot Spark issues faster by gaining access to common log's as well as various Spark metrics
Set the alerts on thresholds that matter to you the most.
HDInsight Spark Linux cluster