|StreamSets Data Collector for HDInsight Cloud||A key step in modernizing your data processing architecture is to upgrade how you move big data from logs, IoT sensors, and other sources through to your enterprise data hub. An integrated solution combining StreamSets Data Collector makes it possible to continually feed your analytics applications consumption-ready data with efficiency, operational control, and agility.
StreamSets Data Collector deploys on top of Azure either as a separate VM, using Cloudera Manager parcel onto your Cloudera cluster or as HDInsight application. It provides a full-featured integrated development environment (IDE) that lets you design, test, deploy, and manage any-to-any ingest pipelines that mesh stream and batch data, and include a variety of in-stream transformations—all without having to write custom code. StreamSets Data Collector lets you build data flows, including numerous Big Data components such as HDFS, Kafka, Solr, Hive, HBASE, and Kudu. Once StreamSets Data Collector is running on edge or in your Hadoop cluster, you get real-time monitoring for both data anomalies and data flow operations, including threshold-based alerting, anomaly detection, and automatic remediation of error records. Because it is architected to logically isolate each stage in a pipeline, you can meet new business requirements by dropping in new processors and connectors without code and with minimal downtime.