StreamSets Data Collector

StreamSets Data Collector is an easy-to-use modern execution engine for fast data ingestion and light transformations. Data Collector's easy-to-use visual tools let you design, deploy and operate streaming, CDC (change data capture) and batch data pipelines data without hand coding, from the full variety of data sources such as Kafka, S3, Snowflake, Databricks, JDBC, Hive, Salesforce, Oracle and many, many more. Fully instrumented ""smart"" data pipelines let you monitor data in-flight, and are designed to handle data drift with built-in detection and handling. Data Collector pipelines are designed to be platform-agnostic, so you can adapt as needed and avoid vendor lock-in.

Easiest to use tool for data ingestion
abstracts away details
single platform for all patterns
full lifecycle, from design to operations

Pipelines that can be ported across on-premise & cloud platforms

Drift resilient, fully-instrumented ""smart"" pipelines

Enterprise functionality and performance

Usage Instructions

It takes a few minutes to deploy the Azure VM. Once the VM is available, it takes about one minute to start the Data Collector service. The StreamSets Data Collector web based UI will be available on port 18630 . To access Data Collector, enter the following URL in the address bar of your browser:

http://[Public DNS of Azure VM]:18630

For example if your Public DNS is 123.123.123.123, enter http://123.123.123.123:18630 in the browser.