StreamPipes has joined the Apache Incubator. Subscribe to our new to stay up-to-date!

Features

Flexible Data Pipelines

  • Easy to use. Web-based user interface to create stream processing pipelines in a graphical editor.')
  • Powerful. Select data streams, add transformation modules and choose a data sink. StreamPipes cares about execution details of your pipeline.')
  • Intelligent. Our built-in semantics recommend you elements you can connect and disallow you to connect elements that wo not work.')

Quickly connect IoT data with StreamPipes Connect.

  • Streams & Sets StreamPipes Connect supports both data sets and real-time data streams.
  • Generic adapters StreamPipes Connect includes a variety of adapters for popular communication protocols and technologies such as HTTP/REST, MQTT, Kafka, OPC UA or ROS. Adapters can be easily configured right from the user interface - without any coding skills.
  • APIs In addition, Web APIs (e.g., Slack, Wikipedia or open data sources) can be used to ingest data into StreamPipes. Specific adapters can be exported and shared.

Rich set of data processors and data sinks.

Data Processors

  • Filter, Aggregate, Enrich. You need less data, more data or other data? Stateless processing components transform your input streams.')
  • Advanced Analytics. Advanced methods (e.g., ML-algorithms) can be used as pipeline elements.')
  • Pattern Detection. We have developed modules supporting pattern detection, e.g., detection of sequences over time.')

Data Sinks

  • Visualize. Create real-time visualizations and observe your KPIs and production systems from web-based cockpits.')
  • Notify. Detect problems or other situations of interest and notify the people in your organization who need to be informed.')
  • Store. Harmonize your data stream and store it in third-party systems, e.g. for offline analysis. We have adapters for Elasticsearch, HDFS, Cassandra and NoSQL databases.')

Visualize Live Data.

  • Real-Time Dashboard. To quickly gather insights, pipelines can make use of the real-time dashboard to visualize live data.')
  • Charts. The live dashboard includes several chart types such as line charts, traffic lights, gauges, tables and number values.')

Easy to extend with the built-in SDK.

  • Analytics Microservices. Pipeline elements in StreamPipes are encapsulated as standalone microservices. This makes it easy to add new pipeline elements once new requirements arise. Data processors and sinks can be easily installed from the Pipeline Element Marketplace.')
  • SDK. The developer-oriented Software Development Kit (SDK) and Maven archetypes make it easy to create new processing elements using one of several available wrappers (e.g., Standalone, Apache Flink and Apache Spark).')