StreamPipes version 0.63.0 has been released! Following our release plan to alternate between user-faced and backend-oriented releases, this release contains many (more than 70) features and improvements, mainly on the backend side. The new version is available on Docker Hub and can be downloaded using our installer.
For this release, we focused on improving the stability and resilience of StreamPipes, adding many new features "under the hood".
Backend & Data Model
- First, v0.63.0 brings support for binary messaging formats. Prior to 0.63.0, the only messaging format used at runtime was JSON. Now, several other binary formats are supported: Smile and CBOR as binary JSON representations and FST, a fast Java-based serializer.
- Second, we improved resilience of pipelines. Now, when StreamPipes is stopped, all running pipelines are stopped beforehand and can be restarted later on, so that you don't end up with orphaned pipelines anymore.
We also extended the data model and SDK with new features:
- Pipeline elements can now provide their own assets such as documentation. Simply add an icon and a markdown file containing the documentation of the pipeline element and both will be visible directly in the pipeline editor after the installation of the pipeline element.
- We added several new static properties to define required user input: SecretStaticProperties can be used to define secret values (e.g., passwords). In addition, StaticPropertyAlternatives were introduced to let users select between multiple combinations of static properties (e.g., a time window and a count window along with window settings).
In addition, we migrated all pipeline element container and the backend to Spring Boot applications. The triple store used to store pipeline element descriptions has been extracted to an own microservice. Btw, have you seen the new helm chart that simplifies running StreamPipes on Kubernetes?
UI and Pipeline Editor
We improved the interactive user guide, so that it now depends on pipeline elements that are also available in the lite version. Three different interactive guides exist: One that teaches you how to create pipelines, another one that illustrates how to connect data with StreamPipes connect and a third one that shows how to create live visualizations using the built-in dashboard.
As user feedback is absolutely important to help us improving StreamPipes, we added a feature that allows users to directly give feedback from the StreamPipes UI. Simply click the feedback icon in the top navigation bar and submit your anonymous feedback to us!
Finally, there are also some functional improvements in the pipeline editor: Pipeline validation has improved and gives direct feedback. And, finally, you can now also add multiple dashboards sink to a single pipeline :-)
And a nice feature you can't see but which you'll probably feel: All UI files are now Gzip compressed by default, so that StreamPipes will now load much faster.
Pipeline Elements
StreamPipes 0.63.0 includes several new pipeline elements:
- A sink for Apache IoTDB
- A sink for PostgreSQL
- A processor to convert boolean values
- A processor to extract timestamps
- A processor to compute trigonometric functions
StreamPipes Connect
We added more adapters and improved some adapters to StreamPipes Connect:
- An improved MQTT adapter that can handle authentication
- A new MySQL adapter
- An improved OPC-UA adapter
- A new random data generator that eases testing of pipeline elements
In addition, we completely reworked the Connect UI. The schema view now lets you edit events more conveniently and timestamp fields can be directly marked in the UI.
Documentation
We updated and restructured the user guide, which now consists of four parts: Introduction, Tour, Installation and First Steps. We also updated all screenshots to reflect the current look and feel of StreamPipes.
In addition, the developer guide was further extended (e.g., there is now a new tutorial on creating data sinks). Maven archetypes are now the recommended way to create new pipeline elements.
Other features and improvements
- An improved installer that lets you choose between two StreamPipes versions for different hardware setups (full or lite, previously named big data and desktop)
- We updated some core libraries like Apache Flink and Apache Kafka to newer versions.
- Improved support for Firefox and Edge: Besides Chrome, StreamPipes should also behave and look similar in Firefox and Edge browsers.
See the release notes for a complete list of new features and improvements.
Feedback
We are absolutely open to your suggestions for further improvements! Let us know (by mail, slack or twitter) and we'll consider your feature request in the next release!