Skip to main content

Moving to the Apache Incubator

· 4 min read

8 minutes to read

Great news: StreamPipes is moving to the Apache Incubator!

The StreamPipes journey so far

We started to build the early foundations of StreamPipes back in 2014 as part of the EU research project ProaSense. Our goal was to make event processing accessible to end users with a focus on the Industrial IoT, without the technical expertise required by stream processing tooling. After some very early prototypes (see an early screenshot below :-)) demonstrating the usefulness of the pipeline editor and the underlying semantics-based data model in two manufacturing-related use cases, we open-sourced StreamPipes in 2017 and continued development in various research projects. Since then, we presented and showcased StreamPipes at various events: fairs, meetups, conference talks and so on. For instance, in 2019 we attended over 20 events where we gave demos or talked about StreamPipes. It is great to see that the solution is now steadily getting increased traction in the Industrial IoT space.

1st prototype of the StreamPipes UI1st prototype of the StreamPipes UI

Moving to the Apache Incubator

Earlier this year, we discussed how to continue the development of StreamPipes. For the core team at FZI, our main goal was to widen the developer base, attract external developers and to make StreamPipes a real community-driven open source project. We had a look at various open source foundations (namely Apache, CNCF and Eclipse), but from the beginning, there was no doubt that we wanted to see StreamPipes in the Apache Software Foundation. Using a lot of Apache Big Data projects under the hood (Apache Kafka, Apache Flink and others), an emerging Apache IoT ecosystem, and the community-driven, open development process of ASF projects, this was an easy decision. Finally, a few weeks before we went to ApacheCon North America in Las Vegas in September this year to talk about StreamPipes, we discovered the awesome Apache PLC4X project, which perfectly aligns with StreamPipes tackling the hard task to connect proprietary industrial protocols with modern software tooling. We met Chris Dutz together with other people from PLC4X and other Apache projects in Las Vegas and were overwhelmed by the Apache Way and the great collaboration between the various projects within the ASF.

So after coming back home, we immediately started to write the proposal to enter the Apache Incubator, which got accepted on 2019-11-11.

New project setup

With the great help from our mentors, we already performed some initial steps required to move our infrastructure over to the ASF, which will continue during the next week:


We would like to thank Chris Dutz for his great support in doing the first steps in the Apache Incubator.

We would also like to thank our employee, FZI Research Center for Information Technology, for the continued great support - starting from giving us the opportunity to present StreamPipes at large fairs such as Hannover Messe, over media support such as creating the great StreamPipes logo, up to the fast legal support we received when we got accepted by the Apache Incubator. If you want to build cool stuff in a great applied research institute while pursuing a PhD at KIT, FZI is surely the best place to be ;-)

Let's continue development and transform Apache StreamPipes (incubating) into the next Apache community!