Skip to main content
Version: Next

Merge stream with same schema


Description

The Merge By Schema processor combines events from two input streams that share the same event schema. It ensures data consistency by only merging events that have identical structures. This processor is essential for:

  • Schema validation
  • Data consistency enforcement
  • Stream synchronization
  • Event structure verification

Required Input

The processor requires two input streams that must have:

  • Identical event schemas
  • Matching property names and types
  • Compatible data structures

Configuration

No additional configuration is required. The processor automatically:

  • Validates schema compatibility
  • Ensures structural consistency
  • Maintains data integrity

Output

The processor forwards events from both input streams, ensuring they maintain their original structure and values.

Example

Input Stream 1 Event

{
"temperature": 25.5,
"humidity": 60,
"timestamp": 1586380104915
}

Input Stream 2 Event

{
"temperature": 26.0,
"humidity": 65,
"timestamp": 1586380105015
}

Output Events

Both events are forwarded as they share the same schema:

{
"temperature": 25.5,
"humidity": 60,
"timestamp": 1586380104915
}
{
"temperature": 26.0,
"humidity": 65,
"timestamp": 1586380105015
}

Use Cases

  1. Data Validation

    • Ensure consistent data structures
    • Validate event schemas
    • Maintain data quality
    • Enforce schema compliance
  2. Stream Synchronization

    • Combine compatible data streams
    • Merge homogeneous data sources
    • Maintain data consistency
    • Ensure structural alignment
  3. Quality Assurance

    • Verify data structure integrity
    • Validate event formats
    • Ensure schema compatibility
    • Maintain data standards

Notes

  • The processor throws an exception if schemas do not match
  • All events maintain their original structure
  • No data transformation is performed
  • Events are forwarded as-is from both streams
  • Schema validation is performed at runtime