Data Engineering · Apache Kafka · Real-time · 2 notebooks
Cold Chain Monitoring
Producers simulate sensors and refrigerated-parcel journeys. Your task is to reconstruct the relationships between those events, determine which parcels are exposed to a temperature anomaly and publish the corresponding alerts.
PythonApache KafkaPandas
Free · No account·Intermediate·2 notebooks
What you'll build
- ✓A connection to a shared Kafka broker and an initial read of both incoming streams
- ✓In-memory state tracking current relationships between containers, sensors and parcels
- ✓Correlation logic that finds the parcels affected by each temperature reading
- ✓Alert messages produced for every parcel exposed to a temperature above the threshold
The 2 lab notebooks
00
Understand and observe the streams
Explore the simulated architecture, connect to Kafka and inspect the first temperature and logistics messages.
01
Build the correlation logic
Maintain current relationships, react to business events, then connect each temperature reading to affected parcels before producing alerts.
Skills covered
- · Reading and producing Kafka messages
- · Event schema interpretation
- · Maintaining current state in memory
- · Heterogeneous stream correlation
- · Handling event arrival order
- · Alert detection and emission
Prerequisites
- · Comfort with Python dictionaries, loops and functions
- · The broker and data producers are preconfigured
- · No installation: online lab