In this talk from the StreamSets DataOps 2019 conference, I provide an overview of the data pipeline for the OmniSci F1 Demo. Using StreamSets Data Collector in concert with Apache Kafka and OmniSciDB, you can create a full real-time data pipeline for telemetry data using only open-source components.
The talk outlines using the UDP listener for StreamSets to collect packets from the F1 2018 game, writing the packets to Kafka, reading from Kafka and using Groovy to parse the packets, and using the OmniSci JDBC driver to insert the data into one of nine OmniSciDB tables. With this workflow, you have a robust platform for accelerated analytics, using the power of GPUs for fast computation.
GitHub: https://github.com/omnisci/vehicle-telematics-analytics-demo
Speakerdeck: https://speakerdeck.com/omnisci/the-f1-demo-streaming-real-time-telemetry-using-apache-kafka-and-streamsets