The JValue Project

Making open data safe, easy, and reliable to use

Job / Abschlussarbeit Model Compilation to Streaming Backends

Wir suchen jemanden kompetent im Compilerbau, der oder die Lust hat, sich eines wichtigen Spezialthemas anzunehmen, nämlich offene Daten nutzbar zu machen. Es folgt eine Aufgabenbeschreibung für eine Abschlussarbeit, aber wir bieten das für alles an: Studentischer Job, Abschlussarbeit, Promotion / Wimi Stelle:

Model Compilation to Streaming Backends

The goal of the thesis is to develop a compiler that turns an ETL pipeline model (the “program”) into a configuration for an event streaming framework (the “target architecture”). To start things easy, we want to compile SQL to Kafka Streams in such a way that an SQL schema definition configures a Kafka Streams instance so that the Kafka instance can load a CSV file and save it into a PostgreSQL database. If this works well, we will increase complexity: Not just a source schema (the “E” in ETL) but also transformation rules (the “T”) and a target schema (the “L”); not just Kafka Streams as a target architecture, but also Spark, Flink, and others.

This thesis is part of the JValue project. The mission of the JValue project is to make open data easy, reliable, and safe to use.

Thesis Description – Model Compilation to Streaming Backend

Bei Interesse gern direkt an mich wenden.

Dirk Riehle