/explore

Click through on any links that interest you or select the planets on the right to continue exploring the Outer Web.
You are here

 risingwave.com
Why Choose RisingWave Over Apache Spark Structured Streaming - RisingWave: Real-Time Event Streaming Platform
| | florimond.dev
Streaming Applications with Apache Kafka: The Opening - Florimond Manca
5.0 parsecs away

Travel
| | I'm starting a series on building streaming apps with Apache Kafka - here's why!
| | mertkavi.com
Building common components in Data Lakehouse is a necessity ยท Mert Kavi
3.5 parsecs away

Travel
| | The article discusses the challenges faced by scalable data teams in managing and processing large amounts of data and proposes a system that includes separating storage, compute, and metadata, using distributed file systems and columnar data formats, real-time data ingestion with CDC tools, and efficient querying through composable components and a joint query/compute engine.
| | quix.io
A very detailed comparison of Python stream processing libraries
3.0 parsecs away

Travel
| | Dive deep into the performance and limitations of Python client libraries to choose the best stream processing solution for your data.
| | tech.scribd.com
Kafka to Delta Lake, as fast as possible | Scribd Technology
6.2 parsecs away

Travel
| Streaming data from Apache Kafka into Delta Lake is an integral part of Scribd's data platform, but has been challenging to manage and scale. We use Spark Structured Streaming jobs to read data from Kafka topics and write that data into Delta Lake tables. This approach gets the job done but in production our experience has convinced us that a different approach is necessary to efficiently bring data from Kafka to Delta Lake. To serve this need, we created kafka-delta-ingest.