Explore >> Select a destination
|
You are here 
|
www.fivetran.com | ||
| | | | |
jack-vanlightly.com
|
|
| | | | | In the world of open table formats (Apache Iceberg, Delta Lake, Apache Hudi, Apache Paimon, etc), an emerging trend is to provide interoperability between table formats by cross-publishing metadata. It allows a table to be written in table format X but read in format Y or Z. Cross-publishing is the idea of a table having: * A primary table format that you write to. * Equivalent metadata files of one or more secondary formats that allow the table to be read as if it were of that secondary format. | |
| | | | |
rmoff.net
|
|
| | | | | [AI summary] This article discusses the evolution of data engineering in 2022, focusing on storage and access methods for analytical data, including the transition from traditional data warehouses to modern data lakehouses and open formats. | |
| | | | |
www.dquach.com
|
|
| | | | | [AI summary] The article discusses the current state of data engineering in Q3 2024, focusing on Open Table Formats (OTFs) like Apache Iceberg, Databricks, and Delta Lake, along with updates on AI prompt engineering, new tools like DuckDB, and cloud provider advancements. | |
| | | | |
jack-vanlightly.com
|
|
| | | In the previous post, I covered append-only tables, a common table type in analytics used often for ingesting data into a data lake or modeling streams between stream processor jobs. I had promised to cover native support for changelog streams, aka change data capture (CDC), but before I do so, I think we should first look at how the table formats support the ingestion of data with row-level operations (insert, update, delete) rather than query-level operations that are commonly used in SQL batch commands. | ||
