/explore

Click through on any links that interest you or select the planets on the right to continue exploring the Outer Web.
You are here

 rmoff.net
Data Engineering in 2022: Storage and Access
| | www.ssp.sh
Data Lake Table Formats (Open Table Formats)
1.8 parsecs away

Travel
| | Prominent table formats include Delta Lake, Apache Iceberg, and Apache Hudi.
| | www.starburst.io
What is a data lake? | Starburst
1.2 parsecs away

Travel
| | A Data lake is a cost effective storage technology designed to house large amounts of data from multiple sources in the cloud or on-prem.
| | jack-vanlightly.com
Table format interoperability, future or fantasy? - Jack Vanlightly
1.1 parsecs away

Travel
| | In the world of open table formats (Apache Iceberg, Delta Lake, Apache Hudi, Apache Paimon, etc), an emerging trend is to provide interoperability between table formats by cross-publishing metadata. It allows a table to be written in table format X but read in format Y or Z. Cross-publishing is the idea of a table having: * A primary table format that you write to. * Equivalent metadata files of one or more secondary formats that allow the table to be read as if it were of that secondary format.
| | ministryofjustice.github.io
Building a transaction data lake using Amazon Athena, Apache Iceberg and dbt - Data and Analytics Engineering - Ministry of Justice
7.3 parsecs away

Travel
| How we leveraged Amazon Athena, along with the Apache Iceberg table format and the dbt SQL management framework, to build robust, scalable and maintainable ELT (extract, load, transform) pipelines.