Snowflake with Iceberg Tables

Snowflake began as a closed warehouse with its own storage format (Snowflake). Its Apache Iceberg 129 tables keep data and metadata as ordinary Iceberg files, either in Snowflake-managed storage or in your own bucket through an external volume, an account-level object holding the IAM credentials for S3, Google Cloud 1 Storage or Azure 6 Storage. The documentation lists Iceberg specification versions 1, 2 and 3. Two catalog modes exist:

Snowflake as the catalog: full read and write, and "Snowflake handles all lifecycle maintenance, such as compaction". Other engines read these tables through Snowflake's catalog, or through Snowflake Open Catalog, the managed Apache Polaris 129 of Apache Polaris.
An external catalog (Glue, Open Catalog, any Iceberg REST catalog) through a catalog integration, or a catalog-linked database that discovers and syncs a remote REST catalog's tables. Snowflake can then write to version 2 and 3 tables, but "does not assume any life-cycle management on the table".

Billing follows the storage choice: with an external volume "your cloud storage provider bills you directly", otherwise Snowflake charges its storage rate. Queries, writes and maintenance consume virtual-warehouse credits plus cloud services, and Snowflake bills cross-region or cross-cloud transfer for Snowflake-managed tables.

BookNest's orders would land in its own bucket with Snowflake as the catalog like this:

An Iceberg table in BookNest's own S3 bucket (from documentation, not run here)

CREATE EXTERNAL VOLUME booknest_lake
  STORAGE_LOCATIONS = ((
    NAME = 'booknest-us-east-1'
    STORAGE_PROVIDER = 'S3'
    STORAGE_BASE_URL = 's3://booknest-lake/warehouse/'
    STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::111122223333:role/snowflake-lake'))
  ALLOW_WRITES = TRUE;
CREATE ICEBERG TABLE orders (order_id NUMBER(19,0), customer_id NUMBER(19,0),
    order_ts TIMESTAMP_LTZ(6), channel STRING, status STRING, total NUMBER(10,2))
  CATALOG = 'SNOWFLAKE' EXTERNAL_VOLUME = 'booknest_lake' BASE_LOCATION = 'orders'
  ICEBERG_VERSION = 3;

For BookNest this is the most familiar path from a warehouse: the tables stay SQL-first and Snowflake-fast, yet Spark 129 jobs can read the same Iceberg files without exporting anything.

MENU