Amazon Redshift adds Apache Iceberg support to eliminate data duplication

This integration reduces the operational overhead for streaming platforms that rely on diverse analytics stacks to process viewer behavior and delivery metrics. By allowing Amazon Redshift to output open-format tables, engineers can avoid the 'data silo' problem where information must be duplicated to be usable in Trino or Snowflake. This move strengthens the AWS ecosystem against specialized data lakehouse competitors by positioning Redshift as a central, interoperable hub for large-scale video metadata. Watch for whether this interoperability leads to a decrease in Redshift Managed Storage usage as teams shift more long-term storage to open S3 buckets.

Amazon Redshift has been building toward deeper Apache Iceberg integration throughout 2025 and 2026. In July 2025, Amazon Redshift announced support for automatic refresh of materialized views defined on external Apache Iceberg tables in Amazon S3, which enabled the service to detect new or deleted data in Iceberg source tables and trigger refreshes without manual intervention. The October 2026 launch extends this trajectory by allowing Redshift itself to produce Iceberg tables as materialized view output, rather than only consuming them as sources.

Five days before the Redshift announcement, AWS separately introduced system-managed write protection for Apache Iceberg materialized views through AWS Glue, which restricts write access so that only AWS Glue can modify the materialized view's data and definition. This governance layer addresses a practical concern for teams sharing Iceberg tables across engines: without write protection, any user with S3 or catalog permissions could alter precomputed results, breaking downstream consumers. The combination of system-managed protection and Redshift's incremental refresh creates a governed pipeline where materialized views remain consistent regardless of who accesses the underlying storage.

Amazon Redshift's Iceberg materialized views support incremental refresh for specific SQL patterns including GROUP BY with COUNT and SUM aggregates, inner joins between Iceberg source tables, and UNION ALL operations, falling back to full recomputation when unsupported constructs are detected. The feature is available on Redshift Serverless and provisioned clusters with RG instance types, though RA3 and DC2 instances are not supported. Source tables must already be in Apache Iceberg format; native Redshift tables, Hive, Parquet, Delta Lake, or Hudi external tables cannot serve as inputs, and Iceberg format version 3 is not yet supported as a source format.