Supabase ETL is a change-data-capture pipeline built in Rust that replicates your Postgres tables to analytical destinations in near real-time. Reading directly from the Postgres Write Ahead Log, ETL ensures your analytics data stays synchronized with your production database.
Key benefits
- Real-time replication: Near real-time data synchronization using Postgres logical replication.
- Analytics Buckets support: Replicate to Iceberg format for large-scale analytics.
- BigQuery integration: Direct replication to Google's data warehouse.
- Complete change history: Captures INSERT, UPDATE, DELETE, and TRUNCATE operations.
- Optimized for analytics: Faster queries and lower storage costs through compression.
- Production isolation: Complete separation of analytics and production workloads.
How it works
ETL uses Postgres logical replication to capture changes. Each replicated table includes a cdc_operation column tracking the type of change. For Analytics Buckets, data is stored in append-only changelog format using Parquet files. For BigQuery, a view is created for each table backed by versioned tables.
Supabase ETL is valuable for:
- Data warehousing and business intelligence
- Historical analysis and audit trails
- Large-scale analytics requiring separation from production
- Compliance scenarios requiring complete data history
Limitations
Tables require primary keys. DDL support (schema changes) is currently in development.
Supabase ETL provides a powerful alternative to Read Replicas for analytics workloads, optimizing performance while reducing costs.