Stage:
Public Alpha
Available on self-hosted:
N/A
Supabase Pipelines creates managed change-data-capture pipelines that replicate your Postgres tables to analytical destinations in near real time. Reading directly from the Postgres Write Ahead Log, each pipeline keeps your analytics data synchronized with your production database.
Key benefits
- Real-time replication: Near real-time data synchronization using Postgres logical replication.
- BigQuery integration: Direct replication to Google's data warehouse.
- Managed destinations: Start with BigQuery while more destinations are evaluated.
- Complete change history: Captures INSERT, UPDATE, DELETE, and TRUNCATE operations.
- Optimized for analytics: Faster queries and lower storage costs through compression.
- Production isolation: Complete separation of analytics and production workloads.
How it works
Pipelines uses Postgres logical replication to capture changes. For BigQuery, a view is created for each table backed by versioned tables.
Pipelines is valuable for:
- Data warehousing and business intelligence
- Historical analysis and audit trails
- Large-scale analytics requiring separation from production
- Compliance scenarios requiring complete data history
Limitations
Tables require primary keys. DDL support (schema changes) is currently in development.
Pipelines provides a powerful alternative to Read Replicas for analytics workloads, optimizing performance while reducing costs.