Automatic Embeddings

Automatic Embeddings provides a system to automatically generate and update vector embeddings using Postgres triggers, queues, and Edge Functions. Embeddings are generated asynchronously with built-in retry logic for failed jobs.

Key benefits

1. Fully automated: No manual embedding management required.
2. Asynchronous processing: Using pgmq (Postgres message queue) for reliable delivery.
3. Retry logic: Built-in error handling for failed embedding jobs.
4. Generic solution: Works with any table and embedding provider.
5. Cron-based processing: Periodic queue processing via pg_cron.

Architecture

Trigger detects INSERT/UPDATE on table. Queue function queues embedding job to pgmq. Cron job processes queue periodically (every 5 minutes). Edge Function generates embeddings via AI API. Update stores embedding back to table.

How it works

1. Create queue using pgmq.create('generate_embeddings')
2. Create generic queue function taking content and column name
3. Set up triggers on tables for automatic queueing
4. Edge Function calls OpenAI or other providers
5. Cron job processes queue at regular intervals

Automatic Embeddings are valuable for:

• Semantic search implementations
• Document similarity systems
• Recommendation engines
• RAG (Retrieval-Augmented Generation)
• Content categorization

Customization

Custom content preparation functions. Different embedding models per table. Batch processing configuration. Custom error handling strategies.

Automatic Embeddings eliminate the manual work of generating and maintaining vector embeddings.