Storage

Vector Buckets

Store, index, and query vector embeddings at scale with similarity search.

Vector buckets enable efficient storage and similarity search of vector embeddings. Built on S3-compatible storage, they provide high-performance semantic search capabilities for AI and machine learning applications.

What are Vector buckets?

Vector buckets are specialized storage containers optimized for vector data. Unlike traditional databases optimized for transactional queries, vector buckets use specialized indexing and distance metrics to perform fast similarity searches across millions of embeddings.

Each vector bucket contains:

  • Indexes - Organized collections of vectors with consistent dimensions and distance metrics
  • Vectors - Embeddings with associated metadata for filtering and enrichment
  • Metadata - Additional context about vectors (text, tags, IDs, etc.)

Key features

  • Similarity Search - Find semantically similar vectors using cosine, euclidean, or L2 distance metrics
  • Metadata Filtering - Filter results by associated metadata before/after similarity search
  • Batch Operations - Insert, update, and query up to 500 vectors per request
  • Scalable Storage - Store millions of vectors in a single index
  • S3 Native - Built on proven S3 infrastructure for reliability and durability

Ideal use cases

Vector buckets excel at:

  • Semantic Search - Find documents or images similar to a query
  • Recommendation Systems - Suggest products, content, or connections based on embeddings
  • Clustering & Anomaly Detection - Group similar items or identify outliers
  • Image Search - Retrieve visually similar images from large catalogs
  • RAG (Retrieval-Augmented Generation) - Find relevant context for LLM queries
  • Personalization - Recommend tailored content based on user embeddings

How Vector buckets work

  1. Create a bucket to organize your vector data
  2. Create indexes within the bucket with specified dimensions and distance metrics
  3. Store vectors with embeddings and optional metadata
  4. Query vectors using similarity search to find nearest neighbors

The system automatically handles indexing and optimization, making searches fast and reliable even with millions of vectors.

Next steps

Get started by learning how to create vector buckets or dive into storing vectors.