Back
BigQuery Wrapper

BigQuery Wrapper

Overview

BigQuery is a completely serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data, with BI, machine learning and AI built in.

The BigQuery Wrapper allows you to read and write data from BigQuery within your Postgres database.

Supported Data Types

Postgres TypeBigQuery Type
booleanBOOL
bigintINT64
double precisionFLOAT64
numericNUMERIC
textSTRING
varcharSTRING
dateDATE
timestampDATETIME
timestampTIMESTAMP

Preparation

Before you get started, make sure the wrappers extension is installed on your database:


_10
create extension if not exists wrappers with schema extensions;

and then create the foreign data wrapper:


_10
create foreign data wrapper bigquery_wrapper
_10
handler big_query_fdw_handler
_10
validator big_query_fdw_validator;

Secure your credentials (optional)

By default, Postgres stores FDW credentials inide pg_catalog.pg_foreign_server in plain text. Anyone with access to this table will be able to view these credentials. Wrappers is designed to work with Vault, which provides an additional level of security for storing credentials. We recommend using Vault to store your credentials.


_15
-- Save your BigQuery service account json in Vault and retrieve the `key_id`
_15
insert into vault.secrets (name, secret)
_15
values (
_15
'bigquery',
_15
'
_15
{
_15
"type": "service_account",
_15
"project_id": "your_gcp_project_id",
_15
"private_key_id": "your_private_key_id",
_15
"private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
_15
...
_15
}
_15
'
_15
)
_15
returning key_id;

Connecting to BigQuery

We need to provide Postgres with the credentials to connect to BigQuery, and any additional options. We can do this using the create server command:

With Vault:


_10
create server bigquery_server
_10
foreign data wrapper bigquery_wrapper
_10
options (
_10
sa_key_id '<key_ID>', -- The Key ID from above.
_10
project_id 'your_gcp_project_id',
_10
dataset_id 'your_gcp_dataset_id'
_10
);

Without Vault:


_13
create server bigquery_server
_13
foreign data wrapper bigquery_wrapper
_13
options (
_13
sa_key '
_13
{
_13
"type": "service_account",
_13
"project_id": "your_gcp_project_id",
_13
...
_13
}
_13
',
_13
project_id 'your_gcp_project_id',
_13
dataset_id 'your_gcp_dataset_id'
_13
);

Learn more on our doc page.

Resources

Details

DeveloperSupabase
DocumentationLearn