Database

Logflare

Logflare is a centralized web-based log management solution to easily access Cloudflare, Vercel & Elixir logs.

The Logflare Wrapper allows you to read data from Logflare endpoints within your Postgres database.

Preparation

Before you can query Logflare, you need to enable the Wrappers extension and store your credentials in Postgres.

Enable Wrappers

Make sure the wrappers extension is installed on your database:

1
create extension if not exists wrappers with schema extensions;

Enable the Logflare Wrapper

Enable the logflare_wrapper FDW:

1
2
3
create foreign data wrapper logflare_wrapper  handler logflare_fdw_handler  validator logflare_fdw_validator;

Store your credentials (optional)

By default, Postgres stores FDW credentials inside pg_catalog.pg_foreign_server in plain text. Anyone with access to this table will be able to view these credentials. Wrappers is designed to work with Vault, which provides an additional level of security for storing credentials. We recommend using Vault to store your credentials.

1
2
3
4
5
6
7
-- Save your Logflare API key in Vault and retrieve the `key_id`insert into vault.secrets (name, secret)values (  'logflare',  'YOUR_SECRET')returning key_id;

Connecting to Logflare

We need to provide Postgres with the credentials to connect to Logflare, and any additional options. We can do this using the create server command:

1
2
3
4
5
create server logflare_server  foreign data wrapper logflare_wrapper  options (    api_key_id '<key_ID>' -- The Key ID from above.  );

Create a schema

We recommend creating a schema to hold all the foreign tables:

1
create schema if not exists logflare;

Options

The full list of foreign table options are below:

  • endpoint - Logflare endpoint UUID or name, required.

Entities

Logflare

This is an object representing Logflare endpoint data.

Ref: Logflare docs

Operations

ObjectSelectInsertUpdateDeleteTruncate
Logflare✅❌❌❌❌

Usage

1
2
3
4
5
6
7
8
9
create foreign table logflare.my_logflare_table (  id bigint,  name text,  _result text)  server logflare_server  options (    endpoint '9dd9a6f6-8e9b-4fa4-b682-4f2f5cd99da3'  );

Notes

Meta Column _result:
  • Data type must be text
  • Stores the whole result record in JSON string format
  • Use JSON queries to extract fields: _result::json->>'field_name'
Query Parameters:
  • Use parameter columns with prefix _param_
  • Example: _param_org_id, _param_iso_timestamp_start
  • Parameters are passed to the Logflare endpoint

Query Pushdown Support

This FDW doesn't support query pushdown.

Limitations

This section describes important limitations and considerations when using this FDW:

  • Full result sets are loaded into memory, which can impact PostgreSQL performance with large datasets
  • Parameter names must be prefixed with 'param' and match the expected endpoint parameters exactly
  • Materialized views using these foreign tables may fail during logical backups

Examples

Basic Example

Given a Logflare endpoint response:

1
2
3
4
5
6
[  {    "id": 123,    "name": "foo"  }]

You can create and query a foreign table:

1
2
3
4
5
6
7
8
9
10
11
create foreign table logflare.people (  id bigint,  name text,  _result text)  server logflare_server  options (    endpoint '9dd9a6f6-8e9b-4fa4-b682-4f2f5cd99da3'  );select * from logflare.people;

Query Parameters Example

For an endpoint accepting parameters:

  • org_id
  • iso_timestamp_start
  • iso_timestamp_end

With response format:

1
2
3
4
5
6
7
8
[  {    "db_size": "large",    "org_id": "123",    "runtime_hours": 21.95,    "runtime_minutes": 1317  }]

Create and query the table with parameters:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
create foreign table logflare.runtime_hours (  db_size text,  org_id text,  runtime_hours numeric,  runtime_minutes bigint,  _param_org_id bigint,  _param_iso_timestamp_start text,  _param_iso_timestamp_end text,  _result text)  server logflare_server  options (    endpoint 'my.custom.endpoint'  );select  db_size,  org_id,  runtime_hours,  runtime_minutesfrom  logflare.runtime_hourswhere _param_org_id = 123  and _param_iso_timestamp_start = '2023-07-01 02:03:04'  and _param_iso_timestamp_end = '2023-07-02';