ClickHouse

Create a connection between Faraday and ClickHouse so that your data is always up to date to make predictions, and your predictions can seamlessly sync back to your warehouse.

In this tutorial, we'll show you how to:

  • Connect your ClickHouse account to Faraday using a connection.

Let's dive in.

  1. You'll need a Faraday Enterprise account — talk to sales to get set up.

Prerequisites

You'll need the following details to create your connection to ClickHouse:

  • Host requiredtextClickHouse hostname. For ClickHouse Cloud, use the host from the connection dialog (without protocol).
  • Port requiredintegerNative protocol port. ClickHouse Cloud secure native is typically 9440. Self-hosted secure is often 9440; plain is often 9000.
  • Database requiredtext
  • User requiredtextClickHouse user identified with Faraday's Ed25519 SSH public key.
  • Secure (TLS) requiredbooleanUse TLS for the native protocol. Required for ClickHouse Cloud (typically port 9440).
  • Load balancer DNS name optionaltextIn case the host is deployed behind a load balancer.
  • SSH bastion optionaltextIn case the host is deployed behind an SSH bastion / jump server. Uses the Faraday SSH public key. This is the address of the bastion including username. For example, faraday@mybastion.example.com
  • SSH public key optionaltextEd25519 OpenSSH public key that should be assigned to the ClickHouse user. Unique per Faraday ClickHouse connection. Set by the Faraday system. Use the rotate_credentials endpoint to regenerate.

Granting access

First, you'll need Faraday access to your ClickHouse account.

ClickHouse connections use SSH key authentication (Ed25519). Create the Faraday connection first so Faraday can generate a keypair unique to that connection. Then create a ClickHouse user with Faraday's public key. The Faraday connection will error until access is set up; after that, use Force Update to refresh.

Cloud and self-hosted

Faraday supports ClickHouse Cloud and self-hosted ClickHouse over the native protocol.

  • ClickHouse Cloud: use the secure native port 9440 and set secure to true.
  • Self-hosted: set host, port, and secure to match your server (common secure native ports are 9440; plain is often 9000).

Please allowlist these official Faraday IP addresses:

  • 34.86.175.54
  • 34.86.252.230
  • 34.145.239.81
  • 35.245.199.181
  • 52.22.91.248
  • 52.23.137.21
  • 52.204.223.208
  • 52.204.228.32
  • 52.204.230.227

This connection type can be placed behind an SSH bastion (aka jump server). In that case, specify the user and host of the SSH bastion.

This connection type can also be placed behind a load balancer. In that case, specify the host of the load balancer.

Setup process

  1. Create your Faraday connection with host, port, database, user, and secure. Faraday generates an Ed25519 keypair and returns ssh_public_key.

  2. Create a ClickHouse user with the public key material from ssh_public_key (the base64 segment after ssh-ed25519, before any comment):

   CREATE USER faraday_user IDENTIFIED WITH ssh_key BY KEY '<public_key_material>' TYPE 'ssh-ed25519';
  1. Grant least-privilege access.

For sources (reading tables and views):

   GRANT SELECT ON your_database.* TO faraday_user;
   GRANT SELECT ON system.tables TO faraday_user;
   GRANT SELECT ON system.columns TO faraday_user;

For targets (full-replacement writes via a staging table and EXCHANGE TABLES / rename):

   GRANT SELECT, INSERT, CREATE TABLE, DROP TABLE, ALTER TABLE ON your_database.* TO faraday_user;
  1. Allow S3 table functions. Faraday exports and imports through ClickHouse s3(...) table functions using short-lived STS credentials. Grant the privileges your ClickHouse edition requires for those table functions (Cloud and self-hosted differ; see ClickHouse docs for s3 / remote function grants).

  2. Test the connection with Force Update in Faraday after the user and grants exist.

Faraday suggests that you use an unguessable string somewhere in the path to your data. This avoids what is called the Confused deputy problem

For example, let's say you were using S3. Instead of naming an S3 bucket s3://faraday-acme/, name it s3://faraday-acme-pwiiprz162ez. This guarantees that malicious actors cannot guess the name and request that Faraday import data from it into their account. The same logic applies to any path that is used to locate data.

Targets

Targets always fully replace the destination table. Provide table_name and a required order_by array of column names used for ENGINE = MergeTree ORDER BY (...). Append and upsert are not supported.

Key management

Each Faraday ClickHouse connection gets its own Ed25519 keypair. The private key stays in Faraday's vault. Rotate credentials with the rotate_credentials endpoint, then update the ClickHouse user with the new public key material before new connection attempts succeed.

Supported types

Faraday maps supported ClickHouse scalar types (integers in a safe range, floats, booleans, dates, DateTime/DateTime64, strings, UUID, enums, JSON). Nullable and LowCardinality wrappers are unwrapped. Maps, tuples, nested values, aggregate states, out-of-range unsigned integers, and oversized decimals are rejected.

Connecting

API via cURL
Dashboard

Use a POST /connections request:

curl https://api.faraday.ai/connections --json '{
  "name": "ClickHouse",
  "options": {
    "type": "clickhouse",
    "host": "...",
    "database": "...",
    "user": "...",
    "secure": false
  }
}'
  1. Wait briefly while Faraday establishes your connection. It shouldn't take long.

Your new connection is now ready to use.