ClickHouse

Create a connection between Faraday and ClickHouse so that your data is always up to date to make predictions, and your predictions can seamlessly sync back to your warehouse.

In this tutorial, we'll show you how to:

  • Connect your ClickHouse account to Faraday using a connection.

Let's dive in.

  1. You'll need a Faraday Enterprise account — talk to sales to get set up.

Prerequisites

You'll need the following details to create your connection to ClickHouse:

  • Host requiredtextClickHouse hostname. For ClickHouse Cloud, open Connect, set Connect with to Native (not HTTPS), and copy the host without protocol.
  • Port requiredintegerNative protocol TLS port. In ClickHouse Cloud, choose Connect with → Native (not HTTPS); the port is typically 9440. HTTPS uses 8443 and will not work. Self-hosted with TLS is often 9440.
  • Database requiredtext
  • User requiredtextClickHouse user identified with Faraday's Ed25519 SSH public key.
  • Load balancer DNS name optionaltextIn case the host is deployed behind a load balancer.
  • SSH bastion optionaltextIn case the host is deployed behind an SSH bastion / jump server. Uses the Faraday SSH public key. This is the address of the bastion including username. For example, faraday@mybastion.example.com
  • SSH public key optionaltextEd25519 OpenSSH public key Faraday generated for this connection. In ClickHouse, create the user with only the base64 segment after ssh-ed25519: CREATE USER ... IDENTIFIED WITH ssh_key BY KEY '' TYPE 'ssh-ed25519'. Unique per connection. Rotate with the rotate_credentials endpoint.

Granting access

First, you'll need Faraday access to your ClickHouse account.

ClickHouse connections use SSH key authentication (Ed25519). Create the Faraday connection first so Faraday can generate a keypair unique to that connection. Then create a ClickHouse user with Faraday's public key. The Faraday connection will error until access is set up; after that, use Force Update to refresh.

Cloud and self-hosted

Faraday supports ClickHouse Cloud and self-hosted ClickHouse over the native protocol with TLS. Use the TLS native port (ClickHouse Cloud is typically 9440; self-hosted with TLS is often 9440).

In ClickHouse Cloud, open Connect, set Connect with to Native (not HTTPS), and copy the host and port from that dialog. HTTPS uses a different port (8443) and will not work with Faraday.

Please allowlist these official Faraday IP addresses:

  • 34.86.175.54
  • 34.86.252.230
  • 34.145.239.81
  • 35.245.199.181
  • 52.22.91.248
  • 52.23.137.21
  • 52.204.223.208
  • 52.204.228.32
  • 52.204.230.227

This connection type can be placed behind an SSH bastion (aka jump server). In that case, specify the user and host of the SSH bastion.

This connection type can also be placed behind a load balancer. In that case, specify the host of the load balancer.

Setup process

Since ClickHouse uses an Ed25519 SSH keypair, you should first create the Faraday connection, which will generate a keypair that is unique to your connection. Then provision the ClickHouse user with that public key. The Faraday connection will error until you have set up access; after that, use Force Update (in the 3-dots menu) to refresh.

  1. Create your Faraday connection using the API or UI with host, port, database, and user. Faraday generates an Ed25519 keypair and returns ssh_public_key.

  2. Get the public key from your Faraday connection. You can find this in the connection details after creation. Faraday shows a full OpenSSH line such as:

   ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... optional-comment
  1. Create a ClickHouse user with only the base64 key material (the segment after ssh-ed25519, before any comment). Do not include the ssh-ed25519 prefix:
   CREATE USER faraday_user IDENTIFIED WITH ssh_key BY KEY '<public_key_material>' TYPE 'ssh-ed25519';

Example: if Faraday shows ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI..., pass AAAAC3NzaC1lZDI1NTE5AAAAI... as <public_key_material>.

  1. Grant least-privilege access. Replace your_database with the database name from your Faraday connection.

For sources (reading tables and views):

   GRANT SELECT ON your_database.* TO faraday_user;
   GRANT SELECT ON system.tables TO faraday_user;
   GRANT SELECT ON system.columns TO faraday_user;

For targets (full-replacement writes via a staging table and EXCHANGE TABLES / rename):

   GRANT SELECT, INSERT, CREATE TABLE, DROP TABLE, ALTER TABLE ON your_database.* TO faraday_user;
  1. Allow S3 table functions. Faraday exports and imports through ClickHouse s3(...) table functions using short-lived STS credentials Faraday provides. Grant ClickHouse source privileges for S3:

For sources (export via INSERT INTO FUNCTION s3(...)):

   GRANT WRITE ON S3 TO faraday_user;

For targets (import via SELECT ... FROM s3(...)):

   GRANT READ ON S3 TO faraday_user;

For both:

   GRANT READ, WRITE ON S3 TO faraday_user;

On older ClickHouse versions that do not support READ/WRITE source grants, use GRANT S3 ON *.* TO faraday_user; instead.

  1. Test the connection by using Force Update in Faraday. The initial connection may fail until the public key is properly configured in ClickHouse.

Faraday suggests that you use an unguessable string somewhere in the path to your data. This avoids what is called the Confused deputy problem

For example, let's say you were using S3. Instead of naming an S3 bucket s3://faraday-acme/, name it s3://faraday-acme-pwiiprz162ez. This guarantees that malicious actors cannot guess the name and request that Faraday import data from it into their account. The same logic applies to any path that is used to locate data.

Targets

Targets always fully replace the destination table. Provide table_name and a required order_by array of column names used for ENGINE = MergeTree ORDER BY (...). Append and upsert are not supported.

Key management

Each Faraday ClickHouse connection gets its own Ed25519 keypair. The private key stays in Faraday's vault. Rotate credentials with the rotate_credentials endpoint, then update the ClickHouse user with the new public key material before new connection attempts succeed.

Supported types

Faraday maps supported ClickHouse scalar types (integers including UInt64, floats, booleans, dates, DateTime/DateTime64, strings, UUID, enums, JSON). Nullable and LowCardinality wrappers are unwrapped. Maps, tuples, nested values, aggregate states, and oversized decimals are rejected. UInt64 values above signed int64 may overflow.

Connecting

API via cURL
Dashboard

Use a POST /connections request:

curl https://api.faraday.ai/connections --json '{
  "name": "ClickHouse",
  "options": {
    "type": "clickhouse",
    "host": "...",
    "database": "...",
    "user": "..."
  }
}'
  1. Wait briefly while Faraday establishes your connection. It shouldn't take long.

Your new connection is now ready to use.