Redshift Serverless

Create a connection between Faraday and Redshift Serverless so that your data is always up to date to make predictions, and your predictions can seamlessly sync back to your warehouse.

In this tutorial, we'll show you how to:

  • Connect your Redshift Serverless account to Faraday using a connection.

Let's dive in.

  1. You'll need a Faraday account — signup is free!

Prerequisites

You'll need the following details to create your connection to Redshift Serverless:

  • Host requiredtext
  • AWS Region requiredtextRedshift tables exists inside of an AWS region, e.g. us-east-1
  • Port requiredinteger
  • Schema requiredtext
  • User requiredtext
  • Load balancer DNS name optionaltextIn case the host is deployed behind a load balancer.
  • Password requiredtext
  • SSH bastion optionaltextIn case the host is deployed behind an SSH bastion / jump server. Uses the Faraday SSH public key. This is the address of the bastion including username. For example, faraday@mybastion.example.com
  • Database requiredtext

Granting access

First, you'll need Faraday access to your Redshift Serverless account.

Amazon Redshift Serverless is the next generation version of Amazon Redshift that doesn't require you to pick a cluster size in advance.

Amazon Redshift is a data warehouse. It is wire-compatible with Postgres and so connections are host-based and require a user and password. We suggest that you create a Faraday-only schema to both send and receive data. Within this schema, Faraday would have full read and write access. Alternatively, you can give Faraday access to certain tables in a shared schema.

This connection type can be placed behind an SSH bastion (aka jump server). In that case, specify the user and host of the SSH bastion.

This connection type can also be placed behind a load balancer. In that case, specify the host of the load balancer. Faraday suggests that you use an unguessable string somewhere in the path to your data. This avoids what is called the Confused deputy problem

For example, instead of naming an S3 bucket s3://faraday-acme/, name it s3://faraday-acme-pwiiprz162ez. This guarantees that malicious actors cannot guess the name and request that Faraday import data from it into their account. The same logic applies to any path that is used to locate data.

Connecting

API via cURL
Dashboard

Use a POST /connections request:

curl https://api.faraday.ai/connections --json '{
  "name": "Redshift Serverless",
  "options": {
    "type": "aws-redshift-serverless",
    "host": "...",
    "aws_region": "...",
    "schema": "...",
    "user": "...",
    "password": "...",
    "database": "..."
  }
}'
  1. Wait briefly while Faraday establishes your connection. It shouldn't take long.

Your new connection is now ready to use.