SFTP

Create a connection between Faraday and your SFTP server so that your data is always up to date to make predictions, and your predictions can seamlessly sync back to your server.

In this tutorial, we'll show you how to:

  • Connect your SFTP account to Faraday using a connection.

Let's dive in.

  1. You'll need a Faraday account — signup is free!

Prerequisites

You'll need the following details to create your connection to SFTP:

  • User requiredtextThe user to connect to the SFTP server as
  • Host requiredtextThe hostname or IP address of the SFTP server
  • Password optionaltextCurrently not supported in favor of SSH keypair authentication (if blank Faraday's private key will be used instead)
  • Port requiredintegerThe port on the SFTP server listening for connections
  • Encryption optionaltextThe encryption cipher(s) to use
  • Compression optionalbooleanThe compression algorithm to use
  • Host key optionaltextThe host key algorithm to use in verifying the host key presented by the remote host

Granting access

First, you'll need Faraday access to your SFTP account.

CSV is a well-known format for transferring data in large batches. Faraday can accept files up to 5gb in size. If you have more than 5gb of data, we ask that you split it into multiple files.

Faraday's CSV support is based on folders. Each CSV in a folder should have the exact same structure. There should be no more than 5000 files in a single folder.

This connection type is used to connect to an SFTP server you control.

Please allowlist these official Faraday IP addresses:

  • 34.86.175.54
  • 34.86.252.230
  • 34.145.239.81
  • 35.245.199.181
  • 52.22.91.248
  • 52.23.137.21
  • 52.204.223.208
  • 52.204.228.32
  • 52.204.230.227

If you would like Faraday to host an SFTP server for you, please contact support - you cannot use this connection type for that.

An SFTP server can be used to both send and receive files. In order to connect to your SFTP server, Faraday needs to first know the hostname or IP of the server (e.g. sftp.acme.com). We will also need the user to connect as and the port which listens for SFTP connections.

Currently Faraday only supports SFTP connections using an SSH key pair, which is more secure than using a password. In order to set this up you will need your SFTP server to contain Faraday's public key. This will give us access to securely pull CSV files from the server or push CSV files directly to it.

Ensure that the following public key is added to your SFTP server's authorized_keys file. This is the real public key, not an example:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC95mNNckQnMVJSqvD9TW44ETN01dZ/X0DNxOXmvVQ7JM6MLmIeGLcEU8TShjAJ2mjVpiaQmq6fPNowId7FLhNPFK0J/GkXBXaCMHZK9P54gTp7cczy6LO5a/0Q1hbyvmkZSTCw5yD2NAqbTXsh9QaJJB5mSWVfVllHE9v+mOKbi7JZTj1Y14X+GvSQil3oj+vY+mHtSY7Fu5Malv3CSShuyh6+MZWPOWFA4VlzquhgwFUteQaIlNUwdf14pr9BF/UpAyYzlXBZpEY7tVTHr6ulkdC81cN8xpjeASvHlSwKrfHGGpNXTtPgi4I/ULu88iGfn/EkHFH/K/rH2FL8LOfTsg+EIvdZEVrY4wJ4fCfD3QnXRo2oAmZasxxxsLpUgicrpW/+tuk5h+Gwjha708khrpybiDWS/YzPubtWEelkBUg8ovGDQTtuw1CPH5fF9JXd3KraRuUqz4qLknYlQXCGEwpMaIQM7avFueooCI4RSUWmL1FtL6VZ6n6tES+PRdMTd/itX6Lcl2B7IAIn5Rq6oJpVuYU9a0pxrsPVtLAGe9/pNRFIrYUuEcULzQ1Ew1eLd+RqQvzLteuZUG23ODfB1Wr6D/4O1iya1v9n6lybWOgawu3eXIpRzr/D/V9tmL6y8OFbYTsm+KO4GY2BJI5DdmzlpV9nJfYef9vflSZm1Q== Faraday incoming keypair (RSA)

Faraday suggests that you use an unguessable string somewhere in the path to your data. This avoids what is called the Confused deputy problem

For example, let's say you were using S3. Instead of naming an S3 bucket s3://faraday-acme/, name it s3://faraday-acme-pwiiprz162ez. This guarantees that malicious actors cannot guess the name and request that Faraday import data from it into their account. The same logic applies to any path that is used to locate data.

Note: If you use a Faraday-hosted SFTP server, please be aware that our servers do not use static IP addresses. If your network configuration requires allowlisting static IPs, contact Faraday support to discuss your options.

Connecting

API via cURL
Dashboard

Use a POST /connections request:

curl https://api.faraday.ai/connections --json '{
  "name": "SFTP",
  "options": {
    "type": "sftp",
    "user": "...",
    "host": "..."
  }
}'
  1. Wait briefly while Faraday establishes your connection. It shouldn't take long.

Your new connection is now ready to use.