sftp
Introduction
Faraday easily integrates with SFTP servers, enabling you to embed AI-powered predictive analytics anywhere else in your stack. Discover your brand's bespoke personas, score your customers for churn risk, find repeat purchasers, and more, enabling you to confidently engage the right customers, at the right time, with personalized, relevant content. The best part? No code—and no PhD—required.
Getting started
Make sure you have a Faraday account (signup is free!) and that it's not in test mode.
Prerequisites
You'll need the following details to create your connection to sftp:
- User required — The user to connect to the SFTP server as
- Host required — The hostname or IP address of the SFTP server
- Password optional — The password for the user (if blank, the private key will be used instead)
- Port required — The port on the SFTP server listening for connections
- Encryption optional — The encryption cipher(s) to use
- Compression optional — The compression algorithm to use
- Host key optional — The host key algorithm to use in verifying the host key presented by the remote host
Connection
Setting up your connection
First, you'll need to grant Faraday access to sftp.
CSV is a well-known format for transferring data in large batches. Faraday can accept files up to 5gb in size. If you have more than 5gb of data, we ask that you split it into multiple files.
Faraday's CSV support is based on folders. Each CSV in a folder should have the exact same structure. There should be no more than 5000 files in a single folder.
Amazon Web Services (AWS) Simple Storage Service (S3) can be used to send and receive files. Access is shared using AWS IAM permissions. We suggest that you create a Faraday-only bucket to both send and receive data. Within this bucket, Faraday would have full read and write access. Alternatively, you can give Faraday access to certain prefixes in a shared bucket.
Which IAM account (or both) depends on use of Targets and/or Datasets:
- Datasets:
arn:aws:iam::113233973114:user/stagecraft-download_s3
:s3:ListBucket
,s3:GetObject
,s3:GetObjectAcl
,s3:GetObjectVersion
- Targets:
arn:aws:iam::113233973114:user/deliver_s3
:s3:ListBucket
,s3:PutObject
,s3:PutObjectAcl
,s3:GetObject
,s3:GetObjectAcl
,s3:GetObjectVersion
Here is an example bucket policy. The top section is for Targets and the bottom section is for Datasets. Delete the sections you don't need.
{
"Version": "2012-10-17",
"Id": "FaradayAccessToBucket",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::113233973114:user/deliver_s3"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::your-bucket-name"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::113233973114:user/deliver_s3"
},
"Action": [
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::your-bucket-name/optional-prefix/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::113233973114:user/deliver_s3"
},
"Action": [
"s3:GetObject",
"s3:GetObjectAcl",
"s3:GetObjectVersion"
],
"Resource": "arn:aws:s3:::your-bucket-name/optional-prefix/*"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::113233973114:user/stagecraft-download_s3"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::your-bucket-name"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::113233973114:user/stagecraft-download_s3"
},
"Action": [
"s3:GetObject",
"s3:GetObjectAcl",
"s3:GetObjectVersion"
],
"Resource": "arn:aws:s3:::your-bucket-name/optional-prefix/*"
}
]
}
Faraday suggests that you use an unguessable string somewhere in the path to your data. This avoids what is called the Confused deputy problem
For example, instead of naming an S3 bucket s3://faraday-acme/
,
name it s3://faraday-acme-pwiiprz162ez
. This guarantees that
malicious actors cannot guess the name and request that Faraday import data
from it into their account. The same logic applies to any path that is used to
locate data.
Creating your connection
Now you can connect Faraday to sftp.
Wait briefly while Faraday establishes your connection. It shouldn't take long.
Your new connection is now ready to use!