Data needed for holdout test

Tips and help for compiling the data needed for Faraday to perform a holdout test

It's time for a holdout test! This doc will help you pull the right data from your systems.

🚧️Do not email us any data

In order to protect privacy and security, we ask that you upload your data to our secure platform using the instructions below.

We'll start with CSV

We've found that using CSV files—which you extract from your systems and then securely upload to ours—provides the fastest, easiest experience. After the holdout test, when we get into a more substantial partnership, we'll use better techniques, like integrations, for keeping data connected.

This solicitation is written with uploaded CSV in mind. If you'd prefer to use another approach (CSVs in buckets, direct connection to your data warehouse, etc.) please feel free to adapt the instructions below accordingly.

Specs

File

  • Format — CSV
  • Column headers — Yes
  • File count — One or multiple

📘Have lots of files?

In some cases, your data may be spread across multiple files. For example, you may have run multiple direct mail campaigns over the test timeframe and have one file per campaign.

  • If multiple files share the same structure and essential information, you can include them in a single Dataset.
  • Files with different structure or information should be uploaded as seperate Datasets.

Timeframe

At least 18 months is preferred.

Identification

Each row should identify the person being described. Identifiers should include some combination of the following:

  • First name, last name, and physical address
  • Email (or hashed email)
  • Phone number

If your data structure normalizes identifiers into seperate tables using a join key (like a customer ID), you can either perform the join when you extract your data, or get help from us (see Getting help section below).

Information

We need to know your leads and conversions. Users generally share this data in one of two ways:

Option 1: Two datasets

  • Leads — A file with one or more rows per person who is considered a lead. This could be a mailing list used for direct mail
    • Include a column containing the date the lead was generated.
    • Feel free to include other columns that might be useful or predictive, such as the source/channel of the lead or any information from a lead capture form.
    • It's OK to include the same person multiple times if they are generated as leads multiple times (e.g. received multiple mailers). Faraday performs identity resolution to consolidate these into a single person.
  • Conversions — A file with one row per person who has converted. This could be a list of customers or closed-won agreements.
    • Include a column containing the date of conversion.
    • Feel free to include other columns that might be useful or predictive, such as the value (in dollars) of the conversion or the product/service purchased.

For both files, include identifiers as described above.

Option 2: One dataset

Leads with conversion indicator — A file with one or more rows per person, where each row represents a lead or lead-generating event.

  • Include a column containing the date the lead was generated.
  • Include a column containing an indication of whether the lead converted. Ideally this would be the date of the conversion, but it could also be a boolean column indicating whether the lead converted.
  • Include identifiers as described above.
  • Feel free to include other columns that might be useful or predictive, such as:
    • The source/channel of the lead
    • Any information from a lead capture form
    • Value (in dollars) of the conversion
    • Product/service purchased

Uploading your data

Like screencast videos? Here's a video walkthrough of our CTO Seamus using the two-file method.

You'll need an account

If you don't already have one, please sign up for a Faraday account.

Get out of test mode

When you first log in, you'll be in test mode. To upload data, you'll need to switch to production mode:

  1. Click on your account name in the top left corner.
  2. Make sure the Test mode toggle is off.

Creating datasets

Now click on the Datasets tab in the left-hand navigation.

Screenshot of Faraday showing empty datasets list

After each dataset is created, there's no need to add additional configuration like identity sets, event streams, or traits. We'll do that with you by screenshare soon.

Start with your Leads dataset:

  1. Click the New Dataset button.
  2. Choose CSV and click Next.
Screenshot of Faraday showing new dataset form, completed
  1. Drag your leads file into the upload area.
  2. Name your dataset something like Lead data.
  3. Click Create dataset.

📘Have multiple leads files?

If your leads data is spread across multiple files, as described above, start with one file (any file), then after creating the dataset, click the "Data" tab and upload the remaining files to the single dataset.

If using Option 2 (one dataset) only:

Create your Conversions dataset:

  1. Return to the Datasets tab.
  2. Click the New Dataset button.
  3. Choose CSV and click Next.
  4. Drag your conversions file into the upload area.
  5. Name your dataset something like Conversion data.
  6. Click Create dataset.

When you're done

Please reply to the email thread you're on with your Faraday contact (or use the shared Slack channel) to let them know you've uploaded your data. They'll take it from there.

Getting help

If you need any help along the way, please don't hesitate to reach out to your Faraday contact. They're here to help you succeed!

  • If you have one, try your Faraday Slack channel.
  • Or reply to the email thread you're on with your Faraday contact.
  • As a fallback, you can reach Faraday support using the chat widget in the bottom right corner of the Faraday dashboard.