Data needed for holdout test

A holdout test is a way to measure how well a predictive model works using your own historical data. You provide at least 12 months of leads and conversions, and we train the model on the first three quarters, then test it on the last one quarter. This shows how accurately the model could have predicted real outcomes in the past before you deploy it on new leads.

Start here

Create an account to access the Faraday dashboard and open the Datasets section.

For a holdout test, you’ll only need Datasets (to upload your files) and Cohorts (to define the evaluation group).

You can ignore the Events console for this process.

👉 Here's a video walkthrough of our CTO Seamus showing how to get the necessary data.

Step 1: Prepare your Leads file

The first dataset you’ll need is a list of leads — people who may or may not have become customers.
We require at least a name and the date the person became a lead.

👉 Use the Leads template.

Step 2: Prepare your Conversions file

The second dataset you’ll need is conversions — proof that a lead actually became a customer.

There are two common ways teams store this information:

Option A: Transaction-level data (preferred)

This format has one row per conversion event (for example, each order, signup, or purchase).

• Required: a unique identifier and the date of the event
• Optional: product or service details, dollar amount. If there are multiple items in a single record, just capture the principal product or a combined value (e.g., “pants and shirt”).

👉 Use the Orders/purchases/transactions template as a guide.

Transaction-level data is the best format for a holdout test, because it lets us train the model on one part of your history and validate on another.

Option B: Customer-level data (fallback)

This format has one row per customer, often with just a “first purchase date” or “last purchase date.”

• This loses the detail of multiple transactions.
• We can accept it, but it usually cannot be used for a proper holdout test.

👉 If this is the only data you have, you can use the Customers template, but note that results may be limited.

Step 3: Upload to the Dashboard

When you’ve prepared your files, go to the Datasets section of the Faraday dashboard, click + New Dataset in top right corner and add your CSV file.

Need help?

If you run into trouble uploading or preparing your files, reach out to support@faraday.ai.