What if you could embed AI-driven predictions into your platform—without the massive investment in data, engineering, and ongoing model maintenance?

Your customers want smarter insights: predictions on which leads will convert, which territories will perform best, and which prospects match their ideal profile. But building AI agents from scratch is a massive undertaking—one that requires specialized expertise, vast amounts of data, and months (or even years) of development. For many SaaS companies, the cost and complexity make it an unrealistic endeavor.

So what is to be done?

[![SalesRabbit partnership with Faraday landing page](/images/blog/pbf-blog/salesrabbit-story-header-image.png)](/images/blog/pbf-blog/salesrabbit-story-header-image.png)


## Introducing Powered by Faraday

This is precisely why we launched **Powered by Faraday**—a turnkey solution that allows SaaS companies to integrate AI-powered customer predictions into their platforms without having to build or maintain machine learning models themselves.

We’ve spent over a decade refining predictive models that help businesses understand and anticipate customer behavior and with **Powered by Faraday**, we bring these capabilities directly into the hands of product teams, enabling them to enhance their software with advanced AI-driven insights quickly and cost-effectively.

## What Powered by Faraday does

With **Powered by Faraday**, companies can:

* **Enrich their customer data** with 1,500+ built-in consumer attributes, adding valuable context to their existing datasets.  
* **Predict customer behavior** by using pre-built machine learning models that score leads, prioritize outreach, and identify high-conversion opportunities.  
* **Accelerate product development** by embedding plug-and-play AI models, cutting multi-year build times down to weeks or even days.  
* **Optimize costs and scalability** with transparent pricing and predictable expenses, avoiding the uncertainty of large-scale computational costs.

For SaaS companies looking to offer predictive capabilities without reinventing the wheel, **Powered by Faraday** is a game-changer.

## Faraday’s competitive edge

Many companies offer AI solutions, but **Powered by Faraday** stands out due to several key advantages:

* [**Seamless integration with a robust API**:](https://faraday.ai/docs/abstractions/traits) Powered by Faraday is designed for seamless implementation. Faraday’s well-documented, developer-friendly API makes embedding AI-driven insights into a product straightforward and efficient.   
* [**FIG (Faraday Identity Graph)**](https://faraday.ai/docs/abstractions/traits): A vast, privacy-compliant third-party dataset with 1,500+ attributes, enabling richer predictions than first-party data alone.  
* **Forward-deployed data science & professional services**: Hands-on support from data scientists who fine-tune models and ensure seamless deployment.  
* **Ethical, privacy-first AI**: A commitment to consumer privacy, avoiding personally identifiable information while delivering accurate predictions.  
* **Industry-specific models**: Pre-trained, vertical-specific machine learning models that accelerate time-to-value.  
* **Flexibility to design tailored solutions:** Faraday enables companies to integrate predictive capabilities without the constraints of rigid, one-size-fits-all AI solutions—offering the perfect balance between ease of adoption and customization.

By combining rich third-party data with hands-on AI expertise, Faraday helps SaaS companies deliver world-class predictive features faster and with greater accuracy than in-house or off-the-shelf solutions.

## Real-world use cases

Many companies have already leveraged **Powered by Faraday** to enhance their platforms. 

Here’s how two of them—[AccuLynx](https://faraday.ai/blog/acculynx-lead-intelligence-powered-by-faraday-api) and [SalesRabbit](https://faraday.ai/blog/salesrabbit)—are making AI-driven predictions a core part of their products:

### AccuLynx: Smarter lead prioritization for roofing contractors

[AccuLynx](https://acculynx.com/), a leading provider of business management software for roofing contractors, needed a way to help their customers prioritize sales efforts more effectively. They integrated [Faraday’s prediction API](https://faraday.ai/docs/reference) into their new [Lead Intelligence feature](https://acculynx.com/roofing-sales-lead-intelligence/), which automatically scores leads within AccuLynx’s platform, based on hundreds of demographic, financial, and property data points.

The result? Roofing contractors using AccuLynx no longer have to guess which leads to focus on. Instead, they get **AI-powered rankings** that help them close more deals efficiently—without needing in-house data science expertise.

As their VP of Product [Mark Rattin](https://www.linkedin.com/in/markrattin/) said, “Lead Intelligence is a game-changer for roofing companies,” … “AccuLynx customers no longer have to guess which leads to call first, or only rely on their experienced sales people to quickly close deals. Now they can improve their sales win rates by focusing their efforts on the best leads.”

Sounds like a win to us\!

[![AccuLynx landing page](/images/blog/pbf-blog/acculynx-landing-page.png)](/images/blog/pbf-blog/acculynx-landing-page.png)

### SalesRabbit: AI-Driven insights for field sales teams

[SalesRabbit](https://salesrabbit.com/), a leader in field sales software, wanted to take their [DataGrid](https://salesrabbit.com/datagrid/) solution to the next level. While their platform already provided canvassers with property and homeowner data, they saw an opportunity to go further—predicting which homeowners were most likely to convert based on behavioral and demographic insights.

By integrating [Faraday’s agentic customer prediction models](https://faraday.ai/templates), SalesRabbit was able to:

* Guide sales reps toward high-potential neighborhoods and properties.  
* Optimize canvassing routes for better efficiency.  
* Improve app performance by caching predictive insights in advance.

By leveraging **Powered by Faraday**, SalesRabbit transformed DataGrid into DataGrid AI, offering their customers a predictive lead scoring tool in a fraction of the time it would’ve taken to build in-house.

[![SalesRabbit DataGrid AI landing page](/images/blog/pbf-blog/salesrabbit-datagridai.png)](/images/blog/pbf-blog/salesrabbit-datagridai.png)

## **Conclusion: AI-Powered predictions without the headache**

For SaaS companies looking to bring predictive insights to their customers, building AI models from scratch is often too expensive, time-consuming, and resource-intensive. Powered by Faraday removes those barriers, making it easy to integrate best-in-class consumer predictions into your product without the heavy lift.

Whether you want to help your customers prioritize leads, optimize sales territories, or enhance customer insights, Powered by Faraday lets you do it faster, more affordably, and at scale.

Want to see how it works? Check out the full stories from [AccuLynx](https://faraday.ai/blog/acculynx-lead-intelligence-powered-by-faraday-api) and [SalesRabbit](https://faraday.ai/blog/salesrabbit), or reach out to [Dave](https://www.linkedin.com/in/davidtoddsmall/) or the [Faraday](https://faraday.ai/talk-to-sales) team to explore how Powered by Faraday can enhance your platform.

AI is revolutionizing how SaaS companies can leverage predictive insights, and at Faraday, we’re making it easier than ever to integrate powerful, ethical AI-driven capabilities without the hefty overhead.

AI-Powered insights for SaaS companies, the fast way

In today's data-driven world, businesses can amass a wealth of first-party information, but this presents only a partial picture of their current and potential customers. Gaps in their data can limit the effectiveness of their marketing strategies, market research, and predictive models. That’s where Faraday’s data appends come in. By enhancing their existing datasets with high-quality third-party attributes, brands can unlock deeper insights to make smarter, more informed decisions.

## What are data appends, and why do they matter?

Data appending is a process by which businesses can purchase and integrate external data into their own datasets to supplement their first-party information. These attributes might include demographic information, behavioral patterns, or even lifestyle indicators, data points that are generally not available through first-party channels alone. By enriching your data with these additional insights, you can gain a fuller picture of your customers and their behaviors. 

But Faraday’s data appends go beyond simple enrichment. Our data is rigorously vetted, privacy-compliant, and built for seamless integration into your existing workflows. Whether refining market research, improving segmentation, or even powering your own predictive models, these appends help businesses make smarter, data-driven decisions. With Faraday, you’re not just adding data—you’re adding intelligence.

## How businesses use data appends

Data appends address various business needs. Here are some common ways companies leverage them:

### Exploratory research

Understanding your market often requires more than just your existing data. Take, for example, a roofing company. They might want to analyze roof sizes across different regions to gauge demand. Instead of sourcing this data themselves (which would be slow and expensive) or purchasing a generic dataset that requires extensive processing, they can use Faraday to buy and append only the relevant geographic and property data, seamlessly. This allows them to conduct targeted research efficiently and make informed decisions—like leveraging predictive targeting with Faraday next.  

### Supplementing existing datasets

Businesses often lack key customer details. Take a retailer that has always relied on email marketing. They would have extensive email lists but likely never collected addresses or phone numbers. If they want to expand into direct mail or call center outreach, they’ll need that missing data. This is where Faraday’s data appends come in, providing the necessary information without the hassle, time commitment, or high costs of sourcing it themselves. 

### Building custom predictive models

 An advanced use case for our data appends is supporting businesses that prefer to develop their own machine learning models. Just as we use this data at Faraday, these companies leverage our high-quality, cleaned, and structured attributes to enhance their predictive outcomes. In this case, Faraday acts as a trusted data provider, delivering the precise insights needed to model a variety of customer attributes—such as likelihood to churn, lifetime value, or purchasing behavior—ready to be seamlessly integrated into customers’ [Python](https://www.python.org/) environments.

## How Faraday’s data appends work

The Faraday Identity Graph (FIG) is the backbone of our data appending services. This robust dataset combines multiple, meticulously cleansed, and feature-engineered data sources into one comprehensive dataset. [With over 1,500 data attributes covering nearly all U.S. adult consumers and their households](https://app.faraday.ai/traits), it offers a deep well of insights into things like demographic trends, property ownership, life events, and financial indicators, just to name a few.

The process to access this data as appends is simple and streamlined:

1. **Select attributes**: Choose the data attributes that align with your business goals (e.g., income levels, household size, location-specific behaviors).  
2. **Match & append**: Faraday will match these attributes to your existing customer records, providing a more complete profile of your audience.  
3. **Integration**: The enriched data is delivered in a clean, structured format that’s easy to integrate with your models and business processes.

[![Data-appends diagram](/images/blog/data-appends-diagram.png)](/images/blog/data-appends-diagram.png)

## Why not just find the data yourself?

Some companies attempt to enhance their marketing efforts by purchasing third-party data directly from various vendors themselves, rather than using packaged and processed services like Faraday’s data appends. However, this approach often presents significant challenges:

**Contracting complexity**: Sourcing offline data can involve lengthy negotiations and large upfront commitments.

**Security risks**: Handling sensitive data requires strict compliance frameworks, such as SOC-2 certification. Unlike many vendors, all Faraday data appends are privacy-compliant, ethically sourced, and adhere to high standards of data protection, ensuring your customers' personal information remains secure while providing actionable insights.

**Regulatory challenges**: Companies storing offline data must navigate consumer privacy laws, including opt-out and deletion requests. Faraday manages these complexities for you, so you can focus on what matters—leveraging new insights to drive revenue.

**Data science overhead**: Raw offline data often requires significant processing, feature selection, and bias mitigation before it can be effectively used. This can result in costly time and resource investments as companies try to integrate third-party data into their models.

Faraday eliminates these challenges by providing ready-to-integrate data that is pre-processed and structured for immediate use. Our data appends come with built-in privacy compliance, so you don’t have to worry about regulatory hurdles or the risk of handling sensitive information. Additionally, the data is feature-engineered and bias-mitigated, saving you the time and resources typically needed to prepare raw data. With Faraday, you can access high-quality, actionable insights without the complexities and overhead associated with managing third-party data directly. This allows your team to focus on refining your marketing strategies and driving better business outcomes.

## How to get started

Getting started with Faraday’s data appends is simple. Once you’ve identified the attributes you need, we’ll work with you to integrate them into your existing systems. Whether you’re looking for a one-time data boost or ongoing data enrichment, our flexible pricing and easy integration make it simple to scale.

To learn more or request a demo, **contact us today**.

Faraday’s data appends enrich your existing customer data with third-party attributes, helping you fill gaps, refine targeting, and make smarter decisions.

Leveraging the power of the Faraday Identity Graph through Data Appends

One of the most critical aspects of model development is ensuring that our models can generalize well to unseen data (i.e. when the predictions are deployed). While it's relatively straightforward to train a model that performs well on training data, the real challenge is creating models that maintain their performance when faced with new, previously unseen examples. This is where cross-validation (CV) comes into play to validate our models' performance and make more informed decisions during the model development process. Throughout this post, we will review different CV variants and how Faraday uses them to train its model and forecast return on investment (ROI) for our clients.

## What is cross-validation

At its core, cross-validation is a resampling procedure used to evaluate machine learning models on limited data samples. Instead of using a single train-test split, cross-validation provides a framework for multiple evaluations of our model using different portions of the available data.

Cross-validation involves partitioning the available data into complementary subsets, training the model on some subsets (training set), and validating it on the remaining subset (validation set). This process is repeated multiple times, with each data point getting the opportunity to be part of both training and validation sets.

The key principles of cross-validation include:

1. Data Independence: Ensuring that training and validation sets are strictly separated in each iteration
2. Comprehensive Evaluation: Using all available data for both training and validation through multiple iterations
3. Statistical Robustness: Generating multiple performance metrics to assess model stability

### K-Fold cross-validation

K-fold cross-validation is the most common and widely used variant of cross-validation. In this approach, the data is divided into k equal-sized folds, and the following process is repeated k times:

1. One fold is selected as the validation set
2. The remaining k-1 folds are used as the training set
3. One model is trained on the training set
4. Predictions on the validation set are saved

Faraday uses a 5-fold CV to evaluate its models. This means that for any given propensity model, Faraday builds 6 variations of the model (5 for the k-fold validation and one final version used as the deployed model.)  

[![K-fold cross-validation diagram](/images/blog/cross-validation-blog/regular_cv.png)](/images/blog/cross-validation-blog/regular_cv.png)


Once all five models are trained, each sample in the data has been scored once. This means that the cross-validation allows us to assign a prediction to each person, allowing us to compute the performance metrics using the reconstructed scores.

Special variants of k-fold cross-validation include:

- **Stratified K-Fold**: Maintains the same class distribution in each fold (useful for imbalanced datasets)
- **Leave-one-out cross-validation**: Sets k equal to the number of data points (useful for very small datasets)
- **Time series cross-validation**: The folds are defined using a rolling window for time-based data.

[![Time series cross-validation](/images/blog/cross-validation-blog/time_cv.png)](/images/blog/cross-validation-blog/time_cv.png)

## The benefits of cross-validation

From the previous section, it is clear that CV implies more computation than training a single model. However, cross-validation offers several crucial advantages that outweigh the added costs and complexity, making it a mandatory step in the training process. Let's explore the key benefits that make this technique so valuable.

### Preventing overfitting

One of the primary benefits of cross-validation is its ability to help prevent overfitting. When a model learns the training data too well, it starts to capture noise and specific patterns that don't generalize to new data. Overfitting leads to poor performance in real-world applications. In the charts below, we compare the effects of three different model architectures on the same dataset:

1. The first model is too simplistic to capture the trend in the data.
2. The third one tries too hard to capture every data point and adds a lot of noise between them.
3. The second model does a good job of capturing the overall trend in the data without necessarily trying to capture small variations in the location of the data points.

[![Underfitting vs overfitting curves](/images/blog/cross-validation-blog/blog_3_curves.png)](/images/blog/cross-validation-blog/blog_3_curves.png)

Cross-validation helps detect overfitting by evaluating the model's performance on multiple subsets of the data. If a model performs significantly better on the training data than on the validation sets, it's a clear indicator of overfitting. This early detection allows data scientists to make necessary adjustments to the model's complexity, feature selection, or hyperparameters before deploying it to production.

If we compute our model performance (ROC, accuracy) on the training data, the third model is the best option. However, when using CV, the second model performs better.

### Computing reliable performance metrics

Traditional train-test splits can sometimes provide misleading performance metrics due to the random nature of the split. Cross-validation addresses this limitation by:

- Providing multiple performance estimates using different data splits
- Enabling the calculation of confidence intervals for performance metrics
- Reducing the variance in performance estimation
- Offering a more robust assessment of model stability

These multiple evaluations give us a clearer picture of how our model is likely to perform in real-world scenarios, helping us make more informed decisions about model selection and deployment.

The content of our outcome report and UI relies entirely on the cross-validation but two charts explicitly depict results per fold. Our “folded ROC” plot shows the ROC curve for each fold. This is useful to ensure the model is stable (i.e. the results across folds are nearly identical).

[![Folded ROC curve](/images/blog/cross-validation-blog/Folded_ROC.png)](/images/blog/cross-validation-blog/Folded_ROC.png)

The lift chart illustrates the volatility of lift, particularly for those with higher scores. This information is crucial for lead generation models, which aim to identify potential customers. Faraday assigns a score to each individual within a specified population, and the client uses only the top X% of scorers.

[![Lift curve](/images/blog/cross-validation-blog/lift_curve.png)](/images/blog/cross-validation-blog/lift_curve.png)

## Technical dive

### Formal definitions

**Dataset definition**  

 - Let $\mathcal{D} = \{(x_i, y_i)\}_{i=1}^n$ be our dataset where:  
  - $x_i \in \mathcal{X} \subseteq \mathbb{R}^d$ are feature vectors for sample $i$  
  - $y_i \in \mathcal{Y}$ is the label for sample $i$  
  - $n$ is the number of samples  
 - Define $\mathcal{P}_{XY}$ as the true joint distribution of $(X,Y)$  


**Model space**   
 - Let $\mathcal{F} = \{f_\theta: \mathcal{X} \rightarrow \mathcal{Y} | \theta \in \Theta\}$ be the hypothesis space  
  - $\theta$ represents the model parameters  
  - $\Theta$ is the parameter space  

**Loss function**  
 - Define $\mathcal{L}: \mathcal{Y} \times \mathcal{Y} \rightarrow \mathbb{R}_{\geq 0}$ as the loss function  
 - The expected risk is:  
```math
R(f) = \mathbb{E}_{(X,Y)\sim\mathcal{P}_{XY}}[\mathcal{L}(f(X),Y)]
```
$R(f)$ is interpreted as the error of the model $f$ on the dataset.  
 - The empirical risk on dataset $\mathcal{D}$ is the average loss over $\mathcal{D}$:  
```math
\hat{R}_{\mathcal{D}}(f) = \frac{1}{n}\sum_{i=1}^n \mathcal{L}(f(x_i),y_i)
```
**K-Fold partitioning**  
 - Partition indices $\{1,\ldots,n\}$ into $k$ disjoint subsets: $\{\mathcal{I}_1,\ldots,\mathcal{I}_k\}$  
 - For each fold $j$:  
   - $\mathcal{D}_{val}^{(j)} = \{(x_i,y_i) | i \in \mathcal{I}_j\}$ (validation set)  
   - $\mathcal{D}_{train}^{(j)} = \{(x_i,y_i) | i \notin \mathcal{I}_j\}$ (training set)  


**Cross-validation estimator**  
The k-fold CV estimate of the expected risk is:  
```math
CV_{(k)}(f) = \frac{1}{k}\sum_{j=1}^k \hat{R}_{\mathcal{D}_{val}^{(j)}}(f_{\theta_j})
```
   where $f_{\theta_j}$ is trained on $\mathcal{D}_{train}^{(j)}$  


### Statistical properties

**Bias of CV estimator**  
The bias of the k-fold CV estimator can be expressed as:  
```math
Bias(CV_{(k)}) = \mathbb{E}[CV_{(k)}] - R(f^*)
```
   where $f^*$ is the true optimal function and $\mathbb{E}[CV_{(k)}]$ is the expected risk over all the folds.  

**Variance decomposition**  
The variance can be decomposed into:  
```math
Var(CV_{(k)}) = \frac{1}{k^2}\sum_{j=1}^k Var(\hat{R}_{\mathcal{D}_{val}^{(j)}}) + \frac{2}{k^2}\sum_{i<j}Cov(\hat{R}_{\mathcal{D}_{val}^{(i)}}, \hat{R}_{\mathcal{D}_{val}^{(j)}})
```
Where $Var(\hat{R}_{\mathcal{D}_{val}^{(j)}})$ is the variance of the risk estimate over a fold. It does not refer to the variance of the risk itself (which is just a scalar value). Instead, it refers to the variance of the estimated risk based on multiple possible samples of the validation set within that fold.  

Therefore, as k increases, the overall variance of the cross-validation risk decreases leading to a less volatile estimation of the model performance post-deployment. 

# Holdout test

### What is it?

A "holdout test" can refer to any process where a model is being evaluated on unseen data (e.g. cross-validation above). At Faraday, a holdout test specifically refers to the process of choosing a date in the past $t_0$, fitting a propensity model with data prior to $t_0$, and evaluating the model on data after $t_0$.

On one hand, a holdout test is just a special case of time series cross-validation with 1 fold, but as we will see there are a few nuances to consider when configuring a holdout test and analyzing the results to inform a business decision.

### Configuring a holdout test in the Faraday platform

A typical scenario where a holdout test can be used is lead scoring. We will walk through how to run a lead scoring holdout test in the Faraday platform, but the same principles apply to a holdout test for other use-cases (e.g. predicting churn).

#### What is needed for a holdout test?

To configure a holdout test a user needs lead and conversion records with PII, dates, and a way to distinguish between the two groups. The setup for this can vary, but perhaps the simplest form is a single table that includes the following:

- PII columns
- a `lead_created` date that captures when a record became a lead
- a `conversion_date` that captures when that lead record became a conversion

NOTE: In our working example our conversions are called `tier_1_leads`.

In addition to the raw data, a user needs to have some prior knowledge of the typical attribution window for conversion which we will discuss in more detail when configuring the cohorts.

#### Datasets and events

Given a dataset described above, we need to configure the PII as one or more identity sets and two event streams, one for leads and one for conversions.

Configuring the leads event can be as simple as mapping a single date column  
[![Lead event](/images/blog/cross-validation-blog/lead_event.png)](/images/blog/cross-validation-blog/lead_event.png)

Similarly for the `tier_1_leads` event  
[![Attainment event](/images/blog/cross-validation-blog/attainment_event.png)](/images/blog/cross-validation-blog/attainment_event.png)

At this point our resource graph has a dataset table that emits two event streams  
[![Datasets graph](/images/blog/cross-validation-blog/datasets_graph.png)](/images/blog/cross-validation-blog/datasets_graph.png)


#### Cohorts

Next, we need to configure groups of people (cohorts) that partition the data into a training and validation set (recall this is time series cross-validation with 1 fold).

The inputs to determine this are:  
[![XYZ categories](/images/blog/cross-validation-blog/xyz.png)](/images/blog/cross-validation-blog/xyz.png)

[![XYZ description](/images/blog/cross-validation-blog/xyz_descriptions.png)](/images/blog/cross-validation-blog/xyz_descriptions.png)

The "prior period" is the training set for the holdout test, and the "holdout period" is the validation set. The amount of data available can influence how long/short these time windows are, but in general, we want to have enough data in the prior period to train a model and enough data in the holdout period to be confident in the results. For example, it would be bad to have zero conversions in the holdout period.

The attribution window is the time it takes for a typical lead to convert. Depending on the situation, this could be a few days to months.

The data lag is the time delta between today and the most recent date of any lead or conversion.

Once these have been determined we can create three cohorts

- prior leads: all records that became leads in the prior period
- recent leads: all records that became leads after the prior period
- conversions: all conversions (`tier_1_leads` in the working example) without any date filtering

Prior leads example configuration  
[![Prior lead configuration](/images/blog/cross-validation-blog/prior_leads_config.png)](/images/blog/cross-validation-blog/prior_leads_config.png)

Recent leads example configuration  
[![Recent lead configuration](/images/blog/cross-validation-blog/recent_leads_config.png)](/images/blog/cross-validation-blog/recent_leads_config.png)
 
Our resource graph now looks like  
[![Cohort graph](/images/blog/cross-validation-blog/cohorts_graph.png)](/images/blog/cross-validation-blog/cohorts_graph.png)

#### Outcome

Once cohorts are configured it is time to build a model and get some output. In Faraday, a propensity model is defined by an "outcome" which takes two cohorts as an input.  

[![Outcome configuration](/images/blog/cross-validation-blog/outcome_config.png)](/images/blog/cross-validation-blog/outcome_config.png)

Note that using the prior leads from the prior period implicitly restricts the conversions (tier 1 leads) to the prior period.

#### Pipeline

A pipeline applies an outcome (propensity model) to a group of people specified by a cohort. In this case we want to apply the outcome trained using data from the prior period to score recent leads records. Additionally, we want to have conversion information available in the output to analyze performance.  

[![Scope configuration](/images/blog/cross-validation-blog/scope_config.png)](/images/blog/cross-validation-blog/scope_config.png)

Here is the full resource graph  
[![Resource graph](/images/blog/cross-validation-blog/resource_graph.png)](/images/blog/cross-validation-blog/resource_graph.png)

This pipeline produces a table whose rows correspond to the recent lead records and whose columns include:  
[![Payload columns](/images/blog/cross-validation-blog/payload_cols.png)](/images/blog/cross-validation-blog/payload_cols.png)

Any of the `fdy_outcome_` columns can be used together with the `fdy_cohort_*_member` column (which is true when a row is a conversion) to measure model performance.

### Analyzing results

The output from the above pipeline can be used for custom analysis as desired by the user. One example is the automated reporting that Faraday produces to analyze holdout results and potential ROI.

#### Holdout results

First, we look at the internal performance metrics of the model built using the prior period.  

[![Training data used](/images/blog/cross-validation-blog/training_data_used.png)](/images/blog/cross-validation-blog/training_data_used.png)

These metrics are based on cross-validation described above and distilled down into a single plot.  

[![Model results](/images/blog/cross-validation-blog/model_results.png)](/images/blog/cross-validation-blog/model_results.png)

Next, we look at the pipeline output to see how the model performed on the recent leads.  

[![Data used for holdout test](/images/blog/cross-validation-blog/data_used_for_holdout_test.png)](/images/blog/cross-validation-blog/data_used_for_holdout_test.png)

Again, we can produce conversion rates by decile. 

[![Holdout test results](/images/blog/cross-validation-blog/holdout_test_results.png)](/images/blog/cross-validation-blog/holdout_test_results.png)

The last step of the holdout analysis is to compare the observed results from the holdout test with the model performance computed at training time via cross-validation.  

[![Comparing model report to test results](/images/blog/cross-validation-blog/comparing_test_results.png)](/images/blog/cross-validation-blog/comparing_test_results.png)
 
If these results are comparable, then it gives the user confidence that the performance metrics observed at training time are reflective of the model performance moving forward.

#### ROI calculation

When additional business context is known, the holdout test results can also be used to estimate ROI. For example  

[![Lead rejection ROI calculator](/images/blog/cross-validation-blog/roi_calculator.png)](/images/blog/cross-validation-blog/roi_calculator.png)

## Conclusion

Cross-validation can be used to prevent overfitting and evaluate a model's performance at training time. A holdout test is just a special case of cross-validation in which a single fold is determined by a point in time. Combining these two approaches with business assumptions enables the user to deploy models confidently, with a sense of expected performance and insight into expected ROI.

Faraday uses cross-validation to evaluate model performance, ensuring accuracy and value by training on multiple data subsets and validating on others. This allows us to make informed decisions and deliver reliable ROI for our clients.

How Faraday validates our predictive models

As businesses race to integrate AI into their software, many turn to first-party data modeling techniques like [RFM (recency, frequency, monetary value)](https://en.wikipedia.org/wiki/RFM_$market_research$) to build predictive models. These models typically rely on user interactions, including website navigation, email engagement, and purchase behavior to anticipate action. But first-party models alone have significant limitations, lacking the depth, speed-to-deployment, and predictive accuracy that businesses need to get ahead in today’s competitive market. 

Faraday’s models solve these problems by integrating rich third-party data from the [Faraday Identity Graph (FIG)](https://faraday.ai/blog/unlocking-the-power-fig), out of the box. The FIG enhances our models by integrating high-quality, consented offline insights into your first-party data, creating a more complete and actionable dataset. By enriching your existing data with verified external attributes—such as demographics, lifestyle indicators, and purchasing behaviors—the FIG fills in gaps, improves predictive accuracy, and speeds up deployment to help businesses reach the right audiences at the right time with the right product.

## Why first-party data alone falls short

First-party data has its advantages, but it also has critical limitations that can hinder AI-driven personalization and prediction.

First-party data is information that businesses collect directly from their customers and audience through owned channels. This includes website activity, purchase history, and email engagement. Because it comes straight from the source, first-party data is highly accurate, privacy-compliant, and essential for personalized marketing and predictive modeling. However, it can be limited in scope, which is why many businesses enhance it with external data for deeper insights.

One major challenge to overcome with first-party data alone is the [cold start problem](https://www.sciencedirect.com/topics/computer-science/cold-start-problem). When a new customer visits your business, there is no past activity to analyze, making it impossible to generate useful predictions without supplemental third-party data. A first-party data-based model will most likely only improve over time as the customer interacts with the brand, but this delay can mean lost opportunities, particularly in industries where real-time data is crucial like financial services.

Even as first-party data accumulates, it often remains noisy and incomplete. Activity data is unpredictable, with various factors influencing when a user engages with your site or makes a purchase, meaning first-party models must quickly adjust to these fluctuations. External factors like age or income level are also essential for understanding customer behavior, but they’re typically not captured in first-party data. Additionally, first-party data only reflects [activity, not affinity](https://faraday.ai/blog/why-first-party-data-and-rfm-is-not-enough). For instance, knowing that someone purchased a pillow tells you little about their broader lifestyle, financial capacity, or long-term interests.

## How Faraday enriches customers’ datasets to power our models

Our data enrichment process uses identity resolution to securely match first-party identifiers—such as email addresses or hashed customer IDs—with the FIG’s extensive dataset before they are fed into our models. This process ensures a seamless connection between your existing data and additional insights, following three key steps:

1. **Secure matching** – Faraday uses identity resolution techniques to match first-party data with offline records, ensuring privacy and compliance along the way.  
2. **Data enrichment** – Once linked, Faraday appends high-quality demographic, behavioral, and lifestyle attributes to each record.  
3. **Model integration** – These enriched profiles are seamlessly fed into our AI models, enabling more accurate predictions and personalized customer experiences. 

[![Data-appends diagram](/images/blog/first-party-merged-w-fig.png)](/images/blog/first-party-merged-w-fig.png)


## Powering better AI with enriched data

Our models’ ability to access the **Faraday Identity Graph (FIG)** solves numerous problems experienced by first-party-only models. Here are some examples:

### 1\. Solving the cold start problem

With the FIG, even first-time visitors are no longer unknowns. By matching first-party identifiers to offline, third-party data, Faraday provides insights into likely interests, purchasing power, and household composition, allowing our models to make informed predictions from the very first interaction.

### 2\. Boosting model stability and accuracy

AI models trained solely on first-party data must constantly adapt to new patterns in an ever-changing environment. The FIG introduces stable, high-quality attributes, reducing reliance on transient, high-variance signals like click-through rates. This results in more durable models that perform consistently over time.

### 3\. Enhancing AI with affinity data

Instead of just looking at past transactions, the FIG integrates data that reflects who customers are, not just what they’ve done. This includes household income, education level, homeownership status, and even media consumption habits.

## The best of both worlds

Faraday’s FIG takes you data to the next level. By seamlessly integrating high-quality, consented third-party insights with your existing first-party information, Faraday empowers you to access more accurate and predictive models through our platform. With this rich array of demographic, lifestyle, and behavioral attributes, you can unlock deeper insights into your customers and make smarter, data-driven decisions. The Faraday Identity Graph goes beyond the limitations of first-party data alone, giving you a clearer, more holistic view of your audience.

Want to see how the FIG can supercharge your AI models? **Let’s talk.**

The Faraday Identity Graph (FIG) allows users to create more accurate, stable, and predictive AI models within our platform with deeper insights into customer behavior and preferences.

Creating better predictive models with the Faraday Identity Graph

It's 4pm on a Monday and your boss has just dropped the news: lead conversion rates are tanking, even as volume has grown dramatically. You need a fix _by Friday_.

Your team is drowning in leads ranging from red-hot to ice-cold—but how do you sort through them?

Luckily you have a tool like Faraday in your belt, which springs into action, collecting and transforming all the relevant data, normalizing it for the situation, extracting representative training sets, engineering features automatically, building dozens of candidate models for the problem, tuning hyperparameters on the fly, and applying the right model to each lead at the right time to gauge its quality.

It’s not just a predictive AI tool, what you have is an AI agent. And it's the ultimate asset for engaging customers at scale, whether that's managing leads, messaging new offers, or personalizing an on-site experience.

Faraday has been building and operating AI agents for thousands of brands for more than 12 years. Now that popular terminology has caught up, we thought we'd share a bit more on the agentic phenomenon.

## A little context

AI agents have actually been around for quite some time. In fact, the idea of machines acting independently to achieve goals dates back nearly a century to science fiction stories like Karel Capek's 1920 play Rossum's Universal Robots (which coined the term, ‘robots’) and Isaac [Asimov’s 1940 classic short story, “Robbie,](https://www.wired.com/story/artificial-intelligence-minds-science-fiction/)” But these robot dreams didn’t take long to transition from science fiction to reality.

[![Shakey the robot full body](/images/blog/ai-agents-blog/shakey-the-robot.jpg)](/images/blog/ai-agents-blog/shakey-the-robot.jpg)
Shakey the Robot, photo credit: computerhistory.org

Indeed, as early as the 1960s, real-world examples of rudimentary robots, such as the [Shakey the robot](https://www.sri.com/hoi/shakey-the-robot/) were already starting to appear. Shakey was the first mobile general-purpose robot and could perceive its surroundings and reason with complex commands, breaking them down into basic chunks without needing explicit instructions for each step. [Shakey was also capable of planning and navigation, and significantly influenced modern robotics and AI techniques.](https://www.computerhistory.org/revolution/artificial-intelligence-robotics/13/289) But in reality, Shakey was, by modern standards, a little shaky.

Shakey could solve problems and utilized symbolic reasoning like modern agentic AI but was massively [limited by weak computational power and rigid programming frameworks.](https://aiworldjournal.com/the-evolution-of-agentic-ai-from-concept-to-reality/) For these reasons it couldn’t compare to the complexity and reasoning ability of modern systems. But despite these limitations, Shakey and [other early robots](https://community.hpe.com/t5/ai-unlocked/a-short-history-of-artificial-intelligence/ba-p/7172315) like it marked major steps forward in our ability to produce machines that could solve problems and make decisions independently. However, the progress was slow, the hardware was clunky, and Shakey and his friends all required an entire room of computers to perform simple tasks—so it wasn’t exactly the autonomous future Asimov had imagined.

[![cover of Asimov's Robbie the Robot, short story](/images/blog/ai-agents-blog/robbie-the-robot-cover.jpg)](/images/blog/ai-agents-blog/robbie-the-robot-cover.jpg)

But while Shakey was still breaking down commands, fiction had already imagined AI with greater capabilities. Take Robbie, the robotic nanny from Asimov’s story. Unlike Shakey, Robbie made independent decisions, learned from experience, and adapted his behavior. Though he doesn’t run on data pipelines or neural networks like modern AI agents, at his core, Robbie functions much like they do, by making decisions and fulfilling tasks independently (and, admittedly, with a cool metal body, but we’re getting there). And where Shakey fell short, today’s AI agents don’t face the same limitations, and the gap between fiction and reality continues to shrink.

Ironically, the main theme of this story was the misunderstanding of new and emerging technologies. With that in mind, let’s break down what AI agents really are, no fiction, just facts.

## Why AI agents are everywhere now

Fast forward to 2025, and AI agents are no longer just science fiction. They’re all over the place: managing supply chains, handling customer service, writing emails, and yes, optimizing marketing campaigns. But why the sudden explosion in AI agent adoption?

A few reasons:

- **Computational power**: Advances in cloud computing and GPUs mean that what used to require a warehouse of machines now runs on your laptop.
- **Better data availability**: AI thrives on data, and today, there’s more of it than ever before.
- **Automation hype**: Everyone wants AI to do the boring stuff so they can focus on the big picture (or maybe take a longer lunch break).

## How AI agents get jobs done

Modern AI agents operate by processing vast amounts of data, learning from patterns, and making decisions based on predefined goals. They typically rely on machine learning models, neural networks, and sophisticated algorithms to analyze information, predict outcomes, and take action.

Some agents function through reinforcement learning, continuously improving their performance based on feedback, while others follow rule-based systems with explicit instructions. Whether it’s a chatbot answering customer inquiries, an autonomous robot navigating an alien environment, or even a predictive model sorting through data, scoring it and then forwarding it to [Shopify](https://www.shopify.com/), AI agents work by combining perception, decision-making, and execution—often in real time—to accomplish tasks with minimal human intervention.

## Common misconceptions about AI agents

But as uncle Ben almost said, with great hype comes great misunderstanding. Here are a few common misconceptions about AI agents:

- **AI agents are sentient.** Nope. They analyze data, follow rules, and optimize for objectives, but they don’t “think” or “feel” in any meaningful way (sorry Robbie, we still love you).
- **AI replaces humans.** Not exactly. AI agents assist and augment human decision-making rather than fully replacing it. The best use cases involve AI handling repetitive or overly complex tasks while humans focus on strategy.
- **More data \= better AI.** This is sometimes true, but as the saying goes: 'Garbage in, garbage out.' More data doesn’t always mean better AI and quality matters more than quantity, especially when you’re looking for actionable business results.
- **AI agents are just fancy chatbots.** This one’s a little off too\! While chatbots are one application, AI agents go far beyond simple conversations. They can analyze data, make decisions, and automate complex tasks in a variety of fields.

## AI agents for the customer experience

Faraday is a flexible and fully composable platform that automatically processes and analyzes your data, delivering actionable insights to a range of targets. By using data and AI agents to make informed decisions and automate complex tasks, Faraday helps businesses streamline processes, optimize strategies, and drive growth. We tap into the true potential of AI agents to help businesses make smarter, faster, and more data-driven decisions, without needing a PhD in machine learning.

So how does Faraday work as an AI agent? Let's break it down:

1. **Independent decision-making**: AI agents act autonomously to achieve specific goals. Faraday independently analyzes consumer data, identifies patterns, and makes predictions without constant human oversight.
2. **Goal-oriented actions**: Agents perform tasks to accomplish defined objectives. Faraday's purpose is clear: to predict consumer behavior and optimize customer acquisition strategies to generate value. View some of [our customer stories](http://faraday.ai/stories) to see how much impact this can make.
3. **Learning and adaptation**: True agents improve over time. Faraday continuously refines its models based on feedback and new data, getting smarter with each interaction and iteration.
4. **Environment interaction**: AI agents observe and respond to their environment. Faraday processes vast amounts of customer data, market signals, and business metrics to inform its decisions.

In short, Faraday builds and operates AI agents for businesses that want data-driven growth, without the guesswork. Unlike Robbie the robot, it won't babysit your kids, but it will make sure your marketing efforts aren't stuck in the past (and if you insist, we do have a founder named [Robbie](https://www.linkedin.com/in/robbieadler/)).

Want to see how Faraday's AI agents can optimize your customer acquisition strategy? Let's talk.

AI agents, like the ones Faraday produces, are transforming marketing by automating predictions, optimizing outreach, and continuously learning from customer data to improve engagement and ROI.

Faraday agents: Why we changed the way we're talking about our core products

In the world of predictive analytics, data is everything. The more accurate, comprehensive, and up-to-date your data is, the better your models perform. That’s where the [**Faraday Identity Graph (FIG)**](https://faraday.ai/docs/abstractions/traits) comes in—a massive, carefully curated dataset designed to help businesses make smarter decisions with ease.

## What is the Faraday Identity Graph?

The Faraday Identity Graph is a powerful combination of multiple data sources, meticulously cleansed, de-duplicated, and feature-engineered to create one comprehensive dataset. It consists of [1,500 data attributes](https://faraday.ai/features/faraday-identity-graph) covering virtually all U.S. adult consumers and their households, offering a deep well of insights into demographic trends, property ownership, life events, financial indicators, and more.

And what’s even better is that it comes built-in with our services.

## So where does the data come from?

We partner with **best-in-breed data providers** like [Epsilon](https://www.epsilon.com/us) to compile a rich dataset that encompasses:

* **Demographic information** (age, education, income, etc.)  
* **Property details** (homeownership, property value, etc.)  
* **Life events** (moving, marriage, new child, etc.)  
* **Financial indicators** (credit attributes, equity, etc.)  
* **Buying patterns** (how they spend across dozens of categories)  
* **Shopping behavior** (e.g. in-store vs online)

By combining multiple premium data sources, we eliminate the need for businesses to juggle multiple vendors—saving them time, effort, and money.

## How frequently is the data updated?

The Faraday Identity Graph is updated regularly throughout the year as we receive new data from our vendors. The frequency of updates varies based on data type, but customers can rest assured that the data contained in our graph is as up to date and relevant as possible to ensure the best predictive outcomes.

## Can I buy the data directly?

Yes\! We offer direct data licensing, allowing businesses to purchase cleansed, consolidated, and competitively priced data. If you're looking for a variety of data sources, without the hassle of amalgamation and the time and expense of working with a range of vendors, we should talk—it’s likely we can save you all that time and money by bundling everything you need into one seamless package, and since we already made that package, you could be accessing that data tomorrow\!

## How can businesses use this data?

Our clients use the Faraday Identity Graph in a variety of ways but here are two core use cases:

* **Data appends:** Businesses can purchase specific data attributes to enhance their own internal models and processes.  
* **AI agents:** We can combine our data with your first-party data to build agents that answer key questions, such as:  
  * **Where are your best customers?** Read about our [market opportunity analysis service](https://faraday.ai/docs/abstractions/MOA).  
  * **Who is most likely to buy?** Check out our [lead scoring quick start guide](https://faraday.ai/docs/quickstarts/leads/lead-scoring).  
  * **What are they most likely to purchase?** Dive into our [product recommender](https://faraday.ai/docs/abstractions/recommenders) system, which predicts the best products or services for each individual.

Whether you're looking for raw data or full-service predictive modeling, Faraday makes it easy to harness the power of data-driven decision-making.

## The Faraday advantage

What makes this data even more valuable is access to our team of forward-deployed data science experts. Our team works alongside you to ensure that you are accessing not just the highest quality data, but the right data for your specific needs. With the addition of this service, whether you're looking for raw data or full-service predictive modeling, Faraday makes it easy to harness the power of data-driven decision-making. 

Additionally, Faraday delivers a broad suite of other services like real-time scoring, giving you instant access to dynamic, data-driven insights that enable rapid decision-making and model explainability, ensuring you can trace every prediction back to its data-driven roots—this transparency enhances model trust and guides smarter adjustments. 

What's more, with immediate access to this rich data from day one, you can avoid the cold start problem, quickly deploying predictive models without sacrificing accuracy. Lastly, Faraday’s flexible API integration allows you to seamlessly incorporate our data into your existing systems, providing a scalable solution that evolves alongside your needs and analytics workflows.

## Questions?

Sounds pretty great right? We think so too. If you want to learn more, reach out to talk to our sales reps or even our data team to learn how Faraday’s data can push your business to the next level\!

The Faraday Identity Graph (FIG) is a massive, meticulously curated dataset of 1,500 attributes covering virtually all U.S. adults, enabling businesses to enhance their models, streamline data sourcing, and make smarter decisions with ease.

Unlocking the power of the Faraday Identity Graph (FIG)

At Faraday, we often hear from clients about their experiences working with our competitors. These conversations give us valuable insights into how our platform stands out in the market, and where we have room for improvement. Recently, one of our clients shared a specific experience that really resonated with us, and we wanted to take the opportunity to highlight it to our audience.

This client, who recently worked with [Think Unlimited](https://thinkunlimited.com/) before moving to Faraday, pointed out some key areas where we outshine our competition. As she put it:

*"The transparency of the data science behind the modeling \[at Faraday\] was refreshing. There were no secrets. It was our data, and it was given back to us in a digestible fashion. We could also easily access the experts behind the science to ask questions."*

We believe that transparency is crucial, and we’ve built our platform with the goal of making data science accessible and understandable for our users, so this was rewarding to hear.

This client also observed a significant difference in the quality of predictions between Faraday and Think Unlimited, noting that Think Unlimited's data science just didn’t seem as robust under the hood. This aligns with our commitment to providing high-quality, validated predictive models that you can trust.

But those observations are just the start. Let’s take a deeper dive into some of the key differentiators we've observed ourselves and how they contribute to Faraday’s unique value proposition.

## Faraday vs. Think Unlimited: Our core differentiators

### 1\. Real-time scoring with Faraday's Lookup API

One of Faraday's core strengths is our ability to score leads in real time. Thanks to our powerful [lookup API](https://faraday.ai/docs/features/lookup-api), users can get immediate feedback on lead quality, before bids are even placed. By receiving immediate feedback, businesses can optimize decisions in real-time, leading to better outcomes in high-pressure or fast-moving situations, such as the [ping post process](https://faraday.ai/blog/so-what-is-a-ping-post-anyway). Think Unlimited, on the other hand, lacks this level of real-time scoring capability, which can slow down the workflow, impact decision-making timelines, and in the case of the ping post process, result in lower quality leads being purchased. 

### 2\. A more robust data set for better predictive power

Our built-in Faraday Identity Graph is a rich third-party [data set](https://faraday.ai/features/faraday-identity-graph) that goes far beyond basic metrics by incorporating a wide range of insights that enhance the accuracy and depth of our predictive models. In addition to traditional data points, we include rich demographic, lifestyle, and behavioral data and this combination of static and dynamic data allows us to understand not just who a customer is, but how they engage with products, services, and brands. Using these third-party insights, Faraday’s models can create a dynamic, multi-dimensional picture of potential customers, leading to more accurate predictions of future actions, such as conversion or product interest.

Faraday’s robust data set also helps solve the [cold start problem](https://www.sciencedirect.com/topics/computer-science/cold-start-problem) by providing this rich data from day one, meaning that brands don’t have to wait for a build-up of their own first-party data before accurate scoring can commence. It also allows them to immediately target new customer segments or groups that they don’t have existing data for, facilitating expansions and allowing them to quickly roll out new campaigns and products.

The depth of our data enables Faraday to generate more accurate, quick, and actionable insights compared to our competitors, empowering businesses to make data-driven decisions with confidence. 

### 3\. Score explainability: know exactly why leads get their score

Faraday emphasizes clear [score explainability](https://faraday.ai/blog/faraday-score-explainability) with all of our models. This means that users can actually know exactly why each lead receives a particular score. This level of transparency is crucial for businesses that need to understand the "why" behind their results. Having this clear visibility into the model’s scoring also helps businesses gain confidence in their decisions and ensures they can optimize their approach based on a better understanding of the process.

That earlier client specifically highlighted this point as being one of the biggest advantages she appreciated with Faraday compared to Think Unlimited, where such transparency can be harder to come by.

### 4\. Flexible integration across use cases

Faraday isn’t just limited to lead scoring. Our platform is adaptable and can fill a range of other use cases, including [product recommendation](https://faraday.ai/docs/how-to/use-cases/customers/next-best-offer), [lead suppression](https://faraday.ai/templates/lead-suppression), and even [churn prediction](https://faraday.ai/templates/churn-scoring), [plus a bunch of other options](https://faraday.ai/templates). This versatility allows businesses to leverage Faraday’s insights across different departments and applications, and also sets them up for success when expanding their business or launching new types of campaigns they haven’t undertaken before. Think Unlimited’s capabilities are more narrowly focused, which can hinder adaptability and expandability.

### 5\. Access to experts

At Faraday, we take our data science seriously. But we also understand that predictive modeling is inherently complex and that just presenting where data comes from doesn't necessarily mean that the client will be able to understand it. 

Score explainability measures are a critical first step but sometimes, the best way to ensure a score is actually explainable is pretty straightforward: just have an expert explain it. That’s why we made the game-changing choice last year to forward deploy our entire data science team, making them directly accessible to our clients—our CTO even works with a range of client accounts to ensure that they’re getting the most possible value out of their investment. This level of open communication and transparency is something Think Unlimited lacks and is the ultimate failsafe to ensure users are able to understand their predictions, and the models behind them.

### 6\. Fully API-based for seamless integration

[Faraday is fully API-based](https://faraday.ai/docs/reference), meaning it can be integrated into any system programmatically, offering unmatched flexibility and customization. This structure not only simplifies integration but also provides the flexibility to adapt the models into any custom workflows, making it an ideal choice for companies looking for a solution tailored to their unique needs. Our API is also well-documented, which facilitates integration into clients’ existing tech stacks and ensures easier operation throughout the customer lifecycle. 

These features dramatically increase speed-to-deployment, with some clients seeing major cost savings or revenue growth [within their first month](https://faraday.ai/blog/optimized-direct-mail-outreach-in-one-month) after signing on. Think Unlimited, in contrast, lacks the API-based framework that makes Faraday so adaptable to a wide range of use cases and workflows.

## Quality predictions and transparency make the difference for our clients

At the end of the day, the quality of the predictions you rely on is what matters most. Our predictions are not only proven to be accurate and value driving, but they’re constantly monitored and revalidated to make sure we’re delivering as much value as possible to our clients. Additionally, Faraday’s data science, transparency, and flexibility make it the clear choice for businesses seeking high-quality predictive analytics that can scale and adapt to their needs, all while remaining understandable and accessible. Think Unlimited may have its place, but when it comes to powerful, explainable models and real-time scoring, Faraday leads the way.

Faraday outshines competitors like Think Unlimited with real-time scoring, a richer data set, transparent predictions, flexible integrations, expert support, and a fully API-based platform.

Faraday vs. Think Unlimited: Our core differentiators

[Direct mail remains a core acquisition strategy for many financial services providers](https://www.lob.com/blog/the-benefits-of-direct-mail-marketing-in-todays-economy), but it comes with a hefty price tag. Each mailer represents an investment, and when sent indiscriminately, costs can spiral with little return.

When this client first came to us at the end of 2024, like many in the industry, they relied primarily on instinct-based direct mail strategies, meaning a significant portion of their outreach was reaching consumers who were unlikely to convert. They knew they were overspending but lacked the tools to pinpoint which mailers were worth sending—and which could be cut without hurting revenue.

This client needed a solution that could optimize their outreach, reduce waste, and maintain strong enrollment numbers. Most of all, they wanted confidence in whatever decision they made—all without disrupting their existing workflow.


## Lead suppression with Faraday

And that’s where Faraday came in. As soon as we identified their challenges, we quickly devised a targeted strategy to help. 

The answer? Faraday’s [lead suppression](https://faraday.ai/templates/lead-suppression/recharge) template.

To enact this strategy, we started by inputting the provider’s recent lead data into our system (maintaining rigorous data privacy standards like HIPAA, GDPR, and SOC-2 compliance throughout the process). This data was then enriched through our [identity resolution](https://faraday.ai/blog/identity-resolution) process with 3rd party data from the built-in Faraday Identity Graph (FIG), which [includes over 1,500 attributes on more than 240 million U.S. consumers and their households](https://faraday.ai/features/faraday-identity-graph), and features nuanced demographic and behavioral indicators that influence consumer actions—ranging from property ownership to life events and purchasing habits.

From there, new predictive models were created and trained on this historical enrollment data (now enriched with 3rd party insights), identifying patterns and trends that distinguished high-value leads from low-value ones. Then, using advanced machine learning techniques, the system assigned each lead a predictive score, ranking them into ten distinct deciles based on their likelihood to convert. With this scoring system in place, a [cut-off threshold](https://medium.com/swlh/determining-a-cut-off-or-threshold-when-working-with-a-binary-dependent-target-variable-7c2342cf2a7c) was established to determine the point at which the cost of printing and sending a mailer outweighed the expected return, ensuring that outreach efforts remained cost-effective.

Finally, Faraday delivered an optimized, high-fit mailing list back to the provider and armed with this refined list, they were able to move forward with confidence, ensuring that every mailer sent had a greater likelihood of driving enrollment. For a visual breakdown of this process, check out our recent blog on lead suppression, [here](https://faraday.ai/blog/holdout-testing-with-faraday).

So how did it go? 


## The results (so far)

If you noted the timestamp on this article, you might be thinking to yourself, “Well, they reached out at the end of 2024 and this was published on Feb 21, 2025, so did they even get the models rolled out yet?” 

Well, despite the short timeframe, they not only deployed the models and sent direct mail based on their predictions, but have also already seen a significant reduction in costs, all while maintaining strong engagement with high-quality leads. 

Indeed, by suppressing low-likelihood recipients, they were already able to send over **80,000 fewer mailers**, producing **a net savings of nearly $40,000**, demonstrating the immediate financial benefit of a more targeted approach.

These early financial wins were certainly affirming, but it had only been a month. How did we know that they were actually the result of our model and not just an uptick consumer interest this year?

### Ongoing model validation

At Faraday, we take the [validation](https://faraday.ai/docs/reference/getscopeefficacy) of our models very seriously. We do this before predictions are deployed through our [holdout testing process](https://faraday.ai/blog/holdout-testing-with-faraday) and continue testing on an ongoing basis even after successful implementation. In this case even though we were already seeing positive results, we wanted to see data that confirmed (or challenged) our assumptions. So we conducted further analysis on our models after we started getting results back to confirm that our predictive scoring was actually aligning with real-world outcomes. 

And we found what we were looking for: Leads with the lowest predictive scores showed below-average conversion rates, validating the suppression strategy. Meanwhile, leads in the highest-scoring segments consistently outperformed the average, reinforcing the value of prioritizing the right audience. And what’s even better is that this correlation is only expected to strengthen as more data flows through the model, further refining predictions and maximizing efficiency.

While these financial results would be impressive on their own, they also highlight how quickly Faraday’s models can be deployed and optimized. The contract was signed in late-December, and within weeks, the provider was already seeing meaningful returns. This quick turn around, combined with the ability to continuously refine and improve targeting over time, positions them for even greater efficiency and cost savings in the months and years ahead.

## Next steps

With direct mail suppression already demonstrating clear value, the provider is now looking to further optimize their customer outreach efforts. A key next step? [Call center list scoring](https://faraday.ai/stories/call-center).

Similar to how Faraday’s predictive models helped refine direct mail targeting, they can also help [prioritize leads](https://faraday.ai/templates/lead-prioritization) for call center campaigns. By focusing call center agents’ efforts on high-likelihood prospects, the provider can reduce wasted calls, improve conversion rates, and increase overall campaign efficiency.

We can’t wait to see how they use this technology to generate even more value for their business\!

## In conclusion

In just a short time, this financial services provider has revolutionized its direct mail strategy, reducing unnecessary costs while maintaining strong engagement with high-value leads. Faraday’s predictive modeling has enabled them to move beyond broad, instinct-driven targeting to a data-backed approach, ensuring each mailer sent delivers greater value. And with tens of thousands of dollars saved in only a few weeks, the provider has already realized the immediate financial benefits of a more targeted outreach strategy.

What’s more, looking forward, the provider is well-positioned to refine their targeting approach even further. By continuously updating and optimizing their predictive models, and exploring additional use cases like call center list scoring, they can maximize the efficiency of all customer acquisition channels. 

For this company, this is just the beginning of a smarter, more efficient approach to customer engagement—one that will continue to drive results well into the future.

If you'd like to know how Faraday can produce similar results for your business, **sign up for free**!

In their first month with Faraday, this financial services provider used lead suppression to cut 80,000 low-value mailers and save nearly $40,000.

How this financial services provider optimized their direct mail outreach in just one month

[Klaviyo](https://www.klaviyo.com/) made a big splash last year, bringing entry-[level predictive scoring](https://faraday.ai/blog/why-you-need-predictive-lead-scoring-today) to their massive audience. And we love it—the industry is catching up to what we’ve been doing for years. With AI-powered tools like [predictive analytics](https://faraday.ai/templates/lead-prioritization), [personalized product recommendations](https://faraday.ai/templates/next-best-offer), and advanced [sentiment analysis](https://faraday.ai/blog/seamus-bigquery-interview) for their emails lists, Klaviyo is giving marketers powerful capabilities to forecast customer behavior, personalize messaging, and automate campaigns. But if you’re looking to take it to the next level, there’s a critical missing piece: [deep consumer insights powered](https://faraday.ai/features/faraday-identity-graph) by [third-party data](https://faraday.ai/docs/abstractions/traits).

## Klaviyo’s predictive tools, and their limitations

[Klaviyo’s AI tools](https://www.klaviyo.com/features/ai) offer a solid foundation for email marketers, helping to predict engagement and automate messaging. But recent changes—including increasing costs and [stricter policies around active contacts and list sizes](https://help.klaviyo.com/hc/en-us/articles/24263920096027)—are making efficiency more important than ever.

Though Klaviyo’s AI is powerful, their analysis is also stymied by its lack of inclusion of third-party data. This type of rich demographic and behavioral data inherently improve the depth, nuance, and critically, speed-to-deployment of predictive models. It also radically improves the accuracy and value that they can be expected to produce.

Only with this type of rich third-party data can marketers expect to achieve the highest levels of  speed and precision with their predictive marketing, and that’s where Faraday’s built-in integrations comes in.

## Optimizing Klaviyo lists with Faraday personas

Integrating Faraday’s [personas](https://faraday.ai/docs/abstractions/persona_sets) and [customer segmentation](https://faraday.ai/docs/how-to/connections/segment) capabilities into your existing Klaviyo stack takes your email lists to the next level, allowing you to gain deep insights that Klaviyo alone can’t provide.

With Faraday, you can segment your email lists not just by basic engagement metrics but by [demographics, life events, property details, and other behavioral signals](https://faraday.ai/features/faraday-identity-graph). This level of insight empowers you to tailor content, messaging, and campaigns that are perfectly suited to your audience. Whether it’s understanding the impact of a new home purchase or identifying consumers in a particular life stage, Faraday’s data-driven approach helps you reach the right people at the right time with the most relevant messaging.

By adding these third-party insights to your Klaviyo platform, you unlock smarter segmentation, faster insights, and a deeper understanding of your customers—making your email marketing more efficient, relevant, and impactful.

### The inclusion of this this data improves your Klaviyo lists by:

* **Building smarter segments** based on real-world demographics, life stages, and property data.  
* **Refining automated journeys** to align with consumers’ actual behaviors and preferences.  
* **Improve conversion rates** by sending the right message to the right people at the right time.

## The cold start problem—and how to overcome it

An additional benefit of integrating third-party data into your Klaviyo stack is the ability to avoid the [cold start problem](https://www.sciencedirect.com/topics/computer-science/cold-start-problem), a classic challenge faced by many marketing tools, including Klaviyo’s out of the box predictive scoring. While these tools are able to generate generally accurate predictions on long time customers using only first-party data, they quickly run into issues when targeting leads or recently acquired customers with very little observed behavior. 

In these cases, the model struggles to quickly generate meaningful insights, as it lacks the foundation of prior data to build from. This delay in data accumulation can significantly limit the effectiveness of campaigns until enough data has been collected.

Faraday inclusion of our third-party data eliminates this barrier by enabling you to build highly targeted customer segments right from the start. Instead of waiting months for data to accumulate, Faraday’s built-in data provides immediate insights, empowering you to craft campaigns optimized for engagement and conversion from day one. Without waiting for slow data accumulation, you can launch campaigns quickly with confidence.

## So who will actually convert? Let’s find out.

Your Klaviyo lists might still be growing, but are they optimized for conversion? Are you maximizing their value-generating potential? With Faraday, you can build a free persona model and see for yourself. Think of all the ways you could use this—better segmentation, more precise messaging, and automated journeys that actually resonate.

Klaviyo is a great start for any email marketing, but Klaviyo \+ Faraday together can take your email marketing to the next level\!

Enhance your Klaviyo lists with Faraday’s third-party data and personas to unlock deeper insights, smarter segmentation, and higher conversion rates—solving the cold start problem and optimizing your email marketing from day one.

Leveling up your Klaviyo lists with Faraday personas

If you are reading this, you are probably talking to Faraday about a potential contract and have a healthy amount of skepticism as to whether we can actually do what we say we are going to do.

And here’s the thing: you’re right to be skeptical. With tight budgets and increasing pressure to show ROI, you can't afford to bet on unproven solutions or spend countless cycles figuring out if a tool is really going to live up to the promise. You need a way to separate legitimate predictive tools from the noise.

Every week, we get to talk to dozens of CMOs, CTOs, & CEOs across industries who are trying to discern the same thing: which tools are legitimate and which ones are not. As a leader of a high profile brand, you are inundated with promises of predictive insights and transformation, but you need more than just promises – you need guarantees.

That's where our holdout evaluation comes in. It's not a proprietary methodology we invented – it's an industry-standard approach used by data scientists to validate predictive models.

**What makes us different is simple: we run this validation before you sign any contract.** Because frankly, if we can't prove we'll deliver real results for your business, we don't want to waste your time.

Below we’ve outlined what we are doing “behind the scenes” and how you can evaluate if we will add the value you need. 

## What is a holdout evaluation?

A holdout evaluation is a method for validating predictive models by comparing their predictions to real-world outcomes. We **set aside a portion of recent conversion data**—data the model has never seen before—and use it as an independent benchmark, against which predictions can be compared.

This “holdout set” acts as a real-world proving ground. It is untouched until after the predictive model has been trained. 

Meanwhile, the rest of your historical data is combined with rich third-party insights from the [Faraday Identity Graph](https://faraday.ai/docs/abstractions/traits) ([which includes 1,500 attributes on 240M+ U.S. consumers and their households](https://faraday.ai/features/faraday-identity-graph)). We use this combined data to train the new predictive model.

Training a predictive model simply means finding patterns in conversion behavior. In other words, what can we learn from your previous customers to identify future customer behavior? Once trained, the model generates predictions—scoring leads based on their likelihood to convert and only after these predictions are made do we compare them to the **real conversion rates** in the holdout set.

If the highest-scoring leads actually convert at higher rates than lower-scoring ones, the model is validated as accurate and reliable. 

Holdout evaluations ensure predictive models aren’t just theoretical—they work in the real world, using real data, to deliver real results.

## How holdout validation works at Faraday

We've streamlined this validation process to be transparent and straightforward:

1. ![customer conversion data](/images/blog/holdout_testing_diag/holdout_testing_diag-01.png)

   #### Prepare your data and set aside a holdout group

   We start with your recent [lead conversion data](https://faraday.ai/blog/first-party-vs-third-party-data-marketing). Before any processing occurs, we reserve a portion of your most recent leads as a "holdout set"—data that the model will never see during training. This ensures we have an independent benchmark for evaluating predictions.

2. ![merged with FIG](/images/blog/holdout_testing_diag/holdout_testing_diag-02.png)

   #### Combine data with Faraday Identity Graph

   With the holdout set reserved, we then combine remaining data with Faraday’s [built-in, rich  consumer data](https://faraday.ai/features/faraday-identity-graph) sourced from the Faraday Identity Graph. This expanded dataset provides a deeper view into key predictive signals, like property and demographic information, life events, buying habits, and more.

3. ![model is trained and predictions begin](/images/blog/holdout_testing_diag/holdout_testing_diag-03.png)

   #### Train the model and begin predictions

   Using this enriched dataset (excluding the holdout set), we train a predictive model to identify patterns in conversion behavior—learning from traits, behaviors, and other signals.

   Once the model is trained, it scores each lead based on its likelihood to convert, generating a ranked list of opportunities—from high-value leads to those less likely to convert.

4. ![model is validated](/images/blog/holdout_testing_diag/holdout_testing_diag-04.png)

   #### Compare predictions to reality to validate model

   Finally, we put the model to the test: we compare its predictions against the real outcomes in that untouched holdout set. If the highest-scoring leads don't convert at significantly higher rates, we go back to the drawing board.


## How to know if a holdout validation was successful

A holdout validation is considered successful when the following criteria are met:

1. **Clear correlation between predicted probability and actual conversion rates**  
   The model’s predicted likelihood to convert should directly align with the actual conversion rates. If the predictions are accurate, high-probability leads should convert at a noticeably higher rate than lower-probability ones.

2. **High-scoring leads convert at significantly higher rates**  
   The top-scoring leads should consistently perform better than those ranked lower. This demonstrates that the model is identifying truly valuable leads.

3. **Consistent performance across different customer segments**  
   The model should show reliable results across various segments—whether you’re targeting different demographics, regions, or other customer profiles. If the model works well across a broad set of data, it proves its reliability and scalability.

When a model meets these criteria, it’s ready for deployment—whether that means optimizing [lead buying](https://faraday.ai/docs/how-to/use-cases/leads), refining your [direct mail strategy](https://faraday.ai/templates/lead-suppression), or enhancing your email [personalization](https://faraday.ai/templates/personalization) approach.

## Questions you should ask us

We want you to push us on what matters most for your business. Here are key questions to evaluate if we're the right fit:

1. **How does this model's performance compare to the lift you are seeing with  similar businesses in our industry?**

   While we can’t share client specifics, we can provide insight into broader trends and expected performance benchmarks.

   → **Why this matters:** Understanding industry benchmarks helps you gauge whether predictive modeling is truly a competitive advantage for your business.

2. **How much historical data do we need for reliable predictions?**

   The answer varies by business model and conversion volume. We’ll help determine whether you have enough data—or how to supplement it if needed.

   → **Why this matters:** Predictive models work best when trained on strong data. Knowing the threshold for reliability ensures you're making data-driven decisions with confidence.

3. **What happens if the model doesn't perform as expected?**

   We have a structured iteration process to refine models when necessary, ensuring they reach meaningful predictive power.

   → **Why this matters:** No model is perfect on the first run. Understanding the process for continuous improvement ensures long-term success.

4. **How will this integrate with our existing marketing stack?**

   We’ll provide technical specs, discuss implementation timelines, and outline the resources needed to get up and running.

   → **Why this matters:** Seamless integration means faster time-to-value and less friction in your workflow.


## Why holdout evaluations work

The businesses who win this year will be those who successfully incorporate predictive modeling into their core workflows. It's no longer a nice-to-have – it's a need-to-have. But we know you can't afford to make decisions based on hunches or unproven tools. That's why we're eager to validate our value up front. 

At the end of the day, three things matter to us:

1. **Confidence before commitment:**  
   You see proof before you commit

2. **Higher marketing efficiency:** 
   Your marketing dollars go to the leads most likely to convert

3. **Stronger sales performance:** 
   Your sales team focuses on opportunities worth their time

If we can't demonstrate value, we'll be the first to tell you we're not the right fit. But if we can – and we usually do – we want you to know the exact savings or additional revenue you can expect to see over the course of the next year with Faraday. Interesting in generating value you can trust? Reach out to learn more about what we can do for your business\!

Faraday uses holdout testing to validate predictive models before you commit, ensuring they deliver real, measurable results.

All posts