👩‍🎤 Are you building your infrastructure for the next 5 years?

It might look like all of the clouds are equivalent&hellip;

- Focused on [Snowflake](https://snowflake.com)? You can run it on any cloud - [Azure](https://azure.microsoft.com/), [AWS](https://aws.amazon.com/), or [Google Cloud](https://cloud.google.com/).
- Committed to [Segment](https://segment.com/docs/connections/storage/warehouses/), [mParticle](https://www.mparticle.com/platform/detail/warehouse-sync/), [Ometria](https://ometria.com/) or [Amperity](https://amperity.com/)? They can give you Customer 360 while syncing to/from any of the big 3 warehouses ([BigQuery](https://cloud.google.com/bigquery), [Redshift](https://aws.amazon.com/redshift/) or Snowflake).
- Bought into composable, native architecture? [Hightouch](https://hightouch.com/platform/customer-studio) will give you amazing audience builders and match boosting - and [MessageGears](https://messagegears.com/) is trusted by Super Senders like [Home Depot](https://www.homedepot.com/) - and they also run on any cloud.
- Ready to turn your customer data into powerful predictions? - [Faraday](https://faraday.ai) has hundreds of customers on all the clouds.
- And the [Department of Defense chose all the clouds](https://www.defense.gov/News/News-Stories/Article/Article/3345260/dod-makes-headway-on-cloud-computing/). You can build zero trust anywhere.

**So how do you choose your cloud?**

- [Snowplow](https://snowplow.io/) and [RudderStack](https://www.rudderstack.com/) will push to any cloud&hellip; but [GA4 only pushes to BigQuery](https://support.google.com/analytics/answer/9358801?hl=en).
- [Azure lets you use open source to build midsize machine learning models](https://learn.microsoft.com/en-us/azure/azure-sql/managed-instance/machine-learning-services-overview?view=azuresql). But [BigQuery ML will give you 10x scale](https://cloud.google.com/bigquery/docs/bqml-introduction) because things like [XGBoost](https://xgboost.readthedocs.io/en/stable/) are compiled directly into [Google's distributed engine, Dremel](https://research.google/pubs/pub36632/).
- [Databricks lets expert Python users use generative AI right next to their data](https://www.databricks.com/product/machine-learning/large-language-models)&hellip; but [BigQuery's ML.GENERATE_TEXT() does it from plain SQL](https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-generate-text).
- Faraday is [Built on BigQuery](https://cloud.google.com/blog/products/data-analytics/faraday-uses-bqml-to-make-predictions-for-any-consumer-brands) and [Built on Google Cloud AI](https://cloud.google.com/solutions/ai-partners) because it was the best place to build an AI/ML application.

## Heading to Next?

[Faraday CTO Seamus Abshere](https://www.linkedin.com/in/seamusabshere/) will be joining [Google's Christoper Crosbie](https://www.linkedin.com/in/christopher-crosbie-81434961/) to discuss [generative AI inside of the data warehouse](https://cloud.withgoogle.com/next/session-library?session=ANA211#view-all) at **10:15am on August 31st.**

There's a lot to consider when building your infrastructure for the next few years, with AI at the forefront. Listen to Faraday CTO Seamus Abshere speak at Google Cloud NEXT to discuss generative AI and the safest place to build AI.

The safest cloud for AI

It’s no secret that [personalized direct mail](/blog/personalized-direct-mail-strategy) marketing is a great way to engage with your customers. Whether you’re interested in driving [repeat purchases](/predict/category/repeat_purchasers) from current customers, [finding new ones](/predict/category/likely_buyers), winning back old ones, or something else entirely, direct mail has proven to be an [increasingly effective strategy](https://www.postgrid.com/direct-mail-statistics/) to make your brand stand out in an era awash with the same old email marketing tactics. In this feature guide, we’ll walk through how to use Faraday to enable direct mail marketing campaigns that are highly personalized, and highly effective.

## How can Faraday be used for direct mail?

Faraday can be used for your direct mail campaigns by creating targeted cohorts, or groups of people, based on attributes from the Faraday Identity Graph, such as their ability to receive mail via, as well as their physical address. In this guide, we'll walk through how to **create a cohort** for direct mail, and how to use it when **deploying a pipeline**.

## Creating a cohort for mailable U.S. customers

To create a new cohort:

1. Navigate to cohorts via the menu bar on the left-hand side and click **+ New cohort** in the upper right.

2. Optionally, select an **event** for the cohort. In this specific direct mail use case, we're going to skip this step. For more info on this, check out our [cohorts documentation](/docs/abstractions/cohorts).

3. Next, click the **Add trait** button, which is the most important part in designating that this cohort of people are able to receive your direct mail.

[![Faraday add cohort trait location](/images/blog/direct_mail_guide/direct-mail-guide-add-trait.png)](/images/blog/direct_mail_guide/direct-mail-guide-add-trait.png)

4. In the **Add trait** popup, we're going to find & select the following trait:

- **Record quality code**: To ensure that the residence can receive mail at all. Choose the option to "select exact values from list" to select mailable.

[![Faraday direct mail cohort traits](/images/blog/direct_mail_guide/direct-mail-guide-cohort-traits.png)](/images/blog/direct_mail_guide/direct-mail-guide-cohort-traits.png)

<Callout type="info" title="Restricting by geography">

Optionally, add a _geography_ trait such as Postal code/ZIP to target specific locations. When using a cohort with a geographic restriction, like "Direct mailers in Burlington, VT," in a pipeline, the deployment will only include people in that locations.

</Callout>

5. Enter a unique name for the cohort, like _Direct mailers_, and click **Create cohort**.
6. Once finished, click **Save changes**. The cohort will begin building and will display with a _calculating_ message.

## Creating a pipeline to predict likely buyers for direct mail

While our _Direct mailers_ cohort is building (if we select it in a pipeline, the pipeline will build once the cohort is ready), we're ready to create a pipeline and deploy it via CSV, which will let us plug our direct mail cohort right into our preferred mailer platform. For a more detailed look at creating a pipeline, check out the [pipelines documentation](/docs/abstractions/pipelines).

1. Navigate to [Pipelines](https://app.faraday.ai/pipelines), and click **+ New Pipeline** in the top right to create a new pipeline.

2. In **population to include**, we'll grab the cohort we just created, _Direct mailers_, as we're only interested in the group of people we've selected: people in the U.S. who are able to receive direct mail.

3. In **population to exclude**, we can optionally add our _Customers_ cohort if we’re only looking for net-new customers.

4. For our **payload**, we'll select:

   - Our **propensity** (likely to buy) outcome. This should be a standard likely buyers outcome, with your _Customers_ cohort as the attainment, and _Anyone in Faraday’s identity graph_ as the eligibility. For more info on creating that outcome, see the [outcomes documentation](/docs/abstractions/outcomes). We’ll filter this outcome a little further on to target the top 20% most likely to buy people in this payload so that we're saving resources by not sending our direct mailer to people who aren't likely to buy.

   - Optionally, we can apply our **customer personas** to the people in this pipeline so that we can tailor creative & copy to their wants & needs.

[![Faraday direct mail pipeline](/images/blog/direct_mail_guide/direct-mail-guide-pipeline.png)](/images/blog/direct_mail_guide/direct-mail-guide-pipeline.png)

5. Enter a unique **name**, and click **Save pipeline**. The pipeline will begin building, as indicated by a loading bar. You'll receive an email when the pipeline has finished building.

[![Faraday direct mail pipeline building](/images/blog/direct_mail_guide/direct-mail-guide-pipeline-building.png)](/images/blog/direct_mail_guide/direct-mail-guide-pipeline-building.png)

6. Once the pipeline is complete, head to the **Deployment** section inside the pipeline, and select **+ Add** under **Hosted CSV**.

7. We're presented with the choice of _Hashed, Referenced, Identified_, and _Aggregated_ as options for our predictions. Since we're creating a direct mail pipeline, select _Identified_ so that identities are not hashed, and we can see physical address information, and so we can filter by residence below.

   - In **Individual identification options**, we'll select _One row per residence (address)_ so that we don’t send mail to more than one person in a household.

[![Faraday direct mail pipeline deployment](/images/blog/direct_mail_guide/direct-mail-guide-deployment.png)](/images/blog/direct_mail_guide/direct-mail-guide-deployment.png)

8. Select **human friendly** for column headers that are easily readable, and click next.

9. Now we’ll expand **filter**, then click **add payload filter** so that we can specify that we’re only interested in the top 20% of likely buyers, ensuring we’re not wasting time and money on bad fits.
   - In the payload filter, select **propensity to buy - outcome percentile** to sort by score percentile, or the distributions of propensity scores within the _Direct mailers_ cohort we targeted.
   - Change the operator to **greater than or equal to (>=)** and type **81** into the next field. Since the scoring is inclusive, this means that, in this deployment, we’ll filter it in a way that we only receive the scores for the top 20% of likely buyers.

[![Faraday direct mail pipeline deployment](/images/blog/direct_mail_guide/direct-mail-guide-deployment-filter.png)](/images/blog/direct_mail_guide/direct-mail-guide-deployment-filter.png)

<Callout type="info" title="Filtering deployments">

If you have a specific budget to stay under, expand the **limit** option to add a cap on the number of rows you receive. For example, maybe each direct mailer costs \$10, and you have a budget of \$50,000. You’d set the limit to the top 5,000 rows.

</Callout>

10. Last, click **finish** to complete the deployment, after which it will begin building. We can use the download button to download the CSV of a completed deployment for upload into our direct mail platform, which will include the top 20% of likely buyers that are able to receive mail in the country, (optionally) with their predictive personas included so that we can tailor copy & creative appropriately.

[![Faraday direct mail pipeline deployment download](/images/blog/direct_mail_guide/direct-mail-guide-deployment-download.png)](/images/blog/direct_mail_guide/direct-mail-guide-deployment-download.png)

**Ready to try it out? [Create a free account](https://faraday.ai/signup).**

How to use each feature in Faraday to create predictions that enable you to accurately target direct mail campaigns through propensity predictions.

How to target direct mail campaigns using Faraday

To salespeople and marketers, Salesforce is a very well-known commodity. Not only is it a CRM, but it has a suite of convenient addons and features for practically every use case under the sun, from forecasting to predictive lead scoring. Since these are addons and features directly within Salesforce, and they’re easy to integrate into the Salesforce part of your workflows.

In this article, we’re going to walk through the drawbacks of Einstein lead scoring (especially for B2C companies), and how Faraday’s approach to [B2C lead scoring](/blog/predictive-lead-scoring-b2c) offers faster, more versatile, and more holistic predictive lead scores that make use of all your data, not just what you have in your Salesforce account.

## Einstein lead scoring doesn’t quite cut it for B2C companies

While the seemingly just-add-it-on implementation of Einstein lead scoring is without a doubt eye-catching, the drawbacks that come with simple addons are inevitable when compared to feature-complete products that are built to accomplish the same task. The shortcut is rarely worth the time and effort invested when the results are often lackluster. Einstein lead scoring falls short for a few different reasons:

- **The cold start problem**: Einstein can’t make predictions on net new leads. It requires time for data to be collected on the individual before a score can be assigned. This leaves you either sitting on your hands while the lead is looking around at your competitors.
- **Limited data**: Data is the foundation of any effective revenue-driving strategy. With Einstein, only your first-party data is able to be used. For leads that you already have data on, you’ll have a rough idea of how they’ve interacted with your brand, but you won’t know what kind of person they are, how they shop, or what makes them tick.
- **Labor-intensive**: If you have non-sales data in disparate systems, like product data in Amplitude or Mixpanel, you’ll need to spend a significant amount of time manually plugging in your key metrics & datapoints to map out how leads should be scored.

## What’s the best B2C lead scoring software for Salesforce?

Here's the thing: the best lead scoring software is purpose-built, like Faraday, not an add-on, like Einstein. If you take the shortcut, it’ll show signs of rust quickly, leaving you scrambling for an alternative and wondering why you ever bothered in the first place. The good news is that Faraday offers a suite of customer predictions for every stage of the customer journey - one of which is lead scoring.

### Solving the cold start problem

Like I mentioned earlier, Einstein lead scoring leaves your reps sitting on their hands as new leads come in. You have no data on them, so you don’t know how to engage with them–or if they’re worth engaging at all. This delay leaves your leads plenty of time to check out your competitors’ offers, and to bite at the [personalized direct mail](/blog/personalized-direct-mail-strategy) postcard they just got in the mail.

Using Faraday, you can solve the [cold start problem](https://blog.hive.co/solving-the-problem-of-cold-start-ecommerce-product-recommendations/#) by scoring leads with propensity models for their likelihood to convert as soon as they come in, enabling your reps to act with them immediately. Whether it’s to prioritize the lead because their predictive lead score is high, or to disregard because it’s low, your reps’ time–their most valuable asset–is being used effectively. Lead prioritization is clear from the minute the lead first interacts with your brand, made possible by unifying your data with Faraday’s.

### Built-in consumer data

Your first-party data alone isn’t enough to not only solve the cold start problem, but also give you clear indicators for how to engage leads you already know about. You’re unlikely to know what the leads’ shopping preferences are, what kind of home they have, or their annual household income. With Faraday’s hundreds of [built-in, cookieless consumer attributes](/features/data), including demographics, psychographics, financials, property information, and more, every interaction you have with your best-scoring leads can be tailored just right.

Using this data in conjunction with your own lead data, you can even [personalize your outreach](/blog/predictive-personalization-machine-learning) with copy & creative that’s resonant with each individual, leaving you with campaigns that are both highly targeted to your best leads, and highly personalized for maximum impact. Use predictive personas to enable talk track alignment so that all of your reps hit the same key notes on every lead where it’s relevant. Similarly, you might find certain reps more effective than others with certain personas, leaving clear coaching opportunities to bring others into the fold. goals, it’s worth the effort to retain them.

### Sits anywhere in your stack

While Einstein lead scoring involves pulling your disparate data into Salesforce from various pieces of software, Faraday sits [anywhere in your stack](/integrations) that you need it to. Your data’s easily ingested from any software you use, used to make accurate, impactful customer predictions, and deployed back to [Salesforce](/predict/pipelines/lead_scores_in_salesforce) just as easily.

## How to implement lead scoring in Salesforce with Faraday

With Faraday, you can implement predictive lead scoring in Salesforce in a day, including scoring brand new leads in real-time as they come in–something Einstein lead scoring can’t do–regardless of where your data is. Here’s how it works.

### Upload your lead and customer data from Salesforce

To start, you plug in your data by mapping your data fields to Faraday properties. You’ll match customer & lead data like names, emails, and addresses to match their identities into Faraday.

[![Faraday dataset](/images/blog/einstein_lead_scoring/einstein-lead-scoring-dataset.png)](/images/blog/einstein_lead_scoring/einstein-lead-scoring-dataset.png)

Then, in cohorts, you’ll utilize both your data and Faraday’s built-in consumer data to group your data into interesting groups of people, which in this case are your [customers](/docs/cohorts/customers_from_salesforce) and [leads](/docs/cohorts/leads_from_salesforce).

[![Faraday leads cohort](/images/blog/einstein_lead_scoring/einstein-lead-scoring-cohort.png)](/images/blog/einstein_lead_scoring/einstein-lead-scoring-cohort.png)

### Predict personas to reveal rich insights

If you really want to make your brand’s engagement stand out to your leads, you can predict personas to reveal who your leads are as people and as consumers. Through trait breakdowns like age, gender, household income, hobbies, shopping styles, and more, you can turn your newest lead, Jane Doe, whom you know nothing about, into Jane Doe, married mother of 3, Amazon-centric shopper, and dog mom with a household income of \$200k.

Your talk tracks & creative couldn’t be easier & more resonant.

[![Faraday customer personas](/images/blog/einstein_lead_scoring/einstein-lead-scoring-personas.png)](/images/blog/einstein_lead_scoring/einstein-lead-scoring-personas.png)

### Build a predictive outcome to score leads in Salesforce

Next, you’ll build your predictive outcome by defining your attainment cohort as your **customers**, as that’s your prediction’s finish line–you want more people like your customers. Here, you’re defining the propensity to convert, or the likelihood that a lead is going to convert, so you’ll choose your **leads** as your eligibility cohort.

[![Faraday lead conversion outcome](/images/blog/einstein_lead_scoring/einstein-lead-scoring-outcome.png)](/images/blog/einstein_lead_scoring/einstein-lead-scoring-outcome.png)

With your lead scoring prediction defined, create a pipeline to make it actionable by choosing your **likelihood to convert** outcome and (optionally) your **persona set** as your payload, and your **leads** cohort as your population to include.

[![Faraday lead scoring pipeline](/images/blog/einstein_lead_scoring/einstein-lead-scoring-pipeline.png)](/images/blog/einstein_lead_scoring/einstein-lead-scoring-pipeline.png)

Next, Faraday’s machine learning algorithms then get to work predicting how likely people in your leads cohort are to convert and become customers, then they’re assigned scores on an individual basis that you can filter & activate on.

### Deploy predictive lead scores back into your Salesforce instance

With your predictions made, deploy your predictive lead scores to your Salesforce account so that each lead is enriched with their predictive lead score–and optionally, their predictive persona–so that, at-a-glance, your reps know exactly how and when to engage for the highest impact in order to drive revenue through conversions.

[![Faraday lead scores in Salesforce](/images/blog/einstein_lead_scoring/einstein-lead-scoring-salesforce.png)](/images/blog/einstein_lead_scoring/einstein-lead-scoring-salesforce.png)

**Ready to try it out? [Create a free account](https://faraday.ai/signup).**

Salesforce’s Einstein lead scoring addon is more attractive on paper than in practice. Learn how an alternative enables you to incorporate more than just lead data, while also scoring net new leads.

An Einstein lead scoring alternative for B2C companies on Salesforce

## What is predictive personalization?

Many brands are familiar with personalization, and the positive impact it can have on your marketing strategy. In recent years, advancements in machine learning and data activation have opened the door for more [authentic personalization at scale](/blog/unlocking-authentic-personalization-at-scale).

Predictive personalization is a term that involves using various machine learning predictions to create customized, personalized experiences for customers, leads, prospects, or any other interesting group of people. It enhances [personalization strategies across the entire customer journey](/blog/personalization-strategies-across-buyers-journey). Whether you’re focusing on–or looking to incorporate–product recommendations, lead scoring, churn prevention, repeat purchasers, or something else altogether, predictive personalization strengthens those strategies and provides clear indicators for how to engage so that every interaction your customers have with your brand is a delight.

## Benefits of using predictive personalization for personalization at scale

On its own, creating personalized content that’s relevant to each individual customer is a long, data-hungry process. Using machine learning for predictive personalization, however, simplifies the process and provides an array of benefits that enable you to achieve personalization at scale.

### Enhanced insights through vivid consumer data

Your first-party customer data includes key datapoints that describe your people your brand is interested in–your customers, leads, prospects, and more–through their interactions with your brand. This can help you identify the products & services that they’re interested in, so that you can create offers that are relevant to them.

Predictive personalization involves the unification of your first-party customer data with [rich, third-party, cookieless consumer data](/features/data)–after all, your customers exist outside of interactions with your brand. The end-result is a clear 360º view of your customers, leads, prospects, and any other group of people you’re interested in that gives you a firm understanding of who your customers are and what they’re like. [Insights](/blog/how-to-create-a-customer-insight-strategy) via demographics, psychographics, financials, hobbies, and more give you easy guidance for how to personalize your engagement in a highly resonant way.

### Proactive engagement

Regardless of the interaction in question, it’s always better, if given the option, to be proactive rather than reactive. Predictive personalization gives you the ability to personalize your content & creative before you’ve even interacted. You’ll find insights like how likely they are to churn and how much they’re likely to spend via their predicted LTV, so that you’re always one step ahead of the next action your customers are going to take.

[![Predictive personalization mock](/images/blog/predictive-personalization-mock.png)](/images/blog/predictive-personalization-mock.png)

### Personalization at scale

By using machine learning for predictive personalization, the effort involved in scaling personalization across your entire customer base–past, present, and future–isn’t as herculean as it sounds. Machine learning algorithms can segment your customers into bespoke, cohesive personas that you can deploy [anywhere in your stack](/integrations), allowing you to create individualized, personalized campaigns for any given subgroup of people you’re interested in.

### Internal talk track alignment

While the benefits of predictive personalization on external-facing engagement are clear–[see how Sealed did it](/blog/sealed-lead-optimization)–there’s an additional benefit that’s equally as impactful: internal alignment of understanding who your customers are, and what makes them tick. Personas enable you to craft clear talk tracks on a persona-level basis so that, regardless of salesperson, prospects and leads will receive content that highlights the same resonant features. Predictive personalization can also help identify which of your salespeople are best at speaking to specific personas so that you can set priorities and internal coaching appropriately.

## Implementing predictive personalization with Faraday

Customer prediction platforms like Faraday allow you to implement predictive personalization with ease–without a data science degree, and without a single line of code. Let’s dive into Faraday to see just how easy it is.

### Connect data and define customers

First, you’ll map your customer data to the matching Faraday properties. Then, in cohorts, you’ll build a cohort for your [current customers](/docs/cohorts/customers_from_hosted_csv) by defining the event that makes them a customer, such as a purchase.

[![Predictive personalization customers cohort](/images/blog/predictive-personalization-cohort.png)](/images/blog/predictive-personalization-cohort.png)

### Build personas to reveal rich insights to enable predictive personalization

Predict personas with the help of Faraday’s rich, cookieless, opt-in consumer data on top of your own first-party customer data. Discover trait breakdowns like hobbies, shopping styles, gender, household income, and more to understand the nuances of how different portions of your customer base stand apart from each other. These breakdowns give you crystal-clear guidance on how to speak to your customers in a way that makes them feel understood.

[![Predictive personalization customer personas](/images/blog/predictive-personalization-personas.png)](/images/blog/predictive-personalization-personas.png)

### Use customer predictions to create proactive, relevant campaigns

Take predictive personalization to the next level by predicting likelihood to churn, repeat purchase, and more so that your content is not only personalized to match who your customers are, but that it’s also attuned to how they’ve interacted with–or _will_ interact with–your brand, and at any scale you need, wherever in your stack you need.

**Ready to try predictive personalization out? [Create a free account](https://faraday.ai/signup).**

Explore the benefits of predictive personalization & the impact it can have on your marketing strategy through rich, effective customer insights.

Predictive personalization: Using machine learning to personalize customer experiences

If you've ever used AI to make predictions, chances are you've asked yourself "How do I know these are . . . correct?" It's a fair question, and healthy skepticism is a critical component of any Responsible AI practice.

Luckily there are plenty of great options for confirming that the predictions you're getting are mostly right. First, a bit of background.

## Prediction basics: How do AI models make predictions?

To set the stage, let's review how prediction works. Throughout this post, I'll use the example of a company wanting to predict which of its leads are most likely to "convert" (become a customer). In Faraday you'd represent this prediction objective with an outcome like so:

![Lead conversion outcome configuration](/images/blog/testing_predictions/outcome.png)

### Finding patterns

Now we'll take you behind the scenes. To predict something, a system must comb through historical examples of that thing happening to find patterns.

Once you know the patterns, you can compare newly observed behavior to the crucial patterns that signal "true" - if they are similar, you predict "true," otherwise "false."

So, to follow our example, the first step is to pull together some historical lead data, including some leads that converted and some that didn't.

![Training data diagram](/images/blog/testing_predictions/training_data.png)

Here, the top row represents your leads over time at their first capture point. Some of them—the **<span style={{color: "green"}}>green</span>** dots—"convert" by becoming customers: we call these _examples_. Others—the **<span style={{color: "red"}}>red</span>** dots—go "stale" and represent failures: we call these _counterexamples_. It's too early to say what the few remaining gray dots will do, so we'll just ignore them.

One useful thing to note is that 6/13 of our leads converted, or about half the time. This will come in handy down the road.

Your prediction system will then use one of a variety of methods to look for interesting patterns. Is there something in common among the green leads? What about the red ones? How are they different? This identification of patterns (the model) is what lets the system make predictions about new leads.

### Using patterns to make predictions

So now, let's take those pesky remaining gray dots, as well as some new ones that might have popped up since we figured out our patterns. I've put a dark border around them:

![Data diagram including new leads](/images/blog/testing_predictions/new_leads.png)

Your prediction system will scrutinize each of these possibilities to see which patterns in your list it follows. If it follows a lot of the patterns associated with conversion, the system will predict that it will convert. If it follows a lot of the patterns associated with non-conversion, the system will predict non-conversion.

![Diagram showing predictions](/images/blog/testing_predictions/prediction.png)

### Beware silly patterns

Easy, right? Too easy you might say. The reason to be skeptical here is that the patterns your prediction system found may be coincidences rather than reflect some underlying truth.

For example, your prediction system may say:

> Here's a pattern I found! Leads with one of these exact email addresses convert, all others don't: `amy@example.com, leon@example.com, ..., pat@example.com`

Sounds silly, but it happens all the time when the wrong kind of data is examined for patterns. The methods your prediction system uses aren't always clever enough to distinguish between goofy patterns and serious ones. Based on the data available, this pattern is actually incredibly powerful! In fact, it's 100% accurate.

Problem is, it will never work with any new leads, because they'll all have different email addresses and the system will therefore predict non-conversion for all of them.

## Three holdout testing methods to see if your predictions are accurate

The way to avoid this kind of silliness is to "test" your patterns to see if they're right. If they pass the test, you can trust them going forward.

Think about your tests in school: your teacher asked you questions and you provided answers. Your teacher knew the true answers. If your answers matched the teacher's true answers, you got an A.

The hardest part of all this is that somebody has to "know" the "true" answers. If only we had some lead data where we knew the right answer . . .

But of course we do! We know which leads actually converted and which didn't. We just can't use the _same_ data to both find patterns and put those patterns to the test.

There are a few great ways to split up your data for these two simultaneous purposes. They all rely on a "holdout"—setting aside a portion of the data to use exclusively for testing. This allows you to see whether the system "understands" enough to assess new data versus only being able to regurgitate answers for the questions it has already seen. Let's dive into these methods now.

### Method 1: random holdout

#### Setup

This is the most common approach, and it involves choosing a random subset of your historical data to set aside as a holdout. Only then do you use your prediction system to find patterns among the remaining data.

![Cross validation diagram](/images/blog/testing_predictions/random_holdout.png)

Here, we've used 50% of our data as a holdout. The system will find patterns exclusively among the remaining leads.

#### Testing

Now we're ready to "take the test." The leads in the holdout are the questions, and their true disposition (converted or not) is the answer key. But for the time being we pretend we don't have the answer key and we treat the holdout leads like new leads. What can the patterns we have tell us about these leads?

I'll illustrate those predictions with colored borders on the holdout leads:

![Prediction step of cross validation](/images/blog/testing_predictions/random_holdout-prediction.png)

Once we've used the patterns to make predictions about the holdout leads, we can dramatically reveal the answer key and see how accurate the patterns really are. Let's see how we did:

![Cross validation results](/images/blog/testing_predictions/cross_validation.png)

We got 4 right and 2 wrong for about 67% accuracy. Is that good or bad?

#### Evaluation

In real life you'll rarely get an A on this test. In some cases you'll predict conversion when the lead didn't actually convert (false positive). In other cases, you'll predict non-conversion when the lead did actually convert (false negative).

Recall that our actual lead conversion rate is about 50%. That means that a really dumb prediction system could always predict "convert" and get it right about 50% of the time, or flip a virtual coin and get the same result. By this measure, 67% is quite a bit better!

So assuming you got more right than wrong, it's an improvement. Before, your patterns were always right about your historical leads, but were useless for assessing new leads. Now, maybe your patterns aren't as perfect, but at least they're useful!

#### Conclusion

Finding the right balance between false positives and false negatives is part of the art of prediction and ultimately comes down to risk versus reward. It's out of scope for this post but I'll be writing one on that soon—subscribe below to get notified.

The upshot here is that the patterns the system found on the non-holdout leads appear to be effective in making predictions about the holdout leads. This is a very good sign that the patterns themselves are "real" and not based on silly coincidences.

The name for this approach is _cross-validation_. In practice, you actually perform this random holdout method multiple times to _really_ make sure your patterns are real. The Faraday platform automatically uses 3 passes like this every time we look for patterns.

### Method 2: recent holdout

#### Setup

Another approach is to use your most recent data as your holdout, a method called a _backtest_. As a refresher, here's what we started with:

![Training data diagram](/images/blog/testing_predictions/training_data.png)

To use this method, we temporarily "rewind" time to a previous day, like so:

![Rewound training data diagram](/images/blog/testing_predictions/recent_holdout.png)

From here, the process is much like above. Our prediction system attempts to find patterns looking backwards from our "pretend now," ignoring the lead data that has transpired since then, which becomes the holdout.

#### Testing

Then it uses those "old" patterns to examine recent leads. Again, predictions here are represented by the colored borders on the holdout leads:

![Backtest prediction step](/images/blog/testing_predictions/recent_holdout-prediction.png)

And finally we use our answer key to see how we did:

![Backtest results](/images/blog/testing_predictions/backtest.png)

#### Evaluation

Here again, we went 4 for 6, or 67%. By the same logic as above, not bad.

The big advantage of the recent holdout versus the random holdout in my mind is that it's easier to understand and, ultimately, trust.

The main disadvantage of this type of holdout is the system can only look for patterns within data spanning a shorter period of time, so if the important patterns take time to appear, then using a recent holdout won't perform as well as using a random holdout.

Faraday has this feature built in as an additional option if you want it, although it's very compute-intensive so it's not available on all accounts by default. If you don't see it and want to use it, just ask support.

### Method 3: secret holdout

You may hear other folks call this a blind holdout, although I hope to see the end of that ableist terminology soon.

The idea here is that you go to extreme lengths to hide your holdout from the prediction system—or even the company running the prediction system!—in order to boost your confidence even further.

Here, the principal doesn't even trust the teacher with the answer key! The diagram looks something like this:

![Secret holdout](/images/blog/testing_predictions/secret_holdout.png)

The user has pre-emptively removed a random portion of the data to serve as the holdout and shared the remainder with the prediction system, which has no idea what's in the holdout.

The system finds patterns using all the data it has. Then the user performs the test by presenting the holdout leads to the system _without their true dispositions_. The system makes its predictions as usual:

![Secret test prediction step](/images/blog/testing_predictions/secret_holdout-prediction.png)

The user then secretly compares these predictions to the "true" answer key to determine accuracy.

In practice the secret holdout is exactly the same as the random holdout, just a heck of a lot more complicated and time-consuming. Nevertheless, you could consider it if you don't trust your prediction system for one reason or another.

Faraday supports secret holdout testing. When connecting your data, make sure to remove your holdout first, so Faraday never sees it. Configure your account as usual. Then upload your holdout—minus disposition—as a CSV dataset with a `holdout` trait, create a matching **Holdout** cohort, and use it as a the population in a new pipeline. Add the outcome you want to test to the payload, and deploy an identified CSV target. Now match Faraday's predictions back to your original holdout and analyze to evaluate accuracy.

## The real deal

Prediction systems finding silly patterns is a real problem—you could even say it's _the_ problem of machine learning. Luckily there are lots of ways to avoid these kinds of shenanigans, and good prediction systems employ them automatically.

Behind the scenes, Faraday runs random and/or recent holdout testing constantly to confirm it's finding legitimate patterns that are practically useful going forward.

I know all this (I helped build the system!), but I still treasure my ongoing skepticism, and you should too. Hopefully understanding how pattern validation works, and seeing how it operates behind the scenes, will help you develop a careful confidence in the predictions you build your workflows around.

Three holdout testing methods you can use to evaluate the accuracy of a model’s predictions.

How to set up holdout tests to evaluate prediction accuracy

## What is customer churn?

Churn refers to the point at which a customer decides to cease using a product or service that your company offers. Churn can occur for a variety of reasons, but the bottom line is that a churned customer brings no revenue to your business. When it comes to churn, no business can ever completely eliminate it, but there are strategies to consider that can help your business with churn prevention and mitigation.

## Why is churn prevention so important?

Studies have consistently shown that it costs more to acquire new customers than it does it to retain existing ones. On top of that, it’s often easier and more profitable to retain existing customers; you have a [60-70% chance of selling](https://blog.hubspot.com/service/statistics-on-customer-retention) to a current customer vs a 5-20% chance of making a new sale, and current customers spend [67% more](https://www.business.com/articles/returning-customers-spend-67-more-than-new-customers-keep-your-customers-coming-back-with-a-recurring-revenue-sales-model/) on average than new customers. Despite this, [44% of businesses](https://www.semrush.com/blog/customer-retention-stats/) focus on customer acquisition, while only 18% focus on customer retention. While you shouldn’t drop acquisition to solely focus on retention, a smart, effective strategy should include a healthy mix of acquisition and retention.

## What causes customer churn?

When a customer churns, it’s key to analyze your data to understand _why_ they churned so that you can take steps to ensure customers like them don’t churn in the future, or–if they’re not ideal customers, and therefore likely to churn anyway, avoid them entirely. Maybe one of your customers received poor customer service when they chatted in for support, or they received an email that looked like generic spam and included offers that weren’t relevant or resonant. Or, perhaps you’re acquiring too many discount shoppers - people who are only biting because you offered them steep discounts.

## Churn prevention strategies

Churn prevention strategies begin with having a strong foundation of customer data, and are solidified by being smart about how you use that data.

### Mitigate churn higher in the funnel

The other key benefit of defining and identifying your best-fit, high-LTV customers is that it can help you find more of them. Discount purchasers that only buy when you send them a 75% off coupon aren’t ideal customers, and are likely to churn once their promo window is up. With a solid understanding of who your ideal customer is, you can find more of them to target so that you’re less likely to need to worry about them churning in the future.

### Use data to identify causes of churn

Segmenting your customers into unique subgroups can have an impact on your overall strategy that’s twofold: first, you’ll find different attributes and traits that enable personalization across each individual subgroup, leading to frictionless engagement that includes personalized offers, copy, & creative, ensuring your customers feel understood.

Second, by analyzing the data that you have on your customers, you can discover patterns and similarities in those that churn, which can help you identify why they opted to make that critical decision. The same applies to the other side of the coin–you can segment out your ideal customers so that you can define what makes them ideal; maybe they’re big spenders, or simply consistent ones.

### Create personal and frictionless experiences

All it takes is one bad experience to turn a loyal customer into a churn. Ensuring that each experience your customers have with your brand is personal and frictionless goes a long way to keeping your customers on board. Your engagement, regardless of channel, should include content that’s relevant to who they are and what kind of shopper they are. This results in experiences that seamlessly deliver the exact information, products, and services that your customers are looking for, so that they never have the excuse they need to find an offer elsewhere.

### Invest in customer loyalty programs

[58% of customers](https://www.semrush.com/blog/customer-retention-stats/#loyalty-program-statistics) that are members of a brand’s loyalty program make a purchase at least once per month. Loyalty programs are an incredible way to keep your customers engaged with your brand on a regular basis. They’re also a two-way street: not only do they reward your customers for their loyalty through discounts, bundles, and other similar offers, but they reward your brand with customers that stay subscribed and keep coming back for more.

### Provide excellent customer service

Nobody likes having to contact customer service, but when they absolutely _have_ to, ensuring their experience is a delight is critical in order to keep their business. With excellent customer service, a customer can go from frustrated at something not working as expected–churn in the back of their mind–to relieved, satisfied, and still on board.

## How machine learning enhances churn prevention efforts

Manual churn prevention strategies are doable, but they’re time-consuming, resource-intensive, and data-hungry. Fortunately, churn management software utilizing machine learning can take the workload off of your teams’ shoulders, while providing you with actionable insights for a variety of use cases.

### Segmentation & insights

Traditional segmentation provides a limited level of insight into who your customers are due to the fact that your data is likely limited to just their interactions with your brand. In adopting machine learning through churn management software, your data is unified with [rich, third-party consumer data](https://faraday.ai/features/data) to reveal distinct subgroups within your customers, leads, and any other interesting group of people–these are your personas. These insights include demographics, psychographics, property, financials, hobbies, & more. Your customer–Jane Doe–who recently purchased a dog toy, becomes Jane Doe, dog owner, married, household income of over \$200k, has children in the household, and prefers to shop in-store rather than online. This level of insights serves you personalization guidance on a silver platter.

Machine learning also significantly simplifies the process of analyzing your segmented data. Instead of manually poring over every datapoint in an attempt to reveal similarities between churners–as well as those between ideal customers–machine learning accomplishes this for you in a fraction of the time, leaving you with a predictive model that’s easily interpretable and [actionable through predictive model reports](https://faraday.ai/blog/faraday-model-reports).

[![Faraday outcome report](/images/blog/churn-prevention-features.png)](/images/blog/churn-prevention-features.png)

_The above image displays a Faraday outcome report, detailing the features most important to the predictive model._

### Personalization

With rich, persona-level insights on every individual that your brand is interested in, all of your outreach and engagement can be personalized to a tee. This means that each engagement–from email to direct mail, and everything in between–provides resonant copy & creative that leaves your customers feeling understood and heard. Customer Jane Doe, a member of the Millennial May persona, can receive a direct mail postcard that includes creative that incorporates some of the traits mentioned above–dogs and children, for example–and copy that resonates with Millennials, capped off with directions to your nearest brick-and-mortar store to match their in-person shopping style.

### Churn prediction using machine learning

What if you could know whether or not a customer was going to churn, before they even knew they were considering the choice? Churn prediction using machine learning provides you with churn scores on an individual level, based on how likely the individual is to churn. This kind of insight gives you the ability to proactively reach out to customers that are likely to churn with deeply personalized outreach so that you can keep them on board. On top of that, churn predictions help you prioritize who to reach out to so that you’re not wasting time on bad fits.

### LTV prediction

To add another layer of depth and effectiveness to your churn prevention strategy, LTV prediction enables you to define cutoffs for how deep a discount someone needs to receive for retention to be profitable. If a customer has a churn score of 75–pretty likely to churn–but they have a predicted LTV that’s in line with your goals, it’s worth the effort to retain them.

## How to predict and prevent churn using Faraday

Churn management software can help you create an effective, [impactful strategy](/blog/churn-predictions-workshop) for finding & retaining your best customers. Let’s take a look at how easy it is to predict churn and deploy churn predictions to your stack with Faraday.

### Connect data and define customers

To kick things off, plug your customer data in by mapping your data fields to matching Faraday properties. Then, in cohorts, you’ll utilize both your customer data and Faraday’s built-in consumer data to build a cohort for your [current customers](/docs/how-to/cohorts/customers), and your [churned customers](/docs/how-to/cohorts/churned-customers).

[![Faraday churned customers cohort](/images/blog/churn-prevention-cohort.png)](/images/blog/churn-prevention-cohort.png)

### Build personas to reveal rich insights and improve your personalization efforts

Predict personas with the help of Faraday’s rich, cookieless, opt-in consumer data. Utilize trait breakdowns like age, hobbies, shopping styles, and more to understand the nuances of how different portions of your customer base stand apart from each other–every little piece of personalization in your engagement matters.

[![Faraday customer personas](/images/blog/churn-prevention-personas.png)](/images/blog/churn-prevention-personas.png)

### Build predictive churn outcomes to detect churn risks

Next, you’ll build your predictive outcome by defining your attainment cohort as your churned customers, and your eligibility cohort as your current customers. With these selections, you’re predicting how likely it is that any member of your customers cohort is going to enter your churned customers cohort by churning.

[![Faraday churn prediction outcome](/images/blog/churn-prevention-outcome.png)](/images/blog/churn-prevention-outcome.png)

Now that your predictive outcome is created, make your predictions actionable by creating a pipeline with your **likelihood to churn** outcome in the payload–alongside **personas** for easy personalization–and your **customers** as the population to include.

[![Faraday churn prevention pipeline](/images/blog/churn-prevention-pipeline.png)](/images/blog/churn-prevention-pipeline.png)

Faraday’s machine learning algorithms then predict how likely each of your current customers is to churn, while also revealing which people belong to what persona on an individual basis so that you know how to engage with them.

### Deploy churn predictions to your stack

With your pipeline created, deploy your churn scores–AI-generated personas attached–anywhere in your stack that you need them. Activate on your predictions by targeting your customers that are most likely to churn, and use their personas as guidance for how to engage with them in a way that’s highly resonant.

### Scale churn prevention by finding more of your best customers

The best churn prevention strategy is to acquire best-fit customers and ensure that they have a frictionless experience with your brand at every stage in the journey. Find more of your ideal customers by following the above guide for [building an outcome](/blog/churn-prevention-machine-learning#build-predictive-churn-outcomes-to-detect-churn-risks), but use your **customers** as the attainment cohort. Implementing both a retention strategy through churn scores and an acquisition strategy that focuses on only your best-fit customers can give you a serious leg up on your competitors.

**Ready to try it out? [Create a free account](https://faraday.ai/signup).**

Learn the benefits of focusing on churn prevention, churn prevention strategies, and the importance of predicting churn.

Churn prevention: How to reduce customer churn using machine learning

Are you searching for effective ways to engage with your customers? Look no further than [direct mail](/blog/personalized-direct-mail-strategy)! In today's digital age, where consumers are bombarded with marketing emails, direct mail campaigns are emerging as a refreshing and impactful strategy.

Join us for an interactive workshop where we'll unveil the remarkable benefits of using direct mail in today's competitive landscape. Watch and learn as the team explores the resurgence of direct mail and its unique ability to shine in the digital era!

**From this workshop, you'll learn**:

- Why are so many modern brands using direct mail?
- How to optimize direct mail targeting and personalization with Faraday
- Top use cases for direct mail in B2C marketing

Join Faraday's Directors of Accounts Alex Noyes and Nick Haggerty, as well as Director of Marketing Perry McDermott, where they discuss modern direct mail targeting and personalization tactics you can use to get better results from your campaigns.

Direct mail marketing tactics you need to know about

## What is direct mail marketing?

Direct mail marketing is the act of a brand engaging with its customers and prospects through physical mail. It can come in various forms, from postcards, to letters, to branded water bottles, and everything in between. The most effective direct mail campaigns are highly personalized to resonate with a brand’s customers. Before email and digital engagement became the norm, direct mail was one of the best ways to reach your customers. While digital has since pushed direct mail marketing to the side, it’s had a resurgence in recent years because, in the age of digital media, personalized direct mail _stands out_–assuming you’ve done your due diligence and personalized it appropriately to match your customers’ wants and needs. Take [PostcardMania](https://www.postcardmania.com/blog/direct-mail-statistics-2023) for example, who has seen their revenue grow 17.56% each year post-pandemic in comparison to 4.67% pre-pandemic.

## Benefits of personalized direct mail

Personalized direct mail has a variety of benefits over digital to prove that it’s a great supplement to your engagement strategy. To start, the response rate of direct mail marketing is 9%, in [comparison](https://www.postgrid.com/direct-mail-statistics/) to the 1% of email, paid search, and social media. Additionally, it has a 29% ROI compared to 23% for paid search and 16% for online display. The benefits of personalized direct mail stem from two key features:

The first is that personalized direct mail can be tailored to a greater extent than its alternatives. Say that you know a group of your customers are avid runners that regularly buy running gear from your store. A digitally-focused brand might launch a campaign between major purchases for that group of people, where they want to offer them a free water bottle for being a loyal customer. An email offering the freebie will look a lot like other emails they receive, so it’ll likely be discarded, even if it’s genuine. With a brand savvy in personalized direct mail, the water bottle could be at their doorstep–delivered in a box personalized with running-focused creative–then in their hands as it becomes their daily drinker that they freely advertise for you.

The second is that, in the digital age, people are used to receiving all manner of marketing emails, to the point that it’s almost second nature to mark something as spam, or delete it. Personalized direct mail is more distinctive, and is much more likely to be seen by your customers. After all, 77% of people sort their physical mail immediately [according](https://www.pb.com/docs/US/pdf/Microsite/Nonprofit/ed_np_getyourmailopened_05MailMoment.pdf) to the USPS. With a chance that high for your mail to be seen, making use of personalized direct mail that resonates with your customers is the key to standing out.

## How can direct mail be personalized?

Personalized direct mail can be accomplished through several different means that utilize your first-party data, such as merge tags and creative variations. With machine learning and third-party consumer data on top of your first-party data, you can accomplish it more effectively, and at scale.

### Leverage data and machine learning for insights to guide personalization

To start, your customer data needs basic identifiers, like name and address, so that you know where to send your personalized direct mail. On top of that, you need to leverage all of your first-party engagement data, like products purchased, abandoned carts, subscriptions, and so on, to ensure any offers you make are relevant. You wouldn’t want to find a postcard with a discount in your mailbox for cat food when you don’t own a cat, would you?

Regardless of how strong your first-party data is, it provides limited insight into who your customers are beyond their interactions with your brand. With [rich, third-party consumer data](https://faraday.ai/features/data) that includes demographics, psychographics, lifestyle, financials, and more, you can take your segmentation to the next level, enabling you to customize your copy & creative so that it resonates.

In addition to leveraging both first-and-third-party data, using machine learning to identify distinct groups–personas–allows you to create personalized direct mail at scale, with each group can be targeted with its own resonant, relevant engagement.

### Use insights to inform your personalization

A uniquely powerful way to personalize direct mail is to make use of merge tags. With merge tags, you can personalize on an individualized basis so that any data you have that’s relevant to your customers can be utilized. This way, beneath the persona-level segmentation, your personalized direct mail is dynamic enough to include datapoints like, say, someone’s name, subscription level, or the city they live in. On top of that, if you have physical locations, merge tags can be used to point customers to your nearest storefront.

Personas also enable distinct creative variations that are relevant to each persona. For your Exercise Guru Elayne persona, you could include imagery of women running alongside a dog and while pushing a stroller, because the majority of the persona are dog owners and have children in the household.

Take your personalized direct mail even further by adding customized product offers and discount codes that are relevant to each persona. Maybe, utilizing your customer data, you know that a specific group of your customers have already purchased a pair of running shoes. You know they won’t need another pair for a while, so a promo for that will bounce off. An offer for a pair of running socks or shorts, on the other hand, is more likely to catch their attention.

![Postcard with personalized discount](/images/blog/direct-mail-discount.png)

_Image via [TheMailShark](https://www.themailshark.com/product-samples/postcards/)._

## Pillars of a personalized direct mail strategy

There are three pillars to a successful personalized direct mail strategy.

- **Insights**: Identify distinct customer segments and their related insights by generating [AI-powered personas](https://faraday.ai/predict/category/persona_assignments). Through these insights, you have clear indicators for how to engage your customers and leads in ways that are highly resonant with their lifestyles.
- **Customization**: Determine how you’ll customize direct mailers based on your insights, data, and promotions. The most impactful personalized direct mail campaigns will include a mix of products, imagery, discounts, and merge tags, ensuring you’re including the right content. Brands that offer personalized direct mail services, provide a range of these types of services.
- **Targeting**: The cost of direct mail can add up–especially if you’re not targeting the best fits. Predict for likelihood to [buy](https://faraday.ai/predict/category/likely_buyers), [convert](https://faraday.ai/predict/category/lead_scores), [repeat purchase](https://faraday.ai/predict/category/repeat_purchasers), [churn](https://faraday.ai/predict/category/churn_scores), or any other relevant datapoint so that you’re spending your time and money where they matter most.

## How to implement a personalized direct mail strategy with Faraday

With a customer prediction platform like Faraday, you can identify personas to gain rich insights and predict behaviors so that you can deliver resonant, personalized direct mail to likely-to-act audiences. All it takes is some historical data about your customers. Here’s how it works.

### Connect data and define customers

To start, you plug in your data by mapping your data fields to Faraday properties.

[![Direct mail dataset](/images/blog/direct-mail-dataset.png)](/images/blog/direct-mail-dataset.png)

Then, in cohorts, you’ll utilize both your customer data and Faraday’s rich, built-in consumer data to build the exact [cohort](https://faraday.ai/docs/cohorts/customers_from_hosted_csv)–or interesting group of people–you want. In Faraday, you’ll create a cohort for your customers so that you can find more people like them, and you’ll also build a cohort of people with mailable addresses, so that you’re also targeting unknown audiences. You'll do this by choosing the following traits when building a cohort:

- **Record quality code**: one of: mailable

[![Faraday direct mail cohort traits](/images/blog/direct_mail_guide/direct-mail-guide-cohort-traits.png)](/images/blog/direct_mail_guide/direct-mail-guide-cohort-traits.png)

### Predict personas to reveal rich insights

While you likely have customer data that indicates, in some ways, who your customers might be, predicting personas tells you who they really are, and what makes them tick. Dive into trait breakdowns like age, gender, household income, hobbies, and more–each one guiding you toward engaging with the right messaging.

[![Direct mail persona set](/images/blog/direct-mail-personas.png)](/images/blog/direct-mail-personas.png)

### Predict propensity to buy to target best fits

Next, you’ll build your predictive outcome by defining your attainment cohort as your customers, as that’s your prediction’s finish line–you want more people like your customers. In this example, we’re defining the propensity to buy, or the likelihood that anyone in the U.S. is going to become a customer by making a purchase, but you can predict for virtually any action that someone is likely to take, such as a conversion, repeat purchase, or churn.

[![Direct mail outcome](/images/blog/direct-mail-outcome.png)](/images/blog/direct-mail-outcome.png)

With your prediction defined, create a pipeline to make them actionable by choosing your **likely buyers outcome**, your **persona set**, and your **mailable cohort**.

[![Direct mail pipeline](/images/blog/direct-mail-pipeline.png)](/images/blog/direct-mail-pipeline.png)

Faraday’s machine learning algorithms then predict how likely people are throughout the U.S. to buy your product, while also revealing which people belong to what persona on an individual basis.

### Deploy personalized direct mail predictions to your stack

Likely buyer scores revealing your best fits across the country? Check. AI-generated personas providing clear guidance for personalization? Check. With these insights in-hand, deploy them anywhere in your stack you need them–for example, to a direct mail provider–then, design your copy & creative so that it caters to the personas you’re engaging with.

**Ready to try it out? [Create a free account](https://faraday.ai/signup).**

Learn the benefits of personalized direct mail marketing, how to personalize your direct mail campaigns, and how to use predictive analytics to maximize impact.

Personalized direct mail: How can direct mail be personalized at scale?

In the world of machine learning, we often encounter two important concepts: the raw propensity score and probability. Though they may sound similar, they represent distinct aspects of a machine learning model's predictions.

## Example of propensity scores vs predicted probabilities

Imagine you have a model that predicts whether it will rain tomorrow. The raw propensity score is like the model's initial hunch or gut feeling about the outcome. It's a numeric value that the model calculates based on various factors like historical weather data, humidity, temperature, and more. The [propensity](https://faraday.ai/blog/propensity-models-predict-customer-behavior) score indicates how likely the model thinks it will rain or not. However, the propensity score itself doesn't give us a clear understanding of how confident the model is in its prediction.

This is where [probability](/blog/faraday-probability-scores) comes into play. The predicted probability aims at providing an answer to the question “how likely is the prediction?” It represents the likelihood of a particular outcome occurring. For example, a score of 0.75 for rain means the model is fairly confident it will rain tomorrow. However, a probability of 0.75 indicates that there is a 75% chance that it will rain tomorrow.

In summary, the raw propensity score is the model's initial guess, while probability quantifies the model's confidence in that guess. Understanding this difference can help us interpret machine learning [predictions](/features/predictions) and make better-informed decisions based on the model's level of certainty.

## Understanding model calibration

The model calibration process refers to the alignment between a model's predicted probabilities and the actual likelihood of events occurring. A good calibration ensures that the predicted probabilities are close to the true theoretical probabilities. Not all machine learning models are perfectly calibrated after training. Some models might be overconfident, providing probabilities that are too extreme, while others might be underconfident, consistently underestimating or overestimating probabilities.

### Importance of model calibration

Calibration is crucial because it impacts decision-making. If a model is well-calibrated, we can trust its probability estimates and use them to make informed choices, such as whether to take an umbrella based on the probability of rain.

Calibrating a model involves post-processing its probability scores to bring them in line with the true probabilities. Various calibration techniques can be employed, such as Platt scaling or isotonic regression, which adjust the raw propensity scores to produce well-calibrated probabilities. These techniques are typically applied during model validation and fine-tuning to ensure that the model's predictions are more accurate and reliable.

The calibration process ensures that when the model predicts a probability, it reflects the true likelihood of the event occurring. For example, if a model predicts a 70% probability of rain, it should be accurate in approximately 70% of the cases where it makes that prediction.

In summary, model calibration in machine learning is all about making sure a model's predicted probabilities accurately reflect the real-world likelihood of events happening. A well-calibrated model is more reliable and can lead to more effective decision-making.

## Benefits of using predicted probabilities

Using predicted probabilities instead of raw propensity scores offers several significant benefits that enhance the interpretability, decision-making, and overall performance of the models:

### Interpretability

Predicted probabilities provide a more intuitive and understandable representation of a model's output. Raw propensity scores are often abstract values that lack clear context, while probabilities are easily interpretable as the likelihood of an event occurring. This makes it easier for stakeholders and end-users to comprehend the model's predictions and build trust in its results.

### Uncertainty quantification

Predicted probabilities convey the model's level of certainty about its predictions. Models can provide probability scores ranging from 0 to 1, indicating their confidence in the predicted outcomes. This uncertainty quantification is valuable in decision-making, as it allows users to consider the risk associated with a prediction and adjust their actions accordingly.

### Threshold selection

Predicted probabilities offer flexibility in choosing decision thresholds. Depending on the specific application and requirements, decision-makers can adjust the probability threshold for classifying positive and negative outcomes. This adaptability allows for a fine-tuning of the model's behavior to suit the desired trade-offs between false positives and false negatives. For example, in the case of a lead conversion, a profitability threshold can be established by taking into account cost of acquisition for each lead, expected profit from actual customers, and probability of conversion.

### Calibration and reliability

When models are well-calibrated, predicted probabilities accurately reflect the real-world likelihood of events. Calibrated probabilities provide more reliable estimates, leading to improved decision-making and better performance in critical applications.

### Ensemble and stacking

Predicted probabilities facilitate model ensembling and stacking techniques. By combining the probabilities from multiple models, ensemble methods can often yield better overall performance compared to individual models or raw propensity score-based approaches.

## Benefits and drawbacks of raw propensity scores

Using raw predictions from models instead of calibrated scores has its own set of pros and cons. Let's explore them.

### Pros of using raw propensity scores:

**Simplicity and Speed**: Raw predictions are the direct output of the model without any additional post-processing, making them easier and faster to obtain. In real-time applications or scenarios where efficiency is critical, using raw propensity scores can be advantageous.

**Suitable for Ranking**: Raw propensity scores can still be useful for ranking predictions. Even though they may not directly represent probabilities, they can help rank instances based on their likelihood of being a positive or negative outcome, which is often sufficient in certain applications.

### Cons of using raw predictions:

**Inconsistent Ranking**: Raw propensity scores might not consistently reflect the true likelihood of events occurring. Two instances with similar raw propensity scores might have significantly different actual probabilities, leading to inaccuracies in ranking and decision-making.

**Suboptimal Decision Thresholds**: Using raw predictions often means relying on fixed decision thresholds (e.g., 0.5 for binary classification), which might not be suitable for all applications. This can lead to suboptimal trade-offs between precision and recall.

**Misinterpretation of Scores**: When raw propensity scores are not calibrated, users might incorrectly interpret them as reliable probabilities, leading to misguided decisions and a lack of trust in the model's predictions.

In summary, while using raw predictions can be straightforward and computationally efficient, it comes with the trade-off of lacking calibrated probabilities and potential inconsistencies in ranking. In many real-world applications, calibrated scores are preferred as they provide interpretable probabilities, better decision thresholds, and improved trust in the model's predictions.

## When should you use propensity score percentiles vs. predicted probabilities?

In the field of machine learning, the choice between using percentiles and probabilities depends on the specific context and the nature of the problem being addressed.

### Using propensity score percentiles

**Outliers are important**: If you are concerned about extreme values or outliers in your data, percentiles can be more robust in representing the distribution. For instance, the 95th percentile captures the value below which 95% of the data lies, making it less sensitive to extreme values.

**Data skewness**: When dealing with heavily skewed datasets, especially those with long tails, percentiles can be more informative than probabilities. Percentiles provide a clearer understanding of the data's spread and central tendencies, even in the presence of skewed distributions.

**Interpretability**: Percentiles can be easier to interpret and explain to non-technical stakeholders. For example, stating that a value is at the 75th percentile means that it is higher than 75% of the other values in the dataset.

### Using predicted probabilities

**Well-calibrated predictions**: Probabilities are essential when you need calibrated predictions. If you want reliable estimates of the likelihood of certain events, probabilities are more suitable. Well-calibrated models can provide accurate and informative probabilities for decision-making.

**Decision thresholds**: In many applications, using probabilities allows you to set appropriate decision thresholds based on your specific needs. For instance, in medical testing, you might want to set a higher threshold to prioritize precision over recall, depending on the consequences of false positives and false negatives.

**Uncertainty quantification**: Probabilities are crucial for quantifying uncertainty around predictions. Confidence intervals and prediction intervals can be constructed using probabilities, giving a range of likely outcomes.

**In summary, use percentiles when you need robust representation of data distribution and are concerned about outliers or skewed data. On the other hand, use probabilities when you require calibrated predictions, decision thresholds, and uncertainty quantification, especially in the context of machine learning models.**

**Ready to try Faraday? [Create a free account](https://faraday.ai/signup).**

Propensity scores and predicted probabilities represent distinct aspects of a machine learning model's predictions. Understand the difference and when you should use them.

Propensity vs probability: Understanding the difference between raw scores and probabilities

Churn looks different for every business, but the ultimate goal remains the same: retaining customers. To succeed in this competitive landscape, it is crucial to have [real-time insights](/predict/category/churn_scores) that can identify customers at risk of churning and engage with them based on their specific needs and preferences.

But where should you start?

Join us in this interactive workshop where we will demonstrate how Faraday's AI capabilities can be leveraged to accurately predict customer churn. By staying ahead in your [churn prevention](/blog/churn-prevention-machine-learning) strategy, you can take proactive measures to retain valuable customers.

**From this workshop, you'll learn**:

- Why modern brands are investing heavily in customer retention
- Ways to reduce churn throughout your customer lifecycle
- Important considerations to effectively predict and prevent churn
- How to get churn predictions into your stack using Faraday
- Activation and engagement ideas to consider

Join Faraday's Directors of Accounts Alex Noyes and Nick Haggerty, as well as Director of Marketing Perry McDermott, where they dive into why brands are investing heavily in retention, and how to do it effectively.

All posts