**See our complete guide:** [Powered by Faraday — embedding predictive AI in your platform](/blog/powered-by-faraday-complete-guide-embedded-predictive-ai).

---

[![DataGrid header image](/images/stories/screenshots/salesrabbit-story-header-image.png)](/images/stories/screenshots/salesrabbit-story-header-image.png)

## Making a powerful tool even smarter

SalesRabbit is the leader in software for field sales teams. Their [DataGrid](https://salesrabbit.com/datagrid/) solution supplies canvassers with household data on specific properties and homeowners in their territories, providing essential context as they knock on doors. However, the SalesRabbit product team recognized that static data points on each property provided limited context for sales reps.

They realized that, if they could identify which attributes of a homeowner or property could predict the likelihood that a prospect would become a customer, then they could use that information to inform how their sales reps approached their day-to-day responsibilities. For example, solar clients would not only know which houses had the right amount of sun exposure for solar panels, but also which of them most closely fit the model of historical solar adopters. Pest control clients would not only know what material houses were made out of, they could also know if the homeowner would be more or less likely to say yes to regularly scheduled inspections or not, and so on.

Furthermore, if they had a way to predict which neighborhoods had the highest concentration of likely customers, then DataGrid could direct sales reps' efforts more efficiently.

## The challenges of building predictive AI models from scratch

SalesRabbit identified three main obstacles to them building effective predictive AI models in-house:

- A lack of data about consumers.
- The long ramp time to build effective predictive models.
- Resources to test models for accuracy and continue to provide transparency about the models efficacy.

### Licensing more customer data

Predictive models need a deep well of data to find an accurate signal. Despite the wealth of data SalesRabbit had on homeowners, adding an ML layer to DataGrid would require thousands more data points to ensure accurate models. Psychographic data, shopping behaviors, demographic data, financial information, and more would all need to be vetted, licensed, and reconciled with their existing data before predictions could be made.

### Extensive time and resources dedicated to prediction

To build a customer prediction pipeline in-house, the SalesRabbit team would also need to grow their existing data science team. They couldn't just reorganize other team members for a temporary sprint. Not only is the typical timeline for an ML program multiple years, they’d need to invest to continuously maintain those models over time.

### Limited transparency and scalability of cost

For the C-level team at SalesRabbit, it was critical for the solution to their challenge to be scalable. The original timeline for building this feature entirely in-house was multiple years and required a major investment from their data science team to get started.

Additionally, SalesRabbit's customer base includes both enterprise home service companies and small, regional installers. Creating a customer prediction pipeline comes with a tremendous computing cost, and running hundreds of millions of scenarios to build a model that works quickly can grow that cost exponentially–and unpredictably. How do you forecast the budget for the computing cost when the number of predictions made monthly for large and small organizations varies dramatically from month to month?

## The partnership

[![DataGrid mock](/images/stories/screenshots/datagrid-mock.png)](/images/stories/screenshots/datagrid-mock.png)

Realizing that an in-house build carried these risks, the SalesRabbit team shopped for existing best-in-class infrastructure that they could build on.

Faraday was the perfect fit for the team to implement customer predictions quickly, and at scale.

### Built-in consumer data for predicting customer behavior

Faraday has over [1,400+ attributes](/features/faraday-identity-graph) built into its platform, which mean all of this data was available on day one. Even better, it could be seamlessly combined with their existing data sets–no identity resolution required.

Employing this font of data on nearly every U.S. adult, SalesRabbit had everything they needed to understand the patterns and trends of their customers' ideal homeowners, regardless of which vertical their customers work in.

### Preserving existing data science resources and cutting product build time with Faraday recipes

Faraday’s plug-and-play [customer behavior predictions](/features/customer-behavior-predictions) meant there was no need to pull their data scientists off of other projects to build a customer prediction pipeline from scratch. SalesRabbit used pre-built prediction recipes as an accelerator to test and refine models before incorporating them into their product. This allowed them to shrink a multi-year roadmap to a matter of days, and free up their data science team to focus on other efforts.

### Peace of mind with transparency and scalability of cost

To the product team at SalesRabbit, building a customer prediction pipeline from scratch came with no small level of forecasting uncertainty. How much should they invest to launch? How much will it cost to scale up as their customer base grows?

With Faraday, they found pricing certainty, which allowed SalesRabbit to know exactly what to expect as they scale up.

## The results

[![DataGrid UI screenshot](/images/stories/screenshots/datagrid.jpg)](/images/stories/screenshots/datagrid.jpg)

To SalesRabbit, choosing to embed customer predictions into their software rather than build it in-house was a game-changer. Once SalesRabbit connected their data and created their initial predictive models, Faraday’s developer API made it easy to start embedding Faraday’s predictive features into DataGrid, and in a fraction of their original timeline.

Launched as “DataGrid AI,” SalesRabbit’s predictive lead scoring tool guides canvassing teams to best-fit territories, helps with route planning, and leaves reps with clear guidance on which households to prioritize when in the field. And because all of Faraday’s predictions are cached in advance, overall app performance improved once they were integrated.

Launched as “DataGrid AI,” SalesRabbit’s predictive lead scoring tool guides canvassing teams to best-fit territories, helps with route planning, and leaves reps with clear guidance on which households to prioritize when in the field.

How SalesRabbit uses Faraday to enhance their field sales app with data & AI

**See our complete guide:** [Personalization tactics that actually drive conversions](/blog/personalization-tactics-that-drive-conversions).

---

Customer experience is THE differentiator for a modern, successful business. Being able to provide a compelling, valuable CX can lead to a variety of benefits, including improved customer satisfaction, better retention, and improved business performance.

Today's businesses are well aware of the critical nature of the customer experience in their future. **[80% of companies](https://www.zippia.com/advice/customer-experience-statistics)** that focus on CX report seeing an increase in revenue, while brands that are more customer-centric and focused on the customer experience report **[60% higher profits](https://cxindex.com/company/blog/20-statistics-to-understand-the-rise-and-value-of-cx/)**. Companies that update their CX to be more data-driven find that by improving the customer experience they can **[reduce customer support costs by up to 33%](https://cxindex.com/company/blog/20-statistics-to-understand-the-rise-and-value-of-cx/)**. And companies are putting their money where it counts when it comes to the customer experience. Global CX spending is expected to reach **[$641 billion in 2024](https://cxindex.com/company/blog/20-statistics-to-understand-the-rise-and-value-of-cx/)**.

It's not just businesses that want a quality, value-driven CX. Consumers are willing to pay good money when interacting with a high-level customer experience, so long as they get something in return for it. For instance, consumers are willing to pay a **[16% price premium](https://cxindex.com/company/blog/20-statistics-to-understand-the-rise-and-value-of-cx/)** in return for a superior customer experience. Data shows that customer spending increases by up to 140% following a positive customer experience.

And don't kid yourselves, today's consumers are more empowered than ever before and will bolt for another brand if they don't have a good customer experience. **[49% of customers](<https://chainstoreage.com/bad-customer-service-puts-brands-risk#:~:text=Roughly%20half%20(49%25)%20of,24%2F7%20customer%20service%20support>)** who left a brand that they are loyal to say it's due to poor CX.

But for brands to be successful in executing a superior CX, they must come to terms with the fact that the foundation of a value-driven customer experience is quality, accurate customer data. The reality is a data-driven customer experience is a better CX.

And while up to now the CX was predominantly fed with more traditional "observable" data, including first-party data like pages visited, clicks, adds to cart, and time on site, for the customer experience to proactively understand customer behaviors it must deploy predictive AI algorithms to deliver to consumers what they desire before they have had a chance to act on it.

## What is predictive AI and generative AI?

The two primary AI models being deployed today in the enterprise business world are predictive AI and generative AI.

Predictive AI uses historical data to find patterns and anticipate future potential outcomes, while generative AI is more focused on the creation of new content assets, like images, text, and video from that data.

With predictive AI, businesses can better understand what their customers need, and what they care about. In this, businesses can finally move beyond making best guesses when detecting churn, scoring leads, recommending next best actions, forecasting spend, and making smarter decisions based on solid predictive data.

These two approaches to AI can be complementary if their algorithms are combined with the right software infrastructure that addresses data-oriented challenges like data quality, unified data integration, data integrity, identity resolution, and global data privacy compliance.

## To be proactive, not reactive

Making decisions from aging data is always going to put a business on the back foot. They will be forever reacting to something that happened in the past. But if an organization can predict future outcomes, it can be prepared ahead of time for the actions of customers, market fluctuations, and even challenging economic headwinds. Predictive AI delivers the knowledge of the possibilities of what might occur and allows a brand to prepare a response before it happens.

To do this companies must invest in the right AI technology infrastructure that will give them robust, dynamic predictive data to feed their CX systems. **[19 out of every 20 CX leaders](https://www.cxtoday.com/data-analytics/96-cx-leaders-are-expanding-their-customer-data-strategies/)** have invested in or plan to invest in data integration, data integrity, or data enrichment technologies.

**[Faraday](https://faraday.ai/)** provides the data and the predictive AI modeling infrastructure (IaaS) that enables organizations to iterate on their business challenges, instead of their technology infrastructure. With our platform, brands, and the software companies who support them, can now develop end-to-end customer experiences, at scale, driven by predictive insights.

In the end, a data-driven CX is a better CX. Faraday is the only predictive AI platform, both for **[software teams](https://faraday.ai/solutions/powered-by-faraday)** and **[brands](https://faraday.ai/features/customer-behavior-predictions)**, dedicated to predicting customer behavior. Everything you need to make customer experiences predictive is built into our software.

## Conclusion

How important is the customer experience to the modern business? According to **[Gartner](https://www.gartner.com/en/doc/3874972-realizing-the-benefits-of-superior-customer-experience-a-gartner-trend-insight-report)**, 80% of organizations expect to compete mainly based on CX. **[90% of businesses](https://cxindex.com/company/blog/20-statistics-to-understand-the-rise-and-value-of-cx/)**, regardless of their vertical market, have said they have made CX their primary focus.

For companies to realize this potential, they must have quality, accurate data to feed the CX system. By only using observed data companies remain inherently reactive, not proactive. To be proactive in serving their customer's needs, brands must deploy predictive AI software technology to deliver the dynamic, enhanced data needed to power the CX.

Through predictive analytics, brands can deliver a superior omnichannel customer experience, anticipate future trends, identify customers' behaviors and future needs, and proactively prevent customer churn.

To deliver the best customer experience, businesses must utilize predictive AI to anticipate and meet customer needs proactively, leveraging quality, accurate data to enhance customer satisfaction and drive business performance.

How to deliver the best customer experience possible with predictive AI

See our complete guide: [Personalization tactics that actually drive conversions](/blog/personalization-tactics-that-drive-conversions)

The customer experience continues to evolve. As consumers become empowered due to digitization, and their interactions with brands become more trackable and data-driven, companies must understand their customers more completely and intimately than ever before to differentiate themselves from their competition. This level of in-depth understanding of the consumer comes from robust, clean, and accurate customer data — both observed and predicted. Today, a data-driven customer experience is a better CX.

To achieve this type of dynamic customer experience organizations are investing in advanced tools like customer data platforms (CDPs), enhanced with [predictive AI software](/), enabling companies to gather and combine data into unified profiles so they can understand their customers more cohesively. Companies use those AI-driven insights to tailor a superior CX which can help improve the quality of customer interactions and resulting behaviors, like reducing churn. The predictive AI / predictive analytics software market is so integral to the development of a dynamic, data-driven CX that it is estimated to grow to $14.52 billion by 2028, according to [Statistica](https://www.statista.com/statistics/1286871/predictive-analytics-market-size/).

Customer acquisition costs (CAC) continue to grow, and in challenging economic times, organizations everywhere are looking to contain costs while maximizing value from their most valuable customers. When one considers that the cost of acquiring a new customer can be as much as five times as to retain one, improving retention strategy should be top of mind for any company that wants to survive and thrive in the modern business world.

By deploying a more AI-driven customer-centric approach to the CX, organizations can meet the rising expectations of today's customers, delivering the right CX at the right time to the right person, according to consulting firm [Capgemini](https://www.capgemini.com/insights/research-library/shaping-the-ai-enabled-customer-experience/).

Companies who deploy an AI-enabled customer experience can deliver a:

- 10% boost in revenue
- 20% increase in customer satisfaction
- 60% reduced effort in audience selection
- 13% decrease in operational costs

Unlike more traditional approaches, AI enables companies to continually tune their understanding of customers and dynamically reshape the CX to provide them, and the business, with true value. All to build long-term value (LTV) with your best customers.

## The challenges of observed data

What type of data is shaping today's modern data-driven CX? A few years ago, that would have been mostly from data mined from traditional survey-based methodology, what is today called zero-party data.

Zero-party data, according to Forrester, is 'data that a customer intentionally and proactively shares with a brand.' In other words, it is solicited data. It's also observed data, which is data that was collected by observing and recording events and behaviors as they occur without manipulation. While zero-party is very desirable data, due to its accuracy and direct relationship with the consumer, it's not the bread and butter of today's CX. That would be reserved for observable first-party data.

The customer data shaping today's CX is primarily made up of unsolicited first-party data like pages visited, clicks, adds to cart, time of day, and time spent browsing. This is the real data making up the data-driven CX. However, this data is reactive, and in the past. It needs to be enhanced with more predictive capabilities to deliver the dynamic data needed to meet a modern consumer's customer experience expectations.

The solution to providing the type of forward-looking data needed to create great customer experiences is for companies to begin leveraging predictive AI capabilities, like predictive modeling and predictive analytics derived via machine learning (ML), to enable companies to anticipate customer behaviors and see CX challenges and exploit opportunities in real-time.

## Predictive data drives the customer journey

Organizations that are powered by predictive insights can understand their customers as full people, interpret their interactions more completely, and trigger them at the right moments in their path along the customer journey for a desired action.

The use of predictive analytics and predictive modeling establishes a fundamental sea-change in how a dynamic customer experience is delivered to consumers. The reality is predictive customer behaviors will increasingly drive the unified customer experience until all customer interactions are predictively driven. Is your organization prepared to leverage AI-powered consumer insights for both business and customer value?

Leveraging consumer data for value is THE key differentiator for a successful modern business. But having quality, accurate data is only part of the challenge. The rubber meets the road when you determine how you will deploy that data to improve business performance, customer value, and ROI.

But all data is not equal. To do more forward-looking activities with your customer data, like train machine learning algorithms with it to inform AI, you will need to enrich it with third-party data and deploy the right software that enables predictive AI capabilities.

## How Faraday can get you there

[Faraday](/) provides the data and the predictive AI modeling infrastructure (IaaS) that enables organizations to iterate on their business challenges, instead of their technology infrastructure. With our platform, brands, and the software companies who support them, can now develop end-to-end customer experiences, at scale, driven by predictive insights. Predicting customer behavior enables more effective and efficient customer engagement within the CX. In business terms, this translates to better conversion rates which in turn creates higher revenue and reduces cost.

In the end, a data-driven CX is a better CX. A quality predictively driven CX will mean better retention for your most valuable customers, helping to reduce new customer acquisition costs in favor of a more long-term value retention strategy.

Faraday is the only predictive AI platform, both for [software teams](/solutions/powered-by-faraday) and [brands](/features/customer-behavior-predictions), dedicated to predicting customer behavior. Everything you need to make customer experiences predictive is built into our software.

## Looking forward

Predictive AI and Predictive Analytics deliver to companies the tools they need to increase customer retention and improve the customer experience. By leveraging customer data and predictive modeling, brands can understand their customer's behavior better than ever before and calculate churn for each customer.

Customer experiences everywhere are becoming more and more predictive, driven by advanced AI. It's only a matter of time until all customer experiences are predictive.

For more information on how to make your customer experience predictively driven by AI, contact [Faraday](/talk-to-sales).

As customer experiences evolve to be more data-driven, companies must leverage predictive AI and customer data platforms to anticipate customer needs, enhance interactions, and improve retention, thereby reducing costs and boosting long-term value.

Predictive customer experience is the future

See our complete guide: [Powered by Faraday — embedding predictive AI in your platform](/blog/powered-by-faraday-complete-guide-embedded-predictive-ai)

_This customer story has been anonymized to respect the company’s competitive strategy._

A leading CRM used by roofers leveraged Faraday’s infrastructure to power their Lead Intelligence tool, which helps their customers see which leads are likely to convert so that no time is wasted on bad fits.

## Giving clients an edge

The product team at this CRM for roofers is focused on helping users find value in their product, and in the five years before adopting [customer predictions](/features/predictions), they observed a 10% drop in their user base’s conversion rate. After investigating why, the product team found that, as the customer's lead generation tools grew more advanced and varied, the lead data quality was reduced, and it became more difficult for their users to see what was valuable to them. With this in mind, they determined that they needed better ways for users to cut through the noise and surface their most valuable leads–and they settled on a solution in AI [lead scoring](/solutions/lead_management).

> “We wanted to be able to help our customers’ sales teams close more deals faster. Faraday’s lead scoring and data could help us do just that.” - **VP of Product at a leading CRM for roofers**

## Finding a solution and time-to-market that fits

Although the product team unearthed an excellent opportunity to help their customers, building AI lead scoring entirely in-house was out of the question, as they didn’t have the resources to do so, and they knew that time was of the essence, so the long timeline wasn’t going to work for them. They also knew they needed purpose-built infrastructure to scale up as they grew.

With Faraday, this roofing software company found an infrastructure designed to predict lead and customer behavior and comes with a [consumer identity graph](/features/data) that includes data on nearly every U.S. adult. This–plus Faraday’s 10+ years of experience–gave their product team the confidence to build and launch a successful product in their desired timeline.

> “If we’d had the resources to build this pipeline ourselves fully, we were looking at a time-to-market of over a year. We integrated with Faraday in about three months, and it’s been worth it for that alone–not to mention the data and expertise we’ve received.” **VP of Product at a leading CRM for roofers**

## Making predictions actionable

The roofing CRM’s software engineers embedded Faraday’s lead scores into their product and provided a way to filter lead results by lead rank. With lead rank, every lead and prospect in the roofing software is assigned a rank based on how likely they are to convert, which is displayed as the lead rank status. Before partnering with Faraday to surface this likelihood, The roofing CRM’s users might spend days nurturing a sales opportunity only to have it fall flat because there wasn’t a good fit.

Their users can easily filter by the highest lead rank, which perfectly fits the ultimate goal at the start–to surface actionable insights that help focus roofing sales teams.

[![Image of roofing CRM user interface](/images/blog/leading-roofing-crm-lead-scoring/roofing-crm-ui.png)](/images/blog/leading-roofing-crm-lead-scoring/roofing-crm-ui.png)

In the first six months since launch, the roofing CRM’s clients who have turned on the new feature close deals 10% faster than those who haven’t–netting an additional $3k+ per month for your average contractor. The product team at the CRM for roofers is thrilled with the results so far and are keen to leverage more of Faraday’s infrastructure to build additional features, focusing on [explainability](/blog/what-is-explainable-ai-why-is-it-important) to give users a clear sense of why an individual lead is receiving its score.

[Create a free Faraday account](https://faraday.ai/signup) today.

A leading CRM used by roofers powers their Lead Intelligence tool with predictive lead scores, which helps their customers see which leads are most likely to convert.

How a leading CRM used by roofers employs Faraday to help clients close deals 10% faster

**See our complete guide:** [Customer personas explained — how Faraday builds them and what they unlock](/blog/what-are-ai-customer-personas).

---

Using machine learning to identify customer personas in your data can help improve your understanding of your customer base, and with the right infrastructure, deliver personalized experiences at a greater scale.

In a nutshell, the process entails using an algorithm to segment your customer data into distinct groups, or personas. Faraday uses a time-tested method called k-means clustering.

## What is k-means clustering?

K-means clustering is an unsupervised clustering algorithm that was first introduced in 1957. An unsupervised algorithm does not require the data to be labeled in order to train the model. This is an important feature of the k-means algorithm because it allows for the discovery of subgroups within the data without any previous assumptions about possible groupings — in this case, personas.

## How does k-means clustering work?

When presented with a data set, the k-means algorithm first introduces random center points around which each data point will eventually be clustered (circles, Fig. 1). Then each data point is assigned to the group of the center to which they are closest.
This happens by calculating the distance from a data point to each cluster center using an equation known as the Euclidean distance measure. For two dimensions, the Euclidean distance is represented by the diagonal line connecting the point to each of the cluster centers. The point is then assigned to the cluster center with the shortest diagonal line (Fig. 2).

Once each point has been assigned to the randomly chosen cluster centers, the centers then "update," moving to the center of the group by recalculating the mean (average) distances to all the points in the group (Fig. 3).

After the cluster centers have updated to their new positions, each point is reassigned to a new group using the Euclidean distance measure again. Most of the points will remain in their original groups, but some points will be assigned to a new group (Fig. 4). So, then the centers must be updated again to be at the center of the new group. This pattern repeats until the centers no longer move.

[![K-means diagram](/images/blog/k-means_diagram.png)](/images/blog/k-means_diagram.png)

## Using k-means clustering for customer personas

All of the above is how k-means clustering worked back in 1957. Since then there have been upgrades allowing us to use something called categorical features. At Faraday, we create persona groups using a mix of these features.

As far as algorithms go, k-means is rather straightforward, but has one major downfall: you have to tell it how many groups to look for, and it will always succeed in finding that many groups. That means if you tell the algorithm to look for 500 groups, it will find them — even if there is no reason for there to be 500 distinct groups within the data.

If you're a marketer, you probably don't want to create 500 unique ads for every campaign. But should you make four? Or six? Or ten?

## How to find the optimal number of personas using k-means clustering

We target the optimal number of groups hidden within the data, rather than guessing and hoping we are right. We’ll share a few methods we use to find the optimal number of personas for our users.

### The elbow method: finding diminishing returns

The most common metric for measuring the success of k-means clustering is called the ratio of sum of squares (rss). At Faraday, we determine the optimal number of personas by comparing the rss for k-means models with two to fourteen personas. We employ the elbow method, which shows the cutoff after which we see diminishing returns for increasing the number of personas. For most clients, the optimal number of distinct personas is usually between four and seven.

**Going beyond the elbow method**

The elbow method is simple, but it has a downside: sometimes, there just isn't a clear single point that is optimal. Maybe three personas could be optimal, maybe six, and we have to intervene based on our client’s business needs to choose one or the other as the optimal number of personas.

Additionally, while reducing variability within each group is an extremely useful way to measure k-means clustering success, it doesn't tell the whole story. We believe that an important aspect of clustered groups is stability, which isn't considered with the elbow method selection process. Our clients expect that their persona groups won't shift every time they convert a lead. This is why we are currently researching a new method for determining the optimal number of personas, which focuses on the stability of the persona group.

### The bootstrap method: focusing on stability

Our first step to assess the stability of k-means clustering involves resampling the client’s data using the bootstrap method. Like k-means clustering, bootstrapping has been around for a while – it was first [introduced in 1979 by Bradley Efron](https://projecteuclid.org/journals/annals-of-statistics/volume-7/issue-1/Bootstrap-Methods-Another-Look-at-the-Jackknife/10.1214/aos/1176344552.full). The basic idea behind bootstrapping is to take data from the pool we have, with the ability to pick the same data point multiple times. This is a method known as sampling with replacement.

The bootstrap method provides us with a powerful tool to measure how much our desired clustering outcome could change with different data. We can get a good sense of how much the personas will change with the addition of new customers. Our goal is to minimize how much influence single customers, or small groups of customers, have on the resulting personas. We want personas that remain relatively unchanged by single customers. In other words, we want more stability.

## Pulling it all together

Once we have the results from bootstrapping, we use the ratio of sum of squares again to measure just how stable the personas are. What we’re looking for is a ratio close to 1, which would indicate that there is no variability within the personas.

The stability metric for determining the optimal number of personas shows promising results for the basic examples in this blog post. At Faraday, we’re actively working on applying this methodology to the more complex persona groupings our clients rely on. Stay tuned for more developments with respect to improving our personas methodology.

Learn how machine learning is used to identify distinct clusters—or personas—in your customer data.

How k-means clustering is used to identify customer personas

_This article is a followup to our [bias reporting](/blog/faraday-bias-reporting) article, and aims to provide a high-level overview of how Faraday users can mitigate harmful bias._

## What is bias mitigation in machine learning?

Bias mitigation in machine learning encompasses a set of methodologies and techniques devised to diminish and ideally eradicate unjust and discriminatory predictions inherent in the decision-making processes of artificial intelligence (AI) systems. This is a key component of [Responsible AI](/features/responsible-ai).

Bias, often unintentional, may infiltrate these systems as a result of the historical data they are shown, leading to inequitable treatment of specific demographic groups. For example, there is a history of [discrimination](https://www.investopedia.com/the-history-of-lending-discrimination-5076948) against specific minority groups in financial lending, and this history can perpetuate bias if it is not intentionally mitigated.

It is important to note that not all algorithmic bias is harmful. Predictions about, for example, the likelihood of buying products that are exclusively relevant to, say, certain age groups will and should discriminate in favor of those age groups. For the remainder of this article, when we discuss bias, we’re excluding innocuous bias like this and focusing on harmful bias.

Bias mitigation ensures that AI systems render decisions characterized by impartiality and fairness, irrespective of variables such as race, gender, or other sensitive attributes. This undertaking is pivotal in the development of AI systems that uphold principles of fairness and equality for all individuals.

The subsequent sections of this article describe how Faraday’s bias mitigation feature allows users to create fair propensity predictions. For more on bias management in Faraday, read our [bias management announcement](/blog/faraday-bias-management-announcement).

## How to reduce harmful bias in machine learning

When predicting propensity, Faraday can mitigate bias in two broad ways:

1. The first step is to ensure that the predictive algorithm considers every demographic group as equal. In order to do so, Faraday rebalances your historical data before submitting it to the algorithm for pattern discovery.
2. Second, when later making predictions, Faraday will adjust the nominal propensity ranking to ensure that the highest scorers for each demographic are ranked highly overall. This ensures that the predictions are meeting the fairness objective while maximizing predictive lift.

Some Faraday terminology will be used here, and will link to documentation.

### Unbiasing historical data

In order to mitigate bias in a client’s existing customer data, Faraday assigns different “weights” to each customer in the data to fine-tune their influence on the model's learning process. The goal is to address and rectify biases present in the data, ensuring that the model gives equal consideration to all relevant examples.

To understand this process, consider an example: a home mortgage company. Given historical disparities arising from discriminatory laws and government policies (among other reasons), some demographics are less likely to appear in the company’s customer data.

Formally, Faraday will discover this by noticing a difference between the “attainment” (all customers) and “eligible” (regional adult population) [cohorts](/docs/product/cohorts) in one or more dimensions of concern. For example, we could find fewer women than men in the historical customer data, even though the regional population is an even split: a disparity that has no rational basis. Without correction, the algorithm would believe (incorrectly) that there is some intrinsic quality which makes men more likely mortgage customers than women.

To unbias the historical data, Faraday would give higher weights to customers in underrepresented or disadvantaged groups, increasing their influence on the algorithm while it is looking for salient patterns. Building on the mortgage example above, Faraday could give higher weight to the proportionally fewer women customers, evening out their influence versus men. This results in more equitable predictions for individuals belonging to these groups.

By adjusting the weights assigned to different customers, Faraday aims to create a more balanced and fair learning process, ultimately leading to AI models that provide fairer outcomes across diverse demographic groups.

### Unbiasing predictions

When you include a propensity [outcome](/docs/product/outcomes) in your [pipeline](/docs/product/pipelines), Faraday can take additional measures to rectify potential biases in the resulting predictions. After the initial predictions are made, this approach evaluates and adjusts them to ensure fairness and reduce discriminatory outcomes.

Given that the first step eliminates bias in the historical customer data, why is this second step even necessary? When we unbias historical data, we can't alter the algorithm to completely ignore the overrepresented population in favor of the underrepresented one–we’re simply telling it to pay more attention to the latter until it has a roughly equal representation. This weighting can have a cascading effect that creates biases of its own, though.

To take the mortgage example even further, maybe the women customers tend to be older than the non-customers, whereas the men customers tend to be younger than their non-customer counterparts–after the weight adjustment, this can make the trait _age_ mean that an individual is more likely to attain the outcome if they are older, unfairly penalizing those without that characteristic, which is why this step is crucial.

In this step, Faraday looks for differences between the demographic makeup of the pipeline’s eligible population and the subgroup receiving the highest scores. The predictions are then adjusted to align more closely with the desired fairness objectives.

This method allows for the correction of biases that may have been present in the original predictions, promoting a more fair distribution of outcomes across different demographic groups. By incorporating fairness considerations after the initial predictions are generated, prediction this re-ranking contributes to developing machine learning models that exhibit improved fairness and reduced problematic disparity in their decision-making processes.

## Mitigating harmful bias with Faraday

As of the publication of this article (February 2024), Faraday currently allows for bias detection and mitigation for the **age** and **gender** dimensions (these features are often referred to as protected classes or Dimensions Of Concern).

When you define a propensity outcome in Faraday, you can choose how you want the system to handle bias:

- **No mitigation**: The outcome fairness is only reported as part of the outcome dashboard. No further transformation is applied to ensure that protected classes are treated fairly. This is perfectly appropriate in cases where the product or service being sold is intrinsically linked to certain demographic groups.
- **Equality**: This mitigation strategy ensures that demographic patterns are similar between the eligible population and the target group (those with the highest scores). For example if “young adult, women” represent 25% of your eligible population, we ensure that any population targeted from the pipeline will contain 25% of “young adult, women”.

  The chart below depicts the effects of historical data bias mitigation using the “equality” setting with three populations (A, B, C). Population **A is underrepresented** in the customer data (attainment population)—30% are customers vs. 47% in the overall population while **B** and **C are both overrepresented**. The mitigation process ensures that in the target group, population **A** accounts for 47% while **B** and **C** account for their original 14% and 39% respectively.

  [![Equality bias mitigation](/images/blog/faraday_bias_mitigation/equality-mitigation.png)](/images/blog/faraday_bias_mitigation/equality-mitigation.png)

- **Equity**: This mitigation strategy “inverts” bias so that populations that were historically underrepresented are now instead prioritized.

  In the example below, the customer group (attainment cohort) consists of 30% of population **A**, 20% of population **B**, and 50% of population **C**. The equity mitigation is going to promote the underrepresented populations **A** and **B** at the expense of population **C**. The target population will contain 37% of **A**, 47% of **B**, and 16% of **C**.

  [![Equity bias mitigation](/images/blog/faraday_bias_mitigation/equity-mitigation.png)](/images/blog/faraday_bias_mitigation/equity-mitigation.png)

Note that when mitigating bias, Faraday will consider all dimensions simultaneously, leading the system to consider subpopulations like “young adult, women” and “middle-aged men.”

## Does mitigating bias hurt performance?

Because the bias mitigation process necessarily alters the ranking and promotes sub-populations with otherwise lower predicted scores, it can reduce lift.

That said, the impact is often small. The top two graphs below depict the impact of **equality** mitigation on Gender. The lift curve for the un-mitigated model is only slightly better than the one for the mitigated model. The main difference lies in the 20% to 40% score percentile range with lift in that range decreasing from ~300% to ~220%.

The two bottom graphs show the mitigation at work. On the left, the un-mitigated model will create a target group with 2x too many males.

After mitigation, the graph on the right shows only a slight discrepancy between the fair proportion of each gender, within 3%. This is a significant fairness improvement over the initial target population breakdown with only a minor reduction in performance.

[![Bias mitigation comparison](/images/blog/faraday_bias_mitigation/mitigation-comparison.png)](/images/blog/faraday_bias_mitigation/mitigation-comparison.png)

## Why mitigate bias?

Bias isn’t inherently harmful. Earlier in this article, we touched on innocuous bias that occurs when products or services are explicitly designed for and marketed toward certain groups, like a womens’ swimwear brand marketing mostly to women. It’s the _other_, harmful–and often hidden–bias that should be surfaced and mitigated.

With Faraday, bias mitigation is easy. If you find that a predictive outcome you created has bias that you want to mitigate, simply create a new outcome using the same cohorts of people, toggle mitigation on for the sensitive dimension in question, select your mitigation strategy, and save the outcome. Once it finishes building, compare the bias report from the un-mitigated outcome with the report from the mitigated outcome. Alternatively, use the **bias_mitigation** object with the API call to [create](/developers/reference/createoutcome) an outcome.

[![Bias mitigation comparison](/images/blog/faraday_bias_mitigation/bias-mitigation-in-app.png)](/images/blog/faraday_bias_mitigation/bias-mitigation-in-app.png)

It’s in the best interest of anyone developing or using AI to do so responsibly. Whether you’re using [generative AI](https://www.bloomberg.com/graphics/2023-generative-ai-bias/) for email copy and creative, predicting customer behavior like we walked through in this article, or something else entirely, you should be wary of the existence of bias in the data that the algorithms are using, and how it impacts who you market and sell to. By employing responsible, [explainable AI](/blog/what-is-explainable-ai-why-is-it-important), you can build a foundation of trust between not only yourself and your customers, but between your business and your data.

Responsible AI isn't just good business–it’s the right thing to do.

Mitigating harmful bias with Faraday is simple, whether through our developer API or in-app. Learn more about how Faraday mitigates bias in your customer predictions.

How Faraday helps mitigate harmful bias in machine learning

Regardless of what part of your business you find yourself in, your team has likely discussed implementing artificial intelligence (AI) of some sort in 2024–and it’s no wonder, with [33% of businesses](https://thesocialshepherd.com/blog/ai-statistics) already using AI and another 42% considering adopting it, the benefits of harnessing AI are too good to be ignored.

As leaders, employees, and consumers alike learn more about predictive models, machine learning algorithms, and various other pieces of the AI puzzle, questions naturally arise about how AI really works. Can the predictive models created be trusted? Can they produce biased results? Making complex machine learning algorithms and their results explainable to humans has proven to be a challenging feat for organizations to overcome in a tangible way, though there are significant benefits for those that do.

Put simply, if you’re going to change the way you interact with customers based on what AI tells you, how can you trust what it’s telling you?

## What is explainable AI?

Historically, AI has been plagued by the [black box problem](https://umdearborn.edu/news/ais-mysterious-black-box-problem-explained), in which artificial intelligence reaches purportedly valuable conclusions—finding high-scoring leads, producing compelling generative content, and so on—and the user is left with the option to take it or leave it. No insight was provided into how the AI reached the conclusion it did, and what data it used to do so, thus the term “black box.”

Explainable AI aims to solve this problem by providing much-needed insight into the inner workings of predictive models so that the end-result of a predictive model offers both the sought-after conclusions, as well as how those conclusions were reached. This means that deploying [customer predictions](https://faraday.ai/features/predictions) and other AI-generated content can be done with confidence, rather than a hand-wave.

## Why does explainable AI matter?

[According to IBM](https://www.ibm.com/thought-leadership/institute-business-value/en-us/report/business-trends-2024), 9 in 10 consumers say trust is the most important deciding factor when deciding which brand to adopt. People have never been more keenly aware of the importance of brand trust, and how quickly it can vanish. The need for consumers to trust brands they interact with has resulted in the need for explainable AI, and for the black box to be removed. IBM also reports that general consumer trust in businesses dropped from 48% in 2021 to 20% in 2023, meaning that businesses need to employ frameworks like explainability that help establish and maintain trust in the relationship for consumers to even consider making a purchase. The relationship between trust and explainability extends far beyond direct-to-consumer goods, too.

In [finance](https://faraday.ai/blog/how-to-optimize-ads-for-compliance-in-finance)—a space that has historically struggled with bias—AI can be used to determine whether or not an applicant should receive a loan, which leaves the door open for their machine learning algorithms to use historically biased data to permit or deny a loan. AI must be explainable in order for lenders, or anyone else building or employing it, to consider it for use.

## Benefits of explainable AI

Explainable AI enables businesses that employ AI pipelines to properly interpret and understand its output without requiring data science degrees. This benefit is twofold: first, the importance of understanding the tools you’re working with rather than just taking their word for it can’t be overstated. Knowledge is power, after all. This leads straight into the second, in that it allows you to not only understand why the AI reached the conclusion it did, but then enables you to take that reasoning a step further by analyzing its context. If you’re using machine learning algorithms to find out which of your customers is [most likely to churn](https://faraday.ai/predict/category/churn_scores), you can dig into its explainability to reveal what traits about them helped make that critical decision. Similarly, explainable AI can help [reveal and mitigate bias](https://faraday.ai/blog/faraday-bias-management-announcement) in your data, which helps you stay compliant with local laws and regulations.

Additionally, a foundation of trust is built (and maintained) between the business and its customers, which is a rare commodity in 2024. With existing customers generating [72% of company revenue](https://www.hubspot.com/hubfs/HubSpots%202024%20Sales%20Trends%20Report.pdf), a happy customer base that you’ve nurtured with trust will keep coming back for more–and they’ll refer their friends, too.

## Challenges of explainable AI

With such impactful benefits on the table, why hasn’t explainable AI become mainstream? The simple answer is that it’s difficult. Entire classes of ML algorithms are fundamentally incapable of effectively explaining their inferences, so you have to choose your methodology carefully. Explainability also requires well-prepared and structured attributes that not every organization is lucky enough to have on the shelf. Finally, in a commercial environment where data science resources are both scarce and overburdened, AI explainability never quite seems to make it to the top of the todo list.

## Explainability is built into Faraday

While businesses that are fully committed to building their own AI pipelines need to overcome the hurdles involved in implementing explainable AI, many businesses are instead adopting platforms like Faraday that are purpose-built for using AI–and in Faraday’s specific case, predicting customer behavior.

Faraday has long been a proponent of [responsible](https://faraday.ai/features/responsible-ai), explainable AI. In fact, it’s one of our founding principles. Every [predictive model](https://faraday.ai/blog/faraday-probability-scores) that Faraday users create allows customization of what traits are used, allows removal of protected class data, and comes with a full report that details things like the most important traits that helped build the model. In 2023, we released [score explainability](https://faraday.ai/blog/faraday-score-explainability), a feature that outlines each of the most influential pieces of data within an individual person’s predictions, and how much influence they had.

[![Image of a Faraday score explainability output](/images/blog/explainable_ai/explainability-output.png)](/images/blog/explainable_ai/explainability-output.png)

Let’s use the above graphic to visualize the churn prediction example in the previous section. Jane Madison has a 75% probability of churning, and the explanation provided in the predictive model that scored Jane said that the fact that Jane is a millennial played a major role in why she’s so likely to churn. Meanwhile, John’s children made it less likely that he would churn. Not only do we know that Jane is more likely to churn than John, but we know _why_.

In the end, explainable AI results in tangible benefits for both the businesses employing it and their customers. Businesses are able to understand their own tools, and use that understanding to reveal impactful pieces of insight like _why exactly_ Jane Madison is going to churn. From a consumer perspective, businesses employing explainable AI stand out as trustworthy and worth interacting with–and worth referring their friends to do the same.

With AI being built into many businesses' workflows, concerns are rightfully being raised about its trustworthiness, and emphasis put on the importance of explainable AI.

Why explainable AI is so important for businesses

Quantifying bias in predictive models is the first step towards building models with fair treatment of protected groups.

Faraday bias reporting: How we measure & report on bias

**See our complete guide:** [Predictive AI for financial services (debt, mortgage, lending)](/blog/debt-mortgage-lending-complete-context-guide).

---

A leading **debt consolidation** company relied on direct mail as a key channel for acquiring new customers. However, a significant portion of their marketing budget was being spent on sending mail to leads who were unlikely to convert. This inefficiency not only increased costs but also diluted the impact of their outreach efforts, making it difficult to connect with high-intent buyers.

## Smarter targeting with Faraday's lead suppression

To address this challenge, the company partnered with Faraday to implement a [predictive lead suppression model](https://faraday.ai/templates/lead-suppression/recharge). This process began with Faraday ingressing their lead data to assess the key characteristics of their existing customer base. Then, through careful analysis of their historical conversion patterns—as well as the inclusion of custom predictive data points, built using [rich 3rd party data and attributes on 240M+ US consumers](https://faraday.ai/features/faraday-identity-graph) that comes preloaded through the [Faraday Identity Graph (FIG)](https://faraday.ai/docs/abstractions/traits)—we were able to predict which of their leads were most likely to convert.

Using this analysis, Faraday’s model identified the bottom 10%—the least-likely-to-convert leads—and suppressed them from the company’s direct mail campaigns. This approach immediately reduced wasted marketing spend and allowed the company to reallocate resources toward higher-value prospects.

## Or see Faraday's lead suppression process broken down visually

1. ![Faraday Identity Graph](/images/blog/lead-supp-panel-0.png)

   ### Don’t start from scratch

   Faraday comes preloaded with 1400 attributes on 240M+ US consumers and their households.

   This powerful dataset provides high-signal data points—including property data, demographics, life events, and buying habits—that can shape smarter decisions.

2. ![client data](/images/blog/lead-supp-panel-1.png)

   ### Your lead data is shared with Faraday

   Your lead data is collected and ingressed into Faraday securely.

   This guarantee is achieved through our robust data security policies, which include HIPAA, GDPR, and SOC-2 compliance.

3. ![model is trained and predictions begin](/images/blog/lead-supp-panel-2.png)

   ### We train predictive model by analyzing client data for patterns

   Faraday’s Identity Graph & AI/ML predictive modeling enables us to identify unique trends in your data set and specifically assess your leads’ likelihood to convert.

4. ![leads are sorted into deciles](/images/blog/lead-supp-panel-3.png)

   ### We identify how likely each lead is to convert

   We identify 10 deciles to be able to help you determine which leads are the best (most likely to convert after receiving mail) and the worst (those that will cost more in postage/print than they will generate in value, or simply won’t convert).

5. ![cut-off threshold is established](/images/blog/lead-supp-panel-4.png)

   ### We establish a cut-off threshold

   Faraday then helps you to calculate the “cut-off” threshold. This determines where the cost to print/send the mailer is greater than the likelihood to convert to revenue.

6. ![targeted client list](/images/blog/lead-supp-panel-5.png)

   ### We return the targeted list

   Once Faraday has removed the low-fit leads, you can use the new list to target your direct mail process and proceed with your next mail drop.

## Optimized results, supported by data scientists

Although this model was already delivering significant value, our data science team wasn’t satisfied.

That was because this company recognized four distinct customer segments within their broader population—each with unique characteristics. Although the one-size-fits-all approach had made sense initially to drive results, we hypothesized that a single model that handled all leads at once was likely not actually able to provide the highest level of precision possible. Instead, we considered creating four additional models which would be individually tailored to each unique customer segment.

While we were confident that this more granular approach would improve results, at Faraday, we prioritize empiricism over intuition so we decided to run rigorous validation tests. And indeed, after comparing conversion trends between the original model alone and all five models together, we were able to prove—rather than assume—the added value of the more granular approach. For the client, this meant even better precision in their targeting and more return on their direct mail spend.

While this process required a deeper investment of our data science team’s time, the outcome was clear: greater accuracy, better optimization, and ultimately, higher value. At Faraday, this isn’t just a one-off added service—it’s how we operate. Strategic data science investment is the only way to unlock insights at this level, and this project stands as a testament to that philosophy.

## The result

The impact of Faraday’s predictive lead suppression was immediate and substantial:

- **At least 600,000 fewer pieces of wasted direct mail** sent to low-intent leads monthly
- **Over $100,000 saved each month** in direct mail costs
- **An additional \$20,000 in monthly revenue, or $240,000 annually**, thanks to these targeted models’ better precision in predicting which customers would convert

With these enhancements, the company is now seeing a **10x return on investment (ROI)** from Faraday’s predictive modeling.

## Beyond cost savings: data-driven growth

By investing in Faraday’s predictive modeling and data science expertise, the company didn’t just cut costs—they redefined the way they acquire customers. Suppressing low-intent leads was just the beginning. By refining their approach with segment-specific models, they turned direct mail into a high-precision growth engine, ensuring that every dollar spent reached the right audience.

With this new strategy, the company isn’t just saving money—it’s driving measurable, sustained growth. Faraday’s predictive lead suppression has become a core part of their marketing playbook, proving that the right data science investment doesn’t just optimize campaigns—it transforms businesses.

Through predictive lead suppression, this company was able to cut at least 600,000 pieces of least-likely-to-convert direct mail each month.

How this debt consolidation company optimized direct mail with lead suppression to save over $100K each month

One of Faraday’s founding pillars is the use of [Responsible AI](/features/responsible-ai), which includes both preventing direct harmful bias and reporting on possible indirect bias. We’ve been building these features directly into the platform over the past several months, and today we’re going to walk through two new features in Faraday with **bias reporting** and **bias mitigation**.

## Bias reporting

To see the bias report on an outcome, head to [outcomes](https://app.faraday.ai/outcomes) and scroll down to the **bias** section to see the breakdown of bias discovered in the _data_, _power_, _predictions_, and _fairness_ categories, which we'll dive into below. These can also be retrieved via the API using [get outcome analysis](/developers/reference/getoutcomeanalysis).

[![Bias reporting on an outcome](/images/blog/faraday_bias_reporting/outcomes-bias-reporting.png)](/images/blog/faraday_bias_reporting/outcomes-bias-reporting.png)

Faraday uses four categories to report on bias for an outcome:

- **Data**: Measures selection bias in the underlying cohorts used in the outcome. In a Faraday outcome, two labels exist for the purpose of this blog post: _positive_, or the people from the attainment cohort that were previously also in the eligibility cohort, and _candidate_, or the people from the eligibility cohort.
- **Power**: Measures outcome performance on a subpopulation compared to baseline performance–for example, Faraday will compare how well the outcome performs on the subpopulation “Senior, Male” compared to everyone else.
- **Predictions**: Measures proportions of subpopulations being targeted compared to baseline proportions in order to see if the subpopulation is over or under-represented.
- **Fairness**: Measures overall fairness using a variety of common fairness metrics in the literature. For example, Faraday will look at whether or not the subpopulation "Senior Male" is privileged or underprivileged.

Each bias category uses groups of people called subpopulations, which are combinations of people defined in the [understanding bias reporting](/docs/product/outcomes#understanding-bias-reporting) section's **sensitive dimensions** dropdown table. Examples of these are "Teens with gender unknown" and "Senior Females." We're launching with two of these sensitive dimensions in **age** and **gender**, but will be rolling out more over time.

If you discover that one of your outcomes includes harmful bias, you can create a new outcome using the same cohorts, but enable bias mitigation options while creating it to compare results.

## Bias mitigation

When creating a new outcome, you can choose to mitigate bias by using the toggles in the **bias** section. Check out our [outcomes documentation](/docs/product/outcomes#mitigating-bias) for info on different bias mitigation strategies.

[![Bias mitigation in an outcome](/images/blog/faraday_bias_reporting/outcomes-bias-mitigation.png)](/images/blog/faraday_bias_reporting/outcomes-bias-mitigation.png)

<Callout type="warning" title="Taking action on bias">
Mitigating bias isn't something you should set-and-forget, meaning it's not recommended that you toggle both **age** and **gender** on for every single outcome you create. For example, if you're a womens' swimwear brand, your data will skew heavily toward women, and mitigating gender would negatively impact your lift.
</Callout>

Always keep the populations that you're making predictions on in mind when considering bias. In the above example, it's _totally_ fine to ignore the biased skew towards a certain gender or age group because they're your ideal customers.

We're thrilled to continue to drive Responsible AI forward by providing agency for our users to both discover and mitigate bias in their customer predictions. Stay tuned for a deep dive into the science behind how Faraday determines bias.

Learn how Faraday's bias management suite enables you to analyze and mitigate bias found in your customer predictions.

All posts