Faraday’s data science team has made significant improvements to the accuracy and interpretability of customer predictions within the platform; we’re going to give you a peek under the hood at how Faraday predictions are made.

The data science team introduced a process called **model calibration**, which measures how well the model's predictions match the actual outcomes. This method of calibration results in **probability scores**; predictions that are even more accurate and easier to interpret. Let’s take a look at a model report to break down the changes.

## Lift and precision before model calibration

In the below **pre-calibration** graphic, note the large dip in the lift curve (red arrow). With this lift curve, the top deciles of propensity scores, or the people you’d generally be targeting with a marketing campaign based on these results, have lower lift in the population (green square), indicating that there’s room for improvement.

A lift curve with strong predictability should show a more gradual dip than you see below. Similarly, the positive attainment cohort _home and living_ (blue square)–or the outcome you want to predict for–is very close to 50%, which could also be improved through model calibration. This would result in pipeline deployments having, say, a Jane Doe with a 0.8 score that could be interpreted as _pretty likely_ to achieve the outcome.

[![Image of Faraday model report lift table pre-calibration](/images/blog/probability-pre-calibration.png)](/images/blog/probability-pre-calibration.png)

_The above graphic displays a predictive [model report](/blog/faraday-model-reports) pre-calibration. Model reports can be found in an outcome's analysis view_.

Because Faraday uses multiple predictive models to create an outcome, it makes more sense to compare scores across models rather than treat scores from different models equivalently.

## Lift and precision with model calibration

Now, with model calibration implemented in the below graphic, the lift curve’s dip (red arrow) is much less significant for the top deciles, resulting in an outcome that has a higher positive attainment (blue square) in _home and living_.

[![Image of Faraday model report lift table post-calibration](/images/blog/probability-post-calibration.png)](/images/blog/probability-post-calibration.png)

_The above graphic displays a predictive [model report](/blog/faraday-model-reports) post-calibration. Model reports can be found in an outcome's analysis view._

So what does this mean? With probability scores, the groups of your prediction’s population–the deciles–are ranked in decreasing lift values, resulting in higher lift in your top attainers, or the people most likely to achieve the outcome.

With this update, Faraday’s predictions give you probability scores that are directly correlated to the probability of an individual achieving your outcome. If Jane Doe in your population has a 0.8 probability score, there’s an 80% chance that she will achieve the outcome.

Marketing campaigns have never been so predictable.

**[Create a free Faraday account](https://faraday.ai/signup) today.**

How model calibration helps Faraday improve customer prediction accuracy through probability scores.

Faraday probability scores: Improving prediction accuracy through model calibration

## What is market sizing?

Market sizing is a method of calculating a company’s revenue potential in a geographic market, which can provide crucial insights for competitive brands to be aware of when looking to expand, otherwise resources are likely to be wasted.

Market sizing is not a new concept, but more recent innovations in data collection and predictive analytics solutions have opened the door for more accurate market size calculations, enabling brands to make better go-to-market decisions and expand with more confidence and direction than before.

## Top down vs bottom up market sizing analysis

There are two methods of market sizing analysis: **top down**, and **bottom up**. In this article, we’ll look at both, and then explore how using predictive analytics for bottom up market sizing can provide you more accurate estimates while drastically expediting the analysis process, and give you actionable insight into where you’re most likely to have success penetrating.

### Top down market sizing

When using top down market sizing to calculate your market potential, you start at the highest level of the market that you’re in, where you find the largest possible size of the market you could acquire, then multiply it by the percent of the market that your product fits in.

For example, say you’re a brand that sells humidifiers, and you’re planning to launch a new humidifier that’s perfect for maintaining moisture levels in a newborn’s nursery. With the top down approach, your research shows you that the US market for humidifiers is \$700 million, and 65% of humidifiers sold annually are single-room, like your new product.

> **\$700m X 65% = \$455m**

Since, for the most part, the numbers that are relevant to your company and industry are likely already published elsewhere, this approach involves doing some research to find the populations and market values you need rather than calculating them yourself.

**Pros**:

- **It’s fast**. The research involved and the calculation are both straightforward and quick, so there’s not much time investment in a top-level market size estimate.
- **Accessible data**. Regardless of the outlet you want to source your data from, people have been publishing industry reports and analyses for years, so the data you need can easily be found via some time on Google.
- **Helpful at-a-glance**. Since top down market sizing looks at broad trends and market-wide data, it’s useful to be able to know, at a glance, if the space you’re looking to go-to-market in is worth the investment you’d have to make.

**Cons**:

- **It’s very broad**. As mentioned above, it’s useful to run a top down market size analysis for an initial glance at the market, but it’s unlikely to be specific enough for your company’s needs.

### Bottom up market sizing

Bottom up market sizing starts on the opposite end in that it begins with your niche. Using the same humidifier example, you’ll likely want to find people with children in the household to target since your new humidifier is for nurseries.

Through extensive research, data collection, and surveys, you find that there are 4 million people in this target group. To calculate the bottom up market size, you’d then multiply the number of people by the annual revenue you’d expect from them for the product–your customers are unlikely to buy more than one humidifier in a year, and its cost is \$100 per unit, so we’ll use \$100.

> **4 million people X \$100 = \$400m**

So what’s the verdict? Top down or bottom up market sizing? Since it uses data that’s relevant to your company, bottom up market sizing provides a more accurate estimate of your potential market–if your company has the bandwidth to put in the time. Fortunately, predictive analytics can help cut down that long timeline dramatically while providing you with geographies where your most likely buyers live.

**Pros**:

- **Backed by your data**. Since bottom up market sizing uses your own data & analytics through your customers, product, and cost data, results are much more tailored to your company, and they’re more realistic to boot.
- **As niche as you want it**. Rather than relying on industry-wide assumptions, this strategy begins and ends with your company’s niche.

**Cons**:

- **Time-intensive**. Performing the data collection required to accurately run a bottom up market sizing analysis is a long process in that, whether you’re conducting interviews or surveys, you’ll need to wait for results to come in.
- **Resource-intensive**. Assigning teams to create surveys, conduct interviews, act on their results, and so on can occupy a significant portion of their time.

## Calculating market size with predictive analytics

In the end, bottom up market sizing wins out as a more effective approach to calculate market potential. The downsides, of course, are the longer timeline and additional resources required to do it effectively.

This is where predictive analytics comes in–leveraging machine learning to predict individuals’ propensity to buy your products or services. With machine learning, your time-to-value in performing a bottom up market sizing analysis is dramatically reduced because it eliminates the need to design surveys and conduct interviews, conduct research, and wait for results.

Better yet, rather than spending extra time using your analysis to plan where you _think_ your campaigns will have the most impact, likely buyer scores show you _exactly_ where your high-intent buyers are at any geographic zoom that suits your needs.

### Traditional approach to bottom up market sizing

As we reviewed earlier, the typical traditional approach involves multiplying the number of people in the segment by the annual cost of your humidifier, which is simple enough math once you’ve done the required research to find the number of people in the segment.

At the end of the day, there’s still a lot of guesswork involved. In your research, you uncovered the raw number of potential buyers in your market, but the only direction you might have as for where to direct your campaign are surveys, which are unlikely to have reliable penetration for guidance, and assumptions–maybe the 5 driest states in the U.S. are the best bets to market your humidifier?

### Predictive approach to bottom up market sizing

With the predictive approach, you’ll take the segment you created above, then use AI to predict the propensity to buy for everyone in the segment based on real historical data. Machine learning algorithms surface patterns in the data to calculate just how likely each individual in this segment is to buy the product in likely buyer scores, and their forecasted spend over their lifetime as a customer of yours.

These likely buyer scores can be filtered by county, state, city, or metro, so that you have crystal clear indicators where your high-intent buyers are. You can even use your own points-of-interest, like brick and mortar locations, to perform radius-based likely buyer targeting for pin-point accuracy.

With Faraday, the predictive approach can be done in just a few clicks, saving you a significant amount of time and resources, providing more accurate results, and helping align your strategy through clear indicators for the locations you’ll have the highest likelihood of penetration in.

## How to do bottom up market sizing with Faraday

Using Faraday, the predictive approach for bottom up market sizing can be embedded wherever in your stack you need it in less than a day. No developers, no data scientists, and no long research timeline required.

### Connect data and define customers

To start, you plug in your data by mapping your data fields to Faraday properties.

![Image of Faraday dataset](/images/blog/market-sizing-dataset.png)

Then, in cohorts, you’ll utilize both your customer data and Faraday’s built-in consumer data to build the [exact cohort](https://faraday.ai/docs/cohorts/customers_from_hosted_csv)–or interesting group of people–you want. To continue with the nursery humidifier example, you’ll define a cohort for your customers, and a cohort using the trait for _children in household_, sourced from Faraday’s [rich, cookieless, opt-in consumer data](https://faraday.ai/features/data), to target with your propensity to buy predictions.

![Image of Faraday cohort](/images/blog/market-sizing-cohort.png)

### Predict propensity to buy to hone in on likely buyers

With your cohorts defined, build your predictive outcome by choosing your attainment, or your customers. Here, you’re defining your propensity to buy, or the likelihood that anyone in the U.S. is going to become a customer by making a purchase.

![Image of Faraday outcome](/images/blog/market-sizing-outcome.png)

Now that your predictions are defined, create a pipeline to make them actionable by choosing your **likely buyers outcome** and your **children in household** cohort.

![Image of Faraday pipeline](/images/blog/market-sizing-pipeline.png)

Machine learning algorithms then get to work predicting how likely people across the country with children in the household are to buy your new nursery humidifier. You can even further your bottom up market sizing analysis by generating customer personas and their predicted lifetime value with your brand.

### Deploy likely buyer scores to your analysis tools

Finally, deploy your likely buyer predictions back to your stack, via CSV file for versatility, or right back into your database or data warehouse for analysis in your BI tools. When deploying, filter to whatever aggregation level makes the most sense for your strategy: county, state, city, or metro. In this example, perhaps choosing state will reveal that the driest states in the country have the highest propensity to buy individuals.

![Image of Faraday pipeline](/images/blog/market-sizing-deployment.png)

With the predictive approach to bottom up market sizing, you have access to likely buyer scores anywhere in your stack in just a few clicks, providing insight into what geographic locations your company has the most high-intent buyers in so that your market size is more than just a rough estimate–it’s actionable and data-backed.

**Ready to try it out? [Create a free account](https://faraday.ai/signup).**

What is bottom up market sizing? What are its benefits? How can B2C brands accurately analyze markets without investing vast resources?

Bottom up market sizing: how to calculate market size using predictive analytics

If you’re a marketer in the finance industry, you know you have some heavy regulations to work with on [Google](https://support.google.com/adspolicy/answer/9917652?hl=en), [Facebook](https://www.facebook.com/business/help/2408667629202904), and other platforms, where consumer protections are put in place to help prevent discrimination. These regulations can make it difficult to reach your ideal customers when targeting new audiences, which likely means that you’re spending more than you need to on inefficient acquisition.

Complying with financial advertising regulations doesn’t mean you need to throw in the towel and accept worse performance. You can respect your customers’ privacy and be smart with your marketing strategy at the same time–you just need to think out of the box. That’s where built-in [third-party consumer data](https://faraday.ai/features/data) and [responsible AI](https://faraday.ai/features/responsible-ai) come in.

The data you have on your customers might be good, but purchases, adoptions, and clickstreams aren’t robust enough to give you a true leg up on your competitors, nor is it enough to ensure you’re not unknowingly discriminating against protected classes. Third-party data, when unified with your first-party customer data, can give you a 360º view of your customers, leads, and virtually any other group of people your business cares about.

With this breadth and depth of data in your pocket, you can then use AI to optimize your ad spend by surfacing your most likely-to-buy customers while remaining compliant by:

- Excluding protected classes from your predictions to ensure no unintended bias
- Targeting at the right aggregation level for your industry (counties, rather than zip codes, for example)
- Removing both human bias and error from the equation by using AI
- Monitoring and minimizing bias in predictive models through bias reporting

Faraday enables you to do all of these things–without a data science degree or programming knowledge–and in under 15 minutes. Here’s how.

## Connect your customer data to combine it with third-party attributes

To start, you plug in last quarter’s adoption data in by [mapping the data in Datasets](https://faraday.ai/docs/cohorts/customers_from_hosted_csv#option-a-each-row-represents-a-transaction-event).

![Image of identity set definition](/images/blog/finance-identity-set-definition.png)

Next, you use this new data to define a cohort–a group of people you’re interested in–of these new adopters.

![Image of cohort definition](/images/blog/finance-cohort-definition-booked-date.png)

## Build a predictive outcome to identify likely buyers

Now it’s time to predict which leads are most likely to convert. To do that, you’ll define an outcome using your _customers_ as your attainment cohort because that’s your end-goal–you want people to become customers and enter your _customers_ cohort.

Faraday makes this easy with step-by-step guides for [connecting to Google Ads](https://faraday.ai/docs/how-to/connections/google-ads) and setting up a [likelihood-to-convert prediction](https://faraday.ai/docs/how-to/predictions/likelihood-to-convert). Once your outcome is defined, Faraday will generate scores that tell you which individuals or locations are most likely to become customers.

![Image of new outcome screen](/images/blog/finance-new-outcome.png)

## Exclude protected classes with trait blocking

When describing your outcome, you choose _trait blocking_ to remove protected class traits. This ensures you’re not only complying with financial fair lending regulations, but are also using AI responsibly.

![Image of trait blocking checkbox](/images/blog/finance-trait-blocking.png)

## Choose the right aggregation level for your likely buyer predictions

Once your predictions are ready, set your aggregation level based on how high you’d like to zoom out of a location–such as by _county_ to remain compliant with Google’s restrictions on zip code targeting.

![Image of pipeline aggregation level](/images/blog/finance-pipeline-aggregation-level.png)

## Push high-scoring locations to your ad platform

With your highest-scoring locations in-hand, you plug them into your Google Ads campaign:

![Image of Google Ads campaign location upload](/images/blog/finance-google-ads-campaign-locations.png)

And your Facebook campaign:

![Image of Facebook campaign location upload](/images/blog/finance-facebook-campaign-locations.png)

That’s it! Your best-bet locations are plugged into Google and Facebook, and fewer than 15 minutes were spent making your predictions.

## Real results from a customer in the financial services space

A Faraday client in finance recently followed these exact steps to kick off a new acquisition campaign for an underperforming insurance plan, and they’re expecting a **3.2x lift in conversion rate** in the top 10% of individuals alone.

![Image of Faraday lift table](/images/blog/finance-lift-table.png)

Next up for them is adding direct mail into the mix, but the possibilities are endless.

[Sign up](https://faraday.ai/signup) free today.

In finance, remaining compliant to fair lending regulations doesn’t mean you need to accept worse acquisition performance. Learn how to use AI and third-party data to optimize ad spend by surfacing  your most likely-to-buy customers in under 15 minutes, while ensuring you're compliant to any relevant regulations.

How to optimize ad targeting and comply with financial advertising regulations

Predictions are complex, and consumer predictions for marketing aren’t an exception. In many cases, you might find yourself given the green or red light to go after a lead, prevent a churn, or approach a customer in another way that’s specific to a business goal your org has. Having access to that knowledge is great, but it’s a black box–simply knowing whether or not a lead will bite isn’t enough if you want to truly understand your customers and the actions they take.

In Faraday, every predictive model you create for your business is scored based on how well you can expect the model to perform, and includes a full technical report describing how the model was built. While the full technical report includes a swathe of info for data science-minded users, for the sake of this guide we’re going to take a look at how **features of importance** and **directionality** impact a sample **lead conversion** model, which you can find in every outcome and recommender.

## Features of importance

![Image of features of importance in a Faraday outcome](/images/blog/product/feature-importance.png)

In each outcome and recommender, you’ll find a **data** section describing the top features of importance for that model broken down into tenures, or how long an individual existed within the eligibility cohort before entering the attainment cohort. The features that had the most impact in creating the model are listed in the chart, sorted by the level of impact. In the above image, you can see that features such as high ticket home decor quintile, activity, total online dollars quintile, and household income had a major impact on the model. This means that when thinking about who you’re targeting with these predictions, you can tailor creative and messaging appropriately–but that still feels a little broad. Let’s take a look at directionality to see exactly how these features of importance break down.

## Directionality

![Image of directionality in model report](/images/blog/product/feature-directionality.png)

When you take a deeper look at a feature of importance in feature directionality, you’ll find it broken down into ranges if the feature is numeric, or selections if not. In the above example, green indicates positive, and red negative. For **household income** (HHI), it’s clear that consumers with an HHI of over 75k rank higher on average (green), whereas all HHI ranges lower than 75k rank lower (red). For **shopping styles**, _luxury offline_ and _prefers online_ shoppers rank higher, while _retail therapy_ shoppers rank lower.

That’s cool and all, but what’s the takeaway?

In short, these breakdowns tell you that in this lead conversion model, the people that are most likely to convert are largely making more than 75k/yr, and prefer to shop online unless it's for luxury items. Using this info, your marketing to them can confidently personalize toward the positive features so that your content is relevant.

Learn how to analyze your predictive models to deeper understand your customer base.

Faraday model reports: Understanding feature importance and directionality

While data security has always been an important part of doing business, in recent years it’s become an increasingly urgent consideration. Public scandals, like those of [Facebook and Cambridge Analytica](https://www.wired.com/story/cambridge-analytica-facebook-privacy-awakening/), and growing government regulations, to the tune of GDPR and CCPA, have consumers on high alert about how their data is being used.

If you find yourself among the many brands looking for ways to build trust with your audience, you’ll need to consider not only how you manage data security at your organization, but also how you communicate with your clients. Transparency is an important part of security—clients and collaborators alike should be able to quickly and easily understand what you’re doing to protect their data. The more effectively you achieve that, the more trust you’ll be able to build.

Read on to explore the three most common questions our customers ask us (and might ask you!) about data security, and how we answer them. Feel free to use these as a blueprint to guide your own practices, though the particulars will vary from industry to industry.

## “How is your company audited for data security?”

_Our answer: Check out our [SOC 2 audit](/security)!_

If you offer a cloud-based product or service, and want potential customers and collaborators to see you as a trusted partner, a clean SOC 2 audit should be your gold standard. SOC 2 audits are exhaustive reviews performed by certified public accountants to ensure that SaaS companies meet strict security requirements and provide a safe operating environment for managing sensitive data. In order to retain a clean SOC 2 audit, companies must renew their audit annually.

The existence of SOC 2 as a process reassures clients, but it also benefits the company. As Seamus Abshere, Faraday co-founder and Chief Technology Officer, puts it, “SOC 2 audits provide a chain of trust going from the client all the way to Faraday and to our vendors (like cloud hosting providers). At every level, auditors are involved to ensure that policies and procedures are followed.”

## “How is my data transferred and stored?”

_Our answer: All data is stored with trusted cloud vendors (Amazon, Google Cloud), encrypted in transit and at rest, and access is controlled using multifactor authentication._

Simply put, encryption is a no-brainer. When customers share data with brands, they’re trusting them to protect it from bad actors, and encryption is the standard practice for doing so. “At this point, using unencrypted data is like a surgeon not washing their hands before surgery,” asserts Abshere. “If you are not encrypting everything, it's just malpractice.”

It's standard practice for clients and vendors to ask about encryption, but at this point, the answer should always be "yes."

## “What data do you actually use, and how?”

_Our answer: We live by the _[_“If we don’t need it, we don’t collect, store, or transmit it”_](https://faraday.ai/blog/using-ethical-data/)_ mentality._

It’s essential for brands to understand what data they have access to in order for them to put the proper safeguards in place. Here at Faraday, where we handle the data of hundreds of leading brands, we’re extremely clear with clients about what we use—and what we don’t.

**“We don’t want your account numbers, social security numbers, health records — any of that,” Abshere says.  “All we need are known identifiers (like billing information) and transaction information. We do not require or accept account numbers, so we don't affect our client's PCI DSS scope.”** In our case, we leverage this personally identifiable information (PII) to match consumers with our identity graph and train machine learning algorithms to make predictions about a brand’s current or potential customers.

Another important step is to prevent any co-mingling of your clients’ data, particularly if your clients are competitors. At Faraday, we understand this is a serious concern, and we’re careful to prevent it. “Data is logically separated, and everyone’s predictions are based only on their own data plus Faraday's identity graph,” explains Abshere. "Mixing client data would be a huge risk."

## Security is always a priority

No matter your role at a given company, you’re always a steward of your customers’ data. Various teams, from security to engineering to marketing, may operate on very different datasets and roadmaps, but it’s crucial that everyone contributes to a trusted relationship with clients and collaborators, working together to efficiently and transparently implement best practices.

Transparency is an important part of your security practices—clients and collaborators alike should be able to quickly and easily understand what you’re doing to protect their data.

Building trust: How brands can answer common data security questions

With summer in full swing, Faraday’s product team and developers have been working hard to create some incredibly exciting updates to the Faraday platform. The _why_ behind these changes is rooted in two of our core product pillars:

1. **AI for all**. The benefits of machine learning should not be reserved solely for the Fortune 500 or those with advanced degrees. The promise of easy AI, scalable cloud-based infrastructure, and a good UI should unlock the benefits of consumer behavior predictions for all brands.
2. **Making data science fun**. Using Faraday should be intuitive and easy. You should feel empowered to ideate, experiment, and deploy, all on your own.

We're thrilled to announce that as of today, four major updates are live. In this blog, we'll walk you through how each one works—here's what we'll cover:

- **Build your own predictions in Outcomes**: Describe the business outcomes you want predictions for, and Faraday will build and maintain predictive models for that outcome, with models getting more accurate over time as they learn from your data.
- **Deploy your predictions with ease in Pipelines**: Choose your likely-to-buy outcomes, your personas, and more; pick a specific population of customers; and target a destination for your deployment, like your data warehouse, cloud bucket, CRM, ESP, etc.
- **Plug into Faraday with Connections**: Plugging in your data is now a simpler process with fewer steps required. Plus, Connections paves the way for a future update where we'll implement true bi-directionality for your data and predictions in the Faraday platform.
- **Create more powerful Cohorts**: Cohort building now features the ability to specify certain trait requirements from your data or the Faraday Identity Graph in order for a customer to enter a cohort, adding an extra layer of depth to your predictions.

Let’s dive in.

## Build your own predictions in Outcomes

Back in December, we released an update that enabled Faraday users to build their own predictive personas within a cohort of their choosing based on various attributes from the [Faraday Identity Graph](/features/data).

That update had been the first major in-platform step toward user autonomy, and, more importantly, toward making AI accessible for all. Continuing on that path, Faraday users can now access an additional self-serve interface, this one for defining propensity (likelihood) predictions: **Outcomes**.

Outcomes allows you to easily describe the business outcomes you’re trying to predict in just a few clicks. Likely to buy? Sure. Likely to churn? You bet. Likely to buy _again_? That, too. Behind the scenes, Faraday takes over from here, applying dozens of strategies to build a range of candidate models, then selecting the one that most accurately predicts your outcome. Over time, Faraday automatically builds and rebuilds progressively more accurate models as you accumulate more data.

Our new Outcomes interface puts the power of experimentation in your hands, allowing you to easily try out different outcomes and view how effective the model will be across your use cases—all without needing a data scientist at your shoulder.

## Deploy your predictions with ease in Pipelines

Once you’ve defined your own prediction objectives, make them actionable with **Pipelines**.

Pipelines completely streamlines the prediction deployment process in Faraday. Previously, you may have used some combination of Reach, Monitor, and Inform to deploy predictions on-demand, on a scheduled basis, and in real-time via API. These three products have been unified into Pipelines for a more accessible and intuitive user experience.

Jump into Pipelines and choose your:

- **Payload:** a combination of your desired propensity outcomes, your personas, and more.
- **Population:** the people you want the predictions on.
- **Targets:** where you want your predictions deployed, including your data warehouse, cloud bucket, or anywhere else in your stack (CRM, ESP, etc).

Faraday takes over from here, keeping your predictions constantly up to date by refreshing scores and persona assignments when new data comes in, ensuring you’re never left wrong-footed when delivering personalized experiences to your most likely-to-buy customers.

With Pipelines, for the first time ever, you’re able to both describe and deploy your own predictions completely autonomously—without a data science degree.

## Plug into Faraday with Connections

Starting today, you’ll notice the Sources and Destinations areas of the Faraday platform have been combined into a single **Connections** view.

With Connections, you’re able to plug Faraday right into your data warehouse or cloud bucket by entering your location-specific details. Once done, it’ll be ready for selection as a deployment target in Pipelines. Dropping the number of setting screens required to complete your end-to-end connection from two to one should save you some clicks, and get your first predictions deployed that much faster.

Keep an eye out for more news here, because Connections is a major building block for a future update that will enable every Connection to be bi-directional—your predictions, wherever you want them, whenever you want them.

## Create more powerful Cohorts

Previously, **Cohorts** in Faraday were defined by event streams, or an event that each person in a cohort has experienced. This included the type of event (such as an order), when the person experienced the event (all orders, last X days, etc), the frequency of the event (such as number of times an order was placed), and finally, the value of the order.

We introduced the Segments console last year to allow users to define a group of people based solely on the Faraday Identity Graph (FIG)-based traits, like geography or income. While this was a great step for offering flexibility in terms of the types of population you could define in the Faraday platform, it was cumbersome having to navigate through both Cohorts and Segments to do so.

With today’s update, Segments and its functionality has been folded into Cohorts. Now, in one console, users can define their cohorts by selecting specific traits from either their customer data, or exclusively from FIG. Event streams are no longer required to build a cohort.

Prior to today, your "Recent bulk purchasers" cohort might’ve been simply based on the number of orders in the last 30 days. You can now refine this cohort further to say "Recent bulk purchasers in CA" or "Recent bulk purchasers with income >\$100k." On the flip side, you might also build a cohort without an event stream, such as "Dog owners in CA with an income >\$100k."

We hope that by allowing you to define your preferred population of people—whether that be based on an event stream like orders, FIG traits, or a combination of the two—gives you another building block upon which you can easily experiment with predictions through Outcomes, Personas, and Pipelines.

## What's next?

Coming up, we’ll be revamping data ingress to make it easier to map your data into Faraday. We see this as a critical step towards democratizing data science and empower you to create and maintain a powerful predictive layer atop your martech stack.

---

\*Explore our [product documentation](/docs/) or [talk to sales](/talk-to-sales) to learn more.

Introducing four major updates that empower users to build and deploy their own predictions in the Faraday platform—no data science degree required.

August 2022 product updates: Outcomes, Pipelines, Connections, Cohorts

Headlines abound on the [opportunities](https://www.techrepublic.com/article/companies-are-doubling-down-on-artificial-intelligence-and-machine-learning-due-to-pandemic/) and [dangers](https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing) of machine learning. Now armed with [plenty](https://www.propublica.org/article/breaking-the-black-box-when-algorithms-decide-what-you-pay) of [examples](https://www.nytimes.com/2019/08/20/upshot/housing-discrimination-algorithms-hud.html), the Data Science community is at least aware that our data lakes and algorithms can make life more difficult for real people if not properly used. But the root problem can be summed up in pretty simple terms:

> An algorithm contains the biases of its builder.

Examples of bias include:

- **Reporting bias** _(at the source)_ — Maybe the folks who filled out your online survey were mostly young, male, and computer-literate.
- **Implicit bias** _(the call is coming from inside the building)_ — Perhaps your image of an ideal job candidate fits into a mental archetype of which you're only semi-aware, and that image excludes those who don't match the archetype.

At Faraday, we have a handful of approaches we use to minimize these effects.

## Data sourcing

It's possible to get your hands on a pretty broad range of data points these days. Everything from the basics of home value, age, and income to the more esoteric pet ownership and liklihood to buy high-end men's apparel. **Faraday actively excludes data points relating to protected classes**. Race, ethnicity, primary language, and religion are excised from our database entirely and therefore excluded from our predictive models and audience definitions.

Of course, seemingly-harmless demographic variables can often act as proxies for these features, but our stance begins with the most effective levers we have available to pull. And further on in the pipeline we do check _results_ against features like race, but we do it at an aggregated level using standardized census data (more on that below).

## Model priors

We investigate the examples in our training data before they become machine-actionable profiles. In particular, we look at how well they're balanced by age, gender, and geography against the population at large. If there's a significant skew versus the baseline, we know there's a chance the resulting predictive model will show bias, so we rework the inputs.

## Model predictions

When a predictive model is fully-built and ready to be put into action, we typically assign a predictive score to everyone in the country as a matter of efficiency. But this also allows us to check whole populations against statistics that reflect the inclusion or exclusion of protected groups at a census block group level.

If we see - for example - that the lowest scores are concentrated in the block groups with the highest proportions of particular racial or ethnic groups, we'll know that _some_ bias has made it into the model, and we'll revise it.

The bottom line is that these strategies are only a start, and **the best defense against unfair algorithms is multi-pronged and in for the long haul.** If you have questions about how your own applications might be veering away from fairness, I suggest checking out:

- Google's [Marketers' guide to ML fairness](https://www.thinkwithgoogle.com/feature/ml-fairness-for-marketers) or [ML bias crash course](https://developers.google.com/machine-learning/crash-course/fairness/video-lecture)
- ["Abolish big data"](https://d4bl.org/videos/19-abolish-big-data) from [Data for Black Lives](https://d4bl.org/)
- The machine bias case studies at [diversity.ai](http://diversity.ai/#case)

How to reduce the effects of implicit bias in machine learning, from data sourcing to model predictions.

Product