It’s tempting to think of ROI analysis as a simple math problem. Plug in the numbers, calculate the return, and call it a day.

But the truly specialized secret to great ROI isn’t in the arithmetic; it’s the methodology and strategy that underlie and inform the analysis. It’s knowing *where* to look, *what* to measure, and *how* to choose the right approach for the situation. 

At Faraday, we don’t start our ROI analyses by creating an equation. We start by picking the right tool from the toolbox. Different tools unlock different kinds of answers. In our last blog, we made the case that [ROI isn’t a scorecard—it’s a strategy](/blog/roi-isnt-a-scorecard-its-a-strategy). This post digs into the next layer: how we measure ROI in a way that actually reflects the individual strategic value of your campaigns. 

And that starts by understanding which is the right method for the job.

## ROI isn't just about math, it’s about methodology

The actual math behind ROI analysis is pretty simple. You’re mostly dealing with addition, subtraction, multiplication, and division. In this case, complexity comes not from the operations themselves, but how we choose to apply them.

[![Comedic GIF showing arithmetic operations](/images/blog/mathishard.gif)](/images/blog/mathishard.gif)

An ROI specialist knows which data points matter, how to structure the comparison, and what kind of baseline you’re using to judge performance. In other words, the real work isn’t in crunching the numbers; it’s in designing the analysis.

And importantly, different ROI methods exist for a reason. Each one comes with trade-offs, and the right choice depends on your goals, your timeline, and the level of precision you need.

## Common "tools" in the ROI toolkit

There’s no single “best” way to measure impact. It depends on the shape of the campaign, the outcome you care about, and how precise your measurement needs to be.

Different methods are used depending on what you're measuring, what kind of attribution you need, and what trade-offs you’re willing to accept.

#### 1\. Holdout testing (A/B testing)

**How it works:** A holdout test randomly splits your audience into two groups, one that receives the intervention, and one that doesn’t. Then simply compare the outcomes.

**Advantages:**

* Delivers highly reliable attribution. You can isolate the effect of the intervention with minimal noise.
* Especially useful when you need clear proof of impact—for example, before rolling out a new campaign strategy.

**Disadvantages:**

* You’re intentionally withholding the (potentially better) treatment from part of your audience, which can mean sacrificing short-term gains.
* Longer-term metrics like LTV can require months or years of tracking, which slows down decision-making.
* Can be difficult to set up for more integrated processes.

**How this might look in practice:** A home services company wants to evaluate the impact of a new predictive lead scoring model. To measure its effectiveness, they randomly split inbound leads into two groups. The test group receives scores from Faraday, and the sales team uses those scores to prioritize outreach. The control group is handled as usual, without scores. After 30 days, the test group shows a 15% higher conversion rate. Because the only variable that changed was the introduction of scoring, the team can confidently attribute the improvement to Faraday’s model—and deploy it more broadly.

#### 2\. Longitudinal studies

**How it works:** These studies measure a key performance indicator (KPI) before and after an intervention, and attribute any change to the strategy.

**Advantages:**

* Easier to implement when the change affects a system or workflow that can’t easily be split, like internal operations.   
* No need to create and manage control groups.  
* Can continue tracking performance indefinitely without missing out on performance benefits.

**Disadvantages:**

* Harder to separate the signal from the noise. External factors like seasonality, sales events, or even weather can cloud the results.

**How this might look in practice:** Picture a credit union using Faraday to launch a predictive direct mail campaign aimed at promoting HELOCs. Since they want to reach the entire audience, they don't create a holdout group. Instead, they measure HELOC applications for eight weeks following the campaign and compare that to the eight weeks prior. Applications increase by 25%, and with no other major marketing efforts running in that window, the credit union sees a strong directional signal that the campaign worked.

#### 3\. Relative performance improvement studies

**How it works:** This method builds on a longitudinal approach, but instead of simply comparing “before” and “after” for one group, you compare the *change* in your test group to the *change* in a baseline group that wasn’t affected by the intervention.

In other words: both groups experience whatever else is happening in the world (seasonality, ad spend changes, etc.), but only the test group is affected by the treatment. By comparing the test group’s performance to the baseline group, you subtract out the “background noise” and can isolate the true impact.

**Advantages:**

* More resilient to outside variables. Since both groups are exposed to the same external forces, the comparison helps filter out unrelated effects.  
* Doesn’t require randomization—useful when you can’t run a traditional holdout test.

**Disadvantages:**

* Can be harder to explain to stakeholders unfamiliar with the method.  
* Requires a valid control group that closely tracks with the test group’s normal performance. If your baseline group isn’t steady or comparable, the whole thing falls apart.

**How this might look in practice:** Imagine a regional furniture retailer running a new predictive direct mail campaign in their Midwest region, while using their Northeast region as a comparison. Both regions are similar in customer base and seasonality, but only the Midwest receives the new campaign. During the test window, Midwest sales rise by 12%, while Northeast sales only increase by 2%. The 10-point gap is attributed to the campaign, with the unaffected region helping control for external factors like seasonal shopping trends or macroeconomic changes.

## Choosing the right tool for the job

Not every situation calls for the same approach, sometimes a "perfect" test isn’t practical or cost-effective. The best ROI analysts know how to match the method to the moment, balancing rigor with speed, and clarity with real-world constraints.

Here’s a quick cheat sheet to guide your choice:

* Use holdout tests when you need the cleanest attribution possible and you have time to run a structured experiment—ideal for testing new acquisition channels or proving the value of a bold strategy shift.

* Use longitudinal studies when A/B testing isn’t practical—like when rolling out changes to workflows, routing logic, or systems that touch your entire organization.

* Use relative performance studies when you can’t randomize, but you *can* find a comparable group or region to act as a baseline—helping you filter out external noise and isolate the true impact.

Choosing the right approach isn’t about checking a box—it’s about working with what you’ve got and building an analysis that supports better decisions.

## ROI is an art of trade-offs

Every method comes with trade-offs: between speed and precision, between statistical rigor and operational feasibility, between what you *want* to know and what you *can* realistically measure.

That’s why ROI analysis isn’t a checklist, but a craft. It requires judgment. True expertise means knowing when “good enough” is actually better than “perfect,” because it lets you act faster, test more often, and make decisions that still move the needle.

At Faraday, we don’t believe in one-size-fits-all measurement. Instead, we tailor ROI analysis to fit the strategy, timeline, and success metrics of each client. Whether that means running a formal holdout test, analyzing past performance, or finding a creative workaround, we help our clients make smart decisions grounded in evidence—not just instincts.

## Conclusion

At the end of the day, calculating ROI isn’t just a mechanical process, it’s a strategic one. The numbers only matter if they’re telling you something useful. And that usefulness depends on choosing the right tool, the right comparison, and the right trade-offs for your business.

Faraday’s approach is built around that idea. We help our clients go beyond surface-level metrics to get a clearer, more actionable picture of what’s working, what’s not, and what to do next. Whether you’re trying to justify spending, test a new channel, or make smarter bets for the future, we’ll help you measure what matters—on your terms.

Want to see how predictive insights and tailored ROI reporting can work together? [Let’s talk](/talk-to-sales).

Faraday’s strategic approach to ROI uses tailored analysis methods—like holdout testing, longitudinal studies, and relative comparisons—to deliver insights that actually reflect each client’s unique goals, constraints, and campaign impact.

Behind the numbers: The methodologies Faraday uses to turn ROI into value

In our last blog, we explored why ROI analysis at Faraday isn’t just about proving value after the fact—it’s also about continually uncovering new opportunities. 

We shared what that looks like at a high level. But this isn’t just a philosophy expressed in our marketing; it’s how we work every day. So today, we’re sharing a story from the \~real world\~ that shows just how much insight you can unlock through our comprehensive ROI evaluation process.

It also drives home an important point: these evaluations can only achieve their full value if you actually engage with them. The top-line numbers might say “Yep, it’s going great,” but just because that’s good news doesn’t mean it reflects the whole story. 

Sure, it’s going great, but how great, and where is there room for improvement? Which parts of the campaign are driving the most value? The least value? Do we need to revise any major assumptions to be more accurate? 

Indeed when it comes to ROI analyses like these, digging deeper is where the real opportunity lies.

## A deeper dive reveals something unexpected

Hi, ROI extraordinaire Chris Waite here. After I came on full-time at the start of the year as Faraday’s Head of Customer Analytics, I kicked off ROI initiatives with several of our existing customers. The goal was simple: move beyond one-time reports and start treating ROI as an ongoing conversation—one that actively drives value, not just records it.

With one financial services client, we began with my usual process: aligning on key assumptions—value per conversion, attribution logic, unit costs—and setting up monthly check-ins to track progress and dig into results over time.

And the returns looked great. Performance was strong, and the model was holding up. But during one of those check-ins a couple of months ago, someone on their team made an offhand remark that caught our attention: some of their hardest-to-convert prospects—sometimes rejected due to their low predicted conversion rates—actually had the potential to generate the *most* value if they did convert.

Our interest was piqued, so we ran the numbers.

We analyzed the relationship between Faraday’s prediction scores and the average conversion revenue. Sure enough, there was a clear pattern: while certain low-scoring groups converted less frequently, they delivered more value when they did.

The selection process didn’t need a full overhaul—but it did need a refinement. We brought the results back to the client, and together we incorporated these insights into their approach. By adding an additional variable into our selection process, we could proactively identify the majority of these high-value individuals and ensure their inclusion in the campaign. Once updated, the campaign began to improve—conversion rates dipped slightly (by design), but total revenue went up.

This is a perfect example of how our ROI reporting process uncovers new and unexpected insights that drive value. In this case, we used it to fine-tune suppression logic and reframe scoring thresholds—without a full process overhaul. The outcome was a smarter, more mission-aligned strategy that helped the client prioritize not just likely conversions, but the ones with the greatest potential overall value.

And through that process, we uncovered something more. These high-impact conversions weren’t just good for business—they were cultural wins. Inside the company, signing one of these clients—often individuals carrying significant debt—was seen as a major moment. It represented the core of their mission: to support people who need it most. The whole team felt it.

So yes, ROI is about financial returns. But sometimes, it's also about morale, alignment, and purpose. In this case, their investment in Faraday didn’t just lead to better outcomes on paper—it reinforced the values that drive their business forward.

## From revenue recap to campaign shift

This paradigm shift is exactly Faraday’s goal when we present our clients with our ROI reports. We can talk about them driving value as much as we want but they drive the most value with buy-in and funding from the teams that use them.

In this case, thanks to the deeper ROI dive, the client was able to:

* Identify underutilized customer types  
* Refine their lead suppression strategy  
* Shift budget allocations across customer segments  
* Drive more high-impact conversions

None of that was the original intent. But that’s the point—when ROI becomes an ongoing process, not just a retroactive check-in, it drives value in real time.

## Why Faraday treats ROI like strategy

We’re not sharing this story to pat ourselves on the back—we’re sharing it because it highlights something bigger: most companies treat ROI as a postmortem. We treat it as a feedback loop.

At Faraday, ROI modeling isn’t a final deliverable—it’s built into every engagement from day one. We co-develop success metrics with our clients, revisit assumptions as new information emerges, and treat every report as a living document meant to drive change in real time. Because when ROI is treated as a collaborative, evolving process—not a static snapshot—it doesn’t just validate past performance. It unlocks new value, sharpens ongoing strategy, and turns every insight into an opportunity for future impact.

## Coming up next

In our next post, we’ll pull back the curtain on how we build some of our ROI models at Faraday—what goes into them, what we measure (and why), and how we handle the gray areas most companies avoid.

Because the better you understand your own ROI, the greater your ability to power new growth.

A routine ROI check uncovered a surprising insight that helped a financial services client shift strategy, boost revenue, and realign with their core mission.

This financial services brand’s ROI assessment empowered a whole new strategy

If you frequent our blog (firstly, thank you), you already know that Faraday loves data, transparency, and [explainability](https://faraday.ai/blog/faraday-score-explainability). These ideas are at the center of our modeling and predictive agents—but it doesn't stop there. That emphasis permeates every aspect of our business.

So when we talk about return on investment (ROI), we’re not just talking about showing a nice number at the end of your campaign. We’re talking about helping you actually understand how Faraday is generating value for your business—not just the number, but where it came from, what influenced it, and how it can best inform your next move.

That’s actually why Faraday hired me.

Hi—I’m [Chris Waite](https://www.linkedin.com/in/christopher-waite-b3548015/), and I’m a data nerd who specializes in evaluating ROI for all our clients. My role is to make sure the value Faraday drives for your business is accurate, understandable, and, most importantly, guiding you towards further insights.

At Faraday, we don’t think of ROI as a final number you get at the end of a campaign. We treat it as a living, evolving data point—a signal that tells a story over time. And just like with any good story, you should be able to follow the plot. My job is to make sure you can.

ROI isn’t only about justifying past spend—it’s about helping you make better, faster, and more confident decisions moving forward. Knowing how things went is good… But knowing where to go next? Even better.

## ROI isn’t a score—it’s a strategy

A lot of companies say they help you track ROI—but what that actually looks like in practice can vary *a lot*.

Some treat ROI like a final grade. You get a slide at the end of a campaign with a nice round number and a chart pointing up and to the right. Others hand you a projection during onboarding that’s... let’s just say, aspirational. It’s a best-case scenario, not a reflection of what’s actually happening in your campaign.

At Faraday, we think that misses the point.

We treat ROI as a process, not a product. The real value of ROI isn’t just in a number at the end—it’s in what you learn along the way.

Done right, ROI helps you:

* Identify what’s working and what’s not  
* Catch issues and optimize while the campaign is live  
* Challenge assumptions you didn’t even realize were there  
* Build confidence in your strategy—not just in hindsight, but in real time

Because in a landscape where budgets are under pressure and every dollar has to count, ROI isn’t just about proving value. It’s about unlocking more of it.

## What to expect from Faraday’s ROI assessment process

Faraday takes ROI personally. Not just because it proves our value, but because—done right—it helps you uncover where your growth really comes from. And that starts long before the first campaign goes live.

When you work with Faraday, ROI isn’t an afterthought or a once-a-quarter deliverable. It’s built into the process from day one. Here’s what that looks like in practice:

#### We start measuring early

ROI modeling is part of onboarding—not something we tack on at the end. From the beginning, we’re thinking about what success looks like and how we’ll measure it.

#### We define assumptions together

ROI analysis is only as good as the logic behind it. We work closely with clients to make sure the “what if” scenarios we’re comparing against are grounded in data, reasonable, and transparent.

#### We track and adjust in real time

Our goal isn’t just to see if something worked—it’s to learn what’s working *while* it’s happening, so we can adapt. Think of ROI as a feedback loop, not a final grade.

#### We keep you involved

You won’t just get a chart in a deck and a polite “any questions?” We walk through the analysis with you, answer your questions, and collaborate on what the numbers actually mean for your business.

We know not every client comes in with a fully built-out internal analytics team. That’s okay. Whether you’ve got a room full of data analysts or none at all, we meet you where you are. Our job is to make ROI not just accurate, but clear and actionable—so you can make smarter, faster decisions with confidence.

## So what’s next

You might’ve noticed this post is the first in a new series covering all aspects of how Faraday approaches ROI—not just as a reporting tool, but as a driver of real, measurable value.

So what can you expect?

We’ll be diving into real client examples where ROI analysis didn’t just prove our impact—it helped uncover new opportunities and improve campaign performance. We’ll also get into the weeds with technical posts that break down the actual methods I use to assess ROI, from modeling assumptions to attribution logic and more.

In our next post, we’ll share a case study that shows how one client’s ROI review sparked new strategic decisions and unexpected insight.

Because if you’re investing in growth, you deserve more than a black-box metric. You deserve to know what’s working, why it’s working, and how to keep improving.

Let’s make that the standard. We can’t wait to see you here for the next one.

Faraday treats ROI not as a final score but as an ongoing strategy—helping clients understand, optimize, and expand on the value generating strategies we're driving throughout the entire campaign lifecycle.