- Business at Scale
- Posts
- Experimentation as a Service Case Study
Experimentation as a Service Case Study
How we run our experimentation framework at DevriX
As PPC, SEO and BDR efforts started to tank last year, we took on a high-ticket client in the solar panels space and scaled it to to 300% ROAS in just a few weeks.
This shaped a consistent (and pretty resilient) framework that we since applied to 8 other companies in our portfolio - and counting.
Whether you’re running a SaaS business, a productized service, or a traditional service business, this mechanism has worked really well this year.
🧑🏫 Here's a breakdown of the DevriX thought process:
1. Campaign debrief and assessing the current strategy
Studying the old process and identifying loopholes. Inefficiencies in paid campaigns, copy, creatives, messaging, landing pages.
Good - we can work these out.
Specific areas we want to explore during this due diligence phase:
Target audience and objectives. What does the current buyer persona look like? Are we effectively targeting the right customer profile? Is messaging/channel alignment in place?
Market research and competitor analysis. Is our channel selection compatible with what the market targets? Is everyone else running Google/Meta ads? What about SEO, reddit, events, podcasts?
Content/creative analysis. Is the quality and messaging aligned with the customer expectations? Most of the time it’s lower (shouldn’t be), but in some cases it needs to look traditional and old-school because of the industry.
Data-driven tracking. All pixels, goals, campaigns, GTMs in place? Heatmaps? Screencast recordings?
2. Interviewing ICPs
A combined effort with us calling 15 leads for 20 minutes each to capture the needs and pain points.
Talking directly to potential customers (and hearing their feedback) gives us valuable insights. We focus on their challenges, what motivates them, and how they make decisions.
This helps us fine-tune our creatives and copy, tailor our offerings, and create marketing campaigns that truly connect.
We may uncover some surprises – maybe unmet needs or new opportunities for growth.
3. Hypothesis
A quick action plan on what to test in a priority order the first 3 months, with a launch plan in 2 weeks.
Gif by kevlavery on Giphy
We pinpoint the most promising areas for improvement and create a clear testing roadmap. This will help us quickly validate our assumptions, identify winning strategies, and start seeing results fast.
4. Landing pages
New landing pages with different copy structure: pain points, logical vs. emotional, CTAs (email vs. phone vs. live chat), look-and-feel tests.
Testing various elements, we identify the most effective messaging, design, and calls to action for the specific audience we focus on. This leads to higher conversion rates and results in more leads or sales from the current campaigns and the new experiments we launch.
5. A/B tests
Starting with 3 pages and testing after a minimum of 50 (ideally 100) actions. The winners get branched out into new landers - testing copy/CTA variations or copywriting framework structure.
What works gets tested against variants.
Winning pages will serve as the foundation for new landing page variations, testing elements like copy, CTAs, and overall structure. We'll continue to iterate, pitting successful elements against new variants to uncover the optimal combinations.
This data-driven approach ensures we're constantly improving and maximizing results.
6. Ad campaign optimizations
We'll optimize your ad campaigns by running them across 3 different channels, each with multiple campaigns featuring copy and creative variations tailored to specific ICPs.
This multi-pronged approach allows us to test different messages and visuals to see what resonates best with each target audience segment.
Keeping an eye on performance data and running bi-weekly reviews, we'll identify winning combinations and continually refine our campaigns to maximize results.
Test fast, fail fast, iterate on the winners
The recipe is simple, what matters is prioritizing the RIGHT tests first and iterating VERY, VERY FAST. 🏎️
Instead of building complex pages and campaigns, we got some launched in days and pivoting weekly, tracking conversions and gathering input from BDRs on conversions and feedback on calls and demos.
In 2024, you can't rely on historical data anymore. Purchasing behavior and priorities are completely different. Getting feedback quickly and acting upon is how you scale and stay afloat.
Our experimentation EaaS retainers span from $5K to $20K monthly on triaging these for B2Bs and are one of the three key services in high demand since September. GTM implementation and SEO come next, with platform engineering (R&D) in the fourth spot.