Bias-Debt Framework: Marketing Bias Audit Template & Instructions

The Bias-Debt Audit Template is a comprehensive Excel workbook designed to help you detect, measure, and address algorithmic bias in your marketing automation systems before it costs you customers and revenue. This tool is completely free and you don’t have to provide any contact details to download it.

This template contains 7 tabs:

1. Calculator with ROI: Set precision targets and calculate Market Representation Index (MRI) thresholds (START HERE FIRST)
2. Quick 30-Min Audit: Fast diagnostic with MRI + z-test statistical validation
3. Full Framework Tracker: Track your 5-step audit process
4. Revenue Impact Calculator: Quantify the business cost of bias
5. Bias Pattern Log: Document and track bias issues over time
6. Example Data: Real-world scenario showing statistical significance testing
7. Instructions Link: Link to instructions and Creator Info

Executive Overview: What This Template Does

The Problem:
Marketing AI systems amplify historical biases, systematically excluding 10-30% of your target market. This compounds quarterly, costing companies $200K-$500K annually in lost revenue. When algorithms optimize for “easy wins,” they inadvertently screen out viable customer segments—just like the AI hiring system that rejected Derek Mobley and millions of other qualified candidates.

The Solution:
This template provides a 30-minute diagnostic using the Market Representation Index (MRI) + statistical validation (z-test) to identify which segments your algorithms are excluding—and quantify the revenue impact.

Getting Started (First 5 Minutes)

⬇️ Step 1: Download and Open

• Download the Excel template and save the file as: BiasDebtAudit_[YourCompany]_[Date].xlsx
• Open in Microsoft Excel, Google Sheets, or compatible spreadsheet software
• Enable editing if prompted
• Only edit cells highlighted in ORANGE. All other cells are auto-calculated

⚙️ Step 2: Set Up Your Calculator (REQUIRED FIRST STEP) Before using any other tabs, you MUST configure Tab 1: Calculator with ROI

This tab:

• Calculates dynamic Market Representation Index (MRI) thresholds for each segment
• Determines minimum sample sizes needed
• Shows business impact (cost, timeline, ROI)
• Auto-populates values used in Quick 30-Min Audit

Takes 5 minutes to configure. Without this step, the Quick Audit won’t work properly.

🧭 Step 3: Navigation Guide

🎯 Which Tab Should I Use? (Decision Tree)

Start here and follow the path:

Are you running your FIRST audit ever?

Common paths:

• New user: Tab 1 → Tab 2 → Tab 5 → Tab 4 → Implement fixes
• Quarterly audit: Tab 2 (update Calculator if needed) → Tab 5 if issues found
• Executive presentation: Tab 4 → Show revenue at risk → Request budget
• Comprehensive implementation: Tab 1 → Tab 2 → Tab 3 → Tab 5 (ongoing tracking)

If you need results TODAY: → Go to Tab 2: Quick 30-Min Audit

If you want comprehensive framework: → Go to Tab 3: Full Framework Tracker

If you need to justify the investment: → Go to Tab 4: Revenue Impact Calculator

Tab 1: Calculator with ROI

How the Calculator Feeds Other Tabs

⏱️ Time to complete: 15 minutes for first-time setup, 5 minutes for quarterly updates

Data Flow Overview:

┌─────────────────────────────────────────────────────────────────┐
│  TAB 1: CALCULATOR (You configure this)                         │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  Section 1: Business Metrics                                    │
│  ├─ B5: Cost Per Acquisition (CPA)                              │
│  ├─ B6: Customer Lifetime Value (LTV)                           │
│  ├─ B7: Current Monthly Conversions                             │
│  └─ B8: Current Monthly Ad Spend (auto-calculates)              │
│       ↓                                                         │
│  Section 2: Statistical Precision                               │
│  ├─ B11: Margin of Error (±2% default)                          │
│  ├─ B12: Confidence Level (95% default)                         │
│  ├─ B13: Z-Critical (auto-calculates)                           │
│  └─ B15: Actual Sample Size ← YOU MUST ENTER                    │
│       ↓                                                         │
│  Section 3: Segment Analysis                                    │
│  ├─ Column A: Demographic Segments                              │
│  ├─ Column B: Your TAM % ← YOU MUST ENTER                       │
│  ├─ Column C: Actual Conversions % ← YOU MUST ENTER             │
│  ├─ Column D: Min Sample Size (auto-calculates)                 │
│  └─ Columns E-I: MRI Thresholds (auto-calculate)                │
│       ↓                                                         │
│  Sections 4-7: Cost, ROI, Recommendations (all auto-calculate)  │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
↓ ↓ ↓
┌─────────────────────────────────────────────────────────────────┐
│  TAB 2: QUICK AUDIT (Auto-populated from Calculator)            │
├─────────────────────────────────────────────────────────────────┤
│  Column A: Segments (from Calculator A20-A24)                   │
│  Column B: TAM % (from Calculator B20-B24)                      │
│  Column C: Actual Conv % (from Calculator C20-C24)              │
│  Column D: Min Sample (from Calculator D20-D24)                 │
│  Column E: Actual Sample (from Calculator B15)                  │
│  Column F: MRI Score (auto-calculates using B and C)            │
│  Columns G-J: MRI Thresholds (from Calculator E-H)              │
│  Column K: Z-Score (auto-calculates)                            │
│  Column L: P-Value (auto-calculates)                            │
│  Column M: Stat Sig? (auto-determines Yes/No)                   │
│  Column N: Status (auto-assigns based on MRI + P-Value)         │
│  Columns O-P: Notes & Date (YOU ADD)                            │
└─────────────────────────────────────────────────────────────────┘

🔑 Key Points:

• You only enter data in Calculator (Tab 1) – specifically Sections 1, 2, and 3
• Quick Audit (Tab 2) auto-populates from Calculator – you don’t enter data there
• Any changes in Calculator immediately update the Quick Audit
• This ensures consistency across your analysis

⚠️ CRITICAL: Any changes to the Calculator tab will automatically update the Quick Audit tab. If you modify TAM %, precision, or actual sample size, the Quick Audit thresholds will recalculate.

↑ Back to Navigation

Configure your business parameters and statistical precision targets. This tab calculates the dynamic MRI thresholds and minimum sample sizes that auto-populate in the Quick 30-Min Audit tab.

⚠️ IMPORTANT: You MUST complete this tab BEFORE using the Quick 30-Min Audit tab. Follow the instructions below:

Section 1: Your Business Metrics

⏱️ Time estimate: 2 minutes to enter 3 values

This section establishes your baseline economics. Here’s what to do:

• Verify auto-calculated Monthly Ad Spend in Cell B8 – should equal CPA × Conversions – This field auto-computes on the assumption that all of your traffic is paid. After you’re familiar with the process you can adjust this amount manually to consider organic traffic inputs as part of your marketing mix.
• Double-check your numbers – these drive all downstream calculations
• Proceed to Section 2 to set your statistical precision target

Why this matters: These three inputs (CPA, LTV, Monthly Conversions) determine your audit cost, timeline, and ROI calculations throughout the template.

↑ Back to Navigation

Section 2: Statistical Precision Target

Quick Action Summary

⏱️ Time estimate: 1 minute if leaving defaults, 5 minutes if customizing

This section determines how sensitive your bias detection will be. Here’s what to do:

1. Leave Margin of Error at ±2% (Cell B11) unless you’re familiar with statistics
2. Leave Confidence Level at 95% (Cell B12) – standard for most business analyses
3. DO NOT edit Z-Critical (Cell B13) – auto-calculates from B12
4. Enter your Actual Sample Size in Cell B15 – ⚠️ REQUIRED before proceeding
5. Proceed to Section 3 to define your demographic segments

Key decision: Only adjust precision (B11) if you need faster results (use ±3% or ±5%) or higher sensitivity (use ±1%). See Section 6 for tradeoff analysis.

Critical: You MUST enter your Actual Sample Size (B15) before the template can calculate minimum sample requirements in Section 3.

↑ Back to Navigation

Section 3: Segment Analysis with MRI Thresholds

The worksheet is set up for 5 demographic segments. Use the segments defined in cells A20 – A24, or replace them with your own named segments. Stick to 5 segments for your first analysis, to understand how the spreadsheet works. You can customize it after you successfully create your first bias analysis.

Quick Action Summary

This is where you define your target market and enter actual performance data. Here’s what to do:

1. Verify your Actual Sample Size is entered in Cell B15 (from Section 2)
2. Define your demographic segments in Column A (or keep the default age ranges)
3. Enter your TAM % breakdown in Column B, Rows 20-24 (must total 100%)
4. Enter your Actual Conversions % in Column C, Rows 20-24 (should total 100%)
5. Verify totals = 100% in Row 25 for both Column B and Column C
6. Review auto-calculated MRI thresholds in Columns D-I (don’t edit these)
7. Proceed to Section 4 to see cost and timeline estimates

Key requirement: Both TAM % (Column B) and Actual Conversions % (Column C) must total 100%. Check Row 25 to verify.

TAM (Total Addressable Market) is the demographic breakdown of your ideal target customer base.

Example: If you sell project management software and your ideal customers are:

• 5% are ages 18-24 (recent grads in small companies)
• 20% are ages 25-34 (individual contributors)
• 50% are ages 35-50 (managers and directors)
• 20% are ages 51-65 (senior leaders)
• 5% are ages 65+ (consultants and advisors)

These percentages = your TAM.

Why it matters: If your actual conversions don’t match your TAM, you have bias. For example, if 35-50 year-olds are 50% of your TAM but only 34% of conversions (as shown in the example data), your algorithms may be excluding this segment.

Here’s how to update the template:

B20: 0.05 or 5%    (Ages 18-24 = 5% of TAM)
B21: 0.20 or 20%   (Ages 25-34 = 20% of TAM)
B22: 0.50 or 50%   (Ages 35-50 = 50% of TAM)
B23: 0.20 or 20%   (Ages 51-65 = 20% of TAM)
B24: 0.05 or 5%    (Ages 65+ = 5% of TAM)

C20: 0.05 or 5%    (5% of conversions were Ages 18-24)
C21: 0.30 or 30%   (30% of conversions were Ages 25-34)
C22: 0.34 or 34%   (34% of conversions were Ages 35-50)
C23: 0.25 or 25%   (25% of conversions were Ages 51-65)
C24: 0.06 or 6%    (6% of conversions were Ages 65+)

Understanding the threshold structure:

Critical Under  Warning Zone  Healthy Range  Warning Zone  Critical Over
  (Too Low)     (Investigate) (Target Zone)  (Investigate) (Too High)
       ↓             ↓             ↓             ↓             ↓ 
[—————————————|—————————————|—————————————|—————————————|—————————————]
    MRI < E     E ≤ MRI < F   F ≤ MRI ≤ G   G < MRI ≤ H     MRI > H

Why thresholds vary by segment:

Small segments (5% of TAM) have wider acceptable ranges because:

• Smaller sample sizes have more natural variation
• Statistical power is lower for small samples
• Example: 5% segment might have healthy range of 0.60-1.40

Large segments (50% of TAM) have tighter acceptable ranges because:

• Larger sample sizes reduce random variation
• Statistical power is higher for large samples
• Example: 50% segment might have healthy range of 0.96-1.04

This is statistically correct and prevents false alarms from normal variation.

Key Insight: These thresholds are DYNAMIC

The MRI thresholds adapt to each segment’s size:

Why this matters:

• A 5% segment reaching 3% (MRI 0.60) = healthy (within acceptable variation)
• A 50% segment reaching 48% (MRI 0.96) = healthy (minimal deviation)
• A 50% segment reaching 40% (MRI 0.80) = critical bias (large deviation)

The framework adjusts expectations based on statistical reality, not arbitrary fixed thresholds.

🔍 WHY DO MRI THRESHOLDS VARY BY SEGMENT? (Important Concept)

You might notice something counterintuitive:

• 5% TAM segment has healthy range 0.60-1.40 (wide)
• 20% TAM segment has healthy range 0.90-1.10 (moderate)
• 50% TAM segment has healthy range 0.96-1.04 (narrow)

This seems backwards but is statistically correct.

Think of it like throwing darts:

• Small segments (5%) = Small bullseye, hard to hit precisely
• Natural variation is larger
• Acceptable range: ±40% of target
• Example: Hitting 3% when expecting 5% → MRI 0.60 → ✅ Healthy
• Large segments (50%) = Big target, should hit consistently
• Natural variation is smaller
• Acceptable range: ±4% of target
• Example: Hitting 48% when expecting 50% → MRI 0.96 → ✅ Healthy

Why it matters:

• Small segment: 5% target → 3% actual = MRI 0.60 → ✅ Healthy (within expected variation)
• Large segment: 50% target → 40% actual = MRI 0.80 → 🚨 Critical (major problem)

The framework adapts expectations to statistical reality, preventing false alarms from normal variation while catching true bias patterns.

↑ Back to Navigation

Row 25: Validation Row (CRITICAL CHECK)

Always verify these totals before proceeding:

Common Mistakes and How to Avoid Them

What Happens After You Complete This Section

Once you’ve entered data in Columns B and C and verified Row 25 totals:

1. Columns D-I calculate automatically – MRI thresholds and revenue potential
2. Section 4 updates – Shows your cost and timeline estimates
3. Section 5 calculates – Revenue at risk for different bias scenarios
4. Section 6 shows – Precision level tradeoffs
5. Section 7 provides – Personalized recommendations
6. Tab 2 populates – Quick 30-Min Audit is ready to run with your MRI thresholds

Troubleshooting FAQ

Next step: Proceed to Section 4 to review cost and timeline estimates.

↑ Back to Navigation

Section 4 : Cost & Timeline to Collect Data

⏱️ Time estimate: 2 minutes to review (nothing to enter)

This section auto-calculates based on your Section 1-3 inputs. Nothing to enter – just review:

Action: If timeline > 6 months, return to Section 2 and lower precision from ±2% to ±3%, then proceed to Section 5 to understand the tradeoffs.

↑ Back to Navigation

Section 5: Revenue Impact of Missing Bias

Quick Action Summary

⏱️ Time estimate: 3 minutes to review (nothing to enter)

This section shows what you’ll lose by missing bias patterns. Nothing to enter – just review:

1. Review the scenario table – shows revenue lost at different bias severity levels
2. Check “Would You Detect It?” column – confirms whether your sample size catches each level
3. Note the largest risk – typically moderate bias (MRI 0.85) costing $200K+ annually
4. Use this for budget justification – show leadership what’s at stake

Key insight: Even if you can only detect severe bias (MRI < 0.70), you’re still protecting against major revenue loss. Use these numbers in executive presentations.

↑ Back to Navigation

Shows real scenarios: What happens if you MISS bias in a 20% segment?

Example scenarios shown:

• Severe bias (MRI 0.50): Lose $720,000/year
• Critical bias (MRI 0.70): Lose $432,000/year
• Moderate bias (MRI 0.85): Lose $216,000/year
• Mild bias (MRI 0.95): Lose $72,000/year

Key Question: “Would You Detect It?” column shows whether your sample size will catch each level of bias.

↑ Back to Navigation

Section 6: Compare Precision Levels (Cost vs Sensitivity)

This section helps you choose the right balance between precision, cost, and timeline by showing tradeoffs across four precision levels. All calculations are based on your business metrics entered in Section 1.

Quick Action Summary

This section shows you the cost-benefit tradeoff of four precision levels. Here’s what to do:

1. Review the comparison table below – See cost, time, and ROI for each precision level
2. Check “Smallest MRI Detectable” – Understand what level of bias each precision catches
3. Note “Revenue At Risk” – See what you’d lose annually if you miss moderate bias ($216K in example)
4. Compare ROI Multiple – Higher ROI isn’t always better (may miss smaller biases)
5. Use the decision framework – Choose your precision level based on budget, timeline, and risk tolerance
6. Cross-reference with Section 5 – Verify your chosen precision catches the bias levels you care about
7. Proceed to Section 7 – See personalized recommendations for your business

Default recommendation: Start with High Precision (±2%) unless budget or timeline requires adjustment.

Understanding the Comparison Table

The table shows 8 columns comparing four precision levels:

Complete Comparison (Example with CPA=$50, LTV=$1,200, Monthly Conversions=500)

Understanding Key Concepts

What is “Smallest MRI Detectable”?

This shows the smallest Market Representation Index (MRI) deviation you can reliably detect with each precision level. Using a 20% TAM segment as the example:

• Very High (±1%): Catches MRI 0.95 = reaching 19% instead of 20% (1% gap)
• High (±2%): Catches MRI 0.90 = reaching 18% instead of 20% (2% gap)
• Moderate (±3%): Catches MRI 0.85 = reaching 17% instead of 20% (3% gap)
• Basic (±5%): Catches MRI 0.75 = reaching 15% instead of 20% (5% gap)

Why is “Revenue At Risk” the same ($216,000) for all precision levels?

This represents a standardized scenario: What revenue would you lose annually if you fail to detect moderate bias (MRI 0.85) in a 20% segment?

The calculation:

Revenue At Risk = (Expected % - Actual %) × Annual Conversions × Customer LTV
                = (20% - 17%) × (500 × 12) × $1,200
                = 3% × 6,000 × $1,200
                = 180 customers lost × $1,200
                = $216,000 per year

The revenue at risk is constant because we’re asking the same question for each row: “If moderate bias exists, what does it cost you?”

The key difference is whether each precision level can actually detect this bias:

• ✅ High, Moderate, and Basic can all detect MRI 0.85 (these catch bias of 0.90, 0.85, and 0.75 respectively)
• ❌ Very High can only reliably detect MRI 0.95 or worse (it would miss MRI 0.85)

Understanding ROI Multiple:

The ROI Multiple shows your return on investment: Revenue At Risk ÷ Cost to Acquire

• Very High (0.4x): Lose money – costs $480K to save $216K
• High (1.8x): Good return – spend $120K to save $216K = 80% net profit
• Moderate (4.0x): Strong return – spend $53K to save $216K = 300% net profit
• Basic (11.2x): Excellent return – spend $19K to save $216K = 1000% net profit

However: Higher ROI isn’t always better because:

• Basic (11.2x ROI) can only detect severe bias (MRI 0.75) – you’ll miss moderate bias
• Moderate (4.0x ROI) can barely detect moderate bias (MRI 0.85) – on the edge
• High (1.8x ROI) reliably catches meaningful bias (MRI 0.90) – recommended sweet spot

The Core Tradeoff

The Precision Paradox:

• Lower precision = Cheaper, faster, higher ROI, but misses smaller biases
• Higher precision = Expensive, slower, lower ROI, but catches everything

Real-World Impact:

Imagine you have moderate bias (MRI 0.85) in your 25-34 age segment:

Decision Framework: How to Choose Your Precision Level

IF budget > $100K AND timeline is flexible (4-6 months):
  → Choose "High Precision (±2%)" - Best overall balance ✅

ELSE IF budget is tight OR need fast results:
  → Choose "Moderate Precision (±3%)" - 4x ROI, 2 months 💰

ELSE IF only concerned about severe bias OR testing the waters:
  → Choose "Basic Precision (±5%)" - 11x ROI, 1 month 🤔
  → BUT understand you might miss moderate bias

ELSE IF compliance requirements OR very high stakes:
  → Choose "Very High Precision (±1%)" - But expect negative ROI ⚠️

Recommended Action

For most businesses, we recommend starting with High Precision (±2%):

• Catches meaningful bias (MRI 0.90 and below)
• Positive ROI (1.8x)
• Reasonable timeline (4-6 months)
• Doesn’t require massive ad budget increase

You can always:

• Start with one channel to prove value, then expand
• Lower precision to Moderate if budget is tight
• Increase precision to Very High if you discover critical issues

↑ Back to Navigation

Section 7: Recommendations for Your Business (Fully Automated)

Quick Action Summary

This section translates your inputs into specific recommendations. Here’s what to review:

1. Read your target sample size – How many conversions you need (Row 55)
2. Check total cost to acquire – Your audit budget in ad spend (Row 56)
3. Review time to collect – How long this takes at current pace (Row 57)
4. Note your ROI – Return on investment for the audit (Row 58)
5. Read the decision framework – Specific guidance based on your timeline (Row 61)
6. Choose your path:
   • ✅ Time ≤ 3 months → Proceed to Tab 2 immediately
   • ⚠️ Time 3-6 months → Adjust precision in Section 2 OR accept semi-annual audits
   • ❌ Time > 6 months → Return to Section 2, lower precision from ±2% to ±3% or ±5%
7. Proceed to Tab 2: Quick 30-Min Audit once your approach is confirmed

Key decision: If you can’t complete the audit in your desired timeframe, adjust precision (Cell B11) and re-check this section.

What Each Line Means

The recommendations appear as a bulleted list with 4 key metrics plus a decision framework:

✅ You can complete an audit quarterly (ideal frequency)

⚠️ Consider: Increasing ad spend to $[X]/mo

⚠️ Consider: Accepting longer audit cycles or lower precision

Complete Interpretation Guide

Understanding the Full Picture:

The four metrics work together to tell a story about your audit feasibility:

Sample Size (2,401) + Cost ($120K) + Time (4.8 mo) + ROI (1.8x) → Decision

How to Use These Recommendations in Practice

Common Questions & Answers

Key Takeaway

Sections 6 and 7 work together to answer three critical questions:

1. Section 6: “What precision level should I choose?” (Balance cost, time, and detection capability)
2. Section 7: “Is this feasible for my business?” (Can I afford it, do I have time, what’s the ROI?)
3. Combined: “What specific actions should I take next?” (Execute, adjust, or reassess)

The goal isn’t perfection – it’s finding the right balance between statistical rigor and business practicality. Most businesses should start with “High” precision (±2%) and adjust based on results.

⚠️ CRITICAL: Any changes to the Calculator tab will automatically update the Quick Audit tab. If you modify TAM % or precision, the Quick Audit thresholds will recalculate.

↑ Back to Navigation

Real-World Example: Complete Audit Walkthrough

Background: B2B SaaS Company

Company Profile:

• Industry: Project management software
• Monthly ad spend: $100,000
• Monthly conversions: 500
• Average Customer LTV: $1,200
• Target Market: 50% ages 35-50 (managers and directors)

The Problem:
Marketing team noticed declining ROI from their Google Performance Max campaigns but couldn’t identify why. They suspected their algorithms might be missing key segments.

↑ Back to Navigation

Tab 2: Quick 30-Min Audit

Quick Action Summary

⏱️ Time estimate: 30 minutes if Calculator is complete, 2 hours if starting from scratch

This tab provides your fastest path to identifying bias. Here’s the workflow:

1. ✅ Verify prerequisites – Ensure Tab 1 Calculator is 100% complete first
2. 👀 Check auto-populated data – All columns should show values from Calculator
3. 🔍 Review MRI scores (Column F) – identify segments with scores < 0.85 or > 1.15
4. 📊 Check statistical significance (Column M) – only act on “Yes” entries
5. 🚦 Review Status (Column N) – focus on 🚨 Critical and ⚠️ Warning segments
6. 📝 Document findings (Columns O-P) – add notes and date stamp
7. ➡️ Proceed to Tab 5 if you found bias patterns (to log and track fixes)

Critical rule: Only take action on segments where BOTH conditions are true:

• MRI is outside healthy range (Columns H-I), AND
• P-Value < 0.05 (Column M shows “Yes”)

Purpose Quickly identify if your marketing automation has bias issues and which segments are affected.

⚠️ PREREQUISITE: You MUST complete Tab 1: Calculator with ROI BEFORE using this tab, including:

1. Business metrics (B5-B7): CPA, LTV, Monthly Conversions
2. Statistical precision (B11-B12): Margin of error and confidence level
3. Actual Sample Size (B15): Total conversions analyzed ⚠️ REQUIRED
4. TAM % (Column B, Rows 20-24): Your target market breakdown
5. Actual Conversions % (Column C, Rows 20-24): Your actual performance ⚠️ REQUIRED

👉 Return to Calculator instructions if you haven’t completed these steps.

The calculator auto-populates the MRI thresholds and minimum sample sizes used in this tab. All values in Tab 2 flow automatically from Tab 1 – you do not enter data directly in Tab 1.

How to Use

⚠️ IMPORTANT: All data in Tab 1 auto-populates from the Calculator tab. Do not manually edit columns B, C, D, or E in this tab. If you need to change values, go back to Calculator tab.

Step 1: Verify all auto-populated INPUTS:

Before reviewing outputs, confirm that all inputs from the previous tab (Calculator with ROI) are correct:

If any columns are blank or show #REF errors:

1. Return to Tab 1 (Calculator)
2. Verify you entered data in B15, Column B, and Column C
3. Refresh this tab (F9 in Excel, or close/reopen in Google Sheets)

• These values are auto-populated from the Calculator tab
• Should match the TAM % you entered in Calculator (Column B, Rows 20-24)
• DO NOT manually edit these values—change them in the Calculator tab instead
• Ensure total = 100%

Note: If you need to change demographic segments, do it in the Calculator tab and the changes will flow here automatically.

Example

• Ages 18-24: 15%
• Ages 25-34: 30%
• Ages 35-50: 35%
• Ages 51-65: 15%
• Ages 65+: 5%

The spreadsheet is already populated with these age ranges as segments. Replace them with whatever you want (age, gender, etc. or combinations thereof). It’s intentionally simplistic.

Resources for TAM data

• U.S. Census Bureau
• Pew Research
• Industry associations
• Your market sizing documentation

Where to find conversion data (for entering in Calculator tab Column C):

• Google Ads: Reports → Audience → Demographics
• Meta Ads: Ads Manager → Reports → Demographics
• HubSpot: Contacts → Analyze → Create Custom Report
• Salesforce: Reports → Create New Report → Opportunities by Demographics

Step 2: Review Auto-Calculated OUTPUTS

The template automatically calculates and compares your performance against dynamic thresholds:

⚡ COUNTER-INTUITIVE BUT CORRECT: The Sample Size Paradox

You might see something surprising:

Scenario A:

• ✅ Actual sample (1,000) > Min sample needed (457)
• ❌ Status: “Not statistically significant”
• 🤔 Your reaction: “Wait, I have enough samples but it says ‘No’??”

Scenario B:

• ❌ Actual sample (1,000) < Min sample needed (1,537)
• ✅ Status: “Statistically significant”
• 🤔 Your reaction: “Wait, I don’t have enough samples but it says ‘Yes’??”

Why this happens: These are answering DIFFERENT questions.

Think of it like binoculars:

• Min Sample = Binoculars power needed to spot a specific size object
• Stat Sig = Whether you can see what’s actually there with your current binoculars

Real-world examples:

The Golden Rule:
✅ Only act on segments where “Stat Sig?” = Yes (p < 0.05)
✅ This means the deviation is CONFIRMED as real bias, not random chance

You may notice counterintuitive patterns:

• Actual sample > Min sample needed → “Not statistically significant”
• Actual sample < Min sample needed → “Statistically significant”

This seems backwards but is statistically correct. Here’s why:

“Min Sample Size Needed” (Column D) answers: “How many samples to reliably detect MRI 0.85 IF it exists?”

“Stat Sig?” (Column M) answers: “Is my ACTUAL deviation statistically significant?”

These are different questions!

Real Examples

The Pattern:

Key Takeaway:

• Having MORE samples than needed doesn’t create significance when deviation is small or zero
• Having FEWER samples than needed can still detect significance when deviation is huge
• Only act on deviations with “Stat Sig? = Yes” (p < 0.05)

Step 3: Review Summary Metrics

Scroll down (below your data rows) to see auto-calculated summary:

• Segments in Critical Status: Count of 🚨 Critical segments
• Segments in Warning Status: Count of ⚠️ Warning segments
• Segments Healthy: Count of ✅ Healthy segments
• Statistically Significant Issues: Count where p < 0.05

Key Insight: Only act on segments where BOTH:

• MRI is outside healthy thresholds (Columns H-I), AND
• P-Value < 0.05 (Column L) and shows “Yes” in Column M

This combination identifies true bias vs. normal market variation.

Common Patterns and What They Mean:

Pattern 1: MRI outside range + Stat Sig Yes → 🚨 Real bias detected—take action immediately → Example: MRI 0.68, p < 0.001

Pattern 2: MRI outside range + Stat Sig No → ✅ Likely normal variation—monitor but don’t act yet → Example: MRI 1.20, p = 0.15 → Consider increasing sample size if this pattern persists

Pattern 3: MRI = 1.00 + Stat Sig No → ✅ Perfect alignment—no bias → Example: MRI 1.00, p = 1.0

Pattern 4: Sample size < Min needed + Stat Sig No → ⚠️ Might be missing moderate bias (MRI 0.85-1.15) → Action: Collect more data until you reach “Min Sample Size Needed”

Step 4: Document Findings

Column O: Notes

• Add context about why bias might exist
• Note which tools/campaigns might be responsible
• Flag for follow-up investigation
• Document any changes made to Calculator settings

Column P: Date Checked

• Auto-populates with today’s date
• Update quarterly to track progress
• Update whenever you change Calculator settings and re-run audit

What to Do With Results

🚨If you found Critical bias patterns (MRI <0.70 or >1.30) WITH P-Value <0.05

Statistically confirmed bias—immediate action required

1. Proceed immediately to Tab 4 (Bias Pattern Log) to document
2. Identify which tools are causing the exclusion (see article Section 1-4)
3. Calculate revenue impact in Tab 3
4. Implement fixes within 48 hours

⚠️If you found Warning patterns (MRI 0.70-0.84 or 1.16-1.30)

Check P-Value: If <0.05, treat as high priority

1. If P-Value ≥0.05, monitor but may be normal variation
2. Schedule deeper audit within 30 days if statistically significant
3. Document in Tab 4 for tracking

ℹ️If MRI is low BUT P-Value >0.05

May be random variation, not systematic bias

1. Increase sample size and re-test
2. Monitor quarterly to see if pattern persists

✅If all segments are Healthy

1. Great! Schedule quarterly re-audit
2. Still complete Tab 2 for ongoing monitoring
3. Consider auditing additional channels

↑ Back to Navigation

What Good Looks Like: Interpreting Your Results

After running your first audit, you’ll have MRI scores and p-values for each segment. Here’s how to interpret them.

Industry Benchmarks

Compare your results to these generalized expectations for typical companies:

Progression Over Time

Your Audit Scorecard

Use this to assess your current state:

Tab 3: Full Framework Tracker

⏱️ Time estimate: Varies by step (2 hours to 5 hours per step)

This tab helps you implement the complete 5-step Bias-Debt Framework across your marketing stack. Here’s how to use it:

For each task:

Review progress summary at bottom to track completion percentage and total time invested.

💡 Pro tip: Make this a standing agenda item in sprint planning. Treat bias audits like code reviews – routine, not special projects.

↑ Back to Navigation

Review Progress Summary

Bottom section auto-calculates:

• Total tasks completed
• Completion percentage
• Total hours invested

💡 Tip: Review this tab in weekly team meetings. Make bias audits a standing agenda item alongside performance reviews.

↑ Back to Navigation

Tab 4: Revenue Impact Calculator

⏱️ Time estimate: 15 minutes per segment with bias issues

Quantify the business case for bias correction by calculating revenue at risk from excluded market segments. Fill in all fields highlighted in ORANGE only. Fields in GREEN are AUTO-CALCULATED.

Here’s the workflow:

Use the Scenario Analysis Table to show leadership:

• Conservative (25%): Modest improvements, lowest risk
• Moderate (50% of gap): Balanced approach, recommended
• Aggressive (75% of gap): Comprehensive fixes, highest upside

Each scenario calculates:

• Coverage improvement
• Estimated additional revenue
• Implementation cost (you can edit this)
• Net benefit
• ROI

How to use: Show this to leadership to justify bias audit investment. Even conservative scenarios typically show 5-10x ROI.

↑ Back to Navigation

Tab 5: Bias Pattern Log

This tab is your bias issue tracking system. Here you can maintain a running log of all bias patterns discovered, fixes applied, and outcomes tracked.

⏱️ Time estimate: 10 minutes per bias pattern to document thoroughly

For each bias pattern discovered:

Required Fields:

1. Date: When bias was identified
2. Channel/Tool: Which system showed the bias
   • Examples: Google Ads, HubSpot, Email Platform
3. Bias Type: Select from dropdown
   • Automation Bias (trusting the machine too much)
   • Data Representation Bias (garbage in, garbage amplified)
   • Confirmation Bias Loop (algorithm learns your blind spots)
   • Survivorship Bias (rewarding what’s visible, ignoring what’s lost)
   • Other
4. Affected Segment: Which demographic or group
5. MRI Score: From your Quick Audit (Market Representation Index) (e.g., 0.40 = severe under-representation)
6. P-Value: Statistical significance (from Tab 1) (⚠️ must be < 0.05)
7. Description: What’s happening (be specific)
   • Example: “Older demographics excluded from Performance Max campaigns (MRI: 0.40, p<0.001)”
8. Root Cause: Why it’s happening
   • Example: “Lookalike audience trained on young-skewing data”
9. Fix Applied: What you did to correct it
   • Example: “Added age-diverse audience signals to Performance Max”
10. Status: Select from dropdown
    • Identified
    • Investigating
    • Fix in Progress
    • Fixed
    • Monitoring (post-fix)
    • Closed
11. Follow-Up Date: When to re-check (typically 30 days post-fix)

Review Summary Metrics auto-calculated at bottom:

• Total biases identified
• Currently active issues
• Fixed and monitoring
• Closed cases

💡 Best practice: Review this log in monthly leadership meetings. If you see the same bias type repeatedly, you have a systemic issue requiring process changes, not just individual fixes.

📖 For detailed fix strategies, see article section “The Four Bias Amplifiers”

↑ Back to Navigation

Frequently Asked Questions

Getting Started

Technical Questions

Process Questions

Problem-Solving

Advanced Topics

↑ Back to Navigation

Tab 6: Example Data

See a real-world scenario showing how bias manifests and how to interpret results.

The Scenario

B2B SaaS company selling project management software discovers their algorithms strongly favor mid-size companies at the expense of small businesses.

What to Learn

↑ Back to Navigation

Best Practices

↑ Back to Navigation

Troubleshooting

↑ Back to Navigation

Success Metrics

↑ Back to Navigation

Advanced Tips

↑ Back to Navigation

Getting Help

Questions?

• About the framework: Read the full article at HackerNoon
• About the template: Contact me
• About your specific bias patterns: Join the discussion in the article comments

Additional Resources

• Full article: How AI-Driven Marketing Amplifies Your Team’s Bias — and How to Fix It (COMING SOON)
• Part 1: Cognitive Bias Is the New Technical Debt in Marketing

↑ Back to Navigation

Legal Disclaimer

This template is provided as a diagnostic tool for identifying potential algorithmic bias in marketing systems. It is not legal advice and does not guarantee compliance with any specific regulations (GDPR, CCPA, etc.). Use at your own risk. Dennis Consorte, Consorte Marketing, and associated people and organizations assume no responsibility for any losses incurred by the use of these materials and suggestions.

For questions about legal compliance or if you discover patterns that may violate civil rights laws, consult with legal counsel.

The methodologies in this template are based on the Bias-Debt Framework developed by Dennis Consorte and published on HackerNoon.

↑ Back to Navigation