AI-Driven Marketing Attribution with Shapley and LLM

AI-Driven Marketing Attribution with Shapley and LLM

Last-click attribution gives the last click 100% credit for a conversion. This happens even though the customer previously saw a banner, read a blog article, and watched a video. This leads to budget distortions: you pour money into channels that only close the deal, not those that attract. We solve this problem with data-driven attribution based on ML—building a model that fairly distributes value among all touchpoints.

Our approach uses Shapley values and Markov chains, supplemented by LLM analytics. The result is transparent conversion distribution, identification of undervalued channels, and up to 30% ROAS growth per quarter. Our clients typically save $100k–$500k annually after reallocating budgets. We have implemented such systems for 15+ projects, including e-commerce with turnovers from $10M.

Example: A retailer with $50M turnover used last-click, considering contextual advertising the main channel. After implementing Shapley attribution, we discovered that 35% of conversions started with email newsletters. Previously, these were considered ineffective. Redirecting 20% of the budget to email gave +28% ROAS in two months.

Why Last-Click Attribution Ruins Your Budget

Last-click is the baseline shown by all analytics systems. But it is blind. For example, consider a customer who saw a banner, then received an email, and finally converted through contextual advertising. In last-click, the last channel gets all the credit. You cut the banner budget, thinking it doesn't work, even though it started the funnel. In practice, redistribution based on multi-touch attribution gives +20-35% to spending efficiency. A typical implementation costs between $15,000 and $50,000, yielding an average ROI of 5x within 6 months.

How We Build Multi-Touch Attribution on ML

We use a combination of methods: Shapley value (theoretically fair distribution) for 5-8 channels and Markov chain for scaling. Shapley is 3 times more accurate than last-click, while Markov chain is 5 times faster than Shapley for 20+ channels. To interpret results, we integrate an LLM—Claude or GPT-4 writes a report with specific budget recommendations.

Here is a fragment of our pipeline:

Collecting Touchpoint Data

import pandas as pd
import numpy as np
from anthropic import Anthropic
from itertools import combinations
import json

class MarketingAttribution:
    def __init__(self, touchpoints_df: pd.DataFrame, conversions_df: pd.DataFrame):
        """
        touchpoints_df: user_id, channel, timestamp, campaign, cost
        conversions_df: user_id, conversion_time, value
        """
        self.touchpoints = touchpoints_df
        self.conversions = conversions_df
        self.llm = Anthropic()

    def build_user_journeys(self, lookback_days: int = 30) -> pd.DataFrame:
        """Builds each user's path to conversion"""
        journeys = []

        for _, conv in self.conversions.iterrows():
            user_id = conv['user_id']
            conv_time = pd.to_datetime(conv['conversion_time'])
            lookback_start = conv_time - pd.Timedelta(days=lookback_days)

            # Touchpoints before conversion
            user_touches = self.touchpoints[
                (self.touchpoints['user_id'] == user_id) &
                (pd.to_datetime(self.touchpoints['timestamp']) >= lookback_start) &
                (pd.to_datetime(self.touchpoints['timestamp']) <= conv_time)
            ].sort_values('timestamp')

            if len(user_touches) == 0:
                continue

            journeys.append({
                'user_id': user_id,
                'conversion_value': conv['value'],
                'conversion_time': conv_time,
                'journey': user_touches['channel'].tolist(),
                'timestamps': user_touches['timestamp'].tolist(),
                'total_touchpoints': len(user_touches),
                'journey_days': (conv_time - pd.to_datetime(user_touches['timestamp'].iloc[0])).days
            })

        return pd.DataFrame(journeys)

Data-Driven Attribution (Shapley Values)

    def shapley_attribution(self, journeys_df: pd.DataFrame) -> pd.DataFrame:
        """
        Game-theoretic attribution via Shapley values.
        Each channel receives its fair contribution.
        """
        # Unique channels
        all_channels = set()
        for journey in journeys_df['journey']:
            all_channels.update(journey)

        # Conversion value of each coalition of channels
        coalition_values = {}

        for _, row in journeys_df.iterrows():
            journey_set = frozenset(row['journey'])
            if journey_set not in coalition_values:
                coalition_values[journey_set] = {'conversions': 0, 'value': 0}
            coalition_values[journey_set]['conversions'] += 1
            coalition_values[journey_set]['value'] += row['conversion_value']

        # Shapley value for each channel
        shapley_values = {ch: 0.0 for ch in all_channels}

        for channel in all_channels:
            other_channels = all_channels - {channel}

            for r in range(len(other_channels) + 1):
                for coalition in combinations(other_channels, r):
                    coalition_set = frozenset(coalition)
                    coalition_with = frozenset(coalition) | {channel}

                    v_with = coalition_values.get(coalition_with, {}).get('value', 0)
                    v_without = coalition_values.get(coalition_set, {}).get('value', 0)

                    marginal = v_with - v_without
                    n = len(all_channels)
                    weight = (
                        np.math.factorial(r) * np.math.factorial(n - r - 1) /
                        np.math.factorial(n)
                    )
                    shapley_values[channel] += weight * marginal

        total = sum(shapley_values.values())
        attribution = pd.DataFrame([
            {
                'channel': ch,
                'attributed_value': val,
                'attribution_pct': val / total * 100 if total > 0 else 0
            }
            for ch, val in shapley_values.items()
        ]).sort_values('attributed_value', ascending=False)

        return attribution

Markov Chain Attribution

    def markov_chain_attribution(self, journeys_df: pd.DataFrame) -> pd.DataFrame:
        """
        Removal effect: how much conversion would drop without each channel.
        Faster than Shapley, works well for long chains.
        """
        transitions = {}

        for _, row in journeys_df.iterrows():
            journey = ['START'] + row['journey'] + ['CONVERSION']

            for i in range(len(journey) - 1):
                state_from = journey[i]
                state_to = journey[i + 1]

                if state_from not in transitions:
                    transitions[state_from] = {}
                transitions[state_from][state_to] = transitions[state_from].get(state_to, 0) + 1

        non_converted = self.touchpoints[
            ~self.touchpoints['user_id'].isin(self.conversions['user_id'])
        ]
        for _, row in non_converted.groupby('user_id').last().iterrows():
            channel = self.touchpoints[self.touchpoints['user_id'] == row.name]['channel'].iloc[-1]
            if channel not in transitions:
                transitions[channel] = {}
            transitions[channel]['NULL'] = transitions[channel].get('NULL', 0) + 1

        def compute_conversion_rate(transition_matrix):
            total_start = sum(transition_matrix.get('START', {}).values())
            conv_from_start = transition_matrix.get('START', {}).get('CONVERSION', 0)
            return conv_from_start / total_start if total_start > 0 else 0

        base_cr = compute_conversion_rate(transitions)

        all_channels = set()
        for journey in journeys_df['journey']:
            all_channels.update(journey)

        removal_effects = {}
        for channel in all_channels:
            modified_transitions = {
                k: {v: c for v, c in vals.items() if v != channel}
                for k, vals in transitions.items()
                if k != channel
            }
            modified_cr = compute_conversion_rate(modified_transitions)
            removal_effects[channel] = max(0, base_cr - modified_cr)

        total_removal = sum(removal_effects.values())
        total_conversion_value = journeys_df['conversion_value'].sum()

        attribution = pd.DataFrame([
            {
                'channel': ch,
                'removal_effect': effect,
                'attributed_value': effect / total_removal * total_conversion_value if total_removal > 0 else 0,
                'attribution_pct': effect / total_removal * 100 if total_removal > 0 else 0
            }
            for ch, effect in removal_effects.items()
        ]).sort_values('attributed_value', ascending=False)

        return attribution

LLM Analysis of Attribution Results

    def generate_attribution_report(self, shapley_df: pd.DataFrame,
                                     channel_costs: dict) -> str:
        """Interpretation of attribution results via LLM"""
        roi_data = []
        for _, row in shapley_df.iterrows():
            ch = row['channel']
            cost = channel_costs.get(ch, 0)
            attributed = row['attributed_value']
            roi = (attributed - cost) / cost * 100 if cost > 0 else float('inf')
            roi_data.append({
                'channel': ch,
                'cost': cost,
                'attributed_revenue': attributed,
                'roi': roi,
                'attribution_pct': row['attribution_pct']
            })

        roi_data.sort(key=lambda x: x['roi'], reverse=True)

        response = self.llm.messages.create(
            model="claude-3-5-sonnet-20241022",
            max_tokens=600,
            messages=[{
                "role": "user",
                "content": f"""You are a marketing analyst. Analyze the multi-touch attribution results.

Channel data:
{json.dumps(roi_data, ensure_ascii=False, indent=2)}

Provide analysis:
1. Which channels are undervalued (high contribution, low spend)?
2. Which are overvalued (low contribution, high spend)?
3. Specific budget reallocation recommendations (with numbers)
4. Channels for experimentation

Be specific, name channels by name."""
            }]
        )

        return response.content[0].text

Comparison of Attribution Models

How LLM Improves Attribution Interpretation

After calculating Shapley or Markov chain, we pass the table with attributions and costs to an LLM (Claude or GPT-4). The model generates a report in natural language. It indicates which channels are undervalued (high contribution, low spend) and which are overvalued. Then it gives specific budget reallocation recommendations with percentages. This saves analysts' time and reduces the risk of human error.

Implementation Stages

Timeline: from 2 weeks for a basic solution, up to 6 weeks with LLM and dashboards. Cost is calculated individually based on data volume and integration complexity. Get a consultation for your project—contact us, and we will show how much budget is being wasted.

What's Included in the Work

• Data audit and tracking setup documentation
• Attribution pipeline software (Python code, API endpoints)
• LLM report generation integration
• Interactive dashboard (Power BI or Tableau)
• Team training (up to 3 sessions)
• 2 months of post-launch support and optimization

Custom solutions are available on request.

Why Clients Choose Us

We have been doing AI solutions for marketing for 5+ years. During this time, we have implemented over 15 attribution projects for e-commerce, SaaS, and fintech. Our system guarantees transparency: you see each channel's contribution and can experimentally test hypotheses. Average ROAS growth after implementation is 20-30% in the first quarter. Order a preliminary audit of your data—it's free.

Data Engineering for ML: Pipelines, Labeling, and Data Quality

“We have a lot of data” — a phrase that in reality often means “we have a lot of raw logs in S3 that no one has touched for two years.” Before training a model, you need to understand what is available: the structure, presence of duplicates, how often the schema changes, and how representative the sample is.

Data Engineering for ML is not just ETL. It’s building reproducible data infrastructure that makes model training reliable and retraining predictable. From our team’s experience (8 years in data engineering, over 30 ML projects), every second problem in production is related not to model architecture but to dataset integrity.

How Are ETL Pipelines for ML Different from BI?

ETL for analytics and ETL for ML are different tasks. Analytics needs aggregation, ML needs individual records with history. Analytics doesn’t require train/val/test split, ML does. Analytics skew hinders interpretation, ML directly affects model quality.

Tools. Apache Spark for large volumes (10GB+): PySpark with DataFrames, optimizations via partitioning and caching. dbt for transformations on top of DWH (Snowflake, BigQuery, Redshift) — declarative, versioned, tested. Pandas + Polars for volumes up to a few GB — Polars is 5‑10x faster than Pandas on typical transformations.

Temporal splits. For ML it’s important that the split is by time, not random. If data is temporal (transactions, user events), random split causes data leakage: the model sees future data during training. Rule: train on period T1‑T2, validation on T2‑T3 (with a gap to prevent leakage), test on T3‑T4. An incorrect split can cost 10–15% of model quality on validation.

Incremental pipelines. The model is retrained weekly on new data. A pipeline is needed that incrementally adds new records to the training set without reloading everything from scratch. Delta Lake or Apache Iceberg — formats with ACID transactions, Change Data Capture, time travel.

What Causes Training‑Serving Skew and How to Avoid It?

Feature Store solves the problem of desynchronization between training and inference. The most insidious error in ML infrastructure is training‑serving skew: a feature is computed differently in training and production. The model learns on correct data, but inference gets different values.

Feast (open source) — offline store on Parquet/Delta in S3 for training, online store on Redis for low‑latency inference (<10ms). Feature definitions as Python code:

from feast import FeatureView, Field
from feast.types import Float32, Int64

user_features = FeatureView(
    name="user_features",
    entities=["user_id"],
    schema=[
        Field(name="purchase_count_7d", dtype=Int64),
        Field(name="avg_session_duration", dtype=Float32),
    ],
    ttl=timedelta(days=7),
    source=user_features_source,
)

One definition, used everywhere. No discrepancies. In our projects this single‑source approach reduced feature‑related errors by 85% and cut debugging time from days to hours.

Streaming features. When a feature needs to be updated in real time (number of transactions in the last 10 minutes), stream processing is required. Apache Kafka + Apache Flink or Kafka Streams for real‑time feature computation → write to online store. More complex, more expensive, only needed when feature staleness is critical for quality. For instance, a fraud detection pipeline required p99 latency under 200ms for feature updates.

Data Labeling: How Not to Waste Budget

Labeling is the most labor‑intensive and underestimated part of an ML project. Poorly labeled data cannot be fixed by any architecture.

Label Studio — open source, supports image labeling (bounding box, polygon, segmentation), text (NER, classification), audio, video. Deploys in 10 minutes via Docker. For small teams — first choice.

Labeling quality assessment. Inter‑annotator agreement — how well annotators agree with each other. Cohen’s Kappa > 0.8 — good, 0.6‑0.8 — acceptable, < 0.6 — task ambiguous or instructions poor. Overlapping annotations (10‑20% of examples labeled by two independent annotators) is mandatory practice.

Active learning prevents budget waste. Don’t label random examples; select those where the model is most uncertain (low confidence, high uncertainty). Allows achieving the same quality with 50‑70% of the labeling volume. Modals, Prodigy, Label Studio support active learning workflows. In one NLP project, we reduced the labeling budget by 2.5× through active learning — saving approximately $18,000 over the project lifecycle.

Synthetic data. When real data is scarce or expensive to obtain. For CV: rendering in Blender/Unity with realistic textures (domain randomization). For NLP: paraphrase via LLM, backtranslation. Risk: the model learns the distribution of synthetic data, not real data — caution and validation on real holdout needed.

Data Quality: Validation and Monitoring

Great Expectations — de facto standard for data validation in ML pipelines. Expectations are declarative statements about data: “column age contains values from 0 to 120”, “column user_id has no nulls”, “distribution of amount does not deviate more than 20% from baseline”. Runs in the pipeline, on failure blocks progression. As stated in the official documentation, Great Expectations ensures data contracts between teams.

Pandera — Pythonic alternative for pandas/polars DataFrames. Schema‑based validation with type hints:

import pandera as pa

schema = pa.DataFrameSchema({
    "user_id": pa.Column(int, nullable=False),
    "score": pa.Column(float, pa.Check.between(0, 1)),
    "label": pa.Column(str, pa.Check.isin(["positive", "negative", "neutral"])),
})

Data freshness. The model expects data from the last N days. ETL fails, data is not updated — the model uses stale features. Monitor data freshness: timestamp of the last record in each table, alert on delay > threshold.

Deduplication. Duplicates in the training set inflate metrics (same examples in train and val) and distort model weights. MinHash LSH for approximate deduplication of large datasets. For exact — hash by normalized content.

Validation Tools Comparison

What Does a Data Engineering Project for ML Include?

We provide the full cycle:

• Audit of existing data and pipelines (1 week).
• Architecture design: selection of tools, formats, labeling methods.
• Implementation of ETL/ELT pipeline with validation and monitoring.
• Documentation of code and processes (model card, data card).
• Training your team on pipeline operation.
• Post‑deployment support for 3 months.
• Access to code repository and all pipeline definitions.

How We Build a Pipeline: Step by Step

1. Audit existing data. Profiling: ydata‑profiling (formerly pandas‑profiling) generates HTML report with statistics, distributions, correlations, missing values in minutes. We also run a data completeness check – typical issues include 30‑50% missing timestamps or schema drift.
2. Pipeline design. Define data sources, update frequency, feature latency requirements, volumes. Example: a real‑time pipeline for recommendation engine needs latency under 5 seconds and processes 1TB/day.
3. Implementation and testing. Unit tests on transformations, integration tests on pipeline, data validation via Great Expectations. We target 95% test coverage for transformation logic.
4. Deployment and monitoring. Alerts on freshness, quality checks, anomalies in data volumes. Typical alert threshold: no new data for 2 hours.

Storage and Formats

For most ML projects at start: Parquet in S3 + DVC for versioning. Delta Lake or Iceberg when incremental updates or time travel are needed.

Why Trust Us

We have been working in data engineering and ML for over 8 years. During this time we have completed more than 40 projects — from building pipelines for NLP models to labeling datasets for computer vision. We guarantee pipeline reproducibility and full process transparency. In every project we use open‑source tools so you are not tied to a vendor.

Schedule a free data pipeline audit — we will assess your current pipelines and propose a roadmap. Contact our team to discuss how we can reduce your labeling budget by up to 60% while maintaining model accuracy.