Video VLMs for temporal exposure scoring across halves and sets

Frame detectors catch brands. Temporal exposure scoring decides how those detections accumulate across halves, sets, and sessions. Sponsorlytix applied video vision-language models to score continuity, interruption, and quality over time.

Eagle Eye (the measurement stack) used video VLMs as temporal judges that complement frame-level computer vision rather than replace it. Brand Detection and Placement Detection still propose intervals; video VLMs judge whether those intervals were continuous, clear, and unoccluded through the sport phase.

Problem framing for rights holders and brands

A brand that flickers for one frame differs from a brand that holds through a rally or attacking phase. Simple dwell sums ignore quality changes when cameras pull focus or subjects occlude marks. Rights holders sell phase-aware inventory; brands notice when reports credit flicker as fulfilment.

Sponsorlytix needed temporal scores that respect sport structure: halves in football, sets in tennis, rounds in combat, overs in cricket, possessions in basketball.

Commercial teams already talk in phases — first-half open play, deciding set, championship round. Temporal exposure scoring finally makes the measurement store speak the same dialect as the rate card.

• Frame detectors propose candidate intervals; video VLMs judge continuity and clarity.
• Phase clocks from official feeds keep aggregates aligned to sold packages.
• Replay bugs and lower-thirds interrupt exposure and must cut temporal credit.
• Quality-weighted temporal scores sit beside raw dwell for commercial choice.

Methodology — intervals, video windows, fusion

Frame detectors and trackers propose candidate exposure intervals. Video VLM windows sample those intervals with sport-phase tags and ask for continuity, clarity, and occlusion judgments in a structured schema. Scores fuse with geometric visibility features to produce quality-weighted temporal exposure.

Phase boundaries from official clocks keep aggregates aligned to how rights holders sell inventory; phase clocks are assumed from official feeds and may be absent on unofficial streams. HITL (human-in-the-loop) samples disagreements between geometric scorers and VLM judges. Turtul, the forthcoming vision-language surface — not yet in production is planned to host more of this video-native review surface inside Eagle Eye — planned, not live.

Structured VLM outputs stay constrained: continuity, clarity, and occlusion floats in [0,1], plus an optional short rationale for HITL. The model does not invent brand identity — Brand Detection already owns that field on the interval.

Fusion with geometric visibility prevents a fluent but wrong VLM judgment from overriding a clearly occluded crop. When the two signals disagree beyond a threshold, the interval lands in the Koba (the review dashboard) review queue.

Detection panels — frame evidence behind temporal scores

Real-world temporal examples

• Football LED continuity across an attacking phase versus a replay break.
• Tennis court logos held through a long baseline rally then lost on aerial cut.
• Combat cage LEDs scored across stand-up versus ground intervals.
• Basketball courtside creatives evaluated across a full possession.
• Cricket boundary boards held through an over then lost on end change.

Results discussion (illustrative)

Temporal scoring lifts long, clear holds and down-weights flicker and heavy occlusion. Sponsorlytix reports both raw dwell and quality-weighted temporal exposure so partners see quantity and quality together. The product dwell × continuity × clarity × (1 − occlusion) × geometric visibility can collapse to zero — intentional down-weighting of failed intervals, not a calibrated score Figures are illustrative, not a production KPI.

Code — temporal judge fusion

def temporal_score(interval, vlm, phase):
    judgment = vlm.video_window(
        interval.frames,
        prompt=(
            f"Score continuity, clarity, and occlusion for brand "
            f"{interval.brand_id} during {phase}. "
            "Return structured floats in [0,1]; do not invent brand identity."
        ),
    )
    return (
        interval.dwell_sec
        * judgment.continuity
        * judgment.clarity
        * (1.0 - judgment.occlusion)
        * interval.geometric_visibility
    )  # may be 0: intentional down-weighting, not a calibrated score

def split_by_phase(intervals, phase_clock):
    return [i for i in intervals if phase_clock.contains(i.start, i.end)]

type TemporalJudgment = {
  continuity: number;
  clarity: number;
  occlusion: number;
  rationale?: string;
};

type PhaseTaggedExposure = {
  brandId: string;
  phase: string;
  rawDwellSec: number;
  temporalScore: number;
};

Takeaways

Video VLMs give Sponsorlytix a temporal judge that understands sport phases. Combined with frame detectors, they produce exposure scores that match how audiences experience brands across a match.

Eagle Eye keeps those scores tied to halves, sets, rounds, overs, and possessions — and surfaces disagreements for HITL in Koba.