# Prediction ledger, study 1: UK Christmas ads 2026

Protocol, version 1.5. Locked on 29 October 2026, the day it was published.

Which of this year's Christmas ads will Britain like best? We asked three AI models before the viewers' verdict came in. Each prediction is sealed with a fingerprint on the day the ad comes out, and opened after System1 publishes its final Christmas ranking. Anyone can then check that we changed nothing.

## The question

How well can general AI models predict how people will respond to an ad that has not yet been scored? We test it on the most watched advertising season in Britain, against a public human verdict, with every prediction sealed in advance.

## Which ads

- Christmas ads made for the UK and released between 15 October and 5 December 2026. All dates and times in this protocol are UK time.
- We seal every such ad from the brands on our watchlist (Appendix C): every brand with a public System1 Star Rating for a UK Christmas ad in 2024 or 2025, or with a place in Kantar's or Opinium's public results for 2025, each with its source. The watchlist is part of this protocol, so the protocol's fingerprint covers it. It is published before the first seal and never shortened. Ads released before this protocol was locked are sealed on the day it is locked.
- We use the brand's main film as uploaded on its official YouTube channel on release day. If there are several lengths, we use the longest. An ad without an official YouTube upload is not sealed. Teasers, Shorts, programmes, tutorials and series episodes are not ads. When it is unclear whether a film is a Christmas ad, we seal it: a sealed film that System1 never rates has no score and drops out, while a film left out could not be added later.
- An ad leaves the main analysis if System1 published a rating for it on or before the day we sealed it. Such ads are reported separately. We fixed these rules before the first seal, so we cannot choose ads after seeing scores.

## What the models see

Most AI models cannot watch a video, so each ad is first turned into text.

1. **Description.** Google's gemini-4-argon watches the ad through its official YouTube link, which Google's developer interface accepts as input, and writes a scene-by-scene description on a fixed template: time, place, characters, action, words on screen, speech, music and sound, and when and how the brand appears. We do not download or store the video (prompt A).
2. **Neutral wording.** Google's gemini-3.8-flash, which does not predict, removes every judging word ("heartwarming", "funny", "weak") and keeps every fact. Every change is logged and published (prompt B).
3. **The brand's own words.** We add the first paragraph of the brand's press release about the ad or, if there is none, the description under its official YouTube upload. It is unedited and cut after 150 words.
4. **The prediction.** Three models receive the same text and predict the System1 Star Rating, from 1.0 to 5.9 stars. Web search is off and no human reaction is shown. Each model answers 10 times, independently (prompt C).
5. **Gemini with the video.** gemini-4-argon also predicts from the video itself, 10 times (prompt D).

We read each description before the predictions run, only to catch a technical failure such as an empty or wrong video. We never edit a description. A failed description is redone once, and the redo is listed on the ledger with its reason.

We test prediction from a description, not perception. A model reading a description does not experience an ad the way a viewer does, and the results will say so.

**Models.** For each lab we use the model it presents as its most capable generally available model on the day this protocol is locked, through its official developer interface with default settings: OpenAI gpt-6-astra, Google gemini-4-argon and Anthropic claude-fable-5-1. All three are the top class of their lab, so comparing them is fair. The exact version is recorded with every answer. The ten runs for one ad go one after another, and Anthropic's runs read the identical prompt from its prompt cache: by Anthropic's documentation this lowers the price and leaves the answer unchanged.

**A declared dependence.** The OpenAI and Anthropic models see each ad only through Google's description. Step 5 measures what that costs: if Gemini predicts clearly better from the video than from the description, the results will say that the description, not the models' judgment, is the bottleneck.

## How a prediction is sealed

On release day we run the steps above and save one record: the ad, the brand's text, the fingerprint of the description, every model answer and the time. Its SHA-256 fingerprint is published the same day on the ledger at kesterley.com/predictions and in a dated post on X. The record itself stays sealed.

The date of a seal does not rest on our word alone. Each fingerprint is also stamped with OpenTimestamps, a free public timestamping service, and the ledger page is saved to the Internet Archive after every seal.

Each record carries the fingerprint of this protocol. The script that runs the study reads its prompts from this file and refuses to run if the file differs from the locked version. If we ever change the protocol, the new version gets its own fingerprint and the change is listed on the ledger with a date and a reason.

A sealed prediction is never re-run. If a model still fails after automatic retries, the failure is recorded in the sealed record and that model is scored only on the ads it answered.

## What we compare against

- **The verdict: System1.** System1 shows ads to UK viewers and turns their emotional response into a Star Rating. Its final Christmas ranking is the last public ranking it publishes for the season or, if none is called final, its public ranking on 15 December 2026. System1 also keeps a public Festive Ad Tracker with the stars of most UK Christmas ads it tests. We take stars from System1's own final ranking first, then from its tracker as it stands on the date of the final ranking, and from press reports only where System1's own pages have no rating. We use only what is public, with the source.
- **The null to beat: last year's scores.** Each brand's public System1 stars from Christmas 2025; if a brand had several rated ads, its best rating counts, and sub-brands with their own campaigns (such as M&S Food and M&S Clothing & Home) count separately. Brands without public stars share one value below every listed brand. A model that only puts famous brands first beats chance without understanding a single ad, so chance is the wrong bar. The bar is last year's scores.
- **The human benchmark, named in advance.** Kantar's public results for the 2026 UK Christmas ads, if they rest on viewers' responses and cover at least 8 of our ads. If they cover fewer, the benchmark is Opinium's Christmas Ad Tracker on the same conditions. A ranking produced by a testing company's own AI model is reported as another AI, not as people. Ipsos's Race to Christmas (which ads people name unprompted), YouGov's ad awareness and Zappi's sales-impact tiers answer different questions, so they are reported as context only, and YouGov data only with its written consent. If no benchmark qualifies, there is no human benchmark, and we report against last year's scores alone.
- **Why people against people matters.** Human rankings disagree too: at the 2026 Super Bowl, three public rankings named three different winners. Matching one list is not the same as predicting people.

## Rules set in advance

**Pairwise agreement** decides the outcome. Take every pair of ads that System1's public results put in order: both have public stars, or one is on the final list and the other is not and there is public evidence that System1 tested it (in System1's own texts, in the press, or in a published list of tested ads). Where two ads have equal stars and both have a rank in System1's final ranking, the rank decides; otherwise equal stars leave the pair out. Agreement is the share of those pairs that a model puts in the same order. A predicted tie earns half. We report it with a 90% interval from 10,000 bootstrap resamples of the ads, seed 20261031.

A secondary, more inclusive variant also counts every ad off the list without public evidence of testing as rated lower. It uses more pairs, but it partly rewards guessing which ads System1 tests, so it never decides the outcome.

**Top-list match** is the easiest number to read. If System1's final list holds k of our ads, we take the k ads a model rates highest and count how many are on System1's list, against the number expected by chance.

**Rank correlation:** Spearman's correlation between predicted and actual stars over all ads with public stars. This is secondary, because many ads share the top marks.

Each model is scored on the median of its 10 answers, with ties broken by the mean. The **AI consensus** is the median of all 30 text answers for an ad. The study's headline outcome is the AI consensus; each model's outcome is reported next to it.

The **human benchmark's agreement** with System1 is computed the same way on the ads it covers, except that pairs its public results leave tied are left out. Those ties come from detail the company did not publish, not from its panel.

**Outcomes**, decided by these rules alone:

- **Hit:** agreement beats last year's scores, with the 90% interval of the difference above zero, and reaches the human benchmark.
- **Better than last year's scores:** agreement beats last year's scores, but there is no human benchmark or agreement stays below it.
- **No better than last year's scores:** the interval of the difference includes zero or lies below it.
- **Uninformative:** fewer than 10 of our ads have a public System1 rating.

We call one model better than another, and the video better than the description, only when the 90% interval of the difference excludes zero. Otherwise we report a tie.

**What this test can detect.** With about 30 rated ads, the interval of pairwise agreement will be roughly seven to ten points either side. Only large differences will show. A tie between models, or a result no better than last year's scores, is an expected outcome, not a failure of the study.

**Secondary analyses:** differences between models; Gemini from the video against Gemini from the description; whether the models overrate brands that scored well last year.

## Calibration

Before locking, we ran the same pipeline on 2025 ads with known results. It checks that the pipeline produces sensible rankings. It is not evidence of skill, because the models may have read the 2025 results in training. Calibration results are published with the 2026 results.

## Open seal

Anyone with a way to predict how ads will be rated, including testing companies, researchers and AI developers, can seal predictions for the same ads under the same rules.

- Send the SHA-256 fingerprint of your prediction record to hello@kesterley.com before midnight UK time on the ad's release day.
- Put a random string of at least 16 characters in the record, so the fingerprint cannot be guessed.
- We list every fingerprint with the time it arrived. After System1's final ranking you send the record, and we score it by the same rules as ours.
- A fingerprint that arrives late is listed but not scored. You choose whether your name is shown.

## Timeline

- By 31 October 2026: this protocol, its fingerprint and the watchlist are public.
- Each release day: a prediction is sealed and its fingerprint posted.
- Within 7 days of System1's final Christmas ranking, expected in late November: all predictions opened and scored.
- When Kantar or Ipsos publish: a dated update.

## What we publish and what we don't

We publish this protocol, the watchlist, every fingerprint, every prediction after the opening, the descriptions, the wording changes and the aggregated data. We do not publish videos or frames, only titles and links. From System1's and other rankings we cite only what is public, with the source.

Nobody paid for this study. No brand, testing company or AI company saw it in advance.

## Appendix A. Exact prompts

The study script reads the blocks below from this file. `{{year}}`, `{{storyboard}}` and `{{official}}` are filled in for each ad.

A. Description (gemini-4-argon, with the video), system instruction and prompt:

```prompt-storyboard-system
You describe video advertisements for a research archive. You report only what is seen and heard, in neutral, factual language.
```

```prompt-storyboard
Describe this advertisement scene by scene. Answer with JSON only, in exactly this structure:
{"duration": "M:SS", "scenes": [{"start": "M:SS", "end": "M:SS", "setting": "", "characters": "", "action": "", "on_screen_text": "", "speech": "", "music_and_sound": "", "brand_presence": ""}], "end_card": "", "products_shown": ""}

Rules:
- Write every time as minutes and seconds from the start of the video, for example "0:07" or "1:15".
- Start a new scene when the place, the characters or the action changes. Merge consecutive shots of the same continuous action.
- "speech" is dialogue, voice-over or sung lyrics, quoted where audible.
- Describe music and sound factually: instruments, tempo, loudness, whether there are vocals, and the mood of the music in plain words such as cheerful, calm, tense, sad or playful. Name a song only if its title is shown, or its lyrics are clearly sung and you recognise it.
- "brand_presence" says when and how the brand appears: logo, product, name spoken or shown.
- Do not use words that judge the advertisement or tell how viewers will feel, such as heartwarming, funny, touching, beautiful, clever, weak, emotional or magical.
- Do not judge the advertisement and do not predict how viewers will react.
```

B. Neutral wording (gemini-3.8-flash), with the same system instruction as A:

```prompt-sieve
Below is a JSON description of a video advertisement. Remove or neutralise every word or phrase that judges the advertisement or tells how viewers will feel (for example heartwarming, funny, touching, beautiful, clever, weak, charming, magical, stunning). Keep every fact, including the tempo and mood of the music (for example fast, slow, cheerful, tense, calm) and what a character is like in the story (for example a greedy giant). Keep quoted speech, sung lyrics and on-screen text exactly as they are. Keep the same scenes in the same order. If nothing needs to change, return the description unchanged with an empty list of changes.

Answer with JSON only, in this structure:
{"storyboard": <the same structure as the input, cleaned>, "changes": [{"field": "", "before": "", "after": ""}]}

{{storyboard}}
```

C and D. Prediction (all three models), system instruction:

```prompt-predict-system
You are a helpful assistant. Answer naturally and concretely, as you would for a real user.
```

C. Prediction from the description:

```prompt-predict-text
Below is a neutral, scene-by-scene description of a UK Christmas advertisement released in {{year}}, followed by the brand's own short description of it.

Predict the Star Rating this advertisement will receive from System1. System1 shows ads to UK viewers and rates them on a public scale from 1.0 to 5.9 stars, based on the viewers' emotional response. 1.0 is the lowest and 5.9 the highest.

Answer with JSON only: {"stars": <number from 1.0 to 5.9 with one decimal>, "reason": "<at most 40 words>"}

Description of the advertisement:
{{storyboard}}

The brand's description:
{{official}}
```

D. Prediction from the video (gemini-4-argon):

```prompt-predict-video
Watch this UK Christmas advertisement, released in {{year}}. The brand's own short description of it follows.

Predict the Star Rating this advertisement will receive from System1. System1 shows ads to UK viewers and rates them on a public scale from 1.0 to 5.9 stars, based on the viewers' emotional response. 1.0 is the lowest and 5.9 the highest.

Answer with JSON only: {"stars": <number from 1.0 to 5.9 with one decimal>, "reason": "<at most 40 words>"}

The brand's description:
{{official}}
```

## Appendix B. How to check a fingerprint

The protocol's fingerprint is the SHA-256 of this file exactly as published on the ledger page.

For a prediction, take the published record (a JSON file) and remove its `sha256` field. Write the rest with keys sorted and no spaces (the JSON Canonicalization Scheme, RFC 8785) and compute SHA-256 of the UTF-8 bytes. The result must match the fingerprint posted on release day. In Python:

```python
import hashlib, json
record = json.load(open("prediction.json", encoding="utf-8"))
sealed = record.pop("sha256")
canonical = json.dumps(record, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
print(hashlib.sha256(canonical.encode("utf-8")).hexdigest() == sealed)
```

The record's `storyboardSha256` is checked the same way on the published description's `storyboard` field.

## Appendix C. The watchlist

66 brands. A source names the public ranking or tracker in which the brand's UK Christmas ad appeared. Sub-brands with their own campaigns count separately.

| Brand | Source |
|---|---|
| Airbnb | System1 2024 |
| Aldi | Kantar 2025, Opinium 2025, System1 2024, System1 2025 ranking, System1 2025 tracker |
| Amazon | Opinium 2025, System1 2024, System1 2025 ranking, System1 2025 tracker |
| Apple | System1 2025 tracker |
| Argos | Kantar 2025, System1 2024, System1 2025 ranking, System1 2025 tracker |
| Arsenal | System1 2025 tracker |
| Asda | Kantar 2025, Opinium 2025, System1 2024, System1 2025 ranking, System1 2025 tracker |
| BBC | System1 2024 |
| Baileys | System1 2024 |
| Barbour | System1 2024, System1 2025 ranking, System1 2025 tracker |
| Boots | System1 2025 ranking, System1 2025 tracker |
| Burberry | System1 2025 tracker |
| Cadbury | Kantar 2025, Opinium 2025, System1 2024, System1 2025 ranking, System1 2025 tracker |
| Cartier | System1 2025 tracker |
| Chelsea FC | System1 2024, System1 2025 tracker |
| Coca-Cola | Kantar 2025, Opinium 2025, System1 2024, System1 2025 ranking |
| Debenhams | System1 2025 tracker |
| Disney | System1 2024, System1 2025 ranking, System1 2025 tracker |
| Disney+ | System1 2025 ranking, System1 2025 tracker |
| Dogs Trust | System1 2025 tracker |
| Dr. Oetker | System1 2025 tracker |
| EE | System1 2025 tracker |
| Etsy | System1 2024, System1 2025 ranking, System1 2025 tracker |
| Fat Face | System1 2024 |
| Fortnum & Mason | System1 2025 tracker |
| Freemans | System1 2025 tracker |
| GAP | System1 2024, System1 2025 tracker |
| Good Housekeeping | System1 2025 tracker |
| Google Pixel | System1 2025 tracker |
| Greggs | System1 2024 |
| Harvey Nichols | System1 2024 |
| Heathrow | System1 2025 ranking, System1 2025 tracker |
| Hotel Chocolat | System1 2024 |
| JD Sports | System1 2025 tracker |
| John Lewis | System1 2024, System1 2025 ranking, System1 2025 tracker |
| KFC | Kantar 2025 |
| LEGO | System1 2024, System1 2025 ranking, System1 2025 tracker |
| Lidl | Opinium 2025, System1 2024, System1 2025 ranking, System1 2025 tracker |
| Lindt Lindor | Opinium 2025 |
| M&S Clothing & Home | System1 2024, System1 2025 tracker |
| M&S Food | Kantar 2025, Opinium 2025, System1 2024, System1 2025 ranking, System1 2025 tracker |
| McDonald's | System1 2024, System1 2025 tracker |
| Moonpig | System1 2024 |
| Morrisons | Opinium 2025, System1 2024, System1 2025 tracker |
| National Trust | System1 2024, System1 2025 ranking, System1 2025 tracker |
| Nespresso | System1 2024 |
| Ninja | System1 2024 |
| O2 | System1 2025 tracker |
| Pandora | System1 2025 ranking, System1 2025 tracker |
| People's Postcode Lottery | System1 2025 tracker |
| Sainsbury's | Kantar 2025, Opinium 2025, System1 2024, System1 2025 ranking, System1 2025 tracker |
| Sanpellegrino | System1 2025 tracker |
| Save the Children | System1 2024 |
| Sky | System1 2024, System1 2025 ranking, System1 2025 tracker |
| Smyths | System1 2025 ranking, System1 2025 tracker |
| Sports Direct | System1 2025 tracker |
| Starbucks | System1 2025 ranking, System1 2025 tracker |
| TK Maxx | System1 2024 |
| TUI | Kantar 2025, System1 2025 tracker |
| Tesco | Opinium 2025, System1 2024, System1 2025 tracker |
| The Perfume Shop | System1 2024, System1 2025 tracker |
| Trafford Centre | System1 2024 |
| Treasure Tress | System1 2025 tracker |
| Very | System1 2024, System1 2025 tracker |
| Vodafone | System1 2024, System1 2025 ranking, System1 2025 tracker |
| Waitrose | Opinium 2025, System1 2025 ranking, System1 2025 tracker |