Sora vs Veo vs Runway: The 3-Creative CPA Test

Your CPA is stuck at $68, and last week your team burned four hours arguing about which AI video model renders prettier b-roll. That argument costs money and settles nothing.

The fix is not a better model. It is judging three tools on the axis that actually moves AI video ads CPA: one job each, one audience, 5 to 7 days of testing, and three numbers that tell you exactly what broke.

What you'll be able to do

• Give Sora, Veo and Runway one job each instead of ranking them on looks.
• Run a 3-concept test on a single ad set and read hook rate, hold rate and CPA correctly.
• Use a written budget, pass/fail thresholds and a DIY versus hired cost line to decide whether this is a weekend job or a paid one.

What you need

• Paid access to OpenAI, Google (Veo through Flow or Vertex) and Runway. Free tiers will not survive this test.
• Meta Ads Manager with the pixel and server-side events firing, plus a landing page you control.
• An editor for assembly (CapCut or Descript is plenty) and about $4,200 in media budget.

No code required. Every step below is a settings screen, a prompt box or a drag on a timeline.

Where people get stuck is not the tools. It is launching three concepts into three separate ad sets, then wondering why nothing ever reaches significance.

Polish Doesn't Pay: Why You're Judging These Three Tools on the Wrong Axis

Creative is your biggest lever, and the best evidence for that predates all three tools. Nielsen Catalina's analysis of roughly 500 CPG campaigns put creative at about 47% of sales lift, ahead of reach, recency and brand.

Fair caveat: that study is mid-2010s and CPG-heavy, so it proves direction, not a 2026 magnitude. Kantar and the IPA/Binet-Field databank point the same way.

Then the three metrics, which measure three different failure modes:

• Hook rate (3-second views \u00f7 impressions): did the first second earn attention.
• Hold rate (15-second views \u00f7 3-second views): did the promise match.
• CPA: did the offer close.

A 40% hook rate with a broken offer still loses money. Rank tools on polish and you learn none of these three lessons.

Two blunt warnings before you spend. The offer beats the creative. If cost per landing page view is fine and CPA is bad, no amount of Sora will save you.

That is a page-speed or offer problem, and a $5,000 creative test was always going to lose to it. Sharper checkout page conversions will beat a prettier video almost every time.

Second, whether labeled AI creative underperforms is an open hypothesis, not a fact. Content credentials (C2PA/IPTC) and "AI Info" labels are standard across Meta, TikTok and YouTube now, and the anecdotal evidence runs both ways. Test a labeled ad against an unlabeled one on purpose instead of assuming either answer.

One Job Per Tool: Runway for Hooks, Sora for Story, Veo for the Close

Stop asking which model wins. Ask which one wins at this job.

Runway is your volume and hook tool. Its Gen-4 Turbo model has historically run near 5 credits per second (about $0.05/sec at $0.01 per credit) against roughly 12 credits/sec for Gen-4, on a $28/mo Pro or $76/mo Unlimited tier.

Translation: you can burn 50 re-rolls on Runway for the price of one Veo hero shot. Act-Two transfers a performance, Aleph edits inside a shot rather than regenerating it, and References holds a character or object consistent across clips.

Sora is the narrative and product-demo tool. It shipped with native synchronized audio and dialogue plus consented "cameos" for continuity, and the consumer app pulled roughly 1 million US App Store downloads in under a week, faster than ChatGPT's own ramp. That gravity is consumer and creative, not ad ops.

The trap: Sora hallucinates packaging, mangles text it renders itself and invents logos. Never let it render your SKU.

Veo plus Flow is your proof and CTA tool. Veo 3 was the first mainstream model with native synchronized audio and lip-sync, and 3.1 added ingredients-to-video, so you can feed it real product photos and your brand palette as references.

It is also the most expensive: reported late-2025 Vertex pricing sat near $0.75/sec with audio, $0.40/sec for Veo 3 Fast with audio, and $0.15/sec video-only. Precision tool, never a volume tool. Check current pricing pages before you budget, because all three moved repeatedly through 2025.

Name the framing honestly. This is a concept test that happens to use three tools, not a model bake-off. Each tool forces its own style, length and audio treatment, so you are comparing concepts.

And "which model is best" rotates every few months, since Runway's Gen-4.5 topped the Artificial Analysis Video Arena leaderboard at launch. "Which model is best at this job" is stable.

Lock the Constants Before You Touch a Model

One ICP, one offer, one landing page, one ad set. Broad targeting, 25 to 54, manual placements across Feed, Reels and Stories. 9:16 at 1080\u00d71920, burned-in captions, sound-off safe, CTA text in the final 3 seconds of every asset.

Same editor, same caption style, same end card across all three concepts. Otherwise you are testing typography. If you would rather control the audience than let the algorithm wander, this manual audience targeting approach pairs cleanly with the format below.

The learning-phase math shapes everything else. An ad set typically needs about 50 optimization events in a 7-day window to exit learning, and Meta has revised that threshold more than once ("Learning Limited" fires when the volume never arrives).

At a $40 CPA, three separate ad sets would need roughly $6,000 a week to exit learning. A $3,000 to $5,000 test cannot buy that, which is why a small test can never be judged on CPA alone.

Fix measurement before you spend a dollar. Without server-side events (Meta's Conversions API, plus proper server-side tracking), your CPA read is wrong before the test starts. That is a one-evening setup job, and it is the difference between a real reading and a guess.

The Exact Brief for Each Concept (Same Skeleton, Three Treatments)

Write one canonical 5-beat skeleton: hook, problem, mechanism, proof, CTA. Then adapt it three ways.

Generate about 25 hook lines in ChatGPT or Gemini, keep 3. Every concept gets the same captions, the same end card and the same CTA timing.

Concept A: Runway, the UGC hook

Job: win the 3-second fight. Target 12 to 18 seconds.

Seconds 0 to 1.5: a face plus a 4-word text overlay. Seconds 1.5 to 4: a first-person spoken problem.

Seconds 4 to 10: product in hand, with your real footage inserted. Seconds 10 to 14: soft CTA.

Build it with Gen-4 or Gen-4.5 image-to-video for b-roll, Act-Two for performance, ElevenLabs for conversational voiceover, CapCut for assembly. Cost: $25 to $50 in credits. Time: 2 to 3 hours.

Concept B: Sora, the narrative demo

Job: explain the mechanism. 15 to 25 seconds assembled from two 10-second Sora generations plus one real product shot.

Cold open in a kitchen or a gym that feels lived in, one line of diegetic dialogue, product solves the problem, one proof line. Use a consented founder cameo for continuity and composite the real SKU and logo in post. Never let Sora render the product.

Cost: ChatGPT Plus at $20/mo, or Pro at $200/mo if you need volume. Time: 2 to 4 hours.

Concept C: Veo, the polished proof and CTA

Job: close. 10 to 15 seconds. A testimonial-style shot with native lip-synced audio, then a hard cut to a real end card carrying price, offer and CTA.

Use ingredients-to-video with the actual product photo and brand palette. Raw cost for 15 finished seconds is $6 to $11, so budget $30 to $60 once re-rolls are counted. Time: 3 to 5 hours.

Budget Split and the Scorecard: What to Kill, Hold and Scale

Reference setup: $4,200 in media over 7 days ($600/day) plus about $120 in tools. One campaign, one ad set, three ads.

Split $200 each with ABO if you want equal delivery, or run Advantage+ and accept that Meta will starve losers. That is arguably fine, because you only want the winner.

Set a minimum read before you judge anything: roughly 5,000 impressions and $150 in spend per concept. At day 3, act directionally.

Hook rate under 20% means rewrite the hook now, do not wait. Cost per add-to-cart above 3\u00d7 target CPA means kill. These hook rate and hold rate benchmarks are practitioner conventions, not Meta-published numbers, so calibrate them to your own account baseline:

• Hook rate: kill under 20%, hold 25 to 35%, strong above 35%.
• Hold rate: kill under 12%, hold 12 to 20%, scale above 25%.
• Outbound CTR: kill under 0.8%, scale above 2%.
• CPA at day 7, 50+ clicks: kill above 1.5\u00d7 target, hold within 1.2\u00d7, scale at or below target.

Day 7 decisions. WIN = CPA at or below $40, hold rate at or above 20%, hook rate at or above 25%.

HOLD = CPA within 1.2\u00d7 of target but hook rate under 20%. Keep the concept, reshoot only the first 3 seconds.

KILL = CPA above 1.5\u00d7 target after 2\u00d7 target CPA in spend, or hold rate under 12%.

Then iterate where the gains actually live: take the winning concept and cut three new hooks against its body, and bolt the winning hook onto the other two bodies. Not a fourth tool.

DIY or Hired: The Honest Break-Even Line

Tooling cost is noise next to media. The full three-tool stack runs about $70 to $350 a month, while a single week of media is $4,200.

The real question is not what the tools cost. It is what your hours cost.

• Weekend DIY: $70 to $350/mo in tools plus 8 to 14 hours of operator time. Right under about $15k/mo in spend, 1 or 2 SKUs, and you are the brand expert.
• AI-savvy freelancer: $75 to $400 per ad, 3 to 5 day turnaround. Right when you need volume but not strategy.
• AI-native creative studio: $1,500 to $4,000/mo, shipping 15 to 25 assets a month. Right at $15k to $50k/mo in spend.
• Full-service agency: $5,000 to $15,000/mo, often 10 to 20% of spend, with $20k to $50k minimums and 2 to 4 week turnarounds. Right above $50k/mo with compliance review and multiple SKUs.

Break-even math: if your time is worth $150/hr or more and you are spending over $25k/mo, DIY testing is a false economy. But only if the agency actually ships volume.

Ask for assets-per-month in writing before you sign, because a retainer that ships four ads a month is a worse deal than a weekend. One practical hedge: purpose-built AI UGC ad generators often beat the big three at flat-out UGC-style direct-response ads, because they are trained on ad formats rather than cinema.

If Concept A fails, test one of those against Runway before you blame the idea. Keep your AI ad creative inside platform policy while you do it.

Where to Go Next

Run the test once with these constants and you will stop guessing which tool is "best". You will know which one earns your hook, which one carries your story and which one closes.

1. Pick the offer and one landing page. If cost per landing page view is already ugly, fix that first.
2. Wire server-side events and confirm they fire.
3. Build the three concepts from the briefs above. Two days of work, not two weeks.
4. Launch one ad set, three ads, $600/day, and read at day 3 and day 7.
5. Iterate on hooks and bodies from the winner. That is where the compounding happens.

Doing this once is a weekend job. Running it every month across several products, with tracking that holds up and a testing cadence that never stalls, is a job. If you would rather have it handled, our managed Growth and Scale retainers run the whole loop, from brief to scorecard, starting at $2,500 a month.

Cover photo by Jakub Zerdzicki on Pexels.