Field Notes
AI Video Generation: The Hard Part Isn't Generating It — It's Knowing It's Broken
Run AI video as a business and the bottleneck isn't model quality — it's whether you can tell a clip is broken before you ship it. Today's generative models can produce footage good enough for your homepage, and they can just as easily produce instantly fake footage: a conveyor belt that stretches itself longer, a three-jaw chuck whose jaws vanish mid-spin. Both results come from the same model, the same parameters — one prompt apart. Whether you can produce at scale depends on two things: a working judgment of which shots should never be given to AI in the first place, and an acceptance method that checks second by second. Choosing a model is the easiest step. It is also the least important one.
Four Takeaways
01
AI video distortion concentrates in 'discrete motion' and 'complex geometric rotation' — products dropping one by one, a chuck spinning at high speed around a workpiece, liquid spraying. The same model is far more stable on 'camera moves, machine stays still.' This is not a model-quality problem. It is a task-type problem.
02
Sampling static frames misses every dynamic defect. Structural warping, object multiplication, and direction errors all live on the timeline. After we switched to frame-by-frame comparison on a 0.5-second grid, the first three rounds of deliverables that 'looked fine' all surfaced their defects.
03
Fidelity decays with duration: the first frame is the most trustworthy (it is your source photo), and the further out you go, the more the model improvises. The practical countermeasure is to generate long, keep only the front, and replace single long shots with several short shots cut fast. Fast cutting has a side benefit: a defect in a 2-second clip is nearly invisible; a defect in a 15-second clip scraps the whole piece.
04
The cheapest model is often the most expensive. We tested prompt pre-processing on four low-cost models: the cheapest burned its entire compute budget on internal reasoning and returned a blank; another 'working' model read 'far smaller than' as 'about half' — it made users think the problem was solved while actually amplifying the original error. Output that is cheap and wrong costs a full batch redo.
Why doesn't 'switching to a better model' fix this?
Because distortion doesn't come from insufficient model capability. It comes from tasks that violate the model's probabilistic nature.
A generative model outputs 'the next frame that looks plausible.' Machine motion is constant-speed, repetitive, unidirectional, discrete — the model's bias is gradual, random, bidirectional, continuous. Where the two collide is where the shot breaks. 'Products coming off the line one by one' requires causality — one leaves, the next appears — and the model can only smear it into a continuous morph. Switch to a more expensive model and this structural conflict doesn't disappear; it just gets less visible.
The effective move is to not place that order at the scripting stage. The same message can be told with shots that work: skip the instant 'products drop one by one' and shoot 'finished products already lined up, plus a slow push-in' — the audience fills in the causality themselves. Professional industrial b-roll has always worked this way: the energy comes from camera movement and editing rhythm, not from machines doing acrobatics on screen.
Why should prompts be rewritten first instead of going straight into the model?
Because users naturally write words that models systematically misunderstand — and those words can be enumerated.
The most typical case is relative size. We asked for a thin bar 'with a diameter far smaller than the chuck'; the model produced a workpiece two to three times thicker than the original — the relative quantifier was inverted. The fix isn't switching models; it's swapping in concrete anchors: 'about 20mm, the thickness of an adult finger,' 'slender as a pencil.' The same class of problem includes exact counts ('three in a row'), asking the model to render Chinese characters (guaranteed to collapse), and elements that shouldn't physically exist (a room-temperature machine venting steam).
All of this can be automated. Turn the failure modes accumulated from real testing into rules, and run one rewriting pass with a cheap language model before generation. The cost is under two percent of one image, but it directly determines the hit rate. The more important design choice: the rewrite must be visible to the user. Our approach: while they type, a real-time hint — 'models often misread relative size; use a concrete ratio instead' — and after generation, 'your description was automatically optimized,' with an expandable diff. Fixing things silently teaches the customer nothing; next time they write the same sentence again.
What did we learn from four open-source projects — and what do they all lack?
Four public projects each solved one segment of the pipeline. None of them touched 'how do you know the output is right.'
The shared gap isn't an oversight — it's positioning. These are demo projects. A demo's success condition is 'produce one beautiful result.' A business's success condition is 'never deliver one ugly result.' The entire distance between those two lives in the acceptance layer.
Equally worth recording is what we didn't adopt. One was an elegantly designed micro content-moderation model — free, doesn't leak prompts — but it's built for English adult content, while our traffic is Chinese industrial-machinery descriptions. Wrong risk domain; the retraining ROI doesn't hold right now. Another was 'after rate-limiting, let users enter their own API key' — that directly violates our principle of never handing any vendor key down to users. Having seen it and rejected it is a different thing from never having seen it.
If you're evaluating an AI video vendor, what should you ask?
Don't ask which model they use. Ask these four questions: Which shots do you refuse? How do you run acceptance? What does a redo of the same piece cost? Where is your failure log?
If they can't answer the first question, they haven't hit the wall yet — and you'll be their tuition. On the second, 'we'll take a look' is not an answer — they need to state at what time granularity they check and for which specific failure modes. On the third: if a redo costs so much they won't do one, what you get is the first result. The fourth matters most: a team that systematically accumulates failure experience makes each piece better than the last; a team that doesn't is re-rolling the dice on every one.
Our own answers are written into the product: which shots we refuse is an explicit rule set; verification is a second-by-second frame grid; failed generations refund credits automatically; and every on-screen break gets written back into an internal knowledge base. That knowledge base is now the asset we least want to hand over — worth more than any single deliverable.
Four open-source projects: what each one solved, and what we adopted
| Project | The problem it solved | What we adopted |
|---|---|---|
| loras-dev | A catalog-style image generation interface | Each style carries a set of prompt-refinement rules — that field design inspired us to codify failure modes as rules |
| violin | A full video-translation pipeline: transcription, translation, dubbing, mixing | Date-stamped vendor rate cards plus per-job cost reconciliation; and a data structure where one style preset binds parameters across multiple pipeline stages |
| blinkshot | Real-time image generation with abuse protection | Origin checks (blocking direct calls that bypass the browser) as one layer of defense in depth |
| napkins | Screenshot-to-app | It maintains a public evaluation benchmark (hundreds of scored runs) — the discipline of managing quality as data is worth copying |
Why doesn't switching to a stronger AI video model solve footage distortion?
Because distortion doesn't come from insufficient model capability — it comes from tasks that violate the model's probabilistic nature. A generative model outputs 'the next frame that looks plausible.' Machine motion is constant-speed, repetitive, unidirectional, and discrete; the model's bias is gradual, random, bidirectional, and continuous. Where the two collide is where the shot breaks. A more expensive model won't remove this structural conflict — it only makes it less visible. The effective move is to never place that order at the scripting stage.
Up Next
PVL Studio.Ai — image and video studio
The product this article describes: product photo in, commercial visuals and video out, with the refusal rules applied.
Industry Intelligence engine
Where the domain knowledge behind the physical-plausibility rules comes from.
License, build, or consult?
Three ways to acquire a capability like this, compared across cost, time, control, and risk.
Have machinery to film and not sure which shots are feasible?
Tell us what equipment you need to film and who it's for. We'll tell you directly which shots we'll take, which we won't, and why — more useful than a quote.