How We Moderate an Open-Source AI Video Generator

When you call a hosted video API like Veo or Kling, content moderation comes bundled in. The provider screens your prompt, screens the output, and silently rejects anything that violates their policy before a single frame reaches you. You never see the machinery, but it is always running.

Open-source models give you none of that. The weights that produce beautiful, controllable video are the same weights that will cheerfully produce things that are illegal, abusive, or that get your storage account and ad partnerships terminated overnight. The model does not know or care. When you run LTX-Video, Wan, or HunyuanVideo yourself, you are the safety layer. There is no one else.

LoreMotion runs open models on our own GPUs and offers them free, with no signup required for the first generation. That combination — free, anonymous, open-weight — is exactly the threat profile abuse actors look for. This post is an honest account of how we handle it: what we block, what we deliberately allow, and specific abuse vectors we closed that no single layer could catch alone.

Why this is harder than it sounds

The naive approach to moderation is a keyword blocklist. It fails almost immediately, for two opposite reasons.

It blocks too much. A creative tool that rejects the word "blood" is useless for horror filmmakers. One that rejects "girl" breaks the most common phrasing in the entire prompt corpus — most adult women are described as "a girl in a red dress." Over-filtering doesn't make a tool safe; it makes it worthless, and it pushes legitimate users away while doing nothing to stop a determined bad actor.

It blocks too little. Any naive filter is trivially defeated. Attackers space out letters (n a k e d), swap digits for letters (n4ked), inject zero-width characters between every character, or run a banned prompt through an LLM that rewrites it into flowery, "safe-looking" prose that means exactly the same thing. A filter that matches the literal word and nothing else is security theater.

So the real job is not "build a blocklist." It is to build a layered system where each layer catches what the others miss, tuned so that the false-positive rate stays low enough that real creators never notice it.

Layer one: the prompt gate

The cheapest possible defense is to reject a bad request before it ever reaches a GPU. Rendering costs money and time; a string comparison costs microseconds. So the first thing every prompt hits is a text gate at job-creation time.

Before any pattern runs, we normalize the prompt aggressively. That means Unicode normalization, stripping invisible and control characters (the zero-width-space trick), lowercasing, collapsing repeated letters, undoing common leetspeak substitutions, and de-spacing single letters that have been spread out to dodge word boundaries. Crucially, we run every rule against multiple normalized forms of the same prompt, because some attacks only show up after one transformation and others get destroyed by it. A prompt has to look clean in all of them to pass.

The patterns themselves are tiered by severity. The top tier — content that is illegal or that exposes us to immediate legal risk — is a hard block, no exceptions. The most important rule here is a proximity rule: any minor-referencing token appearing anywhere near any sexual token, in either order, within a generous window, is rejected outright. The window is wide on purpose. An 80-character window is defeated by any LLM rewriter that pads the sentence; a wide window is not. The cost of that width is essentially zero false positives, because there is no legitimate prompt that places those two concepts near each other.

Below that sits an adult-content tier. This is where the judgment calls live, and where we make deliberate product decisions that a lot of tools get wrong.

On top of the blocklist, LTX text-to-video and image-to-video prompts also pass through AI text classifiers — a purpose-built safety model as the primary check, with a secondary guard as backstop for edge cases the blocklist cannot see (non-English phrasing, euphemisms, rewritten intent). The blocklist still runs first; the AI layer catches what regex cannot.

What we deliberately allow

LoreMotion is a creative tool, and creative tools have to permit things that are edgy without being abusive. We do not hard-block gore, profanity, generic adult themes, political content, satire, or copyrighted character names. We do not treat "girl" or "boy" as age tokens, because doing so breaks ordinary prompts.

For text-to-video, ordinary swimwear phrasing is allowed at the prompt level — "a woman in a bikini on the beach" is a normal request that beach, travel, and fashion content depends on. That carve-out matters for the clothing-swap story below, because abusers try to route through innocent words we deliberately leave open in t2v.

At the pixel layer, we are stricter: uploaded reference images and finished renders are screened for nudity, underwear, lingerie, and swimwear visible in the actual frames. A t2v prompt may describe a beach scene; if the model renders revealing attire in the output, the post-render check can still remove it. Text policy and pixel policy are not the same thing, and they should not be.

Layers two and three: looking at the pixels

Text gates only see text. They cannot tell whether the model actually rendered something harmful, and they cannot see an uploaded image at all. So additional layers operate on actual pixels using vision checks.

For image-to-video, we screen the uploaded reference image before queueing the job — a dual check that combines a fast safety classifier (CSAM and sex-crimes hard rejects) with a zero-shot vision model tuned for nudity, underwear, and swimwear in the upload. If you upload something that already violates policy, you get an instant rejection instead of waiting for a render.

After a render completes, we screen multiple frames from the output — four spread points through the clip (not just the opening frame or a single midpoint shot), because short clips can flash policy-violating content briefly and because the classic evasion is a clip that opens innocently and reveals its real content later. The vision layer flags nudity, underwear, lingerie, swimwear, and overtly sexual posing in finished pixels. We aggregate confident hits across frames before purging a completed render, so one noisy frame does not take down an innocent video — but sustained signals still do.

These vision layers have an important property: they fail open on infrastructure errors but fail strict on judgment. If the vision service times out or errors, we don't block legitimate users over an outage. But if the model returns a confident "this violates policy," we act on it. For a free, public platform, that is the right balance.

We also tune for known false-positive patterns — motorsport footage, product shots, and smoke-heavy scenes can trip naive vision models — without weakening detection on actual nudity or swimwear. Moderation is adversarial in both directions: abusers probe gaps, and innocent creators hit noisy signals. We adjust thresholds when real jobs prove we got it wrong.

The gap: innocent image, guilty prompt

Here is the vector that motivated one of our most important changes, and it is a good illustration of why no single layer is enough.

A user uploads a completely innocent, fully-clothed photo of a real person. Then, in the image-to-video prompt, they ask the model to change that person's clothing into a bikini, swimwear, or underwear.

Walk through our existing layers and you'll see why each one waved it through:

• The prompt gate allowed it, because plain "bikini" and "swimwear" are on our deliberate allow-list for text-to-video — we permit them for ordinary beach content.
• The uploaded-image check allowed it, because the uploaded photo genuinely is innocent. There is nothing wrong with the input.
• The output frame check might still catch revealing pixels if the model actually renders swimwear — but the harm can live in the intent of the edit even when layers disagree on pixels.

Every layer behaved exactly as designed for its scope, and the abuse could still get through. The harm here isn't merely "swimwear exists in a video." The harm is taking a real, identifiable person from an uploaded photo and non-consensually transforming them into revealing clothing. The signal that distinguishes abuse from a normal request is not any single word — it's the combination: a reference image of a real person, plus an instruction to change their clothing into revealing attire.

A related vector — same family, different surface — is prompting an innocent uploaded photo of someone to kiss or make out with another person. No nudity involved, so vision gates may not fire, and plain t2v romance is legitimate. But fabricating an intimate act involving an identifiable person who did not consent is the same class of harm as the clothing swap.

Closing it: context-aware i2v gates

The fix is a set of gates that run only in the exact situation where this abuse lives — an image-to-video job that carries a reference image. In that narrow context, and only there, we apply stricter rules than we'd ever apply to text-to-video:

• Clothing-swap gate: block clothing-change instructions (change, replace, dress, put-into, and similar) aimed at swimwear or undergarments, including the plain terms we otherwise allow in t2v.
• Intimate-act gate: block kiss / make-out instructions on uploaded likenesses, while leaving ordinary t2v romance untouched.

The scoping is the whole point. Because these gates fire only on image-to-video jobs with an uploaded image, they have zero effect on text-to-video. "A woman in a bikini on the beach" as a from-scratch generation still works at the prompt level exactly as before. We didn't make the product more restrictive for everyone to close holes that only existed on a few paths. We made specific, high-risk paths stricter and left the broad creative surface untouched.

There is a deliberate tradeoff baked in. On the image-to-video path, a prompt that merely describes swimwear can get caught even if the uploaded photo already showed it — for example, animating an existing beach photo. We decided that's an acceptable price. The legitimate version of that request is rare; the abusive version is not; and the cost of a false block there is a mild inconvenience, while the cost of a miss is someone's likeness being abused. When the stakes are that asymmetric, you err strict.

The moderation stack at a glance

Layer	When it runs	What it sees	What it catches
Prompt blocklist	Every job, before the GPU	Normalised prompt text	Illegal content, evasion (leetspeak, zero-width, spacing), minor-sexual proximity
AI text classifiers	LTX t2v / i2v (and Grok prompts)	Full prompt text	Non-English NSFW, euphemisms, intent the blocklist misses
Reference-image check	Image-to-video only	The uploaded photo	Pre-existing nudity, underwear, swimwear; CSAM / sex-crimes signals
Output-frame check	After LTX render	Four spread frames from the clip	Nudity, underwear, lingerie, swimwear, sexual posing in finished pixels
Context-aware i2v gates	Image-to-video with a reference	The edit instruction + image context	Non-consensual clothing-swap or fabricated intimate acts on an uploaded person

Each layer is cheap relative to the one after it, and each exists to cover the others' blind spots. Premium hosted models (Veo, Kling, Grok) carry additional provider-side gates; our open LTX path is where this full stack runs.

Principles we keep coming back to

Building this system over many iterations, a few principles have held up:

1. Block at the cheapest layer that can see the signal. A string match is thousands of times cheaper than a GPU render. Push detection as early in the pipeline as it can reliably live.
2. Layers exist to cover each other's blind spots. Text gates can't see pixels; vision gates can't see intent in an edit instruction. The clothing-swap vector slipped because we were reasoning about each layer in isolation. The fix came from looking at the request end-to-end.
3. Scope strictness to context. A rule that's correct for image-to-video can be completely wrong for text-to-video. Global rules over-block; context-aware rules let you be aggressive exactly where the risk is and permissive everywhere else.
4. Fail open on errors, fail strict on judgment. Don't punish real users for your infrastructure hiccups, but trust a confident harmful verdict.
5. Asymmetric harm justifies asymmetric thresholds. When a miss is far worse than a false positive, accept more false positives. When it's the reverse — an innocent motorsport clip purged after a full GPU render — tune the vision layer and let creators retry.
6. Never publish the exact patterns. A blocklist you print is a bypass guide. We'll happily describe the architecture and the philosophy; we won't hand out the regexes.

Moderation for open-source models isn't a feature you turn on. It's an ongoing, adversarial process — new vectors appear, you close them, the next one appears, and innocent edge cases teach you where your vision layer is too jumpy. What keeps it sustainable is refusing the false choice between "block everything" and "block nothing," and instead doing the harder work of figuring out precisely where the line is and defending exactly that line.

If you build on open models yourself, budget for this. The weights are the easy part.

Frequently asked questions

Do open-source AI video models come with content moderation?

No. The weights that produce great video will just as readily produce illegal or abusive content — the model has no built-in safety. Hosted APIs like Veo and Kling bundle moderation invisibly, but when you self-host LTX-Video, Wan, or HunyuanVideo, you are the entire safety layer.

How does LoreMotion moderate AI video generations?

With layered checks: a normalised prompt blocklist and AI text classifiers before any GPU runs, a vision check on uploaded reference images, a multi-frame vision check on finished LTX renders, and context-aware gates on image-to-video jobs to block clothing-swap and fabricated-intimacy abuse on uploaded likenesses. Each layer covers the others' blind spots.

Why not just use a keyword blocklist?

Because it fails both ways. It blocks too much (rejecting "blood" breaks horror prompts) and too little (attackers defeat it with spacing, leetspeak, zero-width characters, or LLM rewrites). Real moderation needs aggressive text normalisation, severity tiers, AI text classifiers for any-language intent, and pixel-level vision checks — not a word list.

Does moderation block normal creative prompts?

Text-to-video beach and fashion phrasing generally works at the prompt level. Pixel checks on uploads and finished videos are stricter about visible nudity, underwear, and swimwear. The strict clothing-swap and intimate-act rules fire only on image-to-video jobs that carry a real uploaded photo, where the harm is transforming an identifiable person — so the broad creative surface for from-scratch generation stays largely untouched.

What happens if the moderation system has an error?

The vision layers fail open on infrastructure errors and fail strict on judgment. If the service times out, legitimate users aren't blocked over an outage; but if the model returns a confident "this violates policy," we act on it. For a free public platform, that's the right balance.