Consistency

Character consistency: fine-tuning vs reference editing

The first thing everyone tries is pinning the seed. It doesn't work: change the prompt even slightly and the face drifts. A fixed seed makes a generation reproducible, it does not make an identity portable — and putting your character in a new scene means changing the prompt by definition.

Approach 1 — fine-tune the model on the character

Train a small adapter — a LoRA, usually — on images of one character, then generate with it. The identity lives in the weights, and within its trained range it holds well. This is the default answer you'll find in most community threads, and for the case it fits it's a good answer.

Where it fits: a fixed cast you control. A studio with a handful of recurring characters trains once per character and reuses them indefinitely — the training cost amortizes across everything they ever produce.

Where it stops fitting: characters created on demand

If your users create their own characters, "train per character" becomes a training pipeline you now operate — every signup queues a job, and someone waits. You've added a second GPU workload with its own queue, failure modes, storage, and cost, sitting directly in your signup flow. The technique didn't get worse; the product shape changed underneath it.

That's the fork. It isn't fine-tuning versus reference editing on quality — it's whether the identity has to exist before the user asks for it, or the moment they ask.

Approach 2 — reference editing (what this kit does)

Generate the identity once as an image, then use that image as the reference for every scene afterward. No training, no per-character pipeline — the anchor is a file.

Stage 1 (once, at character creation) text-to-image ──▶ ONE base image = the identity anchor, saved with the character Stage 2 (every scene, forever after) base image ──▶ edit model ──▶ "same face, new scene" face holds, pose/background change

The same character rendered in three different scenes, with the face unchanged — One base image, three scenes. The face is the reference, not a lucky seed.

The consistency comes from the reference, not the seed. Stage 2 always passes the base image in as input; drop that one input and you get a different face every time — which is, in our experience, the single most common way this breaks.

Three things we learned running it

Prompt the delta, not the person. Re-describing the character in every scene prompt fights the reference and causes drift. Describe what changed — the scene, the pose, the light.
Turn angles gradually. Front to profile in one jump is where identity slips. Ask for intermediate angles instead of the extreme one.
Variety comes from the scene input, not the seed. Randomizing seeds to get variety just reintroduces the drift you were trying to remove; vary the scene instead and pin the rest.

What we're not claiming

We haven't benchmarked fine-tuned adapters against this pipeline on output quality, so we won't pretend one looks better than the other. The claim here is narrower and structural: for a product where users create characters on demand, per-character training puts a GPU training job in your signup path, and reference editing doesn't.

The part that isn't the model

Both approaches leave you with the same infrastructure question: something has to run inference, and it can't be your web server. A generation takes tens of seconds to minutes, so it needs an async job queue, polling to know when the work finished, and storage that serves results without exposing itself.

That plumbing — plus the 2-stage engine above, credits and Stripe, and the 26 traps we hit ourselves — is what OwnStack ships as a working kit and a written course.

See what's in the kit Try the live demo

That demo is the kit, deployed as-is — generate a character and put it in a scene yourself.