focalcut

Cut dance and event footage by who the camera is following: per-subject clips, beat-synced recap reels and editor timelines.

focalcut does two jobs.

• Split a library by subject. Point it at a folder of dance footage, or any video where the camera moves from one person to the next. It finds where the focus changes from one dancer or couple to another and exports per-subject clips plus a timeline for Premiere, DaVinci Resolve, Final Cut or Avid. This path needs no GPU: it uses ffmpeg contact sheets read by a vision model. See Cutting a library by subject.
• Cut recap reels. From a curated folder it builds a landscape and a portrait reel on the music’s beats, told in acts. The reels have velocity edits, motivated transitions, portrait reframing that follows a dancer and favours faces, shake checks, effects and an optional MilkDrop light show. Each comes with a Premiere XML and a review page where the editor picks, bans and reframes shots. The editor’s picks always win. See Recap reels and The editor’s picks.

Every editing rule the planner follows is its own piece of code, listed in Editing concepts, so a teaser, a highlight or a promo can reuse them without the full reel.

• Source: github.com/isaacrowntree/focalcut (MIT)
• Write-up: focalcut on zackdesign.biz

Install

focalcut needs Python 3.10 or later, with ffmpeg and ffprobe on PATH.

The GPU-free pipeline

pip install "git+https://github.com/isaacrowntree/focalcut"
focalcut --help

This installs numpy and OpenTimelineIO and nothing else. It covers contact sheets, the vision-judge library pipeline and timeline export.

Recap reels and effects

pip install "focalcut[reel] @ git+https://github.com/isaacrowntree/focalcut"

The reel extra pulls torch, transformers and the models’ dependencies (SAM 3, BiRefNet, Depth Anything V2, SigLIP 2). The subject pass and the SAM 3 effects want a CUDA GPU; two scoring workers fit a 12 GB card. Set FOCALCUT_DISABLE_CUDNN=1 if your torch build has broken cuDNN engines. Hardware covers what decodes, encodes and renders on which chip.

The SAM 3 engines

focalcut cut uses SAM 3 (facebook/sam3), a gated model. Accept its licence on Hugging Face and run hf auth login first.

pip install "focalcut[sam] @ git+https://github.com/isaacrowntree/focalcut"

SAM 3’s weights are under Meta’s own SAM Licence, separate from focalcut’s MIT licence.

The Claude Code skill

skills/focalcut is a Claude Code skill that lists each editing concept with the code behind it. It isn’t part of the pip package. To install it from a clone:

mkdir -p ~/.claude/skills && ln -s "$PWD/skills/focalcut" ~/.claude/skills/focalcut

From source

git clone https://github.com/isaacrowntree/focalcut && cd focalcut
python -m venv .venv && .venv/bin/pip install -e ".[reel,dev]"
.venv/bin/python -m pytest -q && .venv/bin/ruff check .

Cutting a library by subject

Why this is hard

The hard part is identity under bad conditions. Stage lighting washes every dancer the same colour, lens flare swallows the subject, and dancers spin away from the camera. Naive signals fail. Frame differencing over-cuts on every spin, and colour or even CLIP-embedding similarity can’t tell two magenta-lit couples apart. A SAM 3 detector with an appearance tracker was tried and under-segmented crowded floors. It remains as the optional cut --engine image|video path.

What works is a contact sheet read by a vision model. One thumbnail every few seconds is the most legible identity signal there is: a vision model reading the grid reasons about clothing, hair, faces and recurrence in a way no box metric can.

The passes

All four are built from cheap ffmpeg contact sheets:

1. Coarse: one thumbnail every ~8 s per clip. A judge reads each sheet and returns candidate cut points.
2. Refine: a ±8 s sheet at 2 s spacing around each candidate checks that it’s a real couple change (not flare, a spin or a partner rotation) and gives a ~2 s boundary.
3. Ultra: a ±3 s sheet at 0.5 s spacing pins each boundary to ~0.5 s.
4. Miss-sweep: a sceptical second read of the single-segment clips catches any hand-over the coarse pass missed.

The judge is a vision model fanned out one agent per sheet (Claude, via the scripts in workflows/). On a 50-clip portrait dance library this took about 7 minutes in total and pinned 26 cuts to 0.5 s.

1. Build contact sheets

focalcut library "/path/to/footage" --interval 8 --portrait-only

This writes focalcut/sheets/*.jpg, a sheets.json job list and a skeleton library.json manifest with the cuts left blank.

2. Let a vision judge fill in the cuts

Fan out an agent per sheet using the four passes in workflows/ (01-coarse-judge → 02-refine-cuts → 03-miss-sweep → 04-ultra-boundary). The finished library.json holds, per clip, cuts (seconds), segments and notes describing each couple, so a person can check them.

3. Export a timeline

focalcut export "/path/to/footage"                 # FCP7 XML (default)
focalcut export "/path/to/footage" -f edl          # CMX 3600
focalcut export "/path/to/footage" -f aaf          # Avid Media Composer
focalcut export "/path/to/footage" -f otio         # OpenTimelineIO
focalcut formats                                   # what your install can write

Timelines are built as OpenTimelineIO objects and written through its adapters. Drop-frame timecode, NTSC 1000/1001 rates and mixed-rate conform are handled by the interchange library the post industry already maintains, not by a bespoke XML writer.

--windows-paths (on by default) maps WSL /mnt/d/... to D:/..., so a cut made in WSL opens in Premiere on Windows. Turn it off with --no-windows-paths.

One-off contact sheet

focalcut sheet "clip.mp4" --interval 10

Direct cut (the SAM 3 engines)

focalcut cut "clip.mp4" --clips          # -> clip_dancers/ + clip.focalcut.xml
focalcut cut "/folder" --engine image --clips --copy

Benchmark

bench/ scores predicted cuts against hand-labelled ground truth: boundary precision and recall, over- and under-segmentation, and a manual_corrections_per_min number for how much re-cutting is left. See bench/data/README.md.

Recipe: recap reels

focalcut reel turns a curated footage folder into beat-synced highlight reels — one landscape, one portrait — and a Premiere timeline for each. It was built on a 2.6-hour salsa convention shoot (60 camera files, 97 curated clips) and produces two 3:00 reels from it unattended.

pip install "focalcut[reel] @ git+https://github.com/isaacrowntree/focalcut"
focalcut reel init "/footage/Show 2026" --song track.mp3 \
    --stinger-landscape end_16x9.mov --stinger-portrait end_9x16.mov
focalcut reel run "/footage/Show 2026"

What it expects

Show 2026/
  C2020.MP4 …                       camera files (the reel cuts from these, at full quality)
  export/
    segments/C2020-03_couple.mp4    focal-subject segments (from `focalcut library` + `export`)
    broll/C2006_crowd.mp4           B-roll: used as transitions between dance runs
    matinee/C1990.mp4               any other folder: whole clips, treated as "shows"
  focalcut/library.json             segment and B-roll time ranges

The export folder is your curation: anything not in it is left out. The exported files themselves are not used — each one is mapped back to its source file and time range, so logo intros and stingers baked into the exports never reach the reel.

Everything the recipe writes goes to focalcut/reel/: reel.json (settings), units.json, scores/, moments.json, thumbs/, music.json, plan_<fmt>.json, work/ and out/.

reel.json

focalcut reel init writes the basics; add the rest by hand when a shoot needs it.

The steps

focalcut reel run <folder> --steps … runs any subset; each step reads the previous step’s files.

1. units — one unit per exported clip: source file, time range, role, orientation.

2. score — every source at 640 px: optical flow at 10 fps (camera motion vs. subject motion), people (RT-DETR) and 17-point poses (ViTPose) at 5 fps, sharpness and exposure of the main subject.

3. shake — camera shake per unit, from its export file (GPU-decoded H.264) or the source: corners tracked frame to frame, a RANSAC similarity fit for the camera’s own motion, and what’s left after a ~0.1 s smooth as an RMS over 1 s. A pan or a follow drops out; handheld wobble doesn’t. The planner divides it by the crop’s width (a 9:16 strip shows the shake three times over): too shaky and another part of the clip is used; mildly shaky and the shot is stabilised at render (vidstab).

4. focus — per-person sharpness on the camera’s keyframes at 1280 px: the fallback subject picker.

5. subject — who each shot is about, on keyframes (~1 a second) inside the units: SAM 3 masks every person, BiRefNet says what stands out as the foreground subject, and Depth Anything V2 who is nearest the camera. The subject is the salient, near, large person plus anyone nearly as strong (the couple, or a line on stage), with whether their mask touches the frame edge. Focus alone gets this wrong on stage footage: dancers in motion blur read as soft, still bystanders as sharp.

6. semantic — SigLIP 2 zero-shot on the subject’s crop: hips / butt working (shakes, rolls, the view from behind that animations and shines are about) and expressive faces, against wide, empty or blurry shots.

7. moments — the best non-overlapping moments in each unit, counting only the subject’s moves:

   • spin: the face is seen, then the back of the head, then the face again, within ~1.5 s
   • dip: the torso tilts far from vertical (also catches cartwheels and floor work)
   • trick: a confidently seen ankle above the hips (kicks), or one dancer’s hips above the other’s shoulders (lifts)
   • shake: the semantic hips score, backed by the hips wobbling under a steady torso
   • turn pattern: several turns inside ~4 s with arms passing overhead
   • close-up: a face-forward person whose shoulders span much of the frame
   • travelling shot: the camera moving steadily while the subject stays framed (not marked down as shake)
   • energy and style (arms overhead) fill the gaps; B-roll ranks on steadiness and image quality

   A moment is marked down unless the subject is dancing for the whole stretch a shot plays (about −0.8 to +1.2 s around its peak) and stays in frame throughout. A close-up may hold still, because the face carries it. Someone waiting or walking no longer outranks real dancing. (A VideoMAE Kinetics “dancing” classifier was tried and rejected: it scored a man waiting at 0.76 and real dancing at 0.28.)

   Each moment is also scored per format on how well it frames after reframing (centred, a good size, not cut off — a head at the frame’s top edge scores near zero), and the first quarter of each unit is marked down: routines rarely show their best in the walk-on.

8. faces — a full-resolution frame per moment, face crops from the pose keypoints, and a SigLIP 2 zero-shot expression score (joy, laughter, intensity vs. neutral or turned away). Cached, so re-planning is instant.

9. people — everyone in the footage, for events where people asked to be filmed and should all be in it. Each unit’s export is sampled twice a second through OpenCV’s YuNet face detector and SFace recogniser (built into cv2; MIT / Apache 2.0 models). Faces that move smoothly from one sample to the next are one person whatever the lighting does; tracks that look alike join, inside a unit and then across units. A person counts once seen clearly twice (a confident detection of a usable size). Kisses are found by SigLIP 2 on a frame a second. The plan then makes sure every kiss and as many people as it can are in a shot (cover_people): the moment showing the most people still missing goes in first, in place of the weakest plain shot in its act, or by splitting a long shot when nothing can be replaced – never a shot that is someone’s only appearance, never a unit dropped. review/people.jpg shows one face per person, numbered, to check nobody was missed.

10. music — librosa beat tracking. Salsa tracks at half-time, so one detected bar is an 8-count, which is the cut grid. The window starts on a downbeat and ends on a phrase boundary that keeps the reel inside length. Each beat’s accent (how hard it hits against its neighbours) marks horn stabs and breaks.

11. plan — a story in acts on the song’s bars:

Inside an act a cut gets a transition only when it has a reason, from that act’s own vocabulary and within its caps (grammar.py): a spin cut between two turns, a swipe when the same travel carries across, a zoom pushing in to a face, a flash on a big hit, a glitch at the DJ. Everything else, and every crowd reaction or band insert, is a hard cut on the beat — about 15 transitions in a 3-minute reel. Band players cut in on the hardest hits (members, band_in). Every shot gets a crop that keeps its subject centred: one zoom per shot, the centre gliding with the dancers like a camera operator. Between the subject step’s keyframes (~1 s apart) the crop follows the subject’s own pose boxes at 5 fps, and a portrait strip cut from landscape footage (a third of the frame wide) follows twice as fast. 12. render — 1080p 50 fps (Instagram’s ceiling), effects and transitions applied, music mixed to −14 LUFS. 13. xml — an FCP7 XML per reel for Premiere: plain shots link the original camera files (scaled to fit, with handles); shots with effects, transitions, reframing or rotation use their rendered files (copied to out/<fmt>_media/), with the untouched source on a disabled track above. The song sits on A1 from the reel’s start point, the stinger’s sound on A2. Animated position keyframes are avoided — they don’t survive FCP XML import reliably. 14. review — review/<fmt>.html: a frame per shot, from the rendered reel or else its source. Each shot shows its export file, the time range it uses in that file, and a copyable picks line. Paste the line into focalcut/reel/picks.json and leave it as ban, or change it to prefer with a better range. Then re-run --steps plan,render,xml,review.

• ban: removes moments in that range. "all" removes the whole clip.

• Scrub source on any shot opens the export it came from. The timeline shows that shot, the reel’s other shots from the same video and every moment the planner found. From there you can:

  • mark extra shots anywhere (A)
  • swap the shot to the playhead (S)
  • ban it
  • ask for more from that video

  The page builds the picks.json for you (Copy or Download) and remembers unsaved changes in the browser.

• add: extra shots at those times; each replaces the weakest plain shot in its act.

• more: a number; the clip’s fair-share discount is divided by 1 + more, and its moments rank up.

• prefer: gives moments in that range +1.0. If nothing was detected there, one is made at the middle of the range, with its kind and framing taken from the nearest detected moment.

Serving the review pages

focalcut reel serve "/footage/Show 2026"                          # one project: http://localhost:8765/
focalcut reel serve "/footage/Show 2026" "/footage/Fest 2026"     # several on one port
focalcut reel serve --root /footage                               # every /footage/*/focalcut/reel/reel.json

The server makes the pages save focalcut/reel/picks.json as the editor works (the previous file is kept as picks.json.bak) and answers the source browser’s portrait-crop questions. Each project is served under /p/<slug>/, the slug from its folder name ("SSMC 2026" → /p/ssmc-2026/), and / lists every project’s sources and review pages. Add a project to a running server with curl -X POST -H 'Content-Type: application/json' -d '{"folder": "/footage/New 2026"}' localhost:8765/api/projects, or let --root find it. With a single folder and no --root, the project is also served at the top level as before. Every page has a menu: the project’s sources and reel pages, and All projects (the index at /).

It only answers requests addressed to localhost, and only serves files inside each project’s own folder. A page opened as a plain file still works: changes stay in the browser, with Copy and Download for picks.json.

Times are in the export file’s own timeline. Units record where each export starts in its source (export_t0), so the picks land in the right place.

How it cuts

Each of these is also a standalone concept you can reuse for other clips; the editing concepts list them with the module behind each.

These are the editing rules the planner follows. They come from the editor the recipe was built with, checked against how music videos are cut:

• Every clip, several scenes from each. Each extra shot from a clip has to beat a shot from another clip by a margin (SHARE), a unit’s shots sit at least APART (6 s) apart, and a final pass makes sure every clip in the export appears.
• Card shuffle. An act’s shots are dealt so a clip’s shots spread through the act and interleave with others of the same style: never one clip twice in a row, never two B-roll shots side by side. Static B-roll is held as one long shot instead of several similar ones.
• Cut on the action. A turn cuts to a turn, and travel keeps its direction across the cut. Direction comes from optical flow inside the subject’s box minus the background’s (flow.py, cached in flow_cache.json), so a camera panning with a dancer still reads as the dancer moving. A movement carries two or three cuts, then changes.
• Rhythm in phrases, not one length. Shot lengths come in 8-beat phrases (one salsa 8-count) drawn from varied patterns. Quick patterns go where the music is busier, a long hold falls every fourth quieter phrase, and the night builds into the climax. Moves that need time (turns, travel, patterns) take the long slots, and shakes and hits take the quick ones. Effects go to shots whose slot suits them.
• Hits on the beat. Each move’s peak lands on a beat inside its shot (on the cut itself for a one-beat hit), and every cut comes 2 frames (ANTICIPATE, 40 ms) ahead of its beat.
• The opening is the festival: band and crowd footage with the event’s logo front and centre.
• Velocity edits, not slow-motion blobs. A speed ramp runs fast through the move (never above 1.5x on dancing), eases down to arrive slow exactly as the dancer lands, holds a beat so hair and eyes play out (two in bullet time, more on a close-up hair flip), and eases back up. The landing comes from the subject’s own motion (a burst, then a rest: velocity.settle) and goes on the strongest beat that leaves room for the hold.
• Transitions with a reason. Mostly hard cuts on the beat. Each act has its own short vocabulary and caps (grammar.py): a spin only turn into turn, a swipe only when the same travel carries across, a zoom only to push in to a face, a flash only on a big hit, a glitch only at the DJ. Never two in a row; reactions and band inserts cut straight in. One big subject effect, on the drop.
• Portrait framing. A 9:16 strip from a 16:9 frame follows one dancer of a couple dancing apart (the one showing their face). Shots rank on how the crop holds the subject over the shot (present, wholly inside, a face to the camera); on the social floor a back ranks well down.
• Shake. Shots too shaky for their crop give way to a steadier part of the same clip; mildly shaky ones are stabilised.
• The band. Individual players cut in through the reel on the music’s hardest hits.
• The ending. ending names the reel’s last shot (e.g. the whole room’s hands in the middle); its stretch is kept for it and it runs out exactly at its end time.

Measured against music videos, the first rhythm planner was too even: 33 of 42 show shots were exactly 2 beats, with runs of up to 26 equal lengths, and no move peak landed on a beat. Now no run of equal lengths is longer than 5, every peak is on a beat, and every 8-count starts with a cut.

The research behind these rules:

• Is there a “language of music-video clips”? (548 official music videos, plus interviews with music-video editors and directors).
  • In dance videos the most common shot lasts about one beat (64.5% of clips), with strong bar-level co-occurrence.
  • Editors place cuts with anticipation, just before the downbeat.
  • Editing accelerates near climaxes, and bridges are slower.
• Larry Jordan on cutting montages to music. Clips of one length make the rhythm predictable; cut to secondary beats as well.
• Cutting-rhythm practice: alternate cuts every 2 bars, every 4 bars, and long holds, rather than a single rhythm.

Cuts are beat-exact. Each cut sits on the frame nearest its beat counted from the reel’s start. Rounding each shot to whole frames on its own drifts the cuts by most of a second over three minutes.

Effect modules

focalcut fx list shows them; each works on any clip (focalcut fx apply clip.mp4 -e hero,aura --peak 1.2), and focalcut fx gallery clip.mp4 renders them all side by side.

Subject effects track every “person” with SAM 3 video and keep the subject by the same rule as the subject step (salient, near, large): the couple on a social floor, or the whole line on stage. SAM 3’s masks are coarse once scaled up to 4K, so the edge comes from BiRefNet’s matte, run on a crop around the subject and kept inside SAM’s slightly grown mask. Frames between the tracked ones are blended rather than held, and a little memory stops the edge shimmering.

Hardware notes

Full notes and measurements: Hardware.

• Decoding is the bottleneck. NVDEC before Blackwell can’t decode Sony 4K 10-bit 4:2:2 HEVC. On WSL with an Intel iGPU, the render decodes it with Quick Sync through a Windows ffmpeg build (about 6x less CPU) and encodes with NVENC, several shots at once (FOCALCUT_DECODERS); otherwise it decodes on the CPU. Scoring skips non-reference frames and deblocking, which is about 20% faster and fine for analysis.
• Two scoring workers on a 12 GB GPU. More overflows VRAM; on WSL that pages to system RAM and everything slows to a crawl. Each worker also caps its own threads so they don’t starve the decoders.
• Cap SAM 3 in crowds. Tracking every person in a packed room floods the GPU; the masks module tracks at most 6 confident people at 960 px and scales the masks back up.
• The subject step runs SAM 3 + BiRefNet + Depth Anything on ~1 keyframe a second, about 0.4 s each on a 12 GB card: roughly an hour for a 2.6-hour shoot. Point proxies at H.264 exports of DNxHD / ProRes masters to keep decoding off the critical path.
• Set FOCALCUT_DISABLE_CUDNN=1 if your torch build has broken cuDNN engines.

The editor’s picks

A recap reel is a loop between the planner and an editor. The planner cuts a first reel from the analysis. The editor watches it, marks the shots they want and the ones they don’t, and the planner cuts again. The editor’s picks always win. No rule the planner follows (fair share, framing, rhythm, coverage) overrides a pick.

The pages

focalcut reel serve "/footage/Show 2026"        # http://localhost:8765/

• Sources (review/sources.html) lists every export clip in a big player over a wide, zoomable timeline. The timeline shows a filmstrip, the moments the planner found and where each reel uses the clip. Mark as many in-out selections per clip as you like. In portrait it also shows the planner’s crop and takes focal points you set. The Picks view plays the whole cut, landscape or portrait, with your picks in place.
• Review (review/<fmt>.html) shows one frame per shot of the rendered reel. Each frame has a copyable picks line and a Scrub source button that opens the clip it came from.

Served, the pages save focalcut/reel/picks.json on every change and keep the previous file as picks.json.bak. Opened as plain files, they keep changes in the browser and offer Copy and Download.

picks.json

The keys are export file names. Times are in that export file’s own timeline.

{
  "C2125": {
    "add": ["1:12.40-1:15.10", "5:32.9"],
    "ban": ["0:00-0:20"],
    "prefer": ["0:41-0:46"],
    "more": 2,
    "open": "1:18.6-1:22.8",
    "end": "6:01.0-6:05.5",
    "portrait": [[332.9, 0.50, 0.42], [335.3, 0.55, 0.42]]
  },
  "C1984-05_couple": {"ban": ["all"]},
  "_note": "keys starting with _ are notes"
}

reel.json "picks" takes the same shape and merges over picks.json.

How picks are kept

• Once each, at real speed. An in-out pick plays exactly once, at least as long as marked. It may run in slow motion but is never sped up, and no other shot uses its footage.
• A picked clip gives only its picks. The editor chose its best part, so the rest of that clip is banned. reel.json plan.<fmt>.only_picks: false keeps the rest in play between the picks.
• A pick over a ban wins. The plan log reports it as a conflict.
• The pick check. After every plan, pick_report lists any pick that is missing, short, sped up, doubled, or has another shot over its footage. A clean plan prints nothing.

When there are more picks than reel

An editor can pick more footage than the song holds. The planner then condenses the picks:

• plan.<fmt>.pick_hold (beats): a pick longer than this is cut down to its best pick_hold beats, scored on the scrub’s windows. The editor chose the stretch; the planner chooses the best part of it.
• plan.<fmt>.pick_share (default 0.5): the share of the reel’s beats the picks may take. The rest is the planner’s own footage, for coverage and pacing.
• plan.<fmt>.pick_clip_share (default 1.5): how far above an even share one clip’s picks may go. Raise it when the editor’s picks bunch in a few clips; otherwise those clips lose picks to the cap.

A pick-heavy reel, as a profile:

"profiles": {
  "picks-heavy": {"plan": {
    "landscape": {"pick_hold": 4, "pick_share": 0.9, "pick_clip_share": 4.0},
    "portrait":  {"pick_hold": 4, "pick_share": 0.9, "pick_clip_share": 4.0}}}
},
"use": ["picks-heavy"]

On Electric Nation 2026, with 132 picks bunched in a few long clips, this took the landscape reel from 90 to 127 picks kept.

Review rounds

• reject (reel.json): stretches a review turned down, in picks syntax by export file. The planner treats them as bans, but they live apart from the editor’s own picks.json.
• refill (reel.json, true = the last cut rendered, or a plan file name): re-cut by keeping every shot of an earlier plan and moving only those whose footage is banned now. Each moves along its own clip first, else onto another clip’s footage that passes the rules. A review round then replaces only what was rejected, instead of reshuffling half the cut into shots nobody has looked at.

focalcut reel run "/footage/Show 2026" --steps plan,render,xml,review

Profiles

reel.json "profiles" are named rule sets. The main reel, or any cut, stacks them with "use", in order, then applies its own keys. "footage" and "ban" add up across profiles, "plan" merges per format, and any other key is taken from the last profile that sets it.

"profiles": {
  "no-night": {"footage": ["!night"]},
  "short": {"length": 90}
},
"cuts": {
  "day": {"formats": ["landscape", "portrait"], "use": ["no-night", "short"]}
}

Keep an editor’s standing rulings as profiles. The next event’s reel can then reuse them by name.

focalcut editing concepts

These are the rules the recipe’s editor cuts dance recaps by, checked against how music videos and aftermovies are made. Each section gives the rule, the reason, how focalcut measures or enforces it (module and function under focalcut/ in the repo), the numbers, and the reel.json knob where there is one. Use them together for a full reel, or pick out the ones a new clip needs.

1. Opening: the festival first

Rule: open on B-roll of the band or the packed room, with the event’s logo big and centred over it. Never open on a dance shot. Why: the first second says where we are. Dancing makes sense once the place is set. Code:

• plan.story teaser act: two different clips ranked by opening tags (band-wide, band-crowd, band, crowd).
• The logo goes in on the first shot and holds across the cut (render.cut_shot, logo).
• With no festival B-roll, the opening falls back to the two best moves in slow motion.

2. A story in acts

Rule: tell the day.

Why: a recap that is only highlights has no shape. The acts give it a beginning, a build and a peak. Code:

• plan.story and plan.default_acts; a signature transition at each act change (grammar.SIGNATURE).
• Knobs: acts (bars; put the boundaries on the song’s section changes), split, day, day_shows, night_shows.

3. Use everything: card shuffle and fair share

Rule: every file in the export appears in each format it belongs to, with several different scenes from each. This is a hard requirement: use more, shorter cuts rather than leave a file out. A file is the unit, not the camera clip: one clip split into five social segments is five files, and each one appears. Footage shot in portrait (the social segments) is portrait-only; landscape never uses it.

• A clip’s shots are spread through the reel and interleaved with others of the same style (the matinee routines weave through each other).
• Never play one clip as a block of consecutive shots, cut a clip to itself, or put B-roll next to B-roll.

Why: the people who are in the footage watch the reel. Leaning on a few strong clips leaves people out, and blocks feel like separate videos. Code:

• Picker: every extra shot from a clip must beat other clips by SHARE = 0.3 per shot it already has, and a unit’s shots sit APART = 6 s apart.
• cover orders clips by their best moment as it plays in this format.
• weave deals the act like cards.
• cover_all puts every missing unit in: it takes the place of the weakest plain shot of a unit with several, in its home act (the social floor may also go in the climax), or splits a long plain shot to make room. The band inserts and the editor’s extra shots never take a unit’s only shot.

4. Cut on the action: movement flow

Rule: movement carries across the cut. A spin cuts to a spin, and travel to the right keeps going right. A movement carries 2–3 cuts, then changes. Why: matched action makes cuts invisible and keeps the energy going. Code:

• flow.py: optical flow inside the subject’s box minus the background, so a camera panning with a dancer still reads as the dancer moving. Cached in flow_cache.json.
• plan.flow_link scores a cut from −1 to 2; weave uses it.

5. No same-y runs

Rule: don’t cut between near-identical shots. If footage is static, keep one longer shot; if it is fairly static, break it up with other dancing. Code: B-roll is held long (arrival: 3–5 beats), the same B-roll family is penalised within 3 cuts, and the same move kind is penalised after 2 in a row (weave).

6. Music-video rhythm

Rule:

• Shot lengths come in 8-beat phrases (one salsa 8-count) drawn from varied patterns: quick cuts, bar-length shots, occasional long holds.
• Busier phrases of the music get quicker patterns, and the night builds into the climax.
• Never long runs of equal-length shots.
• Each move’s peak lands on a beat, and cuts come 2 frames (40 ms) before the beat.

Why: in a study of 548 official music videos, dance-video shots mostly last about one beat. Editors cut with anticipation, accelerate near climaxes and slow at bridges. One shot length makes the rhythm predictable (Larry Jordan). Code:

• PHRASES, rhythm, phrase_energy, fit_slots (moves that need time get the long slots), in_point, ANTICIPATE.
• Cut frames are counted from the reel’s start (per-shot rounding drifts most of a second over 3 minutes).
• Salsa tracks at half-time, so one detected bar is an 8-count.

7. Subject: who the shot is about

Rule: the subject is centred and fully in frame, with no head cut off. Effects target the subject, not bystanders. Why: sharpness alone gets stage footage wrong. Dancers in motion blur read as soft, and still bystanders as sharp. Code:

• subject.py / subject_engine.py: SAM 3 masks every person, BiRefNet saliency and Depth Anything nearness pick the salient, near, large person, plus anyone nearly as strong (the partner, or a line on stage).
• frame.track interpolates between keyframes, and frame.framing scores centring, size and edge contact.

8. Moments: the middle, where the dancing is

Rule: for routines and animations, use the middle sections, where the butt shakes and faces are; skip the walk-on. The subject must be dancing for the whole stretch a shot plays. Code:

• moments.py detectors:
  • spin: face, back of head, face, within about 1.5 s
  • dip: torso far from vertical
  • trick: ankle above hips, or a lift
  • shake: SigLIP hips score plus hip wobble under a steady torso
  • pattern, close-up, travel, energy, style
• The first quarter of each unit is marked down, and an activity gate drops people waiting or walking.

9. Velocity edits (speed ramps)

Rule:

• Fast through the move, then ease down to arrive slow exactly as the dancer lands (the end of a turn, a pose hit).
• Hold there so the hair and the eyes play out, then ease back up.
• Dancing never runs faster than 1.5x; any faster and people look sped up, not energetic.
• Super-slow close-ups of hair flips are great.

Why: this is how CapCut velocity edits and Premiere / After Effects time-remap ramps are built. Slowing mid-move gets it backwards: the fast part looks slow and the hold has nothing to show. Code:

• fx/speed.py Speed.curve uses cosine eases with no corners. Presets (fast×, slow×, hold beats, ease beats):

  Preset	fast	slow	hold	ease	Use
  montage	1.5	0.5	0.75	0.25
  hero	1.4	0.3	1.0	0.4
  bullet	1.5	0.15	2.0	0.6	the reel’s peak

  • A close-up slows a further ×0.6 and holds 0.5 beat longer.
  • cap param: 1.5 for dancing, 3.5 for B-roll.

• velocity.settle(project, unit, t) finds the landing: subject motion is a burst in the 0.6 s before and a rest from 0.1 to 0.6 s after, searched over −1.0 to +0.8 s.

• velocity.landing() puts the landing on the strongest accent beat that leaves room for the hold; plan.place applies both.

• One-off: focalcut fx apply clip.mp4 -e hero --peak <landing s> [--closeup].

Knob: heroes, the slow-motion holds in the shows act.

9b. Show the landing (every dance shot)

Rule: when a dancer finishes a trick and comes to rest, the shot stays on them a little longer so the audience sees them land gracefully. Never “dancer does a trick” without “dancer finishes the trick”. Code:

• velocity.landed(project, unit, t) finds where the move around the peak comes to rest (burst then rest, from 0.3 s before the peak to 1.6 s after), or None.
• plan.lands marks each plain dance shot’s landing and, when the shot is too short to show the end of the move (velocity.PRE = 0.5 s) and the rest, takes a beat from a plain neighbour in the same act.
• velocity.hold_in moves the in-point as little as possible so both fit; a one-beat hit too short for both lands 40% in.

Knob: land_hold, beats of rest after the landing (default 1, at least 0.45 s).

10. Transition grammar: cohesive, motivated, mostly hard cuts

Rule: mostly hard cuts on the beat. Each act has its own short vocabulary and a cap on each transition, and every transition needs a reason in the two shots it joins. Never two in a row; crowd reactions and band inserts cut straight in. About 15 transitions in a 3-minute reel, not 45.

Reasons:

• spin: turn or pattern into turn or pattern, from different clips.
• swipe: both shots travelling the same way, flow ≥ 0.12 each side.
• zoom: push in to a close-up that is at least as tight.
• flash: a big musical hit.
• glitch: in or out of the DJ.
• fade: between daytime scenes.

Why: music-video and aftermovie editors use hard cuts for rhythm and save transitions for moments that mean something. A transition cycled from a menu reads as a template, and that random look is the main complaint about CapCut-style edits. Code: grammar.py (GRAMMAR, CAPS, FLASH_AT, SIGNATURE, reasons()), plan.transitions. Knobs: grammar, caps, flash_at.

11. Cohesive effects, not gimmicks

Rule:

• Effects serve the story: one at a time, never on neighbouring shots.
• Each act has its own effects:
  • shows: three slow-motion holds and one spotlight
  • night: one neon glow (the club) and two holds
  • climax: holds building to one bullet-time move in fire (the only subject effect, on the drop)
  • finale: a face, a held move, a hair flip in deep slow motion
• No per-beat punch-ins, orbs or random overlays. No freeze-frame outline effects (they read as gimmicks).

Code: plan.give_slot_fx (skips a shot whose neighbour has an effect), the effect choices in plan.story, and the subject effects in fx/subject.py (aura, neon, spotlight, …).

12. Portrait framing (9:16 from 16:9)

Rule:

• The strip is a third of the frame wide, so track the subject tightly.
• When a couple dances apart, follow one dancer (the one showing their face), not the empty floor between them.
• Strongly favour faces; on the social floor, avoid people’s backs.
• Favour shots where the subject stays wholly in frame and isn’t partly leaving it.

Code:

• frame.follow(d, times, union, cw):
  • If the group is wider than 0.8 of the strip in ≥30% of frames, it tracks one dancer box to box (jumps under 0.12) and holds the last position through gaps.
  • The dancer is scored by face ×2, coverage, √area and sharpness.
  • Otherwise it reports the face of the main (largest) member.
• frame.face_seen: with the unit’s YuNet face scan (scores/faces_<unit>.npz, the people step), a person shows their face when a confident face (score ≥ 0.8, height ≥ 3% of the frame) sits in the head of their pose box within 0.3 s. ViTPose puts confident eyes on the backs of heads on the social floor, so the pose test frame.faces(kp) (nose and both eyes above 0.6, left eye right of right eye) is only the fallback. Run the people step (or at least its face scan) before planning a portrait cut of social dancing.
• plan.face_track: when the body track’s strip doesn’t hold the shot’s face (a crowd shot up close, where the pose boxes lose the person or settle on the one beside them), the strip follows the face instead: the persons step’s dense face scan, only faces in the head of one of the shot’s main people, and only one near the strip already (the planner’s subject, never a bigger face elsewhere). The face sits in the upper third; a second face that fits shares the strip; a face too big for it zooms out. It takes over only when it holds a face for clearly more of the shot (FACE_GAIN 0.25). On Electric Nation 2026’s portrait reel it re-centred 45 of 142 shots.
• plan.reframe returns the crop plus present (≥60% of the subject inside), inside (≥92% inside and not cut by the camera’s edge) and face, each as a fraction of the shot.
  • The crop centre is smoothed over about 0.3 s in a narrow strip and 0.6 s otherwise.
• plan.assess multiplies each moment’s rank:
  • present < 0.6: ×0.04
  • inside:
    • social floor: (0.3 + 0.7·inside)
    • others: (0.45 + 0.55·inside)
  • portrait face:
    • social floor: (0.3 + 1.4·face), and a further ×0.25 if face < 0.3
    • others: (0.5 + 1.0·face)
  • landscape face: 0.85 + 0.3·face
  • The editor’s picks are never dropped.

13. Shake

Rule: measure camera shake. If a stretch is too shaky, use another part of the same clip; if it is mildly shaky, stabilise it. The worst offenders are handheld talking heads (panels) and animation clips. Code:

• shake.py gets camera motion from corner tracking (LK) plus a RANSAC similarity fit, so dancers are outliers. hf is the motion minus its 5-tap (~0.1 s) smooth, as an RMS over 1 s, in frame widths per frame.
  • Pans and follows drop out.
  • It reads the GPU-decoded H.264 export when there is one.
• plan.assess divides hf by the crop’s width (a 9:16 strip shows the shake three times over):
  • above shake[1] (0.006): rank ×0.03
  • above shake[0] (0.0012): up to ×0.65
• place sets stab when the shot’s own jitter is above shake[0] (not on _stabilized sources).
• render.stab runs vidstab (detect shakiness 6; transform smoothing 15, optzoom 1).

Reference levels:

14. Band members interspersed

Rule: individual players (singer, violin, horns, percussion) cut in through the reel on the music’s hardest hits, one at a time. Why: the live band is half the event, and a player on a horn stab ties the picture to the sound. Code:

• plan.band_in: each act gets its count of short inserts.
  • Each replaces a plain dance shot of 2 beats or less on the highest-accent beat near evenly spaced positions.
  • Never next to other B-roll, an effect or another insert, and never with a transition.
• Knobs: members (tag substrings), member_clips, band_in (shows=2, night=3, climax=3).

15. The ending shot

Rule: end on a chosen shot that sums up the event. In the reference shoot (a 2026 salsa congress) that is the demo where everyone puts a hand in the middle and bursts up (C2023 at 126.6–133.4 s). Code:

• plan.reserve_ending keeps ±4 s around it out of other shots.
• plan.ending_record finds it; the finale gives it the remaining beats.
• place sets the in-point so the shot runs out exactly at its to time, with a centred, unzoomed crop at the optional x.
• Knob: ending: [export file, from, to, crop x?], in that file’s own seconds.

16. The song’s start

Rule: the reel starts where the song proper starts, on a downbeat. Downloaded tracks often have an intro: the reference shoot’s track starts about 0:17, on the bar where the music returns after a break. Code: music.analyse(start="auto" | seconds, start_hint=…) picks the bar nearest the hint, and the window ends on a phrase boundary inside max_len (reel.json length). Knob: top-level start_hint.

16b. Several songs, one per act (a festival’s run of sets)

Rule: at a festival with a lineup, each act can play a song by the DJ whose set its footage comes from, in set order, mixed into each other the way a DJ would, so the music never stops or lurches. Each song gives its window around its strongest drop (a few bars of build, then the drop). Into each hand-over the outgoing song rides its tempo to the next song’s over 8 bars (time-stretched, pitch kept; Americano at 123 into Shooting Stars at 125 is inaudible, 112 into 123 sounds like a build), then both play beat on beat for 4 bars centred on the hand-over: the highs crossfade across it and the bass swaps in the half beat before its downbeat, so two kicks never stack. The songs are level-matched. Only songs more than 15% apart in tempo still cut on the downbeat. Code: music.join(songs, max_len, mix=…) (window per song by drops, the low end coming back after a build, and song_window; each section’s source->output beat map, ramp); music.mix renders music_mix.wav (warp lays each song on its beat map with Rubber Band through ffmpeg). The joined grid keeps each song’s real beat times, so plan.clock places cuts on them across a tempo change, and each shot carries its own beat. Knob: top-level songs: [{"song": path, "bars": 16, "pre": 8, "acts": {"teaser": 2, "arrival": null}}, …]; the last song’s bars may be null (whatever fills length). acts names the acts each song carries, in bars (null: the rest of that song). Top-level mix: {"bars": 4, "ramp": 8, "max_ramp": 0.15, "split": 150} (overlap bars, ramp bars, the largest tempo gap to ramp across, bass split in Hz); false cuts between songs.

16c. The headliner’s set saved for the end

Rule: the last set’s footage (the biggest crowd) stays out of the earlier acts, and the climax and finale cut from it first. Code: plan.story with late. Knob: plan.<fmt>.late: categories or B-roll tags (substrings), e.g. ["headline"].

17. Clean effect masks

Rule: subject effects need clean edges. Code: fx/masks.py uses SAM 3 video to track and BiRefNet’s matte on a crop around the subject, kept inside SAM’s slightly grown mask. It blends between tracked frames and adds a little memory against shimmer. It caps at 6 people at 960 px in crowds.

18. Deliverables and the editor’s loop

Rule: always deliver the rendered video plus an FCP7 XML, plus a local HTML review page. The editor answers with picks: ban a range or a whole clip, prefer a range. If nothing was detected in a preferred range, a moment is made there. Code: xml_export.py (rendered shots on V1, sources disabled on V2, song on A1, stinger on A2; no animated position keyframes), review.py, plan.apply_picks.

19. Pick rules: what never makes the cut

Isaac’s rules for which stretch of footage may become a shot. They are for every agent, every event and every format: landscape, portrait and the profile (portrait) segment videos alike. They also cover the planner’s own picks, cover and people shots, recasts, band inserts and openings alike (only the editor’s own picks are exempt). The scrub (windows.py) checks each one over the whole span a shot plays, and plan.snap_windows / plan.window_report keep every planner shot on passing windows. A rule the scrub cannot check yet is a bug in the scrub, not a reason to let the shot through.

The editor’s picks always win, in every format: no rule overrides a pick. A clip with picks is the editor’s call on its best part, so the planner takes nothing else from it, in either format (apply_picks, on by default; plan.<fmt>.only_picks: false lets the rest back in). The planner may still change a pick itself: cut it to its best beats, take a second piece of it, condense a clip’s many picks (pick_hold, pick_share).

The face that counts is the subject’s own: the person the shot is about, and for a couple, duo or group, every person in the action. A face somewhere else in the frame (a cheering crowd behind, a friend at the edge) never makes a back pass.

A shot is never:

• A back. No subject seen from behind for the shot (the back of a head, a back-turned dancer hiding the partner, a singer facing away). Slide along the clip to where the face or the partner shows. (head, body)
• Turned away. A face looking down at a phone, away from the action, or turning its back during the shot fails. Side-on alone is not a back: a profile engaged in the moment (singing, dancing, looking at a partner) passes (EN: C2107, C2091, C2103 kept). Take the stretch where they turn back toward the camera.
• Behind a foreground back. When the largest, nearest person in frame shows their back, the shot fails even if the people beyond them are smiling at the camera.
• Blocked or at the edge. A subject hidden behind their own arm or another person, or half out of the frame at the edge, with a stranger’s head poking in. (blocked, exit)
• A band from behind. A band shot is about a player’s face: the front man, the guitarist, the keyboard player facing the camera mid-song. A front man’s bare back, even with arms up, never stands in for that while the set has stretches with his face in full view. Take the band’s best faces first.
• A group that isn’t together. When half the group has turned away or they’ve drifted apart instead of dancing with each other, it isn’t a group shot. Pick the stretch where they’re together and facing the camera.
• The crowd. A pan off the dancers onto an audience sitting or standing still, or a single bored onlooker as the only face, is never a subject shot. Someone has to be dancing or performing for the whole shot. (idle)
• Blurry. Haze, stage fog and a dark but readable face are fine (EN: C2130, C2125 kept). The subject’s face must not be smeared, though, not just something in the frame: a sharp foreground arm in front of a soft dancer fails. Missed focus, a face smeared by camera motion, or a face pressed up against the lens all fail. The floor is absolute, not relative to the clip, and it applies after speed effects too: frame-blending on a fast stretch must not smear a close face. (soft, blocked)
• Shaky, leaving, sliced or low. Wild camera swings, the subject leaving the frame, a couple split by the strip edge, or the subject far down the frame all fail. (shake, exit, split, low, lost)
• The gap between people. When a pair or a group is wider than the portrait strip, frame one dancer or couple whole and centred and let the others go out of frame. Never centre the floor between two couples and cut both (SSMC: C1991’s yellow couples). Groups take the dancer in the centre by default (frame.GROUP “centre”; reel.json "group": "pan" asks for a pan). B-roll shot among the dancers up close, like a class floor, follows one dancer the same way (plan.close_dancers; SSMC: C1968). Someone the camera’s own edge cuts in half is never one of a couple.
• Bored, blinking or grimacing. Engaged expressions only: smiling, eyes open, in the moment. This is about expression, never appearance. (expr)
• A weaker moment of the same person. If the same clip shows that person smiling and letting loose somewhere else, use that stretch. Rank a clip’s passing windows and take its best, not the first that passes.
• A clip’s weak moments, repeated. Several bad picks from one clip (the dinosaur riders: back, profile, blur) mean its best stretch was never found. Rank the clip’s windows and use the top ones.
• A repeat that’s worse. A person or group coming back for a second shot needs a moment as good as the first, from a different stretch.

Effects follow the same rule: a subject effect (glow, outline, matte) takes in every person in the action or none of them. Never cut through a person, leaving them half in and half out of the effect.

Selection order: scrub every clip densely for windows that pass all of these, then choose moments and coverage from those. Don’t pick by energy and reject afterwards.

Hardware: what decodes, encodes and renders where

focalcut renders with whatever the machine has. Plain ffmpeg on the CPU always works; everything below is optional and found at run time (focalcut/reel/hw.py).

The three chips

Plain shots render several at once (render.DECODERS, or FOCALCUT_DECODERS="qsv=5,cpu=2"); effect shots, which run their own GPU models, go one at a time beside them.

Sony XAVC HS (10-bit 4:2:2 HEVC) and hardware decode

Sony’s 4K 10-bit 4:2:2 HEVC is the hard case:

• NVIDIA NVDEC has no 4:2:2 HEVC before Blackwell (RTX 50). On an RTX 4070 Ti, -hwaccel cuda fails with “Hardware is lacking required capabilities” and ffmpeg falls back to the CPU.
• Intel’s media engine has it (Core Ultra 200S “Arrow Lake” here; Intel iGPUs since Ice Lake). Through D3D12 the Intel driver reports D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_422 at 4K50 into Y210 as supported.
• Mesa’s D3D12 VA-API bridge in WSL doesn’t map it (checked against Mesa main, November 2026: the D3D12 decoder maps only HEVC Main and Main10; its 4:2:2 profiles are for encode). No package or setting changes that; it would take a Mesa patch.
• ffmpeg’s generic D3D11/D3D12 decode doesn’t either: it falls back to the CPU without an error. Only Intel’s own hevc_qsv decoder works.

So on WSL, focalcut decodes those files with a Windows ffmpeg build using Quick Sync, frames copied back to memory for the usual CPU filters, and encodes with NVENC:

Decoded frames are pixel-identical to the CPU decode. -async_depth 1 is required: at the default depth hevc_qsv labels each frame three frames late, so -ss lands three frames early.

With 3 Intel + 3 CPU shots at once on a 24-thread Core Ultra 9 285K, the plain shots of a 140-shot landscape reel took about 4.5 minutes (one at a time on the CPU with libx264: about 30), with the CPU at about 85% and the iGPU’s decoder at about 40%; hence the default of 5 + 2.

Setting it up (WSL)

1. Download a Windows ffmpeg build (for example BtbN’s win64-gpl zip) and unpack it under C:\tools, so that C:\tools\ffmpeg-*\bin\ffmpeg.exe exists, or point FOCALCUT_WIN_FFMPEG at it. FOCALCUT_WIN_FFMPEG="" turns it off.
2. Only files on a Windows drive (/mnt/<letter>/) go through it; stabilised shots stay on the Linux side (their vidstab motion file lives there).
3. Encoding uses NVENC if the GPU has it (constant quality 16, SPS/PPS in every keyframe so shots from either ffmpeg join by stream copy). FOCALCUT_ENCODER=x264 forces libx264. The encoder is part of each shot’s cache key.

VA-API in WSL (for the formats it does support)

WSL has no /dev/dri until the vgem module is loaded, and Mesa’s VA-API driver also needs GALLIUM_DRIVER=d3d12 (without it: “resource allocation failed”):

sudo modprobe vgem                                         # persist: echo vgem | sudo tee /etc/modules-load.d/vgem.conf
sudo chmod 666 /dev/dri/card0 /dev/dri/renderD128          # persist: a udev rule, SUBSYSTEM=="drm", MODE="0666"
export LD_LIBRARY_PATH=/usr/lib/wsl/lib LIBVA_DRIVER_NAME=d3d12 GALLIUM_DRIVER=d3d12
MESA_D3D12_DEFAULT_ADAPTER_NAME=Intel vainfo --display drm --device /dev/dri/card0

That gives H.264, HEVC Main/Main10, VP9 and AV1 decode and encode on either GPU, but not 4:2:2.

Colour tags

Every render is tagged BT.709 (primaries, transfer and matrix) in the H.264 stream itself, through ffmpeg’s h264_metadata bitstream filter (hw.VUI). Shots encoded from raw RGB (transitions, effects, the visualiser) used to come out with primaries and transfer unspecified. Players guess then, and Android guesses BT.2020 for 4K, which oversaturates the picture. The final mux and the media copied for the XML are re-tagged too, so shots cached before the fix are covered.

Other notes

• Scoring decodes every 4K frame on the CPU (about 1.6x real time); it skips non-reference frames and deblocking, about 20% faster and fine for analysis. reel.json proxies can point it at 1080p H.264 copies that NVDEC decodes (see the recipe).
• Two scoring workers on a 12 GB GPU. More overflows VRAM; on WSL that pages to system RAM and everything slows to a crawl.
• Cap SAM 3 in crowds: the masks module tracks at most 6 confident people at 960 px.
• The MilkDrop overlay renders natively with projectM 4 (focalcut/reel/milkdrop/mdrender.cpp, built once by milkdrop/build.sh into ~/.cache/focalcut/bin; needs cmake, g++ and the EGL / GL headers): about 100 fps at 1080p on an RTX 4070 Ti, frames read back raw and encoded on NVENC. On WSL it reaches the GPU through Mesa’s D3D12 driver (GALLIUM_DRIVER=d3d12, MESA_D3D12_DEFAULT_ADAPTER_NAME=NVIDIA, LD_LIBRARY_PATH=/usr/lib/wsl/lib, set by the overlay).
• Set FOCALCUT_DISABLE_CUDNN=1 if your torch build has broken cuDNN engines.
• Another Mac or PC could take shots too (ffmpeg with VideoToolbox, over SSH and a shared folder), but it only speeds up plain shots; measure where a render’s time goes first.

Security Policy

Reporting a vulnerability

Please report security issues privately through GitHub Security Advisories: Report a vulnerability.

Don’t open a public issue for anything security-sensitive. You can expect a first reply within a week.

Supported versions

Only the latest commit on main gets security fixes. There is no support commitment before a 1.0 release.

Threat model

focalcut runs on your own machine, over footage you shot.

• Media is decoded by ffmpeg. Every video, image and audio file goes through ffmpeg / ffprobe from your PATH, so keep that up to date. Treat footage from elsewhere as untrusted input to ffmpeg.
• The review server is localhost only. focalcut reel serve answers only requests whose Host header names this machine (localhost, 127.0.0.1, [::1]). A page on another site that rebinds its DNS to 127.0.0.1 still sends its own host name and is refused. It serves only files inside each project’s own folder, and writes only that project’s picks.json. Don’t expose its port to a network.
• Model weights are downloaded from Hugging Face. The reel and sam extras fetch SAM 3, BiRefNet, Depth Anything V2 and SigLIP 2 on first use. BiRefNet loads with trust_remote_code=True, which runs Python from its model repository (ZhengPeng7/BiRefNet). Pin a revision or review that code if your threat model needs it.
• reel.json and picks.json are trusted. They name files and ffmpeg filters (rotate) that are passed to ffmpeg. Don’t run a reel from a project folder you didn’t make.

Contributing to focalcut

Contributions are welcome — bug reports, footage that breaks the cutter, and pull requests alike.

What I promise you

• A reply within a week. If a PR needs changes, you will hear what they are in one message, not discovered one at a time over months. If I go quiet, ping the PR — that is a failure on my side, not rudeness on yours.
• A decision, not a limbo. If something is not going to be merged, I will say so and say why, while the branch is still fresh in your head.
• Credit. Merged work is credited in CHANGELOG.md under your GitHub handle.
• Small PRs get taken over, not bounced. If a change is nearly right and I have the context to finish it, I will finish it and merge rather than send you round again.

Before you open a PR

• One change per PR. A bugfix and a feature in one branch is two PRs. This is about review, not bureaucracy: a focused branch is one I can read and merge in an evening.
• Run the suite. pytest must pass, ruff check . must be clean. Both run in CI on Python 3.10–3.13.
• Add a test for behaviour you change. Especially anything touching frame timing — a cut landing one frame early is invisible in review and obvious in an NLE.
• Describe it in your own words. A couple of honest sentences about what broke and what you did beats a generated changelog. Use whatever tools you like to write the code — just make sure the description is something you could defend out loud.

Development setup

git clone https://github.com/isaacrowntree/focalcut
cd focalcut
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"        # GPU-free; add ".[dev,sam]" only for the SAM engines
pytest

You need ffmpeg and ffprobe on PATH. The suite synthesises its own fixture clips with ffmpeg, so there is no media to download and no GPU, network, or Hugging Face access required — if a test needs any of those, it is a broken test.

Project shape

A note on timeline formats

focalcut does not hand-write NLE formats. Timelines are built as OpenTimelineIO objects and written through OTIO adapters, so drop-frame timecode, NTSC 1000/1001 rates and mixed-rate conform are handled upstream by the library the post industry already maintains. If you want focalcut to support another editor, the answer is almost always an OTIO adapter rather than a new writer in this repo.

Licence

focalcut is MIT. By contributing you agree your work ships under that licence.

Please do not paste code from a source with an incompatible licence — in particular, anything under a non-commercial or source-available licence cannot go into an MIT project, however small the snippet. Facts about a file format (frame rates, field names, how a rotation flag is signed) are not copyrightable and are fine; someone else’s implementation of them is not.

Changelog

All notable changes to focalcut are documented here.

The format follows Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

Docs

• Docs site at https://isaacrowntree.com/focalcut/ (mdBook from docs/, book.toml, rebuilt by .github/workflows/pages.yml when the docs change): install, cutting a library by subject, recap reels, the editor’s picks, the editing concepts, hardware, security, contributing and this changelog.
• docs/picks.md: the review pages, picks.json (with portrait focal points), how picks are kept and condensed, review rounds (reject, refill) and reel.json profiles. The recipe’s reel.json table gains exposure, reject, refill, cuts, profiles / use and the pick keys.
• SECURITY.md with the threat model (ffmpeg on untrusted media, the localhost-only review server, BiRefNet’s remote code); an issue-template config pointing at private advisories and the docs; package metadata (description, keywords, classifiers, documentation, issues and changelog URLs); a README hero image.

Fixed

• Portrait strips follow the subject’s face when the body track misses it (plan.face_track): on close crowd shots the pose boxes lost the person or settled on the one beside them, and the strip showed a shoulder, a back or the gap between people. Only the shot’s main people’s faces count, and only one near the strip already, so a bigger face beside a masked or turned subject never takes the shot.

• Effect masks keep what the matte joins to the subject. BiRefNet’s edge was clipped to SAM 3’s “person” mask, which stops at the arm, so a skirt flung out, a fan or a serpentine dancer’s wing on its wand was cut out of the glow, outline and aura. Matte pieces joined to the subject now stay in, within the crop around it; pieces apart from it still stay out.

• Renders carry BT.709 primaries, transfer and matrix in the H.264 stream. Raw-RGB encodes (transitions, effects, the visualiser) were untagged, and Android read them as BT.2020 at 4K and oversaturated them.

• People coverage no longer rates footage that is banned anyway (a picked clip’s other footage): a portrait plan with most clips picked went from over 30 minutes to about 2.

Changed

• MilkDrop renders natively with projectM 4 (reel/milkdrop/mdrender.cpp, off-screen EGL, a fixed frame time and the song’s own samples per frame), about 6x faster than Butterchurn in headless Chromium and sharper: frames are read back raw, not as JPEG. milkdrop/build.sh builds projectM and the renderer once into ~/.cache/focalcut/bin; presets are the same .milk sources Butterchurn’s packs were converted from, downloaded on first use. The Chromium path (render.mjs, cdp.mjs, FOCALCUT_CHROME) is gone, and with it the need for node.

Added

• reel.json plan.<fmt>.pick_clip_share (default 1.5): how far above an even share one clip’s condensed picks may go. Raise it when the editor’s picks bunch in a few clips (Electric Nation 2026: 90 to 127 of 132 picks kept).

• reel.json "refill": re-cut by keeping every shot of an earlier plan and moving only those on footage banned since.

• reel.json "exposure" "lift": night subjects that sit dark get a small midtone lift on luma, black and white held.

• Cuts (reel.json "cuts", focalcut reel run --cut <name>): other reels from the same footage and analysis, e.g. a portrait-only night reel of everything shot vertically plus the band ("footage": ["@portrait", "band"]) and a day reel of the rest (["@landscape", "!band"]). Each cut has its own formats, footage, picks (those of its own footage) and setting overrides ("plan" merged per format); out as <name>_<cut>_<fmt>. Footage matches (plan.fits) take @portrait / @landscape for the way a clip was shot.

• MilkDrop overlay "edge_fill" (default 0.35: how much of a corner MilkDrop’s unkeyed ground may cover, the rest keeps the darkened footage) and "bare" (default 0.5: how much deeper the fade gets where a preset lights little of the border).

• Style libraries (focalcut/fx/looks.py): glitch, carnival and edm, each a family of looks for a whole film; reel.json "look" names one (names are unique across libraries). A look can carry its overlay (MilkDrop presets, flickers, strength), used as the default under reel.json’s own "overlay". New carnival library (fx/carnival.py): fiesta glow, sequins, confetti pop, serpentina and pop-art freeze on the dancer; whip pan, confetti cannon, colour flash, serpentina wipe and sun iris transitions; zoom punch, colour strobe, sun flare, mirror ball and halftone accents; looks carnaval, tropical and disco with fireworks, flowers, disco and fire MilkDrop presets. New edm library (fx/edm.py): mainstage (MilkDrop + white/dark flickers) and lasershow (the built-in light show). focalcut fx lookbook --library carnival.

• Picks cut to their best part (reel.json plan.<fmt>.pick_hold, beats): each selection longer than that plays only its best window, scored on the moment curve (moves, quality, framing in that format, activity, face); a clip with many selections is condensed to its best ones, and the picks fill about half the reel (pick_share), a long one earning a second piece when there is room. The picks then take turns with the rest of the footage (the rest of a picked clip included), no act more than ~55% picks, no filler over 4 beats or under 2 outside the climax. The pick check lists what was condensed out. moments.curve exposes the per-frame moment score.

• MilkDrop edges fade cleanly: only MilkDrop’s vivid light fills the edges (its dull ground goes to black), and where a preset lights little of the border the fade to black comes in deeper and wider, so line-art and sparse presets no longer leave a grey haze in the corners. reel.json overlay.milkdrop_edge forces it per preset.

• Opening and ending as picks: the source browser’s pick editor marks one selection as the reel’s opening (under the logo) and one as its ending (picks.json "open" / "end" on the clip); the plan uses them over reel.json and the pick check reports one that isn’t first or last.

• Heavy picks fill the reel: every selection plays its full length; an act they overfill passes its last shots on to the acts after it while those have room, and only then shortens or drops the planner’s shots; an act left short (or a long filler in the finale) gets quick cuts from the picks. An act the picks leave empty no longer stops the plan.

• "keep": true re-cuts the clips the editor left alone from the last cut rendered (render saves plan_<fmt>_rendered.json), so there’s no plan file to copy by hand.

• Glitch looks (focalcut/fx/glitch.py): 7 subject effects on the dancer through the SAM 3 masks (RGB outline, datamosh, pixel sort, ghost trails, a dissolve that reassembles, slit-scan, block damage), 8 transitions (blocks, datamosh cut, slices, RGB tear, VHS roll, signal lost, sort wipe, stutter) and 9 accents, grouped into four looks. reel.json "look" makes the planner say every transition reason and subject effect in that look and put its accents on the hardest hits. focalcut fx lookbook <folder> renders every style and look on the reel’s own footage as one local page.

• Framing follows the action: the crop is on whoever is doing the move (pose activity about the shoulders), not whoever stands together; a couple shares the frame with the follow featured (new role step: SigLIP 2 reads each dancer, woman or man), the follow kept whole when they don’t fit; group routines stay whole.

• Moments that don’t play are switched: the subject walking or standing, a crowd crossing the crop, or someone blocking the camera ranks a moment well down, for planned shots and the people-coverage shots alike.

• Band shots show the band: the role step scores band footage for a player performing, in focus; the plan cuts to the band only there, never the crowd, a back or a blur between the real shots.

• Hardware rendering (focalcut/reel/hw.py, docs/hardware.md): on WSL, camera files only the CPU could decode (Sony 10-bit 4:2:2 HEVC) decode on the Intel iGPU with Quick Sync through a Windows ffmpeg build, frame-exact (-async_depth 1) and about 6x less CPU; shots encode with NVENC and render several at once (FOCALCUT_DECODERS, FOCALCUT_WIN_FFMPEG, FOCALCUT_ENCODER). A 140-shot reel’s plain shots: about 30 minutes down to about 4.5.

• Picks keep their length and bans hold: a pick takes the beats it needs from the weakest plain shots in its act; a ban keeps any shot window out of the banned stretch, not just moments peaking inside it. Picks take precedence: each plays once, never sped up (slow motion stays; no speed curve for long ones), and no other shot reuses its footage.

• Pick check after every plan (plan.pick_report): the plan step lists any pick that is missing, repeated, shorter than marked, sped up or overlapped by another shot, and any ban that did not hold (or that one of the editor’s own picks overrides). Whole-plan regression tests cover the cases that broke on a real reel.

• Bulk drop in the source browser: tick shots in Cut & picks (or drag along the reel lanes in Clips, ✕ on one shot) and drop them all at once; drags snap to the playhead.

• close_only (reel.json): footage that only works close up keeps just the moments with a big face.

• DJ-style song hand-overs (songs): the outgoing song rides its tempo to the next one’s over 8 bars (Rubber Band through ffmpeg, pitch kept), the two play beat on beat for 4 bars with the highs crossfaded and the bass swapped on the downbeat, and the songs are level-matched. Replaces the one-bar crossfade and the hard cut between songs of different tempos (only gaps over 15% still cut). reel.json mix tunes it.

• One review server for many projects: focalcut reel serve <folder> [<folder> ...] [--root <dir>] serves each project under /p/<slug>/ with its own api/picks and api/crop, an index of every project’s pages at /, and POST /api/projects (or --root scanning) to add projects without a restart. One folder still works as before. The pages call the API relative to their project, and the review pages now save picks.json too.

• Source browser (review/sources.html, served by focalcut reel serve): every export on a zoomable timeline; in-out picks, bans and the current cut shot by shot, saved straight to picks.json; portrait focal points (picks "portrait") that move the derived crop, shown beside it.

• Overlay flickers: whole-frame white or dark flickers into each drop and on the hardest hits, or at reel times from reel.json overlay "flicker".

• Show the landing: every plain dance shot keeps the end of its move and about a beat of the dancer at rest after it lands (velocity.landed, plan.lands; land_hold), borrowing a beat from a neighbour when it must.

• Every export file in the reel: coverage is per unit (export file), not per camera clip, so all of a clip’s segments appear; long shots are split to make room. Band inserts and extra shots never take a unit’s only shot.

• Faces from YuNet: portrait framing and ranking read the people step’s face scan when there is one; ViTPose keypoints mark backs of heads as faces on the social floor.

• Recap reels — focalcut reel init|run turns a curated export folder into beat-synced landscape and portrait reels plus Premiere (FCP7 XML) timelines: pose/flow/expression moment detection, coverage of every clip, frame-exact beat cuts and SAM 3 effects. Optional reel extra. See docs/recipes/recap-reel.md.

  • Subject detection (subject step): SAM 3 person masks scored by BiRefNet saliency and Depth Anything V2 nearness pick who a shot is about, with edge contact so cut-off heads are marked down. focus (per-person sharpness) is the fallback; sharpness alone picks still bystanders over dancers in motion blur.
  • Reframing: every shot is cropped around its subject (up to 1.6× from 4K) with a smoothed path.
  • What a shot shows (semantic step): SigLIP 2 zero-shot on the subject’s crop for hips / butt work and expressive faces; adds the shake moment kind and lifts faces. The opening of each unit is marked down.
  • A story in acts: teaser, arrival, shows, night, climax, finale on the song’s bars, with a signature transition at each change of act and transitions picked per cut by content (no one type carries an act).
  • Beat accents in music.json; proxies in reel.json to decode a heavy master from a smaller file.
  • Editing rules from music videos:
    • Shot lengths are varied 8-beat phrase patterns that follow the music’s energy, with holds and a build into the climax.
    • Move peaks land on beats, and cuts come 2 frames ahead of the beat.
    • Shots from the same clips are dealt like a card shuffle, never back to back.
    • Cuts carry the movement on (turn to turn, same direction of travel; new flow.py).
    • Every clip gets a fair share and appears at least once, with several different scenes from each.
  • Velocity edits (velocity.py, fx/speed.py): hero / bullet / montage run fast through the move, ease down to land slow on the dancer’s own landing (settle: a burst, then a rest), hold, and ease back up. The landing sits on the strongest beat that leaves room for the hold. Dancing is capped at 1.5x, and close-ups hold deeper. focalcut fx apply takes --cap and --closeup.
  • Transition grammar (grammar.py): mostly hard cuts. Each act has its own short vocabulary with caps, and a transition goes in only with a reason: turn into turn, the same travel carried across, a push in to a face, a big hit (accent threshold) or the DJ. Never back to back. Replaces the per-act transition share top-up.
  • Shake (shake step, shake.py): camera jitter per unit. Shots too shaky for their crop give way to steadier parts of the clip, and mildly shaky shots are stabilised at render (vidstab). Knob: shake.
  • Portrait framing: a 9:16 strip follows one dancer of a split couple (frame.follow). Moments rank on how the crop holds the subject (present, wholly inside, a face to the camera; plan.assess), so backs on the social floor drop out.
  • Band inserts (members, member_clips, band_in) and a chosen ending shot (ending).
  • Cohesive effects: one effect at a time; one subject effect, on the climax’s final bullet-time move. The punch-ins and orbs are gone.
  • Any cut, not just the reels: plan.place() / plan.shot() time, frame and transition a hand-built sequence. render / xml_export / review take cut=, writing out/<name>_<cut>.mp4.
  • skills/focalcut: a Claude Code skill listing every concept and the code behind it, with an example teaser script.
  • Opening: band / crowd B-roll with the event logo (logo in reel.json) centred across the cut. Daytime B-roll is held long and woven with the demos. The night weaves band shots between its dancing.
  • Tracking: between subject keyframes the crop follows the subject’s pose boxes at 5 fps. A portrait strip cut from landscape footage follows faster.
  • Effect masks: BiRefNet matte refinement inside the SAM 3 mask, blended in-between frames and temporal smoothing, for clean edges.
  • The climax no longer reuses a moment it has reserved for a later slow-motion shot.
  • Activity gate: moments are marked down unless the subject dances, and stays in frame, for the whole shot.
  • Editor picks: ban or prefer ranges per export file, in picks.json or under reel.json’s picks. Units carry export_t0, so the times are the export’s own.
  • Review step: a local HTML page per reel, showing every shot with its export file, time range and a copyable picks line.

• Effect modules — focalcut fx list|apply|gallery: speed curves (montage, hero, bullet, jumpcut, flashin, flashout), SAM 3 subject effects (spotlight, freeze, neon, particles, velocity, echo, strobe, orbs, aura), beat effects (punchin, shake, glitch) and transitions (flash, fade, dipwhite, fadein, zoom, swipe — following the motion’s direction —, spin, lightleak, glitch). Subject effects keep the salient couple, or the whole line on stage.

• Timelines export to Premiere, Resolve, Final Cut, Avid and any EDL-capable editor. focalcut export -f {fcp7,aaf,edl,otio} writes through OpenTimelineIO, so timecode handling comes from the interchange library the post industry maintains rather than from a writer in this repo. focalcut formats lists what your install can produce.

• A test suite — 66 tests covering timing, manifests, rotation handling and the CLI, all synthesising their own ffmpeg fixtures. No GPU, network or model access.

• CI on Python 3.10–3.13, which also asserts the GPU-free install stays free of torch.

• CONTRIBUTING.md and issue/PR templates.

Changed

• pip install focalcut no longer pulls the CUDA stack. torch, transformers and accelerate moved to an opt-in [sam] extra; the recommended contact-sheet pipeline needs only numpy and ffmpeg. Install focalcut[sam] for the SAM 3 engines.
• focalcut export-premiere is now focalcut export --format fcp7 (the default).

Removed

• The hand-written FCP7 XML generators (premiere.py, export.to_fcp7_xml), replaced by the OTIO path above.

[0.1.0]

Initial release: contact-sheet library pipeline with a vision judge (coarse → refine → miss-sweep → ultra), optional SAM 3 engines, and per-subject clip extraction.