--- name: voxel description: Make a narrated vertical reel (1080x1920, 40-55 s) in the "Voxel 3D" format - a blocky three.js night desk by a big city window, an orange Claude block and a blue-cap creator acting out every sentence, a torn paper banner on tape, a counter badge, orange torn-paper word captions and a terminal bar, one ElevenLabs voice, a music bed, rendered frame by frame in headless Chrome and encoded with ffmpeg. Use it when someone asks for a voxel reel, "voxel.md", a 3D explainer reel or a Short in this format, or wants to turn a script into this kind of video. --- # Voxel 3D reel: how to make one You will make a **40-55 second vertical video (1080x1920, 30 fps, H.264 + AAC, at most 60 s and 60 MB)**: one narrator explains something while a voxel toy world acts out every sentence. A blocky orange Claude and a creator in a cream hoodie and blue cap stand on a wooden desk at night, a phone lies on a navy mat in front of them, a huge window full of tiny lit city windows fills the background, everything is a little soft (depth of field), glowing (bloom) and grainy. Over the 3D sit four paper-craft layers: a **torn cream paper banner on two strips of tape** (the beat's chapter label), a **white counter badge**, **orange torn-paper word tags** (the captions, word by word) and a **dark terminal bar** that types the prompt. The camera never stops moving, there is one shot per beat, and when the voice says something changes, the video changes. **`reference.jpg`** (next to this file, if you have it) is a contact sheet of 12 frames from a finished Voxel reel: **this is what it should look like.** If you only have this file, section 3 describes everything in that picture, with numbers. Everything is rendered **one frame at a time** from a three.js page in headless Chrome (`window.setT(t)`), then muxed with a mix made in ffmpeg. This file gives you the look (exact colours, fonts, sizes, positions), the rules that keep it from looking buggy, and the complete, tested source of the engine (Appendix A) plus one example video (Appendix B). Write the engine once, then each video is two files: `scene.js` (the director file) and `cues.mjs` (sound effects). Work in this order and don't skip ahead: **setup -> script -> blocking with guessed timings -> voice -> timings -> beats -> stills -> full render -> sound + mix -> encode -> check -> deliver.** --- ## 0. Rules of engagement (read first) 1. **Be gentle with the user's computer.** One headless browser, on the GPU, at the lowest priority, pinned to one CPU thread, never parallel workers. If WebGL is on the CPU (SwiftShader, llvmpipe, "software"), `cdp.mjs` refuses to render: tell the user to fix the graphics driver. A CPU render of this film pegs every core for half an hour. 2. **Never print secrets.** The key is `ELEVENLABS_API_KEY` (environment, or `.env` in the project folder). Scripts read it; you never echo it, log it or put it in a file you show. 3. **Spend credits deliberately.** `gen-vo.mjs` and `gen-sfx.mjs` are dry runs unless you pass `--yes`. One voice take of the whole script (about 1 credit per character, ~700-950 for a reel). Never regenerate a take that exists; listen first, regenerate only what is wrong. 4. **Stills before film.** Never start the full render until the stills contact sheet is right. Re-rendering 1,600 frames for a one-shot fix is the most expensive mistake here. 5. **Background jobs die when your turn ends.** Render in foreground chunks (`render.mjs ... full 0 900`, then `full 900`), each call under your tool timeout. The render resumes where it stopped. 6. **No real people's content.** No real screenshots, logos, app icons, handles, file names, view counts or other creators' frames. Everything on screen is drawn: the creator, the phone's screen, the props. Counters count only what is said or what is on screen. 7. **Don't publish anything.** You deliver files. Posting is the user's call. 8. If the user's instructions conflict with this file, the user wins. If something is missing (no key, no GPU, no ffmpeg), say exactly what and offer the closest thing. Never fake success. ## 1. Requirements - **Node 22+** (global `fetch` and `WebSocket`), **ffmpeg + ffprobe** on PATH, **Chrome or Edge** (or set `CHROME=`), a GPU. Chrome is driven through ANGLE: d3d11 on Windows, metal on macOS, gl on Linux (if the GPU probe fails on Linux, try `VOXEL_ANGLE=vulkan`). Debug ports 9650-9699 on 127.0.0.1. - **three.js 0.170.0** from npm (pin it; the import map points into `node_modules/three`). - **An ElevenLabs API key.** The free plan (about 10,000 credits a month, API access, premade voices) is enough for a few reels but has no commercial licence; monetised videos need a paid plan. No key yet: do everything with `build.mjs --fake` (guessed timings) up to the stills, and ask for the key. - Fonts and music are free downloads (`setup.mjs` fetches them). Sound effects are made once with ElevenLabs sound generation (`gen-sfx.mjs`), or downloaded free (Mixkit's free sound effects) and saved under the names in section 10. ## 2. Project layout and setup ``` voxel-reel/ one project folder, many videos package.json { "type": "module", "dependencies": { "three": "0.170.0" } } .env ELEVENLABS_API_KEY=... (never print it) engine/ Appendix A: write these files once, never edit them per video film.html main.js the page: fonts -> film.json + scene.js -> window.setT(t) lib.js tex.js world.js helpers, canvas textures, the set + characters + prop helpers post.js camera.js look.js post chain, camera from SHOTS, grade/lights dials + effects overlay.js screen.js the 2D paper layer, the phone's screen common.mjs cdp.mjs setup.mjs render.mjs gen-vo.mjs build.mjs gen-sfx.mjs mix.mjs encode.mjs check.mjs fonts/ music/ sfx/ downloaded / generated once my-video/ one folder per video (Appendix B is a complete one) script.txt video.json the voice text; settings scene.js cues.mjs the director file; the sound effects vo/ film.json stills/ frames/ mix.wav video.mp4 cover.jpg caption.md ``` ```bash mkdir voxel-reel && cd voxel-reel echo '{ "type": "module", "private": true, "dependencies": { "three": "0.170.0" } }' > package.json npm install # write engine/* from Appendix A, then: node engine/setup.mjs # checks Node/ffmpeg/three/Chrome/GPU/key, downloads fonts + music node engine/gen-sfx.mjs # dry run: lists the 20 sounds; add --yes to make them (once, for every future video) ``` **A new video** = a new folder with `script.txt` (the voice text; `#` lines are ignored), `video.json`, `scene.js` and `cues.mjs` (start from Appendix B). `video.json` fields, all optional: `take` (set by gen-vo), `tail` 1.7 (seconds after the last word), `cover` 0.5 (the cover frame time), `capFix` ({ "voxel": "VOXEL" }), `chunkW` 760, `music` ({ file, from: 8, db: -11.5, muffle: [] }), `voice` ({ id, model_id, voice_settings }). Commands for one video (from the project folder; `my-video` is the folder name): ```bash node engine/build.mjs my-video --fake # guessed timings from script.txt: block out shots without a voice node engine/gen-vo.mjs my-video # dry run: characters, words, seconds node engine/gen-vo.mjs my-video --yes # ONE take -> vo/a.mp3 .wav .json ; listen to it node engine/build.mjs my-video # real timings -> film.json, prints every caption chunk with its time node engine/render.mjs my-video stills 0.5,3,7,11,15 --safe # stills + sheet-safe.jpg with the safe zone; fix; repeat node engine/render.mjs my-video full 0 900 # full render in chunks (resumable) node engine/render.mjs my-video full 900 node engine/mix.mjs my-video # mix.wav at -14 LUFS node engine/encode.mjs my-video # video.mp4 + cover.jpg node engine/check.mjs my-video # the final checks ``` --- ## 3. The look, pinned down ### 3.1 The frame, layer by layer (1080x1920 px) | Layer | Where | What | |---|---|---| | Platform UI (keep clear) | top 250 px, bottom 440 px, right 120 px | Instagram/TikTok/YouTube cover these. Nothing important there. | | **Banner** | top edge y 268, centred on x 510, paper 560-880 px wide | Torn cream paper, two tape strips on the top corners, black caps. Changes on the first word of every beat, with a slap. | | **Counter badge** | right edge x 950, centre y 470 (22 px below a 2-line banner) | White card, label + big coloured number. Stays left of x 960, out of the right 120 px. | | **Subject** | the look-at point lands at y ~845 (44 % of the height) | A lens shift (`frameY` 0.44) keeps characters above the terminal and captions. | | **Terminal bar** | centre y 1092, x 66-954, 76 px tall | Dark bar, `>` prompt, types while the voice says the prompt. Only when a prompt is said. | | **Comment box** (CTA) | centre y 1062, the posted comment 150 px above | Instagram-like pill; types the keyword, posts, heart, a paper plane flies off. | | **Captions** | block centred at y 1385 (72 % of the height), always inside y 1150-1480, centred on x 510, at most 820 px wide | Orange torn-paper tags, Cooper Black, 1-3 words at a time, word by word. | | 3D world | everything | DOF, bloom, chromatic edges, vignette, grain. | **Safe zone (checked automatically):** all text in x 60-960, y 250-1480; captions centred on x 510, at most 820 px wide, within y 1150-1480. The engine records every text box each frame; `--safe` stills draw the zone and print anything outside it; the full render logs violations to `frames/safe.json`; `check.mjs` fails on any. **Frame 0 is the cover**: it already holds the subject, the hook banner, the counter and the first caption word. No black, no fade-in. The end is a hard cut 1.7 s after the last word. ### 3.2 Colours | Element | Hex | |---|---| | Banner paper / tape / ink | `#f3eee4` (+ fibre texture multiplied at 55 %) / `#d9c99c` at 86 % / `#141414` | | Banner, dark ("tape") mode for a moody beat | `#17171a`, text `#f2f2f2`, no tape strips | | Badge card / outline 4 px + label ink | `#f8f3e9` / `#2b2420` | | Badge numbers: good / neutral / bad | `#2fae5a` / `#e0702f` / `#d6452b` | | Caption tag / text / text shadow | `#cf6a43` (moody beat `#a9512f`) / `#fdf3e4` / `rgba(70,25,8,.35)` offset 1.5, 3 | | Huge headline tags | `#b8532f` | | Terminal bar / text / `>` + send flash + "↵ sent" | `rgba(16,16,19,.94)` / `#ecebe8` / `#e8784a` | | Comment UI | white, text `#111`, grey `#8e8e8e`, Post `#0095f6` (empty `#b2dffc`, pressed `#00376b`), heart `#ff3040`, avatar disc `#d97757` | | Desk wood / desk front / mat (+ border) | `#a77a62` over warm plank texture / `#4a2716` / `#1c2045` (+ `#e8873f`) | | Window frames / sill / fog | `#161a33` / `#232845` / `#0e1230` from 30 to 70 units | | Night sky / buildings / lit windows | `#141a3e` -> `#27306a` / `#1a2152`, far `#11173a` / `#ffd27a #ffb05a #ffe7a8 #8fb4ff #ff8f5a`, 42 % of 5x8 px windows lit | | Claude / eyes | `#e8845c` / `#111014` | | Creator: hoodie, pocket, skin, cap, jeans | `#f3ecdc`, `#d9ceb8`, `#ffd9b5`, `#3b6fd8`, `#2a3350` | | Accents: lamp, mug, clock | lamp `#2a2f55` + shade `#e8873f` + bulb `#fff1d0`; mug `#f3efe6` + band `#e8873f`; clock face `#f6f1e6`, rim `#e8873f`, minute hand `#d9482b`; sticky notes `#f7d548`, `#f5a3c7` | ### 3.3 Fonts | Use | Original (if installed) | Free stand-in (downloaded by setup.mjs) | Size / weight | |---|---|---|---| | Banner, badge label, in-world signs | Inter Tight ExtraBold | the same (Google Fonts, OFL) | banner 68 px caps, letter-spacing -1 px, shrinks to 50 px, then 2 balanced lines (64-36 px) over 760 px; badge label 22 px, spacing 1 px | | Captions, badge numbers, price tags | **Cooper Black** (ships with MS Office) | **Fraunces 72pt SuperSoft Black** (OFL; inspired by the Cooper series) | captions 64 px (1.42x for "bigger"), huge slams up to 190 px; badge number 58 px | | Terminal, file names | Consolas Bold | JetBrains Mono Bold (OFL) | 36 px, shrinks to fit (min 24) | | Magazine masthead (optional mode) | Bodoni MT Black (+ Italic) | Bodoni Moda Black (OFL) | up to 230 px, letters drop in | | Comment box | Segoe UI | Inter (OFL) | 30-38 px, 400/600/700 | `main.js` registers each family with `local("")` first and the free file second, so the original is used where it exists and the free font everywhere else. Never bundle or redistribute the licensed originals. The free set looks almost identical (tested). ### 3.4 The world All units are world units; the desk top is y = 0 and +z points at the camera. - **Desk** 10 x 6.2 (x -5..5, z -3.2..3.0), warm planks; a dark desk front edge. **Mat** 2.5 x 3.65 under the **phone**, which lies flat centred at (0, 0.012, 0.15), 1.55 x 2.76, 0.13 thick, top surface at **y 0.142**; its screen is a 540x960 canvas the video draws every frame ("the video" with the drawn creator talking, or a card). `R.phonePivot.rotation.x = 1.3` stands it up facing the camera (the CTA). - **Window**: dark navy mullions at x +-1.75 and +-4.8, transoms at y 0.15, 5.2, 10.4, all at z -3.7; a sill. The **city** is a 90 x 45 canvas plane at z -32 (night; sunset and moody versions fade over it), plus two side planes so wide shots never see an edge. Rain streaks on the glass in the moody look. - **Props at rest**: desk lamp back left (-3.3, -1.7) with an orange cone shade and a glowing bulb, mug right (3.0, -0.2), a wall clock hanging in front of the right window pane (3.15, 3.0, -3.5) whose hands the video turns, three sticky notes. - **Lights**: hemisphere sky `#7f8fe0` / ground `#2a1a24` 0.9; **key** spot `#ffd8b0` intensity 60 (distance 30, angle 0.62, penumbra 0.55, decay 1.6) from (4.2, 8.5, 6.5) with soft 2048 px shadows; **rim** directional `#7f9cff` 1.6 from behind (-2, 5, -9), the window's cool light on every edge; **lamp** point `#ffb36b` 7 (distance 9, decay 1.6) at (-2.4, 1.9, -1.2); fill directional `#ffc9a0` 0.5 from (-4, 3, 6). - **Materials**: every surface is a matte `MeshStandardMaterial` (roughness ~0.82) whose colour is multiplied by a blotchy grey value-noise map, so flat colours look hand-painted voxels. Boxes have small rounded edges (RoundedBoxGeometry). Detail comes from light, DOF and bloom, never from texture detail. ### 3.5 The characters - **Claude** (the "Clawd" mascot style of Claude Code, built from boxes, no logo asset): an orange block 1.12 wide x 0.72 tall x 0.78 deep with 0.05 rounded edges, on **four stubby legs** (0.22 tall), **two tall black rectangular eyes** (0.1 x 0.19), **two stub arms** at the sides, **no mouth**. About 0.92 tall, 1.5 wide with arms. Stands at (0, 0, -1.75) facing the camera. - **The creator** ("me", the narrator's avatar): blocky, about 1.8 tall, cream hoodie with a chest pocket, skin-coloured head with two small black eyes, **blue cap with a brim**, navy jeans. Stands at (1.15, 0, -1.25), turned -0.38 rad towards Claude. Extra characters (a rival, a customer) are the same build in other colours (`makeCreator(mat, box, 1, { hood: 0x1b1b20, cap: 0xd6452b })`), mini Claudes are `makeClaude(..., 0.5)`. Accessories (a hat, glasses, gloves) are boxes parented to `R.claude.pivot` or an arm. - **Acting**: Claude has no mouth, so life comes from breathing (squash +-2.5 % at 0.38 Hz), blinks every 2.6-4.2 s, eye darts toward what matters, eyes widening (scale 1.25-1.35), arm raises, and **hops on named words** (a crouch, a parabola, a landing squash). The creator nods, tilts, points, waves. One motion per character at a time. ### 3.6 Props: every claim gets a literal object Props are boxes and thin cards with a canvas texture on the front face, built in `setup()` and moved in `world(t)`: `R.tile({ label, bg: [c0, c1], icon })` an app tile (rounded block) · `R.card(canvas, w, h)` paper, documents, signs · `R.decal(canvas, w, h)` stamps, X marks, check marks, price tags · `R.box(w, h, d, R.mat(color), round)` anything else, then `E.scene.add(mesh)`. Canvas recipes in `tex.js`: `tileCanvas`, `docCanvas` (coloured header, title, mono subtitle, rows), `stampCanvas('FREE', '#2fae5a')` (double border, distressed ink), `tagPriceCanvas('$30')`, `labelCanvas`. **The phone screen** (`phone(g, t)` draws a 540x960 canvas every frame): `S.edited` ("the video": night-to-orange gradient, the drawn creator talking with a rim light, a progress bar, its own small captions), `S.raw` (grey shaky camera footage, REC, timecode), `S.card` (the deliverable: orange pill, 2-3 big lines, a document or list icon, a glow), `S.wipe` / `S.crossfade` between modes, or draw anything else yourself (an app UI, a list, a chart) in the same palette. Never a real screenshot. Rest heights: desk y 0, mat 0.012, on the phone 0.142; a box of height h at scale s rests at surface + h*s/2. Long or wide props go **in depth** (see 7.3). Props never just appear: they drop with a bounce, pop with an overshoot, fly in on an arc, flip in, or slam (stamps scale 2.4x -> 1x in 0.12 s with a camera kick). Proven props: app tiles that fall in and get red-X-stamped, then swept off the desk · document cards (a SKILL.md, a resume, a plan) with a green stamp · a price tag swinging on a box · a cork board with flip-in panels · a cardboard box that bursts · a wall of file cards · a timeline strip with clips snapping in · polaroids floating up · question marks over the creator's head · a trash bin · a crowd of 49 identical mini desks far away ("lots of people do this") · the wall clock spinning ("a few minutes later") · the phone standing up with the deliverable on its screen. ### 3.7 Post (the numbers) | Stage | Setting | |---|---| | Scene | linear HDR half-float target, 4x MSAA, depth texture; no tone mapping in three.js; camera fov 42 default, near 0.1, far 200 | | DOF | 40-tap golden-angle gather; blur px = `ap * abs(1 - focus/z)`, max 16 px; focus = distance to the shot's look-at point. `ap` per shot: wide 4-7, medium 8-10, close-up 11-12, top-down insert 3-4 | | Bloom | bright pass `smoothstep(0.9, 1.5, max(r,g,b))` at 1/2 res; 3 blurred levels at 1/4, 1/8, 1/16, added x0.5, x0.8, x1.1, times strength 0.5 | | Grade | exposure 1.0, ACES (Narkowicz), gamma 2.2, saturation 1.1, contrast 1.04 | | Final | chromatic edges 1.7 (radial, zero in the centre), vignette 0.32, **grain 0.028**, flash capped at **0.24**, the 2D overlay composited last (straight alpha) | | Dials | `warm` 0..1 (sunset city, warmer key, bloom 0.85, halation, warm tint) and `moody` 0..1 (dark blue, rain, desaturated, lamp off); 3D lightning capped at 2.2 | **The caps exist because of real failures:** a 0.55 flash plus a strong lightning light washed three frames white (flash <= 0.24); grain 0.045 doubled the file to 113 MB for 55 s (grain 0.028 keeps it near 50 MB: don't raise it); a 1.2-unit close-up cut Claude off at the sides (minimum framing distance, `fit` 1.8, see 7.3). ### 3.8 The overlay elements, behaviour - **Banner**: one per beat, changes on the beat's first word with a slap sound: pops 1.09x -> 1x in 0.24 s with a small wobble, text fades in over 0.06 s, alternating tilt +-0.018 rad. 2-6 words, caps, at most ~28 characters (longer wraps to 2 balanced lines and pushes the badge down). Modes: `paper` (default), `tape` (black strip, for the dark beat), `mag` (Bodoni masthead, for a "magazine/aesthetic" beat). - **Badge**: `{ label, value, col }`, label in caps (2-3 words), value a number or a short word (`$0`, `7`, `3/5`, `SENT ✓`). Pops in with an overshoot (0.22 s), out in 0.15 s; when the value changes it pops 1.22x. A number on screen in every beat, and it changes inside the beat: counting up on spoken numbers, flipping red -> green on the turn. - **Captions**: chunks of at most 3 words (a break after punctuation, balanced), each word appears at its spoken start (pop + slight tilt), earlier words slide over to make room; the chunk stays up to 0.7 s after its last word. `scale` grows them on "bigger captions"; `huge` slams 1-4 big words in rows on a headline moment (with a thud, a camera kick and an overlay shake). - **Terminal**: slides up 26 px, types the prompt from the first to the last spoken word of the prompt, block cursor, sends 0.1 s after: the outline flashes orange, "↵ sent" fades, Claude hops. - **Comment box (CTA)**: appears on "Comment", types the keyword as it is spoken, posts, the posted comment slides in above with a heart that fills red, a paper plane (the DM) flies to the top right, then the badge says `DM` `SENT ✓`. --- ## 4. Writing the script for this format The user may hand you a script; if not, write one. It is a **narrated explainer**: one voice, first person ("I built", "I stopped"), talking to "you". - **Length**: 40-55 s. Liam speaks about 3 words a second: **120-160 words** (~700-950 characters). The dry run prints the estimate. Over 55 s, cut a beat; never speed the voice up. - **The hook is the first banner and the first line**, complete within 3 s and readable at frame 0 with the sound off: a claim with a number or a stake, under ~12 words. Shapes that work: "N that ." / "I stopped . Claude does it now." / "Never X again." - **Beats**: every sentence (or two short ones) is a beat of 2-4 s, and every beat gets a **banner** (2-6 words, the beat's job, not a transcript), **one shot**, and **one literal visual**. If you can't name the prop or action that shows a sentence on a desk, rewrite the sentence. - **A counter that counts something real** in every beat: something the voice says ("3 apps" -> 3), or something on screen (the 12 file cards -> 12). Never invent statistics. - **A turn at about 60 %** ("But here's the part...", "And it doesn't stop there") opens a second act where attention dips. - **The CTA is the last line**: "Comment KEYWORD and I'll send you ." One word, in caps, that appears nowhere else in the script (the caption fix uppercases every occurrence), ideally the topic's own noun. Promise only something that exists and that will really be sent (the prompt, the file, the list). The keyword appears four times: spoken, in the last banner, typed in the comment box, and twice in the post caption. - **Style**: short sentences, one idea each, concrete nouns, no hype words ("insane", "game-changer", "10x"), no emojis in the voice. Numbers: v3 reads "$30", "7" and "50%" correctly; spell out anything ambiguous (versions, years, ranges, symbols) and use `capFix` in `video.json` to show the written form. Don't say URLs ("the link"). - **A shape that works** (fill it with the user's topic): | Time | Beat | Line (shape) | Banner | Counter | |---|---|---|---|---| | 0-3 s | Hook | "I stopped paying for 3 apps." | 3 APPS I STOPPED PAYING FOR | APPS I PAY FOR 0 -> 3 as they land | | 3-7 s | The claim | "Claude does all three now." | CLAUDE DOES ALL THREE | 3 (red) -> 0 (green) | | 7-30 s | 3-5 points | one concrete claim each, each with its prop | the point in 2-5 words | counts that point's thing | | ~60 % | The turn | "But here's the part nobody shows." | THE PART NOBODY SHOWS | a new counter | | to -6 s | Payoff | what the viewer gets, and that it's free | ... | ... | | last 4-6 s | CTA | "Comment KEYWORD and I'll send you the prompt." | COMMENT KEYWORD FOR THE PROMPT | DM SENT ✓ | - **Write `caption.md`** next to the video: the final script, the post caption (the hook line, 2-4 lines of what it does, "Comment KEYWORD ...", 3-5 hashtags), and a `DM:` line saying exactly what gets sent. --- ## 5. The voice (ElevenLabs) - **Voice: Liam**, premade, `TX3LPaxmHKxFdv7VOQHJ` (labelled "social media creator"), model **`eleven_v3`**, `stability 0.5, similarity_boost 0.8, style 0.6, use_speaker_boost true`, endpoint `/v1/text-to-speech/{id}/with-timestamps` (mp3_44100_128). These are the defaults in `common.mjs`; `video.json` -> `voice` overrides them. Any other premade voice works (e.g. Brian `nPczCjzI2devNBz1zQrb`, Will `bIHbv24MWmeRgasZH58o`; list yours with `GET /v1/voices`). - **One take of the whole script.** v3's character alignment is accurate to about 0.1 s, so one take gives better rhythm than stitched lines and costs one character per character. `gen-vo.mjs` writes `vo/a.mp3`, `vo/a.wav` (48 kHz mono) and `vo/a.json` (text + alignment) and sets `video.json` `take`. Listen to it. Only if it is wrong (a mispronounced name, a flat hook): `--yes --take b`, and keep the old one. - If the API refuses `eleven_v3` for the account, set `"voice": { "model_id": "eleven_multilingual_v2" }` and remove any `[audio tags]`. - `build.mjs` turns the alignment into words: ElevenLabs gives the silence before a word to that word, so each word's start is moved to where its sound really begins (an RMS onset in the wav). Captions then pop exactly on the syllable. ## 6. Timings: film.json `node engine/build.mjs my-video` writes `film.json`: `words[]` (`{ i, w, cap, t0, t1 }`), `chunks[]` (`{ t0, t1, words }`) and `END` (last word + `tail` 1.7 s). It prints each caption chunk with its start time. Use `--fake` before you have a voice: same format, guessed timings, so the whole scene can be blocked out and still-checked first. **Key every beat to words, not to numbers.** In `scene.js`, `at('calendar')` is the start of the first "calendar", `at('my', 2)` the second "my", `end('plan', 2)` the end of the second "plan" (Appendix B). Then a new take re-times the whole video by itself. Re-run `build.mjs` after changing the take, `capFix`, `chunkW` or `tail`, and re-run `mix.mjs` after any change to film.json. --- ## 7. Directing: beats, shots, the edit ### 7.1 How a beat is written `scene.js` exports `setup(E)`, called once; it returns `{ SHOTS, IMPULSE, world(t), effects(t), phone(g, t), overlay(g, t) }`. `E` holds: `THREE`, `R` (the set: `R.claude`, `R.creator`, `R.phonePivot`, `R.minHand`/`R.hourHand`, `R.lamp`, `R.mug`, prop helpers `R.tile/card/decal/box/mat/tex`), `scene`, `camera`, `film`, `END`, `O` (overlay: `banner`, `badge`, `captions`, `terminal`, `comment`, `COL`), `S` (phone screens), `T` (canvas textures, `cv`), `L` (easing, `hop`, `hash`, `keys`), `FX` (`punchIn/punchOut`, `whips`, `flash`, `pulse`, `shake`, `applyFx`), `look(t, { warm, moody, spin, flicker, lightning })` (call it every frame), `rig` (`poseClaude`, `poseCreator`, `blinkAt`, `makeClaude`, `makeCreator`), `V(x, y, z)`. Writing `scene.js` is the long step (an hour or two for a 50 s reel); everything else is minutes. For each sentence, in `scene.js`: 1. **Its start**: `B[k] = at('')`. The banner changes there (`BANNERS`), the shot cuts there (`SHOTS`), a `slap` sound plays there (`cues.mjs`). 2. **One shot** from `B[k]` to `B[k+1]`: camera keys `P` and look-at keys `L` (on the subject: it is also the focus distance), a slow push-in or drift of 4-12 %. 3. **A badge** whose value changes on a word of the beat. 4. **The action on the stressed word**: a prop drops/pops/flies at `at('')`, a stamp slams, Claude hops; add an `IMPULSE` (camera kick, 0.02-0.06) for impacts. 5. **A cut accent** within +-0.1-0.3 s of the boundary: a whip (`FX.whips`, 0.04-0.1) or a zoom punch into the cut (`FX.punchIn/punchOut`, 0.05-0.12). 6. **Its sounds** in `cues.mjs`, on the same word times. ```js // a beat: "It reads my calendar, my notes and my tasks" — a side shot, a doc flies from the phone into Claude on each noun, a counter 1-2-3 { t: [B[3], B[4]], P: [[B[3], [3.9, 1.5, 3.0]], [B[4], [3.5, 1.4, 2.7]]], L: [[B[3], [0, 0.75, -0.7]], [B[4], [0, 0.78, -0.75]]], fov: 40, ap: 9, sh: 0.012, fit: 2.0 }, [B[3], 'CALENDAR, NOTES, TASKS'], // BANNERS { at: B[3] + 0.1, out: B[4] - 0.12, label: 'SOURCES READ', value: t => String(read(t)), col: COL.neutral, minW: 40 }, // BADGES const READ = [at('calendar'), at('notes'), at('tasks')]; // world(t) flies doc i from READ[i] for (const w of ['calendar', 'notes', 'tasks']) ev('swoosh', at(w), -12); // cues.mjs ``` ### 7.2 Shot library (tested positions; Claude at z -1.75, the phone at z 0.15) | Shot | Use | Camera P (start -> end) | Look-at L | fov | ap | fit | |---|---|---|---|---|---|---| | Medium, props in front of Claude | hook, a list of things | (-0.35, 1.75, 4.6) -> (0.3, 1.75, 4.45) | (0, 0.55, 0.2) | 40 | 10 | 1.8 | | High three-quarter | a change of angle on the same set | (1.3, 2.5, 4.4) -> (0.9, 2.35, 4.0) | (0, 0.6, -0.3) | 44 | 9 | 1.8 | | Claude close-up | a key line, a prompt being typed | (0, 1.66, 3.34) -> (0, 1.48, 2.77) | (0, 0.62, -1.75) | 40 | 11 | 1.8 | | Wide, both characters | a reaction, a summary | (-0.6, 2.0, 5.2) -> (0, 1.9, 4.6) | (0.4, 0.8, -1.3) | 46 | 7 | 1.8 | | Establishing wide | once, early | (-1.5, 1.9, 4.0) -> (1.4, 1.95, 3.7) | (0, 0.75, -0.3) | 46 | 8 | 1.8 | | Low along a long prop | a strip, a row of cards | (-3.75, 1.2, 2.95) -> (-3.3, 1.05, 2.85) | (0.1, 0.05, 2.15) | 38 | 11 | 1.3 | | Side: phone -> Claude | something travels to Claude | (3.9, 1.5, 3.0) -> (3.5, 1.4, 2.7) | (0, 0.75, -0.7) | 40 | 9 | 2.0 | | Top-down phone insert | what lands on / plays on the phone | (0.12, 3.75, 2.85) -> (-0.05, 3.55, 2.6) | (0, 0.14, 0.15) | 40 | 3-4 | 0, frameY 0.47 | | A board in front of the window, behind Claude | a cork board, a wall of cards | (0, 2.05, 5.2) -> (0, 2.0, 4.35) | (0, 1.85, -2.9) | 50 | 6 | 1.8 | | The city | "the internet", "everyone" | (0.75, 1.35, 1.0) -> (0.35, 1.5, 0.25) | (0, 2.4, -12) | 46 | 3 | 1.8 | | Wall clock | time passing | (2.35, 2.75, -1.1) -> (2.65, 2.9, -1.75) | (3.15, 3.0, -3.5) | 40 | 7 | 1.0 | | CTA, phone standing | the deliverable + comment box | (0, 1.9, 7.4) -> (0, 1.75, 6.6) | (0, 1.25, 0.9) | 44 | 6 | 1.8 | | End pull-back | the last 2-3 s | (0, 2.5, 8.9) -> (0, 2.35, 8.2) | (0, 1.2, 0.9) | 50 | 4 | 1.8 | Handheld `sh` 0.012 (0.006 for inserts, up to 0.03 for chaos). Shots are `{ t: [start, end], P: [[t, [x,y,z], ease?], ...], L: [...], fov, fov1?, ap, sh, fit?, frameY?, up? }`; keys ease in-out by default. The last shot runs to `END + 1`. ### 7.3 Portrait framing (the lesson that cost a re-render) - At fov 42 a 9:16 frame is only **about 24 degrees wide**. Put props **in depth, not side by side**. A long prop gets a low camera looking **along** it. - **Minimum framing distance:** the camera is pushed back along its line until `fit` world units fit across the frame at the look-at point. Default 1.8, so Claude (1.5 wide with arms) never fills more than ~80 % of the width. Smaller `fit` (1.0-1.4) for small props, `fit: 0` only for screen inserts. A pushed-back shot loses its push-in: when the stills print `framing: shots pushed back`, move that shot's P keys back yourself. - **Lens shift, not tilt**: `frameY` 0.44 puts the look-at point at 44 % of the height, above the terminal and captions; 0.5 for centred inserts. Put `L` **on** the subject: it is the focus. - Don't cut to the same framing twice: change distance by ~30 % or the angle by ~30 degrees. Keep geography: Claude centre, creator to its right, unless a cut moves them. ### 7.4 The edit follows the words Every spoken claim gets its literal visual **at that moment**, and when the voice says something changes, the video itself does it: | The voice says | The video does | |---|---| | a number | the counter ticks to it on that word | | a thing ("my calendar") | that prop appears on that word | | no / never / without / stopped | a red X slams onto it (stamp sound + camera kick) | | free / done / approved / fixed | a green stamp slams; the badge turns green | | a price | a price tag swings in; the badge shows it in red | | "I type / I tell Claude / I say:" | the terminal types the prompt in sync and sends with an orange flash; Claude hops on send | | warmer / aesthetic / golden | `E.look(t, { warm })` 0 -> 1 over ~1 s, a light-leak sweep, shimmer; optional `mag` masthead banner | | darker / moody / serious | `moody` 0 -> 1, black `tape` banner, a lightning pulse, rain, darker caption tags, the music muffled (`muffle`) | | bigger / huge / louder | captions `scale: 1.42`, or a `huge` slam | | later / minutes / hours | the key light swings (`spin`), the clock hands spin, Claude zips around | | lots of people / everyone | a far-away crowd of identical mini desks (a 7x7 grid of small desks with `makeClaude(..., 0.5)` at x = 80, its own lights, its own shot), then one lights up | | comment KEYWORD | the comment box types it, posts, heart, paper plane; badge `DM SENT ✓` | ### 7.5 Pacing (measured on finished reels) - **About 3.7 hard cuts per 10 s** (one per beat, 2-4 s shots, the longest ~5 s). `check.mjs` reports it. - **Something changes every ~0.3 s**: a caption word, a prop, the counter, a stamp, the camera. Nothing on screen holds still for more than 0.5 s (`check.mjs` flags it). - The camera always moves: a 4-12 % push or drift per shot plus handheld. No locked-off frames, no fast whips except as cut accents. - No pause over 0.4 s in the voice except one deliberate beat. ### 7.6 Motion rules (break these and it looks buggy) 1. **Every frame is a pure function of `t`.** No clocks, no `requestAnimationFrame`, no accumulated state, no `Math.random` (use `L.hash(i)` / `L.rng(seed)`). Any frame must render alone, in any order: the renderer reopens Chrome every 150 frames. 2. **Ease everything** (`easeOut`, `easeOutBack` for pops, `easeOutBounce` for drops, `keys()` in-out). Key times strictly increasing. 3. **Ground everything.** Claude's legs on the desk except during a hop; props rest exactly at their surface height (3.6). A prop floating a few hundredths above the desk is the most visible bug. 4. **Characters move only on a cut, or along an eased path** (a hop-walk). No teleport inside a shot. 5. **Set every prop's `visible`, position, rotation and scale in `world(t)` for every t**, not only inside its time window, or it stays where the last rendered frame left it (frames render in any order). 6. **No jitter**: no motion faster than ~2 Hz except a deliberate wave or shake; no raw audio levels driving positions. 7. **No coplanar faces**: offset stacked cards and decals by >= 0.01-0.03 units. 8. **Flashes stay under the caps**; the phone screen and the lamp are the brightest things, never a face. 9. **The camera never enters a prop** and never sees past the side city planes or under the desk; check the first and last frame of every shot. --- ## 8. Stills and the safe zone ```bash node engine/render.mjs my-video stills 0.3,2.9,5.6,8.9,12.2,14.5,16.5 --safe ``` Always include frame 0.3 (the cover), the first and last second of every shot, every stamp/slam, and the CTA. Look at `stills/sheet-safe.jpg` (and the single stills at full size for details) and check: - The output says `✓ all text inside the safe zone`, and no `framing: shots pushed back` line (move those P keys back). - The hook frame reads with the sound off: subject, banner, counter, first word. - No prop cut off at the frame edge in a way that looks accidental; no prop intersecting Claude, the creator or the phone; nothing floating. - Faces (Claude's eyes, the creator's face) are never under the banner, badge, terminal or captions. - Then render the same times without `--safe` and compare with `reference.jpg`: same warmth, same softness, same paper. Expect a few rounds. One round takes about 15 s. ## 9. The full render ```bash node engine/render.mjs my-video full 0 900 # ~0.16 s a frame on a mid-range GPU: 900 frames ~ 2.5 min node engine/render.mjs my-video full 900 # the rest (a 55 s reel is ~1,650 frames) ``` Resumable: it skips frames that exist (> 20 KB). **After any change to scene.js or the engine, delete `frames/` (or the changed range) first.** Blank frames (a lost WebGL context) are retried in a fresh browser. Each frame's text boxes are checked against the safe zone into `frames/safe.json`. ## 10. Sound **The kit** (`sfx/.wav`, made once by `gen-sfx.mjs --yes`; the prompts are in that file; names must stay plain letters) and the levels that work, in `cues.mjs`: | Sound | When | dB | |---|---|---| | `slap` | every banner change (not at 0) | -17 | | `type` (with a max duration) / `enter` | the terminal typing / sending | -14 / -8 | | `boing` | Claude pops up / hops | -8 / -15 | | `clunk` | a prop lands | -6 / -7 | | `stamp` | an X or a stamp | -6 (a big one -4) | | `whoosh` / `swoosh` | a cut with a whip, something sweeps / flies | -9 to -11 / -12 | | `thud` | a huge caption slam | -4 to -8 | | `shimmer` | it turns warm / good | -7 | | `ding` / `plane` | the comment posts / the DM flies | -10 / -10 | | `clack`, `flick` | things snapping into a row / cards flipping | -15 / -17 | | `clang`, `beep`, `spot`, `ff` | a bin, a UI beep, a spotlight, fast-forward | -6, -12, -6, -9 | | `rain`, `thunder` | the moody beat | -11, -4 | **Music**: Mixkit **"Coffee or Tea"** by Michael Ramir C. (Mixkit #1168, `https://assets.mixkit.co/music/1168/1168.mp3`). It is free under the Mixkit Stock Music Free License: personal and commercial projects including YouTube and social media, no attribution required; not for CDs/DVDs, games or TV/radio broadcast, and you may not redistribute the file or register/claim it as your own (check mixkit.co/license before monetising). Start it **8 s into the track** (`music.from`), at **-11.5 dB** (`music.db`), **ducked by the voice** (sidechain), fading out over the last 1.4 s. That puts the bed about **13 LU under the voice**; measure it (`STEM=vo` / `STEM=music node engine/mix.mjs my-video`) if you use another track. A dark beat can low-pass the bed (`music.muffle: [[t0, t1]]`). **The mix**: voice high-pass 70 Hz, compressor (-20 dB, 2.5:1, 8/120 ms, +2 dB), +2 dB presence at 3.5 kHz; music + sound effects; normalised to **-14 LUFS integrated, true peak <= -1 dBTP** (limiter at 0.79). Re-run the mix whenever film.json or cues.mjs change: a stale effect is heard as a stray click. ## 11. Encode, check, deliver - `encode.mjs`: H.264 High, yuv420p **TV range** (JPEG frames are full range: without the conversion players get `yuvj420p`), 30 fps, `-crf 20`, `-maxrate 10M` (lowered automatically so the file stays **<= 60 MB**), AAC 192k 48 kHz, `+faststart`; `cover.jpg` = the frame at `video.json` `cover` (the hook). - `check.mjs` must pass: 1080x1920, 30 fps, yuv420p, High, AAC, <= 60 s, <= 60 MB, -14 +-1 LUFS, true peak <= -1; frame 0 not black or flat; no washed-out (flash) frames; no frozen stretch over 0.5 s; text inside the safe zone on every frame; it prints the cut rhythm (aim 2.5-4 per 10 s). - Then pull ~12 frames from the final mp4 into a sheet (`ffmpeg -ss -i video.mp4 -frames:v 1 ...`) and look once more, and listen to the whole mix once if you can (or transcribe it with Whisper and compare with the script). - **Deliver** `video.mp4`, `cover.jpg`, `caption.md` to a normal folder (for example `Videos/Voxel reels//`), never overwrite an earlier version, then delete `frames/` (~100 MB). Report in a few lines: title, length, shots, the keyword and what the DM must send, any take alternates, and honestly anything you'd still fix. Don't post. ## 12. Checklist - [ ] `setup.mjs` all green (GPU WebGL, fonts, music, sfx kit); the key present, never printed. - [ ] Script: 120-160 words, hook in 3 s, one filmable claim per sentence, a real number per beat, a turn at ~60 %, CTA "Comment KEYWORD and I'll send you ..." for something that exists. - [ ] `build.mjs --fake` -> scene blocked out -> stills look right. - [ ] One voice take, listened to; `build.mjs`; every beat keyed to words (`at()`), banners on first words, badges changing inside beats. - [ ] Stills: safe zone ✓, no pushed-back shots, nothing floating or cut off, frame 0 is a finished cover. - [ ] Full render in foreground chunks; mix at -14 LUFS; encode <= 60 MB; `check.mjs` all ✓. - [ ] `caption.md` (script, caption with the keyword twice, `DM:` line); delivered; frames deleted; short honest report; nothing posted. --- ## Appendix A: the engine (write these files exactly) Tested together (extracted from this file into an empty folder, `npm install`, `setup.mjs`, stills, a mix, an encode and `check.mjs`) on Windows 10 with Chrome, three.js 0.170.0, Node 22 and ffmpeg 7. Paths are relative to the project folder; the render server serves that folder as `/`. ### `engine/film.html` ```html <!doctype html> <html><head><meta charset="utf-8"><title>voxel ``` ### `engine/main.js` ```js // Boot: fonts ->