Simon Willison reverse-engineers Codex Desktop's animated pet feature — gpt-image-2 generates chroma-keyed sprite sheets under the hood

OpenAI's Codex Desktop ships a customizable animated "pet" feature — a Clippy-like desktop companion that surfaces task updates — and the sprite generation pipeline is more interesting than it sounds. Developer Simon Willison (editorial trust: 0.95) reverse-engineered the feature after accidentally activating it, documenting how GPT-5.6 Sol orchestrates multiple rounds of gpt-image-2 calls to produce game-ready sprite assets, complete with a magenta #FF00FF chroma-key background for easy compositing at 192×208px.

The two key skills powering the feature — hatch-pet and imagegen — are open-source under Apache 2.0 in the openai/codex and openai/skills repos. The pipeline starts with a character-reference prompt, then generates individual animation frames (idle, waving, etc.) against the flat chroma-key background, and finally compiles them into sprite sheets and looping GIFs. Willison published every intermediate image, combined sprite sheet, and per-animation GIF in a public GitHub repo (simonw/pedalican). The practical takeaway for AI engineers: gpt-image-2's instruction-following is strong enough to hit pixel-level compositional constraints — fixed background color, bounded canvas, single centered pose — reliably enough for a production sprite pipeline.