The pitch billed as a routine Nemotron weight drop turns out to be a build log: NVIDIA engineer Rick Grandy's team used a stack of existing open Nemotron models to wire up a talking, animated Halloween decoration, published on Hugging Face under the title "nemotron-halloween." No new model checkpoint ships with the post — the news is the reference architecture, not a release.
The system, called MonsterHouse, chains five NVIDIA components into a cascaded voice pipeline. NVIDIA Nemotron 3 ASR (the streaming English checkpoint, nemotron-speech-streaming-en-0.6b.q8_0.gguf) converts visitor speech to text. NVIDIA Nemotron 3.5 Bolt E4B 4GB, a 4-bit quantized small language model built for conversational agents, generates each reply, with a BF16 checkpoint kept around for comparison. NVIDIA Magpie TTS Zeroshot turns that text into speech using a short reference audio sample rather than a trained voice. NVIDIA Audio2Face-3D maps the clean speech onto a facial rig built in Autodesk Maya and exported to Unreal Engine 5. An optional vision layer — NVIDIA Nemotron 3 Nano Omni paired with YOLO and ByteTrack — reads camera frames to describe costumes and track how many visitors are in view. NVIDIA's Nemotron Voice Agent framework coordinates the whole pipeline.
Personality came from a LoRA adapter, not prompting alone. Grandy built a dataset of over 5,000 multi-turn conversations, generated and judged by a separate large LLM, then trained adapters per candidate model — some in about an hour, some taking a couple of hours — using NVIDIA NeMo AutoModel's LoRA recipe for Bolt E4B. He writes that he tested more than a dozen models before settling on Bolt E4B, citing a smaller footprint than Qwen3 30B-A3B and Qwen 3.6 35B-A3B as rivals he considered.
The deployment runs on two GPUs on a single workstation, split by workload: the language and voice pipeline on an NVIDIA RTX PRO 6000 Blackwell Max-Q, and Unreal Engine rendering plus Audio2Face-3D on an NVIDIA RTX PRO 4000 Blackwell. Grandy's post states memory was not the constraint — compute was, since animation frame rate had to stay steady regardless of what the agent pipeline was doing.
The one hard latency number in the post: median end-of-speech-to-audio with Bolt Q4 and TTS runs 1.70 seconds, dropping to 0.72 seconds for cached responses, with a caveat that speech-end detection currently waits for 300 ms of silence before triggering generation. That gap is why Grandy leans on a prepared text and cached-audio bank for common lines — authored text skips language-model generation, and saved audio skips synthesis entirely, so the character can "start talking while the rest of the sentence is still on its way."
The vision layer is explicitly unproven in production. Grandy tested Nemotron 3 Nano Omni's costume recognition on a synthetic video of five costumed sheep, correctly identifying pirate, vampire, witch, pumpkin and ghost outfits, but writes plainly that "the sheep are a synthetic test, not evidence of a successful Halloween night" and that real driveway groups, costumes and night lighting still need testing. He also flags that trendy costume identification and witty retorts "may need some post training," and that MonsterHouse's own application code is not being released — only the underlying NVIDIA models and the Nemotron Voice Agent framework are available for others to assemble.
For teams evaluating open-weight stacks against closed APIs, the actionable path here is narrow but concrete: clone the Nemotron Voice Agent's Generic Assistant example for a working ASR-LLM-TTS cascade with voice activity detection built in, swap in Bolt E4B using the profiles NVIDIA ships for single-GPU deployment, and push conversation state tracking into the application layer rather than asking the model to hold it — that's the one architectural decision Grandy calls out as what kept a 4GB quantized model coherent across a multi-turn exchange on a single workstation.