Built at Fireworks: Deploy with Microsoft & NVIDIA

House Rules

# House Rules — Your safety policy, enforced. A content-moderation pipeline built on open NVIDIA models served by Fireworks. It decides safe / unsafe under an explicit, configurable policy — with an auditable route trail, measured latency, and measured cost per verdict — instead of delegating the decision to a closed frontier API that applies its own policy and sometimes refuses to answer at all. ## What we built Deterministic clean-up + a rule gate strip disguised attacks (base64, leetspeak, homoglyphs, zero-width chars, emoji swaps). A LoRA fine-tuned NVIDIA Nemotron Nano 3 30B-A3B (Fireworks Training API — rank 8, alpha 32, 1 epoch, 19,806 leakage-filtered rows, ~19 min on 2× B200) handles text on a dedicated B200 deployment. Uncertain cases (~5.8% in the demo pipeline) escalate to Nemotron 3 Ultra 550B on Fireworks serverless. ## Headline — held-out NVIDIA-policy test (n = 3,340, same rows for every system) | System | Macro-F1 | Unsafe recall | Over-block | Refused | p50 | $/1M msgs | |---|---:|---:|---:|---:|---:|---:| | House Rules | 95.3 | 97.6% | 5.5% | 0% | 644ms | $56 | | Nano 30B untuned | 82.7 | 93.3% | 27.8% | 0% | 804ms | $64 | | Ultra 550B zero-shot | 84.5 | 86.7% | 3.3% | 0% | 710ms | $108 | | GPT-6.1 Sol | 71.5 | 70.0% | 1.1% | 1.9% | 3,235ms | $1,820 | | Claude Opus 5.5 | 69.0 | 67.6% | 3.9% | 13.0% | 2,966ms | $6,045 | +23.8 macro-F1 vs GPT-6.1 Sol (95% CI +22.1, +25.5; McNemar p = 1.1e-169) and +26.3 vs Claude Opus 5.5 — at 32× and 108× lower marginal cost. Paired on identical rows; refusals scored as errors; every number has a receipt in results/. Fine-tuning the same 30B model: 82.7 → 95.3 (+12.6 F1), wrongly blocked 27.8% → 5.5%. ## Business value At 100M msgs/month: ~$9.8K always-on House Rules GPU vs ~$182K GPT-6.1 Sol vs ~$605K Claude Opus 5.5 — while scoring higher, refusing 0% of requests, and logging every verdict's route, model, and cost for audit. (GPT is cheaper below ~5.2M msgs/month.) Users: product and trust-and-safety teams at chat, social, UGC, and AI-companion platforms who need a configurable, auditable moderation verdict. ## Sponsor tech (entering NVIDIA Best-Use Award) Every model lane is NVIDIA Nemotron on Fireworks: Nano 3 30B-A3B (LoRA specialist), Nemotron 3 Ultra 550B (gray-zone escalation), Nemotron 3.5 Super VL 120B (image lane — wired + unit-tested, not evaluated in this submission), NVIDIA Nemotron-3.5 Content Safety Dataset (CC-BY-4.0). Fireworks surfaces used: Training API LoRA SFT, dedicated on-demand deployments, serverless inference, deployment shape matching. No Microsoft services are used. ## Evaluation & reproduction 3,340-row held-out NVIDIA-policy test + 2,222-row disguised-attack set (8 perturbations from 6 public benchmarks). Deterministic scorer, paired bootstrap CIs, McNemar tests, refusals counted as wrong. 170 leaked training rows removed. Replayable demo (no keys): python demo/server.py → 127.0.0.1:8787. Eval notebook: notebooks/house_rules_eval.ipynb — re-derives every headline number offline from shipped receipts. Repo: github.com/aldridge-fonseca/house-rules ## Honest tradeoffs Over-block is the price of recall: 5.5% vs GPT's 1.1% on-policy, 25.3% under disguise. GPT-6.1 Sol led the earlier disguised-attack run (87.8 vs 82.0 macro-F1). Our advantage is on the target policy — fine-tuning on your own policy is the product.