Running an Insurance Claims Pipeline on Nemotron: 5.3x Cheaper Per Assessment Than Gemini 2.5 Flash

Editor’s Note

Gruve builds AI infrastructure for enterprises that need to run inference workloads at scale, on their terms, on their hardware, under their compliance posture. This post walks through a use case our engineering team built end to end: an AI-powered insurance claims assessment pipeline running on NVIDIA Nemotron. It is a practical look at what it takes to move from prototype to a production-grade agentic pipeline in a regulated industry, and what the token economics actually look like when you swap a proprietary model for an open one.

The Problem

Today’s claims automation typically layers Optical Character Recognition (OCR) and rules engines on top of adjuster workflows that were designed for paper. The fatal assumption is that damage is a text problem. OCR can transcribe a repair invoice; it cannot tell you that the crumple pattern on a front quarter-panel is inconsistent with a reported rear-end collision, or that the odometer reading in a submitted photo doesn’t match the mileage declared on the policy. The moment physical evidence enters the loop (and in auto claims it always does) OCR-first pipelines hit a wall.

The next generation patched this with bolt-on rule engines: if fraud-score > 0.7, escalate. If repair estimate > $15,000, require inspection. Rules are cheap to write and catastrophically expensive to maintain. A regional labor rate spike, a new damage typology from a hail season, a change in OEM parts availability: any of these invalidates thresholds that someone hardcoded eighteen months ago. The rules team is perpetually behind the loss curve, and adjusters spend their days overriding stale outputs.

What’s actually missing is grounded visual reasoning: a model that can look at five photos of a vehicle, reconcile what it sees across images, cite specific visual evidence for each conclusion, and produce a structured output an automation system can act on, without hallucinating damage that isn’t there. Equally critical is the audit side. Regulators and internal compliance teams increasingly require that every automated decision be traceable to specific evidence. A claim approval or denial needs to cite which image, which damage item, which confidence score, and which rule triggered, stored in a form that survives a regulatory examination three years later.

What an AI Claims Assessment Pipeline Is

An AI claims assessment pipeline replaces the manual adjuster workflow with a single, automated sequence: vision-grounded damage analysis, market-rate cost estimation, fraud detection, report generation, and a configurable claim decision, with compliance enforced on every inference call. Open models running on on-prem GPU infrastructure make this viable for regulated buyers who need data residency, audit trails, and cost predictability at scale.

Meet the Model: Nemotron-Nano-12B-v2-VL

NVIDIA Nemotron-Nano-12B-v2-VL is a 12-billion-parameter open-weight vision-language model that processes image and text tokens through a CRadioV2-H vision encoder feeding into a hybrid Mamba-Transformer language backbone. That joint processing lets the model’s chain-of-thought reference image regions directly, rather than working from a text description of them. When the model writes a severity rationale, it is attending to the image content itself.

The /think prefix in system prompts activates explicit chain-of-thought. Responses contain a <think>…</think> block that walks through the model’s visual inspection before committing to a structured JSON answer. In practice, the same model call produces both machine-readable output (damage type, severity, bounding box coordinates, confidence score) and a human-readable reasoning trace that compliance teams can audit. The reasoning trace and the structured output come from the same call, so audit evidence and inference share one artifact.

How AI Assessment Streamlines Claims

Understand the full claim. The pipeline doesn’t just read text. It reasons over photos, claimant narratives, and policy context together. Bounding boxes on damage regions, sourced cost estimates, and confidence scores give assessors evidence they can verify, not black-box verdicts.

Scale to claim volumes. The five-stage pipeline (intake, damage analysis, cost estimation, report generation + fraud detection, claim decision) runs as a state machine. The first three stages execute sequentially because each feeds the next; report generation and fraud detection run concurrently, cutting wall-clock time for the back half roughly in half. Results stream to the assessor UI in real time over SSE as each stage completes.

Surface the evidence. For each damage item, the model returns the exact image filename and a bounding box expressed as normalized coordinates (0.0–1.0 relative to image width and height). This maps directly to percentage-based CSS positioning in the UI. No intermediate geometry calculation, no separate annotation service. Each box is color-coded by severity at render time: red for severe, orange for moderate, yellow for minor. Multiple damage items on the same photo are independently labeled so an assessor can visually verify each finding against the cited region.

Enforce compliance on every call. Every model call in the pipeline returns not just a completion but a set of policy evaluation results attached by Trussed AI’s controller. Each policy result carries a score; if any score exceeds the configured threshold, the call is rejected immediately. The response never reaches business logic. When a call is blocked, the claim is halted, a policy violation flag is set on the claim record, and the specific policy type that triggered the block is recorded. Governance evidence is written to the claim record regardless of outcome, so compliance teams have a complete picture of every check that ran. This makes Trussed AI a blocking gate on every inference call (damage analysis, cost estimation, fraud detection, and the assessor adjustment chat), not a post-hoc audit layer applied after decisions are already made.

Audit log view: per-stage action, performer, timestamp, duration, and justification, queryable across the full claim lifecycle.

Audit Assurance view: governance, compliance, and controlled automation checks across audit logs, topic screening, toxicity & bias, prompt injection, and language coverage.

AI Assessment at Work: The Insurance Claims Pipeline

The pipeline serves three personas (**Claimant**, **Assessor**, and **Admin**), each with a dedicated UI.

Claimant

The claimant submits photos, a narrative, and policy details through the Claims Intelligence Platform. The system ingests evidence (vehicle damage photos, odometer readings, police reports) alongside policy context and incident information.

Assessor

The assessor reviews AI-generated damage analysis, cost estimates, fraud signals, and a structured claim report, with governance evidence visible per pipeline stage.

Damage Analysis. The VLM identifies damage items, maps them to bounding boxes on submitted photos, and scores each by severity and confidence.

Damage analysis view: bounding boxes on vehicle photo, detected damage items table with type, location, severity, and confidence scores.

Cost Estimation. Repair line items are grounded in real-time market pricing (DuckDuckGo), broken down by parts, labor, and paint.

Cost estimation view: repair line items with method, parts/labor/paint breakdown, and grand total ($2,443.50).

Fraud Detection. The fraud detection service scores risk, checks narrative consistency, and surfaces anomaly flags for assessor review.

Fraud detection view: fraud risk score (55/100), narrative match (85%), anomaly flags (incongruent damage pattern, narrative mismatch, pre-existing damage).

Admin

The admin configures decision rules, monitors token consumption and latency across model providers, and reviews aggregate analytics.

Portfolio Analytics. Track claim volume, fraud mix, settlement pace, and portfolio distribution at a glance.

AI Rules and Thresholds. Configurable score cutoffs for routing decisions (auto-approve, manual review, inspection required) without redeploying the pipeline.

Rule Library. Per-service prompt rules that guide how the VLM reasons about damage, cost, fraud, and decisions.

Suggested Rules. The platform analyzes patterns across claims and proposes new rules for admin approval.

The five pipeline stages each write their output to the claim’s own database record as discrete, timestamped entries. Claim decision history is append-only: each time a decision is made or revised, the prior decision is archived, preserving the full adjudication timeline. Every VLM call produces a structured token usage record (service name, model, backend, prompt tokens, completion tokens, thinking tokens, duration, and estimated cost), queryable across all claims in aggregate. The pipeline is NGC-containerized and deployed on Cisco AI Pod for on-prem fit.

Token Economics: How We Chose Nemotron

Below are the token usage and cost estimates for a single AI Assessment run, covering all five pipeline stages, for both models. Rates are sourced from OpenRouter pricing.

Gemini 2.5 Flash

Nemotron-Nano-12B-v2-VL

Note: Nemotron’s thinking token count shows 0 because its reasoning is embedded within the completion tokens, unlike Gemini which bills thinking separately. Rates are sourced from OpenRouter pricing.

$0.0123 vs. $0.0658 per assessment, 5.3x cheaper on Nemotron.

VLM usage view: token consumption and estimated cost per agent across all pipeline runs, with per-call detail.

The cost gap comes down to two factors. First, Nemotron’s per-token rates are significantly lower: $0.20/M prompt and $0.60/M completion vs. Gemini’s $0.30/M prompt and $2.50/M completion. Second, Nemotron embeds its reasoning within completion tokens rather than billing a separate “thinking” tier, so there is no thinking-token surcharge. Gemini’s 12,883 thinking tokens at $2.50/M alone account for nearly half its total cost.

At 10,000 claims per month, that’s $123 vs. $658 in model spend. At 100,000, it’s $1,230 vs. $6,580, before factoring in that Nemotron runs on-premises or on a private cloud endpoint, which matters for carriers with data residency requirements.

For cost estimation, accuracy is grounded in real-time market data (regional labor rates and parts pricing pulled at the time of the claim), making estimates current to the market rather than a static table last updated quarters ago. For damage detection, model-reported confidence scores average 87.5% across 133 damage findings. A structured annotation effort against a labeled ground truth set is the right next step before publishing defensible precision numbers.

The honest trade-off: Gemini 2.5 Flash is currently more precise on damage detection and cost estimation. Nemotron wins on cost, GPU/container fit, and on-prem deployability. How we’d choose: cost per claim, on-prem requirement, latency budget, compliance posture.

Learnings for Regulated AI Workloads

Why LangGraph, not NeMo Agent Toolkit. LangGraph is a code-first orchestration engine where nodes are plain functions, edges are conditional logic, and a typed state schema travels through the entire graph. That made it the right fit for a claims pipeline where each stage feeds the next, partial reruns need to re-inject state mid-graph without replaying from the start, and the human-in-the-loop breakpoint at claim decision needs to pause execution cleanly until an assessor acts.

The NeMo Agent Toolkit operates at a higher altitude: less about defining logic and more about running it reliably in production. It handles guardrails, secure sandboxed execution, hardware-optimized inference via NIMs, and an agent-to-agent protocol that lets independent agent microservices coordinate across services. The natural end state is both together: LangGraph as the brain, NeMo Agent Toolkit as the armor.

What we’d revisit. We’d lean into the NeMo Agent Toolkit earlier rather than treating it as a later-stage concern. The observability it provides out of the box (token tracking, cost monitoring, latency per agent call) is something we built ourselves, and that’s work NeMo would have handled without custom instrumentation. We’d also evaluate NVIDIA NIMs earlier as purpose-built, hardware-optimized inference endpoints that would reduce the latency variance we see across pipeline runs today.

Where this applies next. The underlying pattern (VLM-grounded structured extraction, a typed pipeline graph, per-decision audit trail, and a compliance gate on every inference call) applies anywhere a human expert currently looks at images and produces a structured judgment. Healthcare prior authorization, underwriting, workers’ compensation, and equipment breakdown coverage all share the same structural problem: physical evidence arrives as images, decisions need to be traceable, and static rule engines can’t keep pace with a shifting loss environment.

—

Book a time for a technical walkthrough with Sai and the Gruve engineering team