Which LLM Is Best for Trading? 120,000+ Inference Calls Later, Here Are the Results

Introduction

When we set out to build an autonomous digital asset trading platform, one of the hardest questions wasn't which strategy to trade — it was which AI model should make the decisions. Over the past six months, we've run 120,000+ AI inference calls across 29 different LLM models, generating over 18,000 unique trading signals across multiple tokenized pairs. This article breaks down what we learned.

This isn't a theoretical comparison. Every number in this article comes from production trading logs — real signals, real P&L, real latency measurements.

The Testing Framework

Our trading agent supports three AI architectures: Pipelines (structured multi-step workflows), ReAct (single-model reasoning loops), and Swarm Commander (a commander LLM orchestrating sub-agents). The majority of production data comes from pipeline mode, which proved the most reliable for systematic signal generation. Each model was evaluated under identical market conditions:

• Pairs: ZEC/USDT, LTC/USDT, TRUMP/USDT, TRX/USDT, 1000SHIB/USDT, XMR/USDT, ETC/USDT
• Timeframes: 1h, 4h candles with multi-timeframe confluence
• Leverage: Up to 20x on futures
• Duration: February — May 2026 (4 months production), with prototype data from Nov 2025
• Total Inference Calls: 120,000+
• Unique Models Tested: 29
• Total Signals Generated: 18,000+

The Leaders

Before diving into the full breakdown, here are the clear winners from 6 months of production testing:

🏆 Overall Best: Mistral Medium 3

59.5% win rate · +2.34% avg PnL · 8,064 signals from 17,810 calls

The undisputed champion. Mistral Medium 3 processed the most calls on our production environment and generated the highest signal volume of any model. Its average wins of +8.37% significantly outpaced average losses of -6.42%, yielding a strong 1.30 profit factor. With a 53.4% signal rate (it only traded when confident), it proved the most disciplined high-volume model in the entire benchmark.

🎯 Best Precision: Gemini 2.5 Pro

+6.00% avg PnL · +18.13% avg win · -3.44% avg loss

The "sniper" of AI trading. Gemini 2.5 Pro generated only 21 signals from 93 calls but with remarkable conviction — its average winning trade was 5x larger than its average loss. A model built for high-conviction setups when quality matters more than quantity.

⚡ Best Speed: Nemotron Nano 12B

Sub-second latency · 54,000+ calls processed · 5+ pairs covered

Unmatched throughput — processing at over 100x faster than cloud-hosted models. Ideal as a real-time market scanner. Its 56.0% win rate is positive but its average loss (-10.43%) is too wide for standalone use, making it a perfect first-pass filter rather than a final decision-maker.

💎 Best Newcomer: Mistral Small 4 119B

59.1% win rate · 253 signals · Rapidly growing adoption

The latest addition to our production stack. Despite launching in May 2026, Mistral Small 4 already accumulated 2,704 calls and shows a promising 59.1% win rate with strong risk characteristics. One to watch closely as the data set grows.

Pipeline Models

Pipeline mode runs models through structured multi-step workflows — market data collection, indicator analysis, signal generation, and risk validation. This is where the bulk of our production data lives.

Mistral (Mistral AI)

The strongest vendor in pipeline mode — Mistral dominated our production pipeline from January through May 2026.

Verdict: Medium 3 is the clear production winner with 8,064 signals and a 1.30 profit factor. Small 4 and Medium 3.5 are next-gen replacements still being tuned.

Google Gemini

Google's cloud-hosted models — the highest win rates of any vendor in pipeline mode.

Verdict: Flash models consistently hit 62-64% win rates — the most accurate family in pipeline mode. Legacy Flash has the best average PnL at +3.76%.

NVIDIA (Nemotron)

Self-hosted speed — used as the real-time scanner in the pipeline's first step.

Verdict: Sub-second latency makes it ideal as a pre-screening step in pipelines, but its wide stops (-10.43%) mean it should never be the final decision-maker.

Others in Pipeline Mode

Agent Models (ReAct & Swarm)

Agent mode gives LLMs full autonomy — they reason, call tools, and make decisions in a loop (ReAct) or coordinate as a team (Swarm Commander). This mode was introduced in March 2026.

Google Gemini (Agent Mode)

Verdict: Gemini 3 Flash proved to be the best all-round agent — 64.4% win rate with solid risk discipline across 635 signals. Gemini 2.5 Pro is the "sniper" — low volume but extraordinary signal quality with average wins 5x larger than losses.

Deep Reasoning (Research Agents)

Qwen 3.5 (122B and 397B) and Kimi K2 Thinking were deployed as autonomous reasoning agents (7,200+ combined calls). Their role is multi-factor analysis, strategy evaluation, and pattern discovery — providing deep analytical support rather than direct signal generation. Latency (300-537s per call) makes them impractical for real-time trading but invaluable as the "thinking engine" behind complex trading decisions.

Why Mistral Medium 3 Wins

Let's break down what makes the leader stand apart:

Signal Selectivity. Mistral generated 8,064 signals from 17,810 calls — a 53.4% signal rate on the production server. This means it rejected nearly half of all potential trades, demonstrating genuine selectivity rather than trigger-happy execution.

Risk Discipline. The ratio of average win (+8.37%) to average loss (-6.42%) yields a profit factor of 1.30 — the healthiest among all high-volume models. Compare this to Nemotron Nano's ratio of +7.91% to -10.43%, where losses dwarf wins.

Cross-Symbol Consistency. Mistral performed across ZEC, LTC, TRUMP, TRX, and 1000SHIB — the widest profitable coverage of any single model in production.

Instruction Compliance. In our behavioral analysis, Mistral demonstrated perfect instruction compliance (100/100) and factual grounding (100/100), making it a reliable component in multi-agent workflows.

Behavioral Quality: Beyond Win Rates

Win rate alone doesn't tell the full story. We developed a Behavioral Score (0-100) measuring four dimensions:

Behavioral Scores by Model Configuration

Critical Finding: Every single model scored 0 on Hallucination Free. This means all LLMs occasionally generated price levels, percentages, or indicator values that couldn't be traced back to actual tool outputs. This is the single biggest challenge in LLM-powered trading — the models confidently present fabricated data as market analysis.

Latency vs. Quality Trade-Off

The data reveals a clear spectrum:

• Sub-1s (Nemotron Nano 12B): 56.0% win rate, -0.16% PnL — speed without depth
• 15-80s (Gemini Flash, Mistral Medium 3.5): 51-64% win rate, best balance
• 150-330s (Qwen 3.5 122B): Deep reasoning, too slow for real-time
• 500s+ (Kimi K2 Thinking): Research-grade analysis, not suitable for live trading

The sweet spot is in the 15-120 second range. Models below 1 second lack reasoning depth for reliable decisions. Models above 300 seconds don't justify the latency increase with proportionally better signals.

Monthly Evolution

Watching model usage shift over time reveals the learning curve:

The March 2026 spike was our Nemotron Nano scaling experiment — we ran it at full speed to test whether high-frequency micro-analysis could compensate for lower per-trade quality. Result: it couldn't. Volume without discipline equals losses.

May 2026 marks the beginning of our next-gen migration — Mistral Small 4 (119B) and Medium 3.5 (128B) are replacing the older Medium 3 in the pipeline, bringing improved reasoning at competitive latency.

Practical Recommendations

Based on 6 months of production data:

For Signal Generation (Primary Analysis)

→ Mistral Medium 3 or Gemini 2.5 Flash

Both offer the best combination of win rate, PnL, and reliability. Mistral dominates on volume and risk discipline; Gemini wins on ecosystem integration and tool calling stability.

For High-Conviction Trades

→ Gemini 2.5 Pro

When you want fewer but higher-quality signals, Pro models justify their cost. The +18.13% average win with -3.44% average loss is exceptional.

For Real-Time Scanning

→ Nemotron Nano 12B (self-hosted)

At sub-second latency, it's perfect for continuous market scanning. But never use it as the final decision-maker — always have a higher-quality model confirm its signals.

For Research & Strategy Development

→ Qwen 3.5 or Kimi K2 Thinking

These deep-reasoning models are too slow for real-time trading but excellent for backtesting analysis, strategy optimization, and pattern research where latency doesn't matter.

Models to Avoid for Live Trading

• Chat-oriented interfaces — not designed for structured tool calling, dramatically lower win rates
• Very small models (< 12B) in isolation — they lack the reasoning depth for multi-factor analysis

The Hallucination Problem

The most important finding from our research: every model hallucinates financial data. Our behavioral scoring system detected unsourced claims in 100% of models tested. These include:

• Fabricated price levels that don't appear in any tool output
• Made-up indicator values (e.g., "SMA 20 at $247.38" when no SMA was computed)
• Phantom percentages in risk calculations

This doesn't mean LLMs are useless for trading. It means you must implement:

1. Factual grounding pipelines — validate every price/indicator against real data
2. Tool output verification — cross-check AI claims against actual API responses
3. Guard rails — reject signals where key parameters can't be traced to source data
4. Multi-model consensus — use one model to verify another's analysis

Conclusion

After 120,000+ production inference calls, the answer to "which LLM is best for trading" is: it depends on your use case. No single model wins across all dimensions. The future of AI trading isn't about finding the perfect model — it's about orchestrating multiple models in a swarm architecture where each plays to its strengths.

Our current production stack:

The data speaks clearly: diversified AI is the future of algorithmic trading.

Data sourced from FNXT Platform production logs (Nov 2025 — May 2026). Past performance is not indicative of future results. Trading digital assets involves substantial risk of loss.