Balaji Boominathan
Back

Writing

Local inference

  1. · 3 minTool calling is the benchmark that mattersI benchmarked ten open models for agent work on my own GPU server and a laptop. Tokens per second got all the attention. The result that decided everything was whether the agent loop survived its second turn.

    Try “agents”, “ISO 26262”, “MCP” or “FMEA”.