Brian Sathianathan, Chief Technology Officer and co-founder, Iterate.ai, says instead of chasing scale for its own sake, CIOs should prioritize precision, adaptability and cost control.
By now, every business has heard the pitch for LLMs. What’s still less obvious is understanding when a smaller, faster, cheaper model might do the job better. CIOs are under a lot of pressure to figure out if they need large LLMs, small language models (SLMs) or both.
But the choice isn’t binary and, increasingly, the most effective enterprise AI strategies rely on a combination of the two.
LLMs grab the headlines. Their ability to generate natural-sounding responses across a wide range of prompts makes them attractive for customer-facing use cases and complex reasoning tasks. But SLMs are becoming just as important to business outcomes. These smaller, more targeted models are ideal for faster development, domain-specific use, and efficient scaling. Both models have a role to play, and using them together can lead to smarter, more sustainable outcomes.
In its recent coverage of top challenges for CIOs in 2025, Intelligent CIO emphasized the need for proactive, pragmatic decision-making about AI adoption.
As the pressure grows to deliver both innovation and ROI, the ability to match the right tool to the right task will separate the leaders from the laggards. SLMs are particularly compelling for organizations looking to move quickly. Because of their smaller size (often under 10 billion parameters), they train faster, require fewer resources and can run on common CPUs or NPUs rather than high-end GPUs. Some of the most streamlined models, especially those used in retrieval-augmented generation (RAG) systems, weigh in at just 70,000 parameters. That compact profile translates to lower infrastructure costs, less energy consumption and faster time to value.
Many enterprises are already leveraging SLMs through internal development efforts or platforms that make small model deployment easier. At Iterate.ai, for example, our team works with organizations using Generate, a private AI platform that supports both large and small model architecture for rapid prototyping and production AI workflows. What we’re seeing firsthand is that many problems CIOs are trying to solve don’t require massive scale. They require speed, context and precision.
That’s not to say LLMs don’t have a place. Their strength lies in general-purpose capabilities and the depth of their training. LLMs can handle broad or poorly defined queries, support richer interactions, and adapt more easily to evolving use cases. But that power comes at a cost – and it’s often a steep one.
Running an LLM at enterprise scale requires expensive GPUs, high energy use and ongoing investment to stay current. As major providers compete in the LLM arms race, many enterprises are discovering that mimicking those tactics isn’t going to be all that sustainable.
I see today’s situation mirroring what we’ve seen in early cloud adoption patterns. Enterprises jumped in quickly, then realized they needed to rein in costs and rethink long-term strategy. The same course correction is now playing out with GenAI. CIOs that rushed into LLM deployment are now pausing to assess whether their infrastructure can handle the load, whether their applications are differentiated enough to justify the cost and whether better options exist.
In many cases, better options do exist. Starting with SLMs allows teams to prototype quickly, prove ROI early and grow from there. SLMs are especially effective for internal tools, document summarization, code suggestions and workflows where data sensitivity is high and outputs need tight control. If those applications gain traction and business needs expand, LLMs can be introduced thoughtfully, guided by real usage data rather than assumptions.
Another major benefit of SLM-first strategies is data privacy. Many enterprises still don’t realize the extent that public cloud-hosted LLMs learn from the data you send them. That means proprietary information could end up contributing to responses for a competitor using the same model. With self-hosted SLMs or private LLM deployments, CIOs retain full control over their data, preserving it as a competitive asset rather than commoditizing it. This control also simplifies compliance. Whether it’s GDPR, HIPAA or AI regulations that continue to crystalize, enterprises need to know exactly where data lives, how it’s used and who has access. Private models offer a cleaner path to compliance, especially for industries like healthcare, finance and government, where data governance is non-negotiable.
Importantly, choosing private infrastructure doesn’t mean sacrificing performance. Today’s SLMs are surprisingly capable, especially when fine-tuned with domain-specific data. And thanks to advances in tooling and optimization, many enterprises can run them without dedicated AI engineering teams. With the right framework in place, even modest IT departments can begin experimenting and delivering results.
Of course, some applications will still benefit from the scale and flexibility of LLMs. Use cases like multilingual chatbots, creative content generation or multi-document analysis across unstructured data sets may require larger models to achieve the desired outcome. But these should be the exception, not the default. When LLMs are used, they should be used surgically, not indiscriminately. That means training only on what’s necessary, optimizing for performance and choosing deployment strategies that align with business goals. Whether public or private, these models need to be continuously evaluated for cost-effectiveness and impact.
In 2025 and beyond, the most successful CIOs will take a balanced approach. They’ll treat LLMs and SLMs as parts of a broader toolkit – not as competing options, but as complementary components.
SLMs help teams move fast and stay focused. LLMs expand the frontier once the value is proven. Together, they form a flexible foundation for enterprise-grade AI that scales responsibly and delivers measurable results.
Instead of chasing scale for its own sake, CIOs should prioritize precision, adaptability and cost control.
Start small. Build for what matters. Scale only when you’re sure the foundation is strong. In this time of rapid change, that kind of clarity isn’t optional – it’s the edge that sets leaders apart.
