Why LLMs Alone Won’t Deliver ROI in Financial Services

Discover top fintech news and events!

Subscribe to FinTech Weekly’s newsletter

Read by executives at JP Morgan, Coinbase, Blackrock, Klarna and more

Large language models (LLMs) have been called the electricity of our time, and their arrival has sparked a wave of experimentation in finance. From automated research to customer insights, the potential is vast. But as adoption grows, a clear reality is emerging: LLMs alone are not enough without an agentic layer on top.

LLMs can generate words, but they need agents to guarantee truth. They can summarize data, but without an agentic layer, they cannot decide what matters most for your business. And in a sector where trust, compliance, and speed are non-negotiable, that gap is critical. While LLMs bring power to the system, agentic AI knows when and how to turn on the lights.

LLMs alone are not enough

LLMs are impressive, but they are reactive. They respond to prompts, generate text, and summarize data, but they do not operate with business context. On their own, they lack grounding in organizational definitions, rules, and timelines. Without an agentic layer and a context catalog, these models are powerful but incomplete. They can communicate fluently, but they cannot ensure that what they say is aligned with how the business defines truth. That gap becomes critical in complex financial environments where information must be trusted, organized, and shared consistently.

Agentic AI, combined with a context catalog, provides the missing elements: business context for decision-making and human-in-the-loop learning for continuous improvement. Together, they add autonomy, context, and memory. Agents know what to look for, the context catalog ensures outputs map to trusted definitions, and both operate within clear boundaries. In practice, this enables  financial institutions to:

*   Continuously scan markets, news, and filings for anomalies before humans notice
*   Track customer sentiment over time and connect insights to advisors and product teams
*   Automate reporting and compliance workflows so insights translate directly into decisions

Agents combined with a metadata layer turn LLMs from reactive tools into active participants in financial operations, while humans remain primary decision-makers. They transform potential into performance.

As more businesses adopt AI tools, the organizations that treat AI like a fancy side dish to their strategy won’t see the ROI they’re after. AI strategy is most successful when it’s woven into the fabric of an organization, when it becomes a part of the organization itself.

Building intelligence on top of the model

The history of electricity provides a useful analogy. Early access to power was a competitive advantage. Once electricity became widely available, the advantage shifted to those who designed the systems that used it efficiently. Factories, assembly lines, and lighting systems became differentiators.

LLMs are now at the same stage. They are widely accessible. The real advantage comes from how institutions use them to inform workflows, orchestrate decisions, and support human judgment. Simply deploying a model as a “fix all” is not a strategy. Using intelligence to solve or support a specific goal is what drives measurable impact.

Consider three examples:

*   **Market research**: An LLM can summarize news or filings. An agent, supported by contextual catalog metadata, filters, prioritizes, and highlights what is relevant for investment decisions tailored to an investor.
*   **Customer sentiment analysis**: An LLM reads social posts or surveys. Agents contextualized by the catalog aggregate insights, track trends, and connect results to relationship managers.
*   **Fraud and compliance**: LLMs parse unstructured data. Agents orchestrate anomaly detection using definitions from the catalog, then automate reporting and follow-up tasks to prevent operational risk.

In each scenario, the model provides scale and fluency, but the combination of agent and context catalog creates relevance, focus, and actionability.

Supporting human judgment

Some assume that agents or LLMs will replace humans. In financial services, this is unlikely. Humans provide judgment, oversight, and strategic thinking that cannot be automated. Agents and the context catalog amplify human capabilities by ensuring information is accurate, contextualized, and ready for decision-making. They handle repetitive, time-consuming, or highly distributed tasks.

When combined, LLMs, agents, and the context catalog create a feedback loop: The model generates insight; the agent prioritizes and orchestrates it; the catalog grounds it in organizational truth. Finally, humans make decisions.

The result is faster, more confident, and more precise outcomes. Analysts and leaders spend less time gathering information and more time acting on it.

The competitive imperative

Financial institutions that rely solely on LLMs remain reactive. Those that integrate agents and a context catalog gain proactivity, efficiency, and insight at scale. LLMs are necessary but incomplete. Agents turn them into systems that deliver real value. The catalog ensures those systems operate on trusted definitions and verifiable data.

The financial services industry is at a turning point. LLMs have become a baseline utility. Competitive advantage now comes from designing systems that orchestrate intelligence, provide context, and integrate across workflows. Those who understand this reality will define the next era of fintech innovation.

LLMs provide the power. Agents and a context catalog direct that power and make it useful. Together, they allow financial services organizations to see clearly, act confidently, and make smarter decisions.

About the author

Alexander Walsh is Co-Founder and CEO of Oraion. With a diverse background in strategy, finance, and international expansion, Alexander has spent over a decade driving growth for leading global companies. Before founding Oraion, he served as Director of International Expansion at Via.work, helping scale the company’s global operations and leading it to a successful exit via acquisition to JustWorks. His experience spans roles at Apple, N26, and Silicon Valley Bank, where he specialised in operations, compliance, and data-driven decision-making. Alexander’s expertise lies in business strategy, financial management, and leveraging automation to drive growth and transform businesses.