What the Future Holds: From LLMs to AI Agents
In 2024, we saw a dramatic shift in interest towards AI “agents” and, more generally, systems that allow domain expert humans and AI agents to work collaboratively towards a goal or automate the handling of incoming tasks. Despite having existed as a term in AI for decades (specifically reinforcement learning), “agent” has become a loosely defined term in the post-ChatGPT era, often referring to LLMs that are tasked with outputting actions (tool calls) in response to events. Expected responses to events or chat messages constitute an execution plan for the agent to follow. The combination of tool use, plan compilation, and memory required to go from LLM → agent has necessitated a new agentic AI application architecture. This is a significantly more complex engineering challenge compared to basic LLM chatbots because these applications require state management (retaining the message/event history, storing long-term memories, executing multiple LLM calls in a loop) and tool execution (safely executing an action output by an LLM and returning the result).
Agents and Tools
One of the primary differences between standard AI chatbots and AI agents is the ability of an agent to call tools. In most cases, the mechanism for this action is the LLM generating structured output (e.g. a JSON object) that specifies a function to call and arguments to provide. A common point of confusion with agent tool execution is that the LLM provider itself does not do the tool execution – the LLM only chooses what tool to call and what arguments to provide. The agentic AI application is responsible for safely and securely executing the tools. Agents all call tools via a JSON schema defined by OpenAI, which means that tools can be compatible across different agentic AI applications. Such emerging tooling frameworks include Langchain, CrewAI and Composio.
Memory management
State-of-the-art memory management techniques include the ability to retrieve previous conversation history and manage topics of knowledge content. Each time an action occurs (a chat message or an external event), the agentic AI application moves relevant conversation history and knowledge chunks into the working context. This behaviour is critical to maintaining focus and tight collaboration.
What next?
Agentic AI is unlocking a new era of innovation, with its potential to transform industries through dynamic decision-making systems that combine human intelligence with the speed and precision of AI. While the paradigm is still in its early stages, the rapid pace of development signals that significant changes are just around the corner. Investment managers who invest in understanding and leveraging Agentic AI today will position themselves to lead in tomorrow’s world, where adaptability and strategic integration of AI will define success or failure. Simply directing your IT department to integrate AI is not a recipe for success. An example of unintelligent AI integration would be signing up for Microsoft Co-pilot, implementing it company-wide, and assuming your team members will figure it out independently. Yes, Microsoft makes it easy to give them money, and they may be pre-approved vendors, but do they understand how your business works and who will teach Co-pilot how your department operates? Strategy and an onboarding plan for new AI are critical to user adoption. Now is the moment to gain the know-how and seize the opportunities this revolutionary technology offers.
To learn more about how AI infin8 is implementing Agentic AI in Financial Services, reach out to one of our experts.