Expert Designs Agentic AI on Massive-Scale Cloud Infrastructure

Kasinadhuni
Written By:
Arundhati Kumar
Published on
Updated on

Most enterprise AI suggests. An example is a large financial-services company that gave a software system (built by an engineer) the ability to perform live actions on the identity of its customers. Before allowing a piece of software to operate autonomously, the engineer who built it described what must exist.

Years ago, the method for giving a new partner the key to access the company’s computer systems appeared quite old-fashioned. An engineer would schedule a video conference call. The partner would join the call. And the secret identifiers that allow software programs to communicate with other software programs would be either spoken aloud or shown on a common screen. All participants on the call could see these identifiers. None were recorded in some dependable manner. And adding all of this to the time it already took to add a new partner to the company’s API ecosystem added days of waiting time. For one large financial services company, it took twelve weeks to onboard one external partner to the company’s API ecosystem.

At the same time, today, no one participates in that video call anymore – and there isn’t even a video call. The credentials are automatically generated and given to the partner via a secure portal session. Every action taken during this process is logged. Today it takes two weeks to onboard a new partner. That change was made by an engineer that removed the video call by creating something that the company had never previously allowed -  an AI agent that could take real actions on its own.

This concept is central to Bharadwaj Kasinadhuni’s work. After spending fourteen years designing and implementing distributed systems, he achieved a principal-level engineering position that set technical direction for AI platform architecture. From that position, he developed and implemented the organization’s first production agentic AI system. To-date, most commercial AI is advisory. It creates drafts, summaries and recommendations, which are then carried out by humans. Kasinadhuni created an agent that crossed that line. “Agents do not create suggestions,” Kasinadhuni said. “They implement them.”

PartnerConnect is the name of the agent. It performs the task that previously required both an engineer and a meeting. When an authenticated partner requests production credentials, the agent generates those credentials directly via the identity platform’s administrative interface - the Keycloak Admin API - and delivers them to the partner through the portal instead of showing them on a screen-share. The outcome is an entirely auditable record where there was previously nothing. In six months, the agent successfully completed 269 requests at a 99 percent success rate and never provided a credential insecurely.

Allowing such a significant amount of control over a machine-based system is precisely why most engineering teams have always hesitated. Kasinadhuni asserts that hesitation is misplaced. According to him, the industry continues to treat autonomous AI as if it is solely based upon competing larger models. “Industry is treating agentic AI as a model capability issue,” Kasinadhuni said. “My experience tells me that it is an interface design issue.” Kasinadhuni concludes clearly: “Agentic AI will not scale through better models. It will scale through better tool design.”

Kasinadhuni illustrates his point using how PartnerConnect was constructed. It employs a pattern referred to as ReAct - think, act, observe, repeat. The agent determines what an authenticated partner needs. It selects which tool to use. It executes the selected tool. It reads the results. And repeats. The intelligence people generally believe resides within models reside within Kasinadhuni’s description in the four tools the agent is permitted to utilize: a tool that produces credentials; a tool that requests modifications to a network access list; a tool that files classified service ticket requests; and a tool that answers documentation questions from source materials including cited references. “The total quality of the system relies on what occurs during the ‘act’ phase,” Kasinadhuni states. “And that relies on the tool interface - not the LLM.”

Kasinadhuni’s safety vs. recklessness design decision occurred when each tool was coded in accordance with very strict input restrictions, validation, and error responses in terms of data types. As stated earlier, LLMs are somewhat unpredictable and slightly probabilistic; however, the actions PartnerConnect performs against live banking infrastructure cannot be. Therefore, Kasinadhuni restricted each tool based on how they functioned, and assigned responsibility for determining if anything ultimately happened to the tool - not the model. The credential tool checks if the requesting partner has registered approval prior to performing any functions and also verifies if an approved environment exists prior to executing any functions. If either condition fails, it rejects the request and provides a structured explanation of why it rejected it. “The model decides when to invoke the tool,” Kasinadhuni said. “The tool determines whether or not to execute.” Any production changes also have to route through an approving human before proceeding; however, lower risk functions can proceed independently after preconditions are cleared.

However, the hardest challenge was a specific problem inherent to allowing multiple external partners to share one agent. A malicious partner could easily provide instructions to the agent to perform actions on behalf of another competitor by merely typing it in. Since prompt-level limitations may be circumvented, Kasinadhuni did not depend on those limitations. Instead, Kasinadhuni tied the agent’s identity directly to the authenticated session token at an infrastructure level below any possible conversation layer. The agent retrieves information regarding whom it believes is logged-in from the authentication session and never from any chat session, and literally cannot transfer another partner’s identity to any tool. “Identity is foundational to agentic AI in enterprise environments,” Kasinadhuni said. “Prior to developing your first tool, you must answer: as who is the agent acting?” Over a period of six months, this mechanism silently blocked four requests whose message text specified a partner that did not align with the session.

Safety considerations were incorporated into the roll-out as well. Rather than activating four completely autonomous tools at once, each was individually tested in production for a minimum of twelve weeks before the next tool activated - establishing an incremental approach that became the standard procedure for introducing autonomous functionality for this organization. Kasinadhuni has documented this approach in detail in a technical paper submitted for review to IEEE Software - essentially a forum in which organizations share successful production practices.

He ends his discussion with a gentle criticism of the model size competition. A properly designed tool will enable a modest-sized open-weights model to operate safely and consistently; conversely, poorly-designed tools will cause large-scale frontier models to fail to operate safely and consistently. Kasinadhuni firmly believes that the future of trustworthy autonomous AI will be determined not by whoever develops the largest model but by whoever develops the cleanest environment in which autonomous decisions can be made.

logo
Analytics Insight: Top Tech & Crypto Publication | Latest AI, Tech, Crypto News
www.analyticsinsight.net