.png){kind=logo}
Building the Infrastructure Behind Scalable and Reliable AI Systems
At a time when artificial intelligence is rapidly moving from experimentation to real-world deployment, the infrastructure behind it is becoming just as important as the models themselves. In this interview, Simerus Mahesh discusses how experience across big tech, startups, and research environments has shaped a hands-on approach to building systems that can reliably support modern AI workloads.
Having worked on large-scale production infrastructure at companies like PlayStation and Meta, his experience also spans both high-scale production environments and experimental systems, from building decentralized machine learning platforms to contributing to autonomous racing systems. In this conversation, he discusses the realities of scaling infrastructure, the challenges of observability and reliability, and what the next generation of AI systems will require to operate safely and effectively at scale.
Can you share a brief overview of your professional journey and the experiences that shaped your career in technology and infrastructure engineering?
My path into infrastructure came from working across large companies, startups, and research environments. I was drawn to how systems support millions of users without breaking.
At companies like Google and Meta, I saw how decisions impact systems at global scale, while startups and research pushed me toward building from first principles. That ultimately led me to distributed systems and AI infrastructure, where scale, reliability, and fast-moving technology meet.
Having worked across both large technology companies and early-stage startups, what key lessons have most influenced your approach to innovation and problem-solving?
The biggest lesson I’ve learned is that innovation and problem-solving heavily depend on context. In larger companies, you’re usually building within a much larger system, so it’s not always enough to only understand the thing you’re working on.
You also need to understand what surrounds it, what depends on it, where the constraints are, and how it fits within the broader business. This sort of context matters just as much in startups, but it becomes more abstract since the constraints are still changing. I’ve found that you have to move quickly while understanding why a particular decision matters for your product, the users, and the company. That changed how I solve problems.
What are some of the major challenges you have faced while building and scaling technology infrastructure, and how did you overcome them?
One challenge is that infrastructure issues often only appear under real production pressure, and without observability, debugging becomes guesswork. In one case, I traced a slow service by mapping dependencies and analyzing system metrics across CPU, memory, disk, and network. The issue turned out to be aggressive OS-level memory swapping, not the service itself. Fixing it improved performance, but the bigger takeaway was that strong observability is essential to pinpoint where problems actually exist.
In your opinion, what technical capabilities are essential for engineers to succeed in today’s rapidly evolving AI and infrastructure landscape? Having led multiple startups and innovations, what qualities do you believe every successful technology leader must possess today?
I still think that engineers today need to have a strong foundation in programming and system design. AI can make execution a lot faster, but that makes technical judgment all the more important. You still need to actually understand what the AI is building, why the architecture does or doesn’t make sense, and what tradeoffs are being made before deploying something. For technology leaders, I think it comes down to product judgment, clarity, and resilience
You also contributed to the MIT-PITT-RW autonomous racing team competing in the Indy Autonomous Challenge (IAC). How did working on autonomous systems influence your perspective on real-time distributed infrastructure and AI systems?
This experience gave me a deeper appreciation for infrastructure, as it connected software directly to hardware and to real-time constraints. In a high-stakes competition like the IAC, the underlying system isn’t just running in a cloud environment or traditional backend; it’s coordinating sensors, compute, perception, and control on the actual racecar itself. A lot of the stack had to run close to the car to reduce latency and support faster decision-making. Contributing to MIT-PITT-RW made distributed systems feel much more concrete to me.
As someone actively involved in the global developer ecosystem as a venture scout at Soma Capital and a judge for hackathons associated with organizations such as Meta, UC Berkeley, Caltech, and Carnegie Mellon, what trends are you currently observing among the next generation of AI and infrastructure engineers?
A common trend I’ve noticed while evaluating projects and startups is that younger engineers are no longer treating AI as a separate feature; they’re building products where features like agentic workflows, natural-language interfaces, and semantic search are baked in from the start. One thing that stands out to me is how quickly young teams can turn an idea into a working prototype, often with strong product instincts and AI-assisted development.
Secure and scalable AI execution is becoming increasingly important as AI workloads grow rapidly. In your opinion, what are the biggest infrastructure bottlenecks the industry must solve over the next few years?
I think execution control will be one of the biggest bottlenecks. As AI workloads grow, the challenge is to compute efficiently, isolate workloads safely, and understand the blast radius when something goes wrong. This becomes extremely important as AI systems move beyond generating text to calling tools, interacting with live data, and taking actions. Over the next few years, I believe companies need to prioritize stronger infrastructure around scheduling, observability, efficiency, and security.
Looking ahead, how do you envision AI infrastructure evolving, particularly at the intersection of distributed computing and autonomous AI agents?
I envision AI infrastructure moving beyond just stitching fragmented pieces together. The next layer will be deeper control and governance for agentic applications. Today, we already see many companies assembling the different pieces required to make agentic systems work: model providers, orchestration frameworks, tool integrations, and more. The harder problem is controlling what happens once these agents start taking multi-step actions across tools, data, and systems. To me, the real opportunity is to build infrastructure that enables autonomous systems to do useful work without becoming impossible to control or reason about.
