From Systems to Agents : Rethinking Enterprise Architecture in the Age of AI - with Gautam Nadkarni

FULL TRANSCRIPT

This transcript is based on the episode’s English auto-captions and has been formatted for readability. Please allow for occasional transcription errors in names, acronyms and specialised terms.

[00:00:00]

till till before these new agentic AIs or AI agents uh was something that automation executed now the agents are the ones you know that are able to decide uh for themselves right so that in an agentic system uh what's more uh differentiated is the context that you can actually provide to the agent that becomes even more fundamental uh to supporting an agenting system you know and almost like a central you know to to the architectural nervous system you know of an organization needs to go into production and [music] not just you know it doesn't end you know till the time that the system is designed so that someone will pick it up build a system [music] and it's done. No it's it's not possible. Have human in the loop where required cuz that's what's going to give them the maximum business value [music] maximum agility. Enterprise architects who used to have you know annual review cycles [music] um you know traditional kind of governance structures and so on. It has to be continuous. Hello and welcome to the enterprise tech talk podcast. I am your host Saumitra Kalikar. Now for decades enterprise architecture has been about designing deterministic systems which operated predictably um followed well-defined organization flows and had very clear scope boundaries. But in the age of AI this is changing we are seeing emergence of agentic systems now which not only can reason but can also act autonomously. And that raises an important question as to if the systems we are designing are inherently nondeterministic then how are we going to design the overall technology ecosystem and how are we going to govern that is the focus of today's conversation enterprise architecture for agentic systems and to help me unpack this important topic I am joined today by Gautam Nadkarni um Gautam is the chief architect and head of tech advisory at VRO Australia He's also a member of Forbes Technology Council. Gotham, welcome to the podcast.

[00:02:14]

>> Thank you, Samitra. You know, glad to be here with you. >> Goautam. Um, you are uh professional journey has been interesting. You have worked on various technology domains uh starting from uh service management to uh infrastructure, cloud, architecture and now. Yeah. So maybe to get started um can you provide a brief overview of your professional journey and how you eventually landed in the world of AI? >> Sure. Sure. Thanks uh thanks Audra. Um it's a very good question and the way that now you put it that it just kind of thinks that looking now you know you've been into you know your career you know for a very long period of time but look I think uh to be honest I think I I started off my career with uh um into data warehousing you know and some of the technologies and so on right and working for you know one of the medical and healthcare uh customers you know for us uh but from then on I think you know um every few years I I think you know I got opportunities to um move or choose uh paths you know that uh came across to me and personally I believe in the philosophy of you know going as it goes you know so taking it as it goes uh fortunately I have got opportunities you know which were very interesting uh challenging uh that provided opportunities for learning and so on right and that's how typically I made my choices in terms of um moving through the different either technology or even some of the roles as well, right? Not purely, you know, sticking to you know, one or the other. Um, so so it's been a very uh a very good journey, very learning, you know, uh, learning learned kind of a journey, you know, over through the so many years, right, with uh, data warehousing to service management as you mentioned to infrastructure consulting, got into some of the platform engineering parts of it, you know, got um, opportunities to build some digital banking platforms, you know, on the cloud. um then moving into you know from an architecture overall uh standpoint and so on and so forth and now you know coming to the AI part of it right so which is you know pretty much pertinent and again uh my my for as well you know into the AI world has predominantly been uh more in terms of the opportunities that uh I'm getting to work on right because it's it's a very challenging era that we are into from an AI perspective uh so which is where it's very exciting uh to learn about you know so many new things and you know things that are fundamentally changing uh of the typical you know technology paradigms that we've known all through so many years um and I think even going forward as well right I think you know I would choose you know the way that I see it is um try to keep it simple uh as long as you know there is some you know interesting work to do uh interesting people around you to kind of work with uh it uh you know that the journey becomes actually very interesting and uh enjoyable. Yeah, that's a great backdrop. Um, Gautam, um, so maybe we can started start with a with a grounded conversation. Um, see, uh, this whole term concept around agentic systems is um, uh, is becoming more and more visible and more and more discussed with within the business organizations and rightfully so. Uh, but it also has a bit of mixed interpretation, right? So when you talk to business leaders and IT leaders uh and try to explain what agentic system is, how do you approach and um explain that in a much more simplistic and practical practical way?

[00:05:49]

Um uh sure I think you know so um so look in the most uh you know simplistic way if you have to define um and and if you look at the journey uh that has exponentially grown over the last about uh 2 and a half to 3 years in the AI space uh look AI has been there for quite a bit of time uh but it's it's only I think you know gotten off this in the last about two and a half three years you know after the large language models you know became much more um accessible to everyone at a cost that's actually not accessible to everyone and so on. Right? So which is where uh that capability was actually uh available now and and and it's getting more and more important today because um you know there are a lot of organizations almost now 80% you know in the last uh 2 years if you see organizations that have now using these new AI models in one way or the other uh and that's you know and that has grown you know from almost I think you know where it was about 25 30% about two two and a half years ago to almost 80% plus now uh in you know quarter 1 of 2026.

[00:06:58]

Um but all of that right I think the just shows that look you know the adoption of this fundamentally uh changing you know technology has been uh extremely high and the curve is very very steep you know as compared to any previous changes that we have seen. So uh so what is it that's making this different to answer your question right um uh so so in in most simplistic terms right so know agents are you know systems that are able to do either of these uh four things right one uh are they able to actually perceive what's happening in their environment right so uh either in terms of the data that they're being fed or events that are happening or visuals that are getting presented to them uh or or whatever that they are hearing right so so that's where you know essentially are AI agents that can actually perceive the environment um the second type of agents essentially more around reasoning where the agents are able to not just understand what they are perceiving but able to actually make a judgment call on what to do with that particular perception that they have gathered um the third is essentially now that they have actually perceived they have actually isn't understood you know what to do uh they have now the capability to actually act right so so they can actually act through tools through APIs through other systems and take that action uh and not just actually stop there but also then adapt further uh learn from what the outcome has been and then get into a loop where they actually perceive reason act adapt and it can go on into a continuous loop right so so uh so in the most simplistic terms agents AI agent are um uh are systems that are actually built with underlying um large language model intelligence if I were to you know put it in very you know very kind of layman terms >> uh that are very highly intelligible uh intelligent uh that are able to have a logic which is following uh neural logic which is like the way that our brain works uh and able to perceive reason act and adapt right uh so if you were to really look at it that look you know the automation till till now that we knew uh the automation till till before these new agent AKIs or AI agents uh was something that automation executed now the agents are the ones you know that are able to decide for themselves right so that's where the autonomy comes in >> yeah yeah and you already uh differentiated explained well as to how it differs from the traditional automation because automation as a concept is not new uh we had different the auto automations and this is a bit different but just to clarify a few points for our audience um how would you differentiate agents from let's say microservices and other other automation technologies in the past >> yeah it's a very interesting question I think you know um a bit technical in nature as well depending on the know who uh would look at it right but essentially if you're looking at how how are today's you know AI agents or Gen AI or agent AI different fundamentally different from automations or you know workflows or you know microservices you know all of the systems that we know of today um it would be easier to understand it from their behavior in terms of how uh different their behavior is from the traditional or not I shouldn't say traditional whatever that we know of automation workflow microservices before the new AI 4A uh to now into the agent authentic systems, right? How are they different? And if I were to really kind of simplify it, maybe I would actually put it in five uh different dimensions.

[00:10:50]

Um so the first dimension essentially is the execution path of how that particular system is executing whatever it's supposed to do, right? Um and in this case, right, so the traditional automation or workflow microservices, uh they would typically have predefined, you know, deterministic paths, right? we have defined or programmed it in such a way that it will behave in a in a particular manner. Uh and you'll be fairly certain that login that's how you have programmed it. Um an agent system is something that the execution path that it actually follows is more emerging you know based on the context that it is provided and planned at the runtime right so it will decide in terms of what and how I should behave uh you know while it's actually being executed itself. It's one that execution path is you know fundamentally different. The second part if you look at it for any system is uh where do you think that the system can fail right? Uh so from a failure mode perspective uh typical traditional automation workflows microservices you know they would um the failure mode would be typically you know from a predictable standpoint right so where either it would time out that look you know there is no response from it or there would be an exception a scenario that it has not come across and that's the reason why it will terminate right so in which case that particular transaction or um or or an or or an action or execution that automation is supposed to provide it will fail and stop and it's stop at a particular point. Whereas if you look at agentic system then the failure mode for an agentic system typically is it's a bit more probabilistic. So it will still it might still continue to operate right but it would actually either hallucinate it will actually create stories right or it would drift you know from uh the level of confidence that it's supposed to provide what's the exact you know required output right because there is a inherent intelligent you know judgmental system that's actually running behind it. So that probabilistic behavior around uh failure being from hallucination and drift uh is different uh with respect to traditional automation um systems you know which is more timeout or you know exception based. The third is essentially around testing right so how would you test a typical automation system versus how would we typically test today an agentic system right um so uh very we've been now for past so many years right been used to do you know your unit test system test integration test for whenever these systems are getting developed right your automation workflows microservices APIs etc uh for agentic system however I think the testing needs to be a more about evaluation that look you know what's level of confidence of the output it is getting produced what's the uh level of accuracy uh that it is actually getting produced right uh you'll have to constantly simulate to understand you know whether the system is drifting or not drifting uh you'll have to do behavioral testing in terms of those AI agents in different scenarios how are they behaving uh because there is no you know standard input standard output you know kind of scenarios right because your uh the behavior of the system would depend on what's the context you're providing at the input And based on that even then uh you know the output would be typically different into different scenarios right so the testing is different um the fourth is essentially where it comes with the state uh that the systems follow right so whether a traditional IT systems you know they're either stateless or session bound you know in terms of having a particular state you know with a particular you know set parameters that are there relevant at that point in time okay uh but however agentic systems they have a memory memory that actually goes you know across a time horizon right so in terms of how do you track that you know that's also a state in terms of that it it it has a memory of its own and and it could work either ways right I mean you know sometimes in most times you know it gives the context but in certain times if you want it to behave certainly differently then you would need to ask it to forget what the previous memory was to be able to really get the required output that you need and the last fifth is essentially about the governance right how would you govern agentic systems um your code reviews, change management, you know, typically are uh your governance mechanisms for traditional ID systems. Whereas in case of agentic systems, what you would need to do is, you know, you need to continuously observe their behavior. Uh you need to continuously evaluate the um how that system has been built, how the underlying LLM models are actually being released on a frequent basis and basis that you know is the system kind of changing and so on. So you have to continuously evaluate both before production and after production as well in terms of what's happening with that system. So um and and it's a very sorry a long- winded answer but I think if I were to summarize in a way um that typically you know a micros service you know would typically answer what exactly it is doing right what exactly we need to do a typical workflow would actually help us to execute how essentially a particular process needs to run whereas the now the newer systems agents uh in addition to the what and how it actually actually gets a why as well that it's able to reason and then decide in terms of you know how the what and how have to be performed. >> Okay, that's that's good Goautam. Uh you are clearly I think articulated that u uh agentic systems this is a this is not an incremental change uh in the way automation has done in the past. It is really a fundamental shift uh completely right and and that raises some important implications on how enterprise architecture needs to be done in in the in the in the future because um in in traditional way uh systems the we have uh the technology of platforms the systems which are very much deterministic they follow very well- definfined workflows as as you also mentioned right um but uh and they they had very clearly well- definfined integrations right Um but in the agentic system um the as you said there is a inherent intelligent uh agency um there is uh uh emergence of mechanisms and protocols called like MCP which determine dynamically which which are the tools they should be using for what kind of use case etc right so that raises um important question as to u when you talk about how we do the enterprise architecture traditionally um identifying systems identifying integration points pattern etc. [gasps] What are the shortfalls in the way we used to do enterprise architecture um and so and and what kind of gaps we need to fill in to make enterprise architecture ready for uh agentic systems in the future.

[00:17:45]

>> Yeah. Um I think you're right absolutely uh you know Somra because uh you know till now you know as architects or even as you know overall IT professionals you know we've been used to working with deterministic systems and you know typically these words deterministic and probabilistic you know um have now started to become more prevalent. uh deterministic being that look you know you've uh you you have programmed a system such that uh you know there is a a stated out input and then based on that there will be a stated output and you can be reasonably confident based on how you have programmed in terms of what the input and output could be whereas probabilistic is that look um you the the system has an intelligence of its own. So it will take the input but along with the input you will have to provide as much context as you can provide to that system as an AI agentic system and then based on that it will decide you know what kind of output is relevant. So the outputs as well you know could be probabilistic in terms of whether you know it would be this or that right. So there is a bit of probabilistic nature in there. um and and for that right so if you look at typical traditional enterprise architectural approaches um there are a few uh areas in my mind where uh typically you know things uh fall short of and I think they're they're fast kind of closing the gap uh but I think you know given the steep learning curve uh we are still in a catch-up mode right so first one I would think in terms of um you know some of the architectural frameworks that are there today uh that usually Enterprise architects use and they all you know assume uh fairly you know deterministic systems and structures constructs that are relevant from determinist things like you would have capability maps you know application portfolios you know integration patterns you know all of them uh they assume that look you know given an input X the system will produce an output is Y right but as I explained I think you know when it comes to now the newer uh AI geni agent KI kind of systems you know that con that that assumption is no longer good right because based on input X uh depending on what other context that you give to the system the agent will decide in terms of what the output is and the output could be either Y or it could be Y minus one or Y + one right so in this scenarios what's the best practice from an enterprise architecture framework that to be utilized in such a frame of reference right so uh so that's one part the second is um from a standpoint of uh you know the static blueprints of technology and architectures uh more and more uh all of the systems are actually becoming more autonomous right so where you know previously you did have blueprint where you've got an X system and a Y system and both you know you you have to program it that look you know how are they going to talk to each other in this scenario the X agent and Y agent if there are two different agents they can themselves determine in terms of how they want to talk to each other and how they want to behave with each other, right? So, so that's a runtime behavior. So, um so when we think as architects in terms of how to build such systems, uh how to manage systems systems, how to make sure that these systems um are able to produce the outputs that within a required range. Um that's something different, right? We're not used to that kind of a uh construct and that kind of a way of working from an architecture perspective that look you know how do we get to that state it's really uh very practically and fundamentally uh changing architects behaviors on the ground and that I think architects needs to be very conscious about it um the third is essentially around um from some of the change management uh of the code or the systems right so uh for agents as I said I think you know the behavior can actually shift, can drift um for multiple reasons. You know, an agentic system, right? Either the underlying model that was uh used that changes. Either you use a different model or there's a newer uh released model uh you know, version of this of a newer model. Um you know, all of these things, right? So based on that the system behavior will change or it will change based on the context that it is provided or the guardrails that it has actually you know being uh constructed around it. All of these uh would change the behavior of the system uh even if you don't change the code right. So uh so how do you kind of manage and you know construct all of these things together so that it's not just about the code but you know it's more about the behavior in terms of what we want that particular agent to do and the various surrounding parameters that are changing alongside with it.

[00:22:28]

>> Yeah. Um so just an addition question to that goam. Um uh what does it mean then to way we used to do architecture governance uh right uh because as the the determinism determinism has gone away right there is less assurance to architects as to what the output would be right and most of the governance in the traditional sense has been around getting that assurance making sure there are less risks around outcomes etc right um so what are those implications on architecture governance Yeah. Yeah. Pretty much. So I think um uh and there are uh you know so many dimensions to this question you know to be honest but um uh but look uh again I think going back to what I had said right I think the core uh difference of uh the uh you know probabilistic or autonomous systems of agentic that we have to uh you know deterministics you know previously systems that we've been used to governing uh is uh the same thing that look you know a a system behavior you know previously used to be very predictable in terms of the time and then you know you kind of monitor that for deviation. Whereas in this you know in an agentic world what would happen is you can only make out what the agent's output is you know when uh it's at actually runtime when an actually output has been generated right so uh so that's the that's the fundamental change of the context you know in terms of what's changing right so which is where as a architect or as enterprise architects um what you need to focus on is not just about uh the actual agentic system being designed or the actual agent system as such. But the architectural governance also has to uh shift a bit to the left in terms of before that agent being live what kind of evaluation of the pipelines of the context that it is being provided uh what's the evaluation uh before that as well as once the actually agent is in action you know architectures also have to govern the runtime from a monitoring perspective to make sure that the agent is still behaving within acceptable limits right so which is where uh I don't know how to explain this but essentially in a way uh today for architects right you know typically enterprise architects you go vote go to a governance forum you would say that look you know fine I think you know all of these design parameters are met now it's approved to deploy right because once you even deploy it you know still you you can't say that look you know okay fine now given that I have designed this is how it'll operate right you would still need to look at that particular agentic system in production to keep track of it that whether it is actually operating ing within bounds or if it's not if it's drifting then again go back to it and you know make sure that you're making changes and tweaks to the parameter. So it's like almost a continuous architectural governance that you require in today's agentic AI world.

[00:25:21]

uh if I were to sum up uh if I were to sum up the uh architectural governance for agentic systems and how that's changing um a very good uh you know practical uh you know layman's terms that we could actually correlate to is um that enterprise architecture right it's kind of today moving from uh ctography to uh an air traffic control kind of system if you were to put it into an know air airlines you know kind of scenario Right. So where the enterprise architecture is moving from ctography to air traffic control where uh ctography you know typically used to be you know designed you know focused on designing actually maps drawing maps okay but now the architects right they the maps are just not relevant because they have to kind of more manage the flight paths in real time to know that every aircraft can replan its own you know flight change its own path you know mid-flight right the same way agents behaviors can change actually mid-flight right so you can't actually just release into production and then um you know rely that look you know they are going to actually behave exactly what they're expected. >> Um let's dive a bit deeper into the the the core architecture patterns which are emerging when we talk about um agentic systems right and in particular because we are we don't just don't talk about a single agent. We actually talk about multiple agents, right? And and those multi- aent systems require different um uh patterns, right? How you can architect those. All right? Um so there are different patterns which have been talked about and emerging, right? One is of uh an orchestrated pattern, right?

[00:26:56]

Where where one agent actually can orchestrate the workflow and there are um dedicated sub agents which have un specific responsibilities. Then there are um swarm patterns um right where where it's a swarm of agents which are actually doing their own unique um uh delivering their unique responsibilities right when you look at the patterns across the spectrum. How do you um uh what's the what's would be your guidance to uh u uh business business organizations as to how they should approach thinking these patterns? uh where federated patterns make more sense versus uh uh swarm of agents. >> Sure. Sure. Thanks. Yeah. Yeah. Yeah. S I agree with you. Look, I think there are some broad uh you know about uh two three patterns as you rightly mentioned you know when using um you know today's you know the new age kind of AI systems. Um uh the more simplistic one you know first one essentially is just using a single agent right so and plus you know some kind of tools with it. um it's it's kind of typically most applicable in you know a very singular task or a specific activity that we would want the agent to do right so and typically it works in activities that are uh maybe you know a bit more regulated environment and you know you need to know that look you know you're able to broadly control it properly in which case you just have a single agent so that you have got parameters that you can properly control and monitor um so for example you know you could have a finance agent that does, you know, just the invoice reconciliation by itself. Or you could have a legal documentation agent that's actually churning out uh you know legal contracts uh you know based on whatever context that it has been fed with and designed with right so so very you know single agent kind of structures right so that's where it's kind of typically used and most organizations you know not just regulated but many times when you're actually starting on an agent AI or an AI journey uh you know in in a lot of sense a lot of organizations know that's where they start here right having single agents different point solutions that they want to use. Uh the second is orchestrator or supervisor pattern where you've got an agent orchestrator. You have got like these you know multiple different agents right? So for example if say um there is a customer service uh orchestrator as an agentic orchestrator and it would have uh multiple agents as a under underneath it that it will have to uh orchestrate right things like uh first you know about perception right it will a single agent that's maybe you know hearing what the customer is saying or it's actually reading what the customer is actually writing and understanding what the context is the other agent could be based on that context you try and extract some information, you know, from the backend systems or from the memory that the uh chat or the agent has, right? Uh that could be the second agent. Third agent could be, you know, either do some kind of a uh action for what's actually required uh for that particular customer, right? Either uh extract some kind of a statement, give it to that, do some kind of refund, you know, provide some information or do some escalation and so on, right? So there could be multiple agents and there's an orchestrator sitting on the top. Now this is where you know typically uh today is what we call as agentic system or an agentic AI uh where you've got an agent orchestrator that's actually orchestrating between multiple agents right so and usually it's there you know used when you know you require compliance some bit of predictability uh because you're actually then decomposing your artificial intelligence into different kind of pockets so that you know which system where exactly is actually something that's happening right so so So that's a second orchestrator pattern.

[00:30:39]

Third is a decentralized swarm or you know more of you know agent pure collaboration where you don't require an orchestrator agent or an orchestrator to sit on top. Uh does individual agents have u you know instructions given to them that look you know when it's something as a task that they are not the best capable to handle they can actually look for any other alternative agents you know that's there in their ecosystem who can do that work and they will actually pass on the batn to that particular agent to do that work right so which is like very small amount peer collaboration that typically happens where you require very high speed um you know at scale you know fall tolerant kind of matters and where And so this third pattern is a bit more complex typically uh more used in things like you know examples like you know your real real time you know logistics or root optimizations or uh you know typically your market making you know from a uh automated or you know autonomic trading uh into the markets. Uh those kind of use cases are where you know decentralized you know swarm of peer collaboration can work in most scenarios. However, I think you know where the industry is moving towards most likely uh I say most likely because we don't know and no one is able to today predict that how the whole you know agentic and AI system and world is going to uh evolve into uh but most likely it's a today most more or less I think you know it's a hybrid of the second and the third pattern right where you've got majority of the systems that have following orchestrator pattern and some of them you know would potentially follow a pure collaboration pattern or a decentralized you know kind of a swarm pattern as well.

[00:32:15]

>> Yeah. And and as organizations start thinking about these multi-agent architectures or multi-agent ecosystems, uh there seems to be emergence of what what is being called as control plane. Uh and this is about uh making sure uh we have a right um governance the the observability the security standards etc. all applied consistently across all all the agents. Right. Can you unpack that that concept a bit as to what is the control layer about? What is its uh critical role in the multi- aent ecosystem? >> Sure. Sure. Sure. [gasps] Yeah. Look, I think uh the the the control plane is you know definitely one plane. Uh but I think along with that in today's agentic systems uh uh it's it's important to know that along with the control plane uh you know typically you can classify two other planes that go along with it, right? So one is your data and context plane and the last one essentially is your execution plane right. So uh so in your control plane you know typically you would you know set up your guardrails you know your uh identity and access management of that agent in terms of what all things would can the agent have access to um routing of you know which particular tasks is going to what particular agent you know so all of these typically you know would happen within the control pin and even you know things like you know your continuous evaluation observability or uh you know kill switches you know in terms of you know how to kind of uh make sure that you know this is a parameter when you have to really stop the agent completely. Right? So, so that's the control plane you know on the topmost layer. the the data and the context plane is also equally important where you know you would typically have your embeddings you know vector stores etc etc in terms of and that answers the question in terms of what does the agent know right and where where is it getting the context from and the third one essentially is more about the execution layer where uh the agents themselves right who are actually executing that particular tasks or the tools or APIs that they're invoking you know so that's the execution layer and that actually answers the questions of what's the agent doing or what does the agent do?

[00:34:20]

uh and and you're right I mean you know look across these you know the control um you know data context execution plane you know around some of the APIs tools layers and so on um you know multiple of them that are uh getting evolved you know even you know every day every hour that we actually speak so uh and typically you know the model context protocol as well right as we know it's kind of becoming a standard layer for different agents uh you know talking to each other or different to or agents talking to different tools um you know from a agent to platform kind of uh protocols to talk to them and uh now it's almost reached almost some 10,000 odd kind of MCP servers you know that are today publicly available and which is where uh just worth noting here because you know just because of the sheer scale of the MCP servers that have now come up an enterprise architect or an architect's job also is important to understand which of those are actually relevant and secure right that we need to actually make sure that we can actually make use of indoor enterprise.

[00:35:22]

>> Yeah. And um and probably an addition question and probably a bit too early to honest I would say uh but as organizations will start building this agentic ecosystem. Um there would be at some stage plethora of this this these agents and and you need to make sure that you are effectively controlling them and the the life cycle the version controlling of agents will start becoming important. Right? So uh how do you see the the need for agentic life cycle version management be becoming more important in the future and what are the ways organizations can control the life cycles for the agents. Yeah, look I think this um one of the parts you know from an overall uh the infrastructure layer of uh today's IT systems including the agentic systems as well right so that uh in a way has evolved and and it's continuously evolving you know for the purpose of how do we use our existing systems to manage um the new agentic systems that are actually getting built right so so which is where you know if you're looking at typically from an agent life cycle standpoint point uh it would in some shape or form you know at a principal level um you know still operate you know like u you know typically that you know we would have uh known you know some of the MLOps or you know the machine learning ops you know in terms of how that MLOps life cycle used to kind of operate or even now operates you know so it kind of follows a similar model right so so which is now I think nowadays you know you've got your agent life cycle you've got your MLOps you've got your LLM ops as well in terms of how do you manage the life cycle of LLM MS you know that are underlying the agents. Uh but but usually right I think they follow the similar life cycle stages right and you define exactly what exactly are the agents supposed to do. You build you know around with prompt engineering integrate the agent with tools. Uh the third you know would be around you know evaluating them you know in terms of uh behavioral test suites right because the testing of agents you know works differently you know how do you do red teaming uh what kind of hallucination benchmarks are actually accept acceptable you know within limits all of that's you know about evaluation deploy them you know with the circuit brokers and you know road or roll back kind of parts in terms of once you deploy an agent where do you actually make sure that look this is something that's not acceptable and you should you know kill the agent or stop the agent um observation, you know, observability of uh agents. Uh even more critical in terms of evaluations in production, whether the agent behavior is drifting, you know, just because of the way that the user is interacting with the agent or the way that look, you know, something else is changing as a parameter. >> Uh govern the agent, you know, from a periodic reertification. Very important to make sure that you know at a periodic level it is actually reertified, republished. And then uh the last you know from a life cycle perspective most critical stage around retirement because the agent has actually built context memory uh access uh you know etc etc. Um for from a retirement perspective it's important to make sure that the memory is completely cleaned uh credentials are you know completely revoked uh anything you know that's required from a uh auditability perspective in future uh is make sure that that's kind of archived and you know kept for records. So, so typically you know it would follow if you know I think you know similarly you know to an MLOps or an LLM ops you know kind of life cycle that an agent would follow. Let's move to the the next uh topic that I wanted to unpack with you which is around u data context and uh me the way memory needs to be managed within agentic systems and you you started describing that a bit but I want to unpack that further um because one of the biggest shifts when you talk about agentic architecture is about the context how they effectively manage context right uh that's a key differentiator um um so when you talk about uh the new data architecture that needs to support the management of memory, management of context etc. and how the data architecture has been done traditionally. What are the key key gaps you see how it is done today and what are the key capabilities or improvements are required um for building that data backbone to support um agent take uh the the essentially the outcomes that agents needs to deliver consistently.

[00:39:45]

>> Sure. Sure. Yeah. Yeah. Look, I think uh there are a few foundational elements, you know, that are uh critical, right? Because um as we move from uh you know, determin systems where you know the differentiator is actually your code um in an agentic system uh what's more uh differentiated is the context that you can actually provide to the agent, right? uh how much crisp is the context, how much accurate is the context, the breadth and you know depth of the context that you're able to provide the agent is what's going to determine and that context comes from data right because and that's going to determine you know how effectively the agent is going to perform and the output that the agent can provide and all of this you know has to be you know within the uh making sure that look you know u also about you know the performance you know in terms of what and how the agent can perform right so so uh so the key part from a foundational thing you know that has to be different is uh a shift from a data standpoint is um you know moving from uh code differentiation to the context differentiation that will be provided by the data systems right uh two different two organizations you know having the same foundational model uh the outcomes can be actually radically different based on the context that their agents can access right and that everything is about the data right so which is where the data architecture becomes even more fundamental uh to supporting an agent ing system you know and almost like a central you know to to the architectural nervous system you know of an organization right so which is where important note that look you know the competitive mode is no longer the application code but the contextual data layer so so so to answer I think your first question there in terms of what are some of the gaps uh there would typically expected to be huge gaps right I think your fundamental you know data architectures and the data structures uh data infrastructure that was previously there before the for of agentic AI systems that anyways has to be there right so because your your output of the agents is going to be only as good as the data how accurate is the data uh how you know how how clearly you're able to track the data lineage in case if there is a particular you know question that actually comes you know from an agent and so on right so that fundamental has to be there on top of your you know normal you know traditional you know data foundational elements that we have been used to uh for agentic systems now you know there are more um you know different data capabilities that have started to move in I don't know whether you know that's what you were alluding to but uh you know things like you know vector stores right uh which is important for uh making sure that you know the semantic context for what's actually required for an agent it's it's able to retrieve it through the vector stores that's how you build vector stores you store into the vector stores the information and context that you would that your agents would require The embedding strategy you know from a data perspective uh is again more core from an architectural perspective where a data architect or enterprise architect would need to think about how exactly my data is going to be chunked no and then fed to the agent so that uh making sure that we're able to use uh appropriately the context windows that are available for specific agents based on their capabilities. Right? So, so that embedding of the um you know context and the model selection etc all of that uh would be critical you know from an embedding strategy right so there's a second you know from a gap perspective that is required uh the third gap essentially is from a real-time context pipelines right so because the context of the organization is continuously changing right so the performance of the agent is only become going to be as real time and as accurate and relevant that we need um based on how much of the real-time data or whatever is happening within an enterprise is accessible to this agent right so which is where the things you know where your systems of engagement and systems of records are getting >> [snorts] >> uh refreshed or populated or changed how that change is going to get propagated to the AI agent is going to have you know your real-time context pipelines for the agents and that's going to make your agent behavior more um uh more uh valuable you know to the organization.

[00:44:03]

So so I would think a obviously the data foundational element uh b vector stores embedding strategies you know your real-time context pipelines you know are most critical. Yeah. And another important aspect when it comes to agentic systems um is um the way the memory is handled and managed, right? Um and um look, I I use open client at home for my personal use cases and uh anyone who has used those kind of agentic systems know how importantly those these systems tend to forget uh about you and uh the previous job activities you have done, right? Um but that is not acceptable in enterprise uh context use cases right so you you do need to think of um short-term memories how you manage the long-term memories etc. So maybe that's something where you can potentially provide a bit u uh uh insight to our audience as to how organization should think of different layers of memory memory layers um um and how they should architect architect that. >> Sure. Sure. um I'll try and brief I'll try to be brief and you know answer in a very crisp manner right so the the typical you know three layers you know if you were to look at from a memory perspective in context of agent AI systems right so once you're working memory that within a particular single task or a single chat that we are having um the memory that it you know is that the agent uses you know for the conversation and the context that's going on right uh so that working memory can be typically you know ephemeral right which is like you know short-lived lowrisk you So you don't need to actually do uh you know that's how it the nature of the memory needs to be right and accordingly you need to choose you know what kind of memory system that you choose for that. The second is essentially your episodic memory right so where within a particular session or related sessions to each other. So if you bundle some of the chats into a single particular project then you know for that particular project or an episode the agents would have all of that contextual memory right it would have in you know it'll store information around what what did the agents just do what is it that the user is preferring what exactly some of the data being actually fed by the user to this memory or to to the agent all of this episodic memory can typically be very PII heavy right so your user is going to feed in some of the things and so on so which is where from from a retention of the rules perspective from a data governance standpoint, it is very important to handle it carefully, right? So that because there is a lot of contextual personal information that is being passed on to the agent. The third essentially is where your semantic memory comes through, right? So where the agent gets the organizational knowledge, the learned patterns in terms of how it's expected to behave, that semantic memory is actually common across all the sessions for that for that particular agent, right? So it'll have carry all of it every time all the time it'll have it accessible to it right. So that particular semantic memory is very IP heavy. You will have to access control it very critically so that you know which agents are going to have access to it. Which are the users and what part of that information can an agent disclose to a user. All of those controls need to be put in place. Right?

[00:47:14]

>> So three types of memory short-term long-term working memory memory and semantic memory. >> Cool. Uh let's move to the next se segment which is around governance risk and autonomous systems. I think you have we discussed a little bit around architecture governance but maybe in this section we can expand that a bit to understand the implication from security governance perspective data governance perspective. U then u uh how to uplift the AI governance for in the agentic context. Yeah, you know, in the most simplistic manner, right? Um, your uh the governing systems, you know, for today's uh from an particularly from an architectural governance, if you are looking at it, >> um you know, uh there are maybe no three, you know, specific u concrete mechanisms that are in place, right? Because uh you have to consider the main principle change is that the governance um is shifting from uh trying to gate the change meaning you know be a gatekeeper of whether what change is going to into production or not to uh actually trying to govern that once it is in production what is the limits of that behavior that is acceptable right so that's what the main governance is shifting right so which is where uh the three concrete mechanisms that you know from a governance perspective that are important. Uh one is policy as code right as you rightly mentioned all of the you know the security controls there the guardrails uh access that um is allowed for the agent and who has access to the agent right so as well as you know data and so on right so all of that you know has to be codified you know from a policy perspective uh you know and make sure that look you know that helps to kind of keep the agentic system behavior within the expected boundaries. The second is continuous evaluation. I think we spoke about earlier that um not just during the pre-eployment testing part of it but you know even in production you know evaluation harnesses actually become uh very important because the evaluation is continuous you know continuously you need to keep on evaluating whether the agent is behaving within the specified limits or not right before uh I mean during testing and during production both times. The third is essentially how to make sure that look you know which are the relevant cases where you need to put a human in the loop for sure right so you'll have to able to calibrate the risk uh and based on that risk profile uh you determine what are the kind of high-risk actions right things like you know financial transactions or any kind of customerf facing decisions and so on that require human approval and then you know the agent at that points in times the agent needs to hand over the control to the human right so that the lowrisk loops they can actually run autonomously ly and continuously but the high-risisk ones essentially definitely you need to put a human in the D right so uh to sum up three main controls right from a governance systems policy as code continuous evaluation and human in the loop checks yeah let's move to uh the next uh segment I wanted to have discussion with you on which is around um how aentic systems need to coexist with the legacy the traditional platforms etc because that is not going away So soon right uh you will always have uh ERP systems and CRM and many other systems right which so as you start building the agentic ecosystem um how do you see those integrating with these legacy platforms etc what are the key standards they can follow um uh to drive the the use cases agentic use cases true yeah it's a very good point and uh you know to be honest I think you know this whole space is evolving right and will continue to evolve. Okay.

[00:51:04]

>> Yeah. >> Uh currently the way that we actually know are seeing you know obviously I think you've got uh AI agents AI uh features that are natively available within the uh existing core platforms or legacy platforms by themselves. Right. So everyone has embedded AI >> in their platforms right so that's a one centralized platform where you know everything is there and you know could use that. The other essentially along with that you still require some more you know specific uh agentic systems that are there outside of the core platform uh to make sure that look you know there are other things that would require to work along with the core platform uh to be able to orchestrate all of those right so which is where you require a federated kind of an agent ecosystem right so this is where you know typically um you know some of the from the maturity perspective of agentic ecosystems you know that's where typically people should or you know aspire to be or where mature organizations are they will end up into right so so from a centralized platform standpoint you know um where you know you'll have uh you know a single kind of a platform you know doing all of the AI versus a federated agentic ecosystem is you know where you will have a mix of uh native AI agents working alongside with custom AI agents um you know and then working all of them together.

[00:52:21]

>> Yeah. Okay. Um let's uh also now as we move towards uh wrapping up this episode let's uh talk about the f the path forward and the how the future is looking like right um for many organizations um they have of course most of the organizations by now have worked played with a gen AI most of the organizations in effort by effort 26 have at least started experimenting with agent AI right um but It's also the reality that most of the organizations have not started productionizing those agentic care system well enough right they're at the stage where they want to they are still exploring how to now transition from pilot to now production and how the benefits right so the how would you pro what kind of advice you will provide the these organizations who are just want to transition but they're still kind of struggling or exploring what is the best way to now um start getting benefits of agent IC systems.

[00:53:22]

>> Sure. [snorts] Sure. Yeah. Look, I think you're right and again right the things are changing. It's evolving. Organizations are learning. Um we have now definitely reached however a stage where uh everyone right all almost all organizations have realized that the uh this capability around AI is fundamentally changing and it is here to stay and it is really giving um you know something truly you know valuable uh that people cannot ignore that organizations cannot ignore. So uh but there is a journey to it because as we discussed you know about this deterministic probabilistic part and so on right fundamentally a lot of things are changing right so there is a change part to it um most organizations typically you know they would start with um you know usually take one particular bounded context or one particular use case you know which might be most relevant that will provide maximum you know uh uh value from a business perspective either it could be a you know customer service desk internal IT help desk or you documentation you know kind of agent uh or typically you know chat bots and so on right that I think you know where previously also chat bots are there they now are actually becoming more intelligent with the underlying you know newer more capable LLMs you know which are uh very very close to you know how humans interact right so that's a starting point usually right so and that's what typically as a first starting step is called as an augmented kind of a journey right so you augment whatever currently the humans are doing with AI agents the second essentially is you learn from that you know and then you know start to actually provide semi-autonomy to some of those agents right so you have model gateways evaluations you know you build some scaffolding systems to make sure that the uh as we expand into the agentic systems uh there are proper guardrails around it security around it access around it because observability around it because all of these u uh have a different context when you look at it from an agentic system perspective in terms of how to manage agentic AI Right.

[00:55:18]

So that's the second step to actually provide semi-autonomy. And the last step essentially is really go towards agentic you know AI autonomous uh ecosystems. Right. So where um uh you know you you build truly agentic uh you know workflows or agentic processes end to end right? So where uh that's the maximum benefit that agentic systems can provide given the uh radical you know capability that uh today uh AI agents are actually offering or AI is offering right so where you take complete you know a critical business process or you know some of the critical business process and start to automate them end to end reimagining them end to end and make sure that look you know you only have human in the loop where required because that's what's going to give them the maximum business value maximum agility uh in terms of what and how a matured agentic AI ecosystem needs to work. Okay. So that's the that's where you know organizations of today moving from um you know not having anything to do with agent to moving to a very mature autonomy or autonomous you know kind of agent AI operations. There are however I think you know some organizations I'm assuming you know they will also start to be more kind of AI native which are organizations newer uh that are getting built from scratch you know completely you know AI native models you know that um are people are building businesses around them so so that would be 100% AI native kind of enterprise systems or you know even businesses that are actually getting churned up [snorts] >> yeah yeah that that's that's well said because yeah I think there will be a spectrum as you said uh there are a native startups then there are digital businesses which will be potentially more advanced on the AI journey they will have more risk greater risk appetite for a great experimentation etc whereas uh there are more um companies which are in the more regulated traditional sectors they might be at a different uh stage when in the AI journey they the pace of um getting on board on the AI uh agentic AI uh is is going to be different for different industries as well. So so wrapping up and um just a final question circling back to our original topic about enterprise architecture and architecture teams. So what what does it mean for architecture teams for the architects for the future how they should start thinking to now think um um designing agentic systems?

[00:57:52]

>> Yeah. Yeah. Yeah, look I think um has uh as I mentioned and I don't know I think you know I've had this multiple some of conversations you know with so many architects you know around the same context right and it is still um uh a newer concept that you know people are or architects are trying to grasp okay because I don't think you know they still understand in terms of the fundamental change uh that architects behaviors and expectations in terms of what's expected out of enterprise architects right because they need to really start thinking from uh thinking away from documenting systems to actually operating them right because the architectural governance needs to go into production and not just you know it doesn't end you know till the time that the system is designed so that someone will pick it up build a system and it's done no it's it's not possible right so because the agent is is a continuous live ecosystem so the architects as well need to uh think uh of the AI agentic systems even into production in terms of what's happening with them and continuously know be on track you know with it right so that's a different context altogether um continuous evaluation of agentic systems is critical I think typically enterprise architects you used to have you know annual review cycles um you know traditional kind of governance structures and so on it has to be continuous you know because the agentic ecosystem as I mentioned is a live ecosystem it'll keep changing right um interestingly I think you know for architects there are newer roles as well you know that have started to come up right So um uh agent architects you know AI platform engineers um you know AI evaluation leads you know who who understand that agentic system end to end from an evaluation perspective all of these systems right so where typically uh they're not just normal you know engineering kind of functions in my mind uh but more uh you know senior engineering or you know principal engineering or in a way you know enterprise architecture kind of skill sets to be able to have that maturity to make sure that you're evaluating a production system you know in a proper manner right so um I think yeah so that's what I would think one is change of behavior that enterprise architects need to do uh that the governance is no longer about um uh designing the systems it is uh actually designing the systems that are going to design you know further behavior >> yeah and with that Gautam thank you thanks for your time I think this was incredibly insightful discussion I think there will be lot many takeaways for our audience So uh thanks again and it was great hosting you.

[01:00:24]

>> Sure. Thank you. Thank you so much Somra. You know likewise you know it was a a very uh educative you know kind of conversation. Uh had a you know enjoyable conversation. Thank you so much for inviting me for this session. Thank you. >> If you found this discussion valuable please follow and subscribe to Enterprise Tech Talk and thanks for listening. I look forward to seeing you in the next