From Governance Theatre to Continuous Assurance: Why Transformation Risks Stay Hidden

Governance Theatre to Continuous Assurance

Speaker 1

[00.00.16]

Hello and welcome to the Enterprise Data Podcast. I'm your host Saumitra Kalikar. Large transformation programs rarely fail without warnings, and these warnings are generally visible through delivery delays, undesired dependencies, or the risks which remain open month on month. And yet, the executive dashboards may continue to show the status green till the time problems becomes too significant to to be ignored. So why does this happen? Why are the the decision makers continue to get the wrong information and sometimes outdated information about the status of their programs? And now with AI into the mix are being used more and more into program delivery and governance. Do we expect AI to provide more structure and objective assessment of the risk? Or will AI just automate the existing weaknesses into the programs that exist in the current governance processes? To explore all these questions. I'm joined today by Peter Wardle. Peter is the co-founder of Intelligent Assessments. He has 25 years of experience as management consultant, delivering complex, uh, transformation programs across multiple industries.

Speaker 2

[00.01.35]

He his work focuses on program assurance, governance and what he describes as delivery intelligence. Something I'm keen to explore with him today. Peter, welcome to the podcast. Great to have you here.

Speaker 1

[00.01.49]

Yeah, Mary, thanks very much.

Speaker 2

[00.01.53]

Um, I wanted to start with a very basic question. Uh, that is, um, why seemingly green transformation programs still fail. Because when you think of these programs, the transformation programs generally have very robust governance controls, uh, governance committees, forums and risk management practices. Right. And yet, when you look at the executive dashboards, they continue to show that the programs are tracking. Well, just before all of a sudden you realize that project costs, program costs are exploding. Um, then our, uh, the delivery timelines are shifting. Um, and program is struggling to meet, uh, the intended, uh, objectives and outcomes. Right. So I want you to get your thoughts based on your experience as to why this continues to happen, even when these programs are generally good governance mechanisms in place.

Speaker 1

[00.02.50]

Yeah, I mean, we could probably use the full hour to just discuss that one question, to be honest with you. I mean, one of the main things that I see is, is, is that reporting systems are generally set up to manage how deliverables are being met. Whereas the real question that should be asking is, or more of the focus, it should be on whether the outcomes are going to be achieved. And and I think that that is one of the kind of key things. So I've like I've seen programs with great raid logs, plans, great governance, but the, the underlying position just just gets hidden because to focus on on delivering stuff. And as a delivery person, I heart myself. I'm really engaged about kind of delivering, but sometimes you just missed the focus of it in terms of what is the outcome that you're trying to get from from this. There's also the human element, um, just, you know, people just don't like to give bad news. So they often soften it up and then soften it up and soften it up. So, so by the time you get to a kind of an exit level, uh, you know, the real news is just hidden away. Yeah. Yeah. And I that's true. And I. Because when you see the programs are struggling and they fail. Um, and when you talk to the people on the ground, you get the sense that there were already signs visible. That is going to happen. Right. But uh, until, uh, the, the the decision makers do not get this information on time. Right. Uh, so I want you to understand even the why that happens. And you potentially indicated that people generally so potentially is a behavior issue, as is that people do not want to communicate the, the

Speaker 2

[00.04.43]

the this information upfront very clearly to decision makers provide this keeps happening again and again. Is it more of a B side behavioral issue. Is there any other issues that you you see

Speaker 1

[00.04.56]

I see principally as, as, as a behavioral issue and also, uh, understanding whether uh, news actually kind of travels because you're right. There's there's few instances where I've seen where, you know, a program was suddenly gone from kind of green to red and failure without anybody knowing about it. And people, you know, on the ground have known about it, the, the, the many kind of weeks and months, but the information doesn't travel at a really good example, uh, a few years ago where I was doing a review on a, on a, on a massive, uh, program, and every week they had a stand up sessions. This this was kind of pre-COVID, um, and the big kind of, uh, ceremonies, were they all the various kind of work streams with right up on, on, on whiteboards, what the progress was, what the, uh, the maintenance issues was. But. Ten minutes before. And they exactly used to kind of come in and they used to walk around this, this, this big kind of whiteboards. But what they didn't see is ten minutes before each of the, um, each of these kind of weekly reviews, people would go round and just wipe off anything incriminating evidence. So you'd have work stream, uh, manager one who would who would get incriminated, uh, or kind of called out on worksheet uh, two and he'd rub it out and, and he kind of, you know, not that particular example. People were just living from, from one meeting to the next and surviving from one meeting to the next. And, you know, and the culture of that, that piece of work was, was that they were just every time they, they exposed a risk or an issue, uh, that they were genuinely kind of reach out for help. They got hit over the head. So there's this, there's this, you know, these these big transmission programs need the right environment. Uh, and when the went out for help, they need to kind of have the confidence that they are going to actually get the help rather than this over the head and told to do better.

Speaker 2

[00.07.13]

Yeah. Yeah. Right. True. Yeah. And of course, as you said, no one wants to be that bad guy. Uh, but who who would call out? Uh, the real issues. Right? Yeah, exactly. Because, uh, and especially for people who have, uh, uh, in organizations and wanting to make a career out of it, then, you know, sometimes they build things down because they don't want to. Yeah, yeah, yeah. It also requires a very open and transparent culture and the leadership role in the. Um, one thing I want you to unpack before we get into the, the the intelligence delivery, uh, sorry, delivery intelligence framework that that, uh, you you, uh, uh, propose, um, I want you to get your thoughts on, uh, the the gap. Typical gap between the strategy and the execution. Very high level. Right. And the reason I want to understand is, uh, most of the organizations generally have strategies in place. Then there are some some sort of investment planning mechanism, some sort of portfolio management mechanism. Right. But, um, uh, having strategy is one thing. But then when the strategy hits the ground and faces, uh, the delivery environment, that's where the problem starts. Right. I want you to get your thoughts and based on your experience again, as to how how often do you see that? Yes, there is a good strategy defined, but when it comes to the program definition, the the overall scope, the object, the clear objectives, etc. are not well defined. And that, that that becomes the starting point for potentially, uh, why these programs eventually struggle in the long run.

Speaker 1

[00.09.09]

Yeah. So so so it kind of goes back to the point of the, uh, early in terms of you lose, um, every time you go from kind of, uh, strategy to business case to defining the program to requirements design, delivery, you know, you lose things in, in, in translation. So you may have started off with, with with a really good strategy. And that's exactly why you're doing it and what the benefits were. But as you go down, uh, as we move in the kind of lifecycle assumptions get made and, uh, they don't usually get kind of follow through. So you could get into a position where you got a whole set of deliverables which actually were meeting that kind of strategy. And, you know, when you run these, uh, big pieces of work and you've, you've you've got, you know, the governance around it and you get kind of sign off of things sometimes it just just gets missed. Uh, not not because anybody intentionally wants it, but it just kind of gets diluted. I had a really great example, uh, a few years ago where I took over a program and it had just come out of the Big assurance review. So one of the big, uh, global kind of consultancies have come in. They've done a review that looked at the business case, that looked at the raid log. And, uh, their professional opinion was, was that it was a kind of an A-minus. It was it was one off, uh, being, you know, complete, perfect. I took one look at the and everybody was really pleased about that. And everybody was patting each other on the back. Uh, I took one business case, and I said, well, that's not true. This this is this is invalid. This is an assumption that that that that I personally don't, don't, don't agree with. And there was just so many flaws in it, uh, that we had to completely reshape it. But if I hadn't had that kind of brutal kind of review of it and looked at it from from a kind of strategic point of view, that program would have just been delivered and it wouldn't have met the, the, uh, the, uh, the strategy or the outcomes that, that, that that was the original intent.

Speaker 2

[00.11.30]

Yeah, yeah, yeah. There are so many examples like that across the industry, to be honest. Um, yeah. I want you to now, uh, shift or focus more on the the role of governance in, in, in, in, uh, delivering these programs. And I want you to get your thoughts on, uh, what are the constraints emerging in the traditional governance mechanisms? Um, and when I say traditional governance mechanisms, of course, you think of multiple governance gates, big steering committees or transformation programs, right? Where the the risks and all the progress reports are presented. Uh, right. But the problem with this is that one thing, by the time something comes to that governance forum to present it right, it is generally a bit late in the sense that, um, that that information that is coming to Governance Forum could be a bit stale. Uh, and particularly in the today's environment dynamics where, um, the, the businesses want to be much more agile and most of the programs need to go move faster, right? Organizations want to move faster. Um, this kind of governance kit may seem to be inadequate. Um, now what do you get your thoughts on this? What are if in today's, uh, today's environment, it is a market. If a organization decides to have this kind of traditional governance mechanism or social program, what are the constraints and risks you see?

Speaker 1

[00.13.04]

Okay. I mean, again, going back to my experience, um, I've seen things as bad as, um. Yeah. Go out to do kind of requirements takes two years, and then you come back and all the requirements are kind of invalid. So yeah, you're right. In this kind of fast moving world, we need to kind of rethink it in terms of how we do the governance. And also if if you think about it, the projects we programs usually usually just get a kind of a three color rag rating. When you've got something that's really complex, you know, distilling it down to red, amber and green it, everything usually gets put into it to amber because it's it's a little bit of a cop out. I've done plenty of, uh, organization or portfolio reviews and everything comes out there and it's just like, how do you, how did how do you kind of move on from there? How do you understand what is really hurting, uh, each each project, how do you understand what what is really important. And it's just hard. So, so we just need to think a little bit differently in terms of how we how we assure these pieces of work. No, not not for assurance in the, in the traditional sense because I talk about assurance and people just think you ought it. You know, and that's why I've changed my vocabulary from, uh, continuous assurance into delivery intelligence. Because you're going to make the projects work. Then you need to give give the keys to the project managers. That is the most effective way of doing it. So you need to show them how to kind of unlock things so that they can, uh, have the, uh, the, the knowledge and, and they have the information. Um, and one of the really kind of simple, uh, techniques I, I kind of promoted is just asking, uh, people within the, in a project or a program, uh, how confident are they in the delivery and how confident are they in terms of, of of the status that that is being projected? Because I've, I've seen on plenty of occasions where a project manager is, is reporting things as it was green and, uh, extreme leads are saying, I don't think it's green. And then the people on the ground and even the end users, they say, well, it's definitely a red. We fundamentally disagree with that. And that new travel to to an exact and sometimes it doesn't even travel to. Mhm.

Speaker 2

[00.15.45]

Mhm. Yeah. And I want to unpack. Definitely on the delivery intelligence framework with you. Um but before we go there I want you to couple a couple of questions. Um specific questions. Um, one is um, there is usually this pain area that, um, from a project manager perspective, um, that the information about the progress and the status of the program generally is spread across multiple tools, I would say, or multiple PowerPoints or spreadsheets. Uh, same information is captured in different tools, Jira and so on and so forth. Right. Um, and that what that fragmented information potentially affects the quality of the, the how you report on the translation program and how, I mean, one. My question to you is, what is the best way to get over this? Or is this kind of issue and get the most out of this, this fragmented, um, toolsets to report to the, uh, to the governance forums? I

Speaker 1

[00.16.49]

mean, I don't know if it's too, uh, too early to start talking about AI, but but for me, this is this is the key thing that AI can do, because you're all right? There's there's lots of fragmented data that is available to projects and programs, but usually or traditionally. It's been trying to kind of pull all these things together. Uh, whereas now we've got AI which, you know, which which can look across a whole raft of, of, of data and look at trends. Look at where things aren't, uh, are misaligned. So JIRA could be saying that, uh, we're running behind from, from a test perspective. The test manager could produce something that, that, that says something a little bit differently. And, and, you know, it's a real opportunity to use AI for for what it is good at. You know, uh, people kind of say, oh, uh, is AI going to replace project managers? But, uh, there's definitely a place for, for area of this. And exactly what you just described in terms of fragmented data. Lots, lots of data. It enables us to be able to pull it all together, make sense of it, and highlight things that are going wrong.

Speaker 2

[00.18.15]

Yeah. Yeah. Um, so the thing I wanted to unpack with you was around. Um, what are the important metrics, in your view? Uh, which that should be presented and discussed in at a leadership level about the transformation program progress. Because when you present some, uh, typically when the present, when the progress or the transformation or any program is presented, it is in terms of, um, the, the, the spend. And it is typically a spend analysis. Uh, the progressed was uh, the whatever the deadlines which we have um, uh, progress towards particular milestone etcetera. All these are important metrics. Um, uh, and but these are, these are lagging indicators. I would, I would say, um, to really get to understand how the program is performing and to be able to execute, uh, to help executives make very early intervention and judgment on, on, um, on the progress and, uh, know the program. What other important metrics you think are important to you? Are they the iron triangle of of of of budget? Time. And so cost time and air and quality. Is is is is not the best way to kind of look at it. So I look at things from, uh, if I take 20 to step back and look at things like, uh, our decisions being made, uh, on time, what, uh, how many assumptions are have not been invalidated? Are dependencies actually, uh, confirmed? Uh, what are the stakeholders? Uh, not only thinking, but are they turning up meetings? I've seen a lot of times where, uh, senior stakeholders, uh, they lose confidence and they just don't turn up to meetings anymore. Um, they don't want to be associated with something that, that that that's failing. Also trying to understand things like, uh, what, uh, do the delivery teams think versus what the project manager thinks and, and trying to tease those out in, in, in, in an effective, uh, an effective way, which doesn't cause a project manager to be embarrassed. Also looking at things like what do and customers feel about this. So and if I kind of look back in, in my kind of early career, I wish to ask some of these questions a lot earlier in terms of, okay, we're delivering a big change here. Uh, we've done shows and tells, but are we truly listened to to our kind of end customers and are we doing something about it? Also looking at things like, you know, if you've got kind of temporary workarounds, uh, trying to understand, uh, are we just kind of fudging things to make sure that we appear to be able to be on track? Um, compression? And a

Speaker 1

[00.21.23]

lot of times, uh, I see, uh, the, the person at the end of the of, of, of the chain just gets compressed and compressed, and I think it's going to be magic that they'll be able to kind of, you know, reduce the test time by, by 50% and that those guys are kind of working all weekends and every kind of hour. So yeah, it's it's it's those softer sort of things. You could also look at how things like, uh, churn within the, uh, the team and how, how often are people, uh, do you have a lot of, of kind of turnover within, within the team? Do you have certain individuals that are just working, you know, uh, well, uh, beyond any, any kind of, uh, uh, normal kind of, uh, working sounds and an overall morale. So morale is, is usually one of the, the early indicators of, of, of is it working there.

Speaker 2

[00.22.25]

Right. That's a really interesting point there. Um, so at this point, I really want to get your now, uh, thoughts on the continuation assurance before we get into, um, the, uh, intelligent delivery intelligence as well, because so far what we discuss is the traditional governance and how it should. What are the key successes, etc. and we discuss it and hopefully agree that that is not adequate for today's, uh, fast paced world and, uh, organizations. Can you unpack now, what is continuous assurance is, well, how it is different to the traditional governance mechanism.

Speaker 1

[00.23.09]

Yeah. So so, um, it's probably worth just talking about, you know, the first time I kind of really kind of considered, uh, continuous issues. So I remember, uh, a couple of decades ago now, I took a work and, um, before I actually started, uh, the the sponsor sent me a whole load of status reports, grade logs, and, and it went away, and I did a bit of a desktop analysis. And I came up with a hypothesis in terms of where I thought things were going wrong. And then when I formally kind of joined the program, I started speaking to people and understanding what was was, was, was, was that the actuality rather than what was it? Uh, was was documented. And I found that my initial, uh, thesis was. Not completely wrong, but it was largely wrong. And and that kind of thinking in terms of if you just look at things from a kind of a traditional children's way where you've got usually PMO who give a kind of oversight and challenge, if it's a complex kind of piece of work or it warrants a completely independent, um, uh, review, then you do that. But but what I see is, is, is, is is that the if you go to that, that, that kind of model, uh, your, your PMO review is, is usually on a monthly basis, which just reviews, uh, the articles. The third party assurance is usually on an annual basis. And, and it's almost like what's the point of that? You know, they, they come in they, they, uh, they carry out probably the same activities as the PMO has done. Uh, but but it costs you a lot more. And then they highlight things you already know, know and then delete. And then what happens is the next time it comes around, in six months or a year later, everybody just download all the actions from last time and you repeat the process. And having these snapshot views of governance is is completely the wrong way to do it. Uh, the I've talked before about giving uh, project managers of keys project managers needs to own assurance. And and when I've talked to project manager about assurance they automatically think of of audit and they don't listen. So that's why I've kind of reframed it and talked about delivering intelligence. Because what we're trying to do is, is collectively, uh, not not assure the program as an audit. We're trying to give people, uh, confidence that we're going to deliver what we initially said we were going to do in the strategy. And that I'm going to promote this is let's give the project managers the the tools that they need so that they can do the best job that they can do.

Speaker 2

[00.26.26]

Yeah. Yeah. And what specifically changes in the in the continuous assurance model? Uh, where you and I. Uh, so, as you said, in the traditional ways of governance, yes, there is a monthly or bi monthly, uh, governance forum and there is a lot of preparation work goes by project management, PMO side into to capture the information from different uh, uh, leads on working on the program, uh, synthesize that information and, and share with everyone right. And present that. So that is what, what I'm understanding is from under continuous assurance. That is not the way you should be going about um, you want more on time in, uh, understanding an intersection of what is happening on the ground about the project. Right. So programs, how do you go around on a day to day basis to capture tap into that kind of insights? Yeah, yeah. So so so what I'm sorry, is, um, is is asking people what they think. Uh, so we do a lot of pulse checks, so and do the at various levels. So go out to work, stream leads and get them to anonymously tell you what what what. They go out to execs and again tell them ask them, uh, how things are being run and whether to support a project since so many times we're going to the team steering groups and everybody puts on on the face and says, it's really the support things and the come out kind of conversations. I don't support that, don't really agree. So I want to bring some of these things out in a, in a in a way that doesn't cause embarrassment, but it turns out the truth. Also promote is, uh, rather than having an audit done on, on a project here or here in and and going through a, uh, a review, give the that those those questions to project managers and I've similar on on quite a few occasions where you've got a really talented project manager or, or a program manager, the up and coming and they've given something that that is, is beyond the capability or they just don't have that, that, that experience and everybody's busy. Uh, they don't get the necessary, uh, support or controls that, that, that they need. So, so, so what uh, I do is, is, is, is get them to go through a kind of a self-assessment that allows them in a, a really kind of controlled environment to, to tease out exactly what is going wrong. And then the advice in terms of how they can do something about it. I kind of recall that my, my, my really early career where I was delivering was I was I came into project management almost like as an accidental project project manager. So I didn't have any any kind of formal kind of qualifications in project manager. At the start of the experience, I

Speaker 1

[00.29.45]

delivered a project and hit all the kind of timescales it was in cost. Um, and we delivered the, uh, a new system. But every time you kind of entered information in it, it's 30s to go to the next screen. And it was completely new. And that's because I didn't know about, uh, I didn't know that we had to kind of specify, uh, response times. And, and the people who I work with, they, they were, uh, they equally weren't as experienced. And, you know, that was that that project was seen as a failure because we didn't understand what the objectives of, of the, uh, of the piece. So. Yeah. Yeah. And as you said, the stakeholders from, uh, project managers or PM wanted to work with to extract information. They are fairly busy. The another scenario I would potentially paint is, um, uh, more in in today's organizations, there is a shift towards having more autonomy towards most of these business teams and delivery teams. Um, and they do not want to be governed, so to speak. Uh. Right. They do not want to be told and, uh, and ask too much information about what they are doing. Right. In these scenarios, um,

Speaker 2

[00.31.13]

what are the kind of tips and tricks you can share, potentially for, uh, PMO to get just enough minimum information, but the right information that is required, uh, that can really help the the decision maker.

Speaker 1

[00.31.30]

I think he goes, well, what was I was just discussing because you, a project manager, needs to demonstrate that they've got things under control. And, uh, in order to, to get everybody's confidence. But he's making sure that they've got that kind of right information If they can demonstrate that they truly understand what the outcomes of the project are and that they are delivering to those outcomes, and they've got the cost and time timelines under control then, and they've got the mechanisms to to ask for help if, if they do need help and they've got the as we discussed at the start, they've got that kind of cultural kind of aspect all sorted out. Those are the key things. But but I keep coming back to you asking people in terms of the level of confidence. So level of confidence of delivery and level of confidence in terms of where the benefits are going to be achieved. And if you've got everybody saying exactly the same they're great. But usually tends to happens is, is that you've got a cohort of people who thinks things are going great, and 1 or 2 people who who know that things aren't going great, and it's teasing those things out and do it in a in a way that doesn't cause friction and allows people to kind of recalibrate and get, get, get back on track that, that, that, that that is key to that.

Speaker 2

[00.33.02]

Yeah, yeah. Um, so at this point, I want to understand the your framing of delivery intelligence. Um, um, my understanding is that I want to understand is it more rebranding of continuous assurance or is it is there more to that than just confusion? Is

Speaker 1

[00.33.23]

it there is a branding of continuous assurance. But I look at things in terms of or I've traditionally looked at, uh, things in terms of those three lines of defense. Um, so first line is, is is with uh, project managers, second lines, usually with PMOs. Third line is with a kind of independent kind of audit, uh, party. But if we've been honest about it, the third line of defense comes in too late or it just gives a snapshot. And, uh, the second line of defense and, and usually replicates what is in the second line of defense. The second line of defense happens. But it is um, is it happens usually too late and usually usually have kind of stage gate reviews. Um, but most effective is is is is out. The first line of defense with the project managers. Now I've had to reframe this because if if I'm being if I'm being truthful to myself, uh, I've, uh, in my early career didn't ever have a first line of defense assurance kind of plan. Um, if I go out and ask project managers and program managers now, if they have gotten an assurance plan, they usually point to second line of defense. So what I'm saying is, you know, to, to to to really make sure that that we get from a project and program exactly what we first, uh, what we're asking for. Then you've got to kind of reframe it and you've got to reframe it as, as delivery intelligence, something that actually helps the project manager deliver and helps them give confidence to stakeholders that are going to get exactly what they want. So, so that that's my kind of campaign to, to to kind of reframe things, but to, to genuinely give project managers something that they can use, uh, to, to make sure that the projects are successful.

Speaker 2

[00.35.34]

Yeah. And the second line of defense is there where PMO is the important role.

Speaker 1

[00.35.40]

Usually in all organizations are different in all projects and not all different. Yeah. You know, in in a simple kind of project, uh, organization, PMO, in a program program, you may have a PMO within the program. So that might be your second line of defense, but it's somebody given a kind of a critical view and hopefully help you as well.

Speaker 2

[00.36.06]

But, uh, the program. Yeah. Yeah. Because, um, you're right. Uh, if there is a, let's say, two, three year big transformation program that's happening, uh, to execute towards a strategy, you would invariably see multiple initiatives or projects, whatever you call all those. Right. Within that whole transformation program. Right. Um, and a project manager, uh, may have maybe across 1 or 2, uh, um, projects. You may not have the full understanding of the full transformation, uh, program, uh, from coverage perspective. So some, uh, from one project manager perspective, 1 or 2 initiatives might be progressing well from some other project manager perspective. His initiatives may not be progressing that well. There could be some issues there, and the the interdependency between these initiatives also may not be fully understood. Right. And that's what I am understanding is that's where the second line of defense becomes important, because they can look at the portfolio as a whole, uh, the whole transformation program, they can find the patterns. And what are the systemic issues arising across all the initiatives within that program? Um, is that how you position that the delivery intelligence? Uh, really?

Speaker 1

[00.37.28]

Yeah. So so so so so so that's a good articulation of it in its entirety. But but what I'm trying to and promote is, is giving delivery teams, uh, the, the intelligence so that they can do something about it. So is given the case, given the keys to the project managers so that they can actually do with it, rather than waiting a month or two months or to the next stage, get review and then and then coming out because you've lost, you know, a lot of time there. So it is trying to. Get project managers to think a little bit differently and get information, uh, real kind of tangible information, uh, as early as they can and keep them repeating it in that, that, that information as soon as they can. Because what the, the second line of defense should be. And the third line should be, uh, primarily understanding what that data tells you. So if you've got a, a, an organization with several portfolios and all the projects have you using this kind of consistent kind of mechanism for delivery intelligence, they can quite easily, uh, and understand where things are commonly kind of going wrong in that organization. So there may be a problem with commercial teams and it might be an underlying issue, you know, but but who's actually kind of bringing that to the surface in an organization. You might have morale issues because, you know, a test manager or kind of a function might, might be kind of, um, uh, behaving poorly or may have uh, uh, poor kind of, uh, controls. So it's all trying to kind of tease things out. And in an age where we're using kind of data, more and more is trying to do it in a really kind of sensible place rather than just doing the lazy. Oh, well, uh, how many benefits are in this one? We should be better than that.

Speaker 2

[00.39.37]

Yeah, yeah, I think, uh, I've seen, uh, of course you would have seen, uh, as well that what happens is for transformation programs, you there are certain initiatives or work items. I would say they, uh, they progress, but then we realize that they're no longer relevant to our core objectives. Um, but the delivery perspective, these, these projects still continue where people do not necessarily cause they will stop them. Um, when they when they identify, they are not relevant. Um, how can delivering intelligence help in terms of really identifying, surfacing these kind of initiatives, uh, within the transformation program? Um, so that early on so that, uh, we can, uh, that the decision makers can make decisions better for them and redirect their capacity and resources to more important projects.

Speaker 1

[00.40.37]

Yeah. And that goes back to the those. Those those kind of key questions in terms of do you believe in the delivery plan. Do you believe that that the benefits are going to be achieved. If if you've got people who or if you've got the data that that that says, we don't believe that this project is going to deliver benefits. And you and and it's it's flashing red. Um, and then, then then it needs to be reviewed. It needs to be reviewed by the preacher because he, he, he or she has got the, uh, the right information and they should be brave enough to, to make that decision, uh, backed up by evidence and then add a kind of a second and third line. Uh, they should also be looking at that data and kind of challenging and asking, why are we working on that? Because all organizations are busy. They usually, uh, run too many projects or they're there are over capacity. And if we can free up resources on things that are actually adding value. Then, then, then that thing because it it not only kind of waste time. Waste time doing the things that aren't going to deliver a value, but it actually promotes the success of the things that we do need to care about work.

Speaker 2

[00.42.03]

Oh, yeah. Look. Um. Oh, I now I want to get your focus back on, uh, AI use of AI in project management and governance. Um, and we briefly discuss about this. And use of AI can vary a lot. Right? We you you can use it just to summarize certain certain reports or sort of content from a project reporting perspective all the way. There could be a complex use case where you can actually effectively use AI to predict some some the the success of failure or the future trajectory about the programs as well. In your experience, what kind of use cases you are seeing emerging more and more in, in, in this area project governance and delivery and and what are other potential opportunities are there? Yeah. I mean, I kind of took a few of these areas, uh, in a conversation, but, uh, pattern recognition at scale. So you've got lots and lots of data points. Uh, yeah. And lots of kind of things that are moving about it. AI is great for AI analytics and just highlighting things that, uh, are inconsistent, highlighting kind of, uh, patterns, anomalies, uh, things that are, um, things where people kind of disagree, uh, that the, the thing that, that is, is, is often missed is, is that, you know, if the data isn't good, then the outputs are not going to be good. And it goes back to that that kind of story was telling before about the, uh, the, the whiteboards. If people aren't putting in the information into the red logs, into the progress, you can't expect that AI is going to, uh, produce the truth, uh, based on either incomplete information or misleading information. So it needs to be kind of recognized that, uh, if the inputs are good, you're not going to get the outputs. Also, I get a lot of kind of comments in terms of, oh, can we just automate everything using it? Right. And people talk about, uh, uh, AI uh, hallucinating kind of, uh, things. I look at it a little bit differently. Um, you know, I, I work with all the time. I like things, but I always get up and especially on, on, on kind of important reports. I always get people to, to review it. Because I make mistakes all the time. Um, and I think everybody kind of makes mistakes. If you've got a system that completely relies on on AI, uh, then then it then is flawed. You need to make sure that your governance around what things you put in place because, uh, AI will

Speaker 1

[00.45.04]

use the data that, that that is AI is, is, is continuously kind of, uh, progressing. And, uh, the output from one day may change from, from day to day as AI, uh, improves. But you need to think of it in terms of the way I kind of, uh, use AI is it allows to to kind of, uh, highlight things, but then it's for humans to make decisions based on the data that's been to them. So there's this I think that AI is going to make projects more and Programs more effective, but not replace the management and that kind of human judgment.

Speaker 2

[00.45.48]

Yeah, that's a really important point because, look, um, you're using AI for, let's say, summarizing, uh, meeting notes is one use case, a really simple low hanging use case versus, um, using AI to synthesize all information about the program and create the insights and recommendations for decision makers to executive committee, etc.. That's a entirely different use case, because that's where the stakes are very high, right? And, um, you don't want AI to be by itself doing everything you want it. You want judgment. Um, but at the same time, my question is how if you are using AI for these kind of complex use cases, how do you make sure that, uh, it is backed by enough evidence? Um, so, so to speak, so that, uh, um. Um, the executives get the confidence that these reports, while they are generated, they're still very well, uh, supported, based based on well-established facts and report.

Speaker 1

[00.46.58]

Yeah. I mean, so so the evidence piece is really interesting because you you on on everything you do need to be able to kind of back it up with, uh, and you need to be able to kind of specify to, to, to whatever kind of query or whatever work you get an AI to do to, to make sure that it does, uh, back things up. But it kind of goes a little bit back to what I was just making in terms of just because it works today, does it mean it'll work tomorrow? And does it mean that that the, the, the results will be completely consistent? So you need somebody who is looking over this and making sure that, um, evidence, uh, or evidence that AI produces is, is, is consistent in an ongoing way. So, you know, it's not I will we'll create something and then it'll be, it'll it'll work forever. You need to kind of just make sure that you've got somebody looking over it to, to make sure that, that you have got that level of consistency.

Speaker 2

[00.48.06]

Yeah. Yeah. Look, um, as we start wrapping. Wrapping up now. Uh, Pete, I wanted to get your, uh, your thoughts and, um, and your advice to executives on on how they should, um, start adopting this continuous assurance and delivery intelligence practices. Um, um, so imagine a scenario, for example, uh, a organization where, uh, this, uh, CIO wants to move away from a traditional governance mechanisms for these big programs, um, uh, governance mechanisms to more continuous assurance capabilities. Right. Uh, what kind of approach you would suggest? What are the main where they should start? What is what would be a 36, 90 day plan at a high level for them

Speaker 1

[00.48.58]

to offer something simple? So I'd given three, three things. So firstly, uh, ask them to ask a question in terms of what evidence sits underneath the current reports. So if something is great ask them what evidence they've got to, to to to to say why it's great. And similarly if it's under or Or read. Then secondly, go back to the delivery teams and and other execs and ask them to tell them if they see the same picture. So if you've got a green colored piece of work, go to the delivery teams and ask the question, what do you think? And if they pick something else, then you start kind of picking away and understanding why. And then thirdly, uh, establish uh, a set of, of kind of leading indicators, uh, that you can continuously monitor rather than establishing a new kind of reporting function or anything like that. That will give people confidence. And, you know, the simple things that I've kind of mentioned here in terms of what confidence levels are there that will deliver confidence levels of the the benefits will will be delivered are two really, really simple ones. You know, you can go really kind of complex and um, but if you can get those, those, those two kind of lead indicators on a continuous way, then you it'll give you some really great kind of insight.

Speaker 2

[00.50.36]

Yeah. Yeah. Cool. On that note, Pete. Thanks. Thanks for your time. Um, I think this was a really wonderful discussion. Um, one key takeaway for me is, um, I would say governance. Uh, delivery governance is not the same as delivery confidence. Um, right. Because through governance, you can create all kinds of reports and so on and so forth. But that does not necessarily translate into how effectively your program is actually getting executed, whether it is meeting your objectives. Right. So moving towards continuous assurance seems to be the better mechanism to to look at how you actually manage the these programs. So thanks. Thanks for your time. And um, it was really pleasure hosting you

Speaker 1

[00.51.25]

a pleasure. So my thanks.

Speaker 2

[00.51.29]

If you found this discussion valuable, please follow and subscribe to enterprise TikTok. And thanks for listening. I look forward to seeing you in the next episode.