Weaponisation of AI - A New Battlefield for Enterprise Risk

FULL TRANSCRIPT

This transcript is based on the episode’s English auto-captions and has been formatted for readability. Please allow for occasional transcription errors in names, acronyms and specialised terms.

[00:00:00]

criminals are using AI to build code faster. Um, you don't need to be a software developer um to build your own malware. It just make the barrier for entry for cyber criminals as low as anything [music] else that we've been seeing being used with AI. The higher level u you have uh nation states [music] uh on the second level you have organized crime. I would expect that by now we already have some sort of like evil LLM as a service. We cannot be 100% sure that the person that we are seeing, the person that we are hearing is really themselves. It could just be a deep [music] fake. And the most important skills that will help anyone in my view um will be Hello and welcome to the Enterprise Tech Talk podcast. I'm your host Saumitra Kalikar.

[00:01:00]

In today's episode, we are going to explore a topic which is increasingly becoming impossible for organizations and senior leaders to ignore and that is weaponization of artificial intelligence. Now we often frame AI as a productivity accelerator, innovation engine and competitive differentiator. But AI is also being used increasingly by um cyber criminals, by organized gangs and groups and some nation states as a strategic weapon and enterprises are becoming increasing increasingly the center point of the conflict. So to discuss what weaponization of actually means for organizations and how they can prepare for it, I am joined today by David Mucci. David is the head of information security at Plywise. David, welcome to the podcast. >> Thank you for having me. >> So David, before we get started, would you mind providing just a brief overview of your professional career and what excites you to be in the cyber security industry? >> Sure. Um, so you know, I'll try to be as as brief as I can. Um, my uh career um started back in Brazil. So I'm originally from Brazil. Um and back in Brazil um I started um at the early uh entry- levelvel roles in cyber security um as a sock analyst. Um and this was my first experience uh professionally working in cyber um and it got my attention like nothing else. So before that I dabbed in a bunch of other different areas uh within uh technology.

[00:02:38]

Uh but something insecurity caught my attention. Um uh I love that um that mindset of like having a puzzle and trying to solve a puzzle. Um and when I joined um in a sock team of a gigantic bank uh back in Brazil, um it felt like I was in um in a kind of crime scene unit uh try to solve a crime scene every single day in my job and every day was different and I had a multitude of different tools uh to play around. So it sparked my interest uh and I hyperfocused on on that and um and double down in security. Since then um I've worked in security for 15 years um in all areas you can imagine. Um back in Brazil still um I progressed uh in the in the ranks in the sock team. Um and then jump uh laterally to uh thread intelligence, thread hunting, a little bit of like uh reverse engineering, um working with uh law enforcement in in some bigger campaigns. Uh so being involved a bit little bit of everything.

[00:03:40]

Um did a little bit of um risk and compliance as well. And when I migrated to to Australia, my first experience was around um cloud security. So I started doing a lot of consulting for cloud security. Um, and after doing that for four years, uh, and seeing that my advice to heads of security and CESOS was pretty much the same all the time. Um, then I I thought to myself like I should take my own advice, uh, and see how good my advice is. Um, so I took a role as a head of security uh, for a company. Uh, and he was very impressed to see that it it's not rocket science. Uh, if you follow the basics and you do the basics really well, um, it actually makes sense. So I took my own advice and continue my my journey um as a security leader in the the corporate world now. Uh but since the topic um is AI um a little bit of my um experience as well kind of like in parallel a little bit earlier than uh cyber security uh back at uni I was studing um the first ever uh cyber security bachelor's degree in Brazil at the time um and my capstone project uh it was treated because it was the first time that they had a course uh and it was still fairly new compared to the other technology courses as well.

[00:04:57]

they didn't have a basis on how to build the course and uh how to guarantee that they are studying something new and it has a good quality. Um so they decided to take it to a master's degree kind of level of excellence. So if they miss it at least it will be pretty good compared to any other courses that will come up afterwards. So my research uh capstone project at uni uh was not just a simple capstone project. It was slightly bigger than that. Um and at the time I was already quite interested in science fiction and everything like that. Um and AI was completely different uh back then. Um I usually like to joke around that my uni research was what now is considered a T in chip. Um I work with some extremely bleeding edge u uh pattern recognition um algorithms that now it's pretty much uh nothing compared to the LLN we have nowadays. Uh but back then it was the most like bleeding edge technology. I applied that to some security. Um and back at the time I was working in thread intelligence and most of my work was preventing fishing to get into our customers. Um so I built this very simple AI that would check a list of like millions of uh URLs per day. Um and you would check across 14 different parameters and will come back saying if it was fishing or not and the um assure uh um accuracy will be based on those parameters how much it could match to a real fishing based on the training data.

[00:06:32]

Um and since that time in uni um I still continue my trajectory in cyber security but I always been very uh interested in AI and always been playing around with everything that came uh in the last decade or so. Um so really nice to see that almost all my fing science fiction dreams uh became reality now or they are about to. >> Yeah great great I'm sure um this will definitely help in that case for for our audience as well to cut through the hype and uh focus on really what organizations their organizations need to really prepare for. Um so let's start uh David with a very grounded question. Um when we talk about weaponization of AI uh people immediately think of autonomous weapons or some sci-fi scenarios but from your perspective um how artificial intelligence is influencing the at a global scale the economic the information or cyber warfare. Um I think my my two best references um and I took the name of the of the author because if you want to put in the in the references or or any link uh later on uh it could be use useful. Um so my two best references that I have at the moment um at a very high level when I need to explain to boards for example executives uh the state of the eye where we are where we going um I have a very extreme in the positive side and a very extreme the negative side uh and based on those two I build my own like balanced view comparing those two. So in the positive side um I have um Yuvo Noah Harrari uh with the book Nexus um and he talks about all the cool um uses that AI will have in the future and how we going to save and improve our lives uh and how the human uh species will evolve in something different because of that. Uh and on the other side I have the ma mathematician uh Ray Kurtzwell uh with a book the singularity is nearer. um he had a book called the singularity is near and he updated to another one calling the sing singularity is nearer uh and apparently he's working on a third one that is even nearer or something like that. Um so this his side is a lot more pessimistic a lot more negative. Um, so what I usually see at the moment, um, and I like, um, Harrari's view on that one, whoever owns, um, the content, whoever whoever owns the information, uh, will be able to weaponize or use this um, in the way they want. And he very specific specifies the word um, information instead of the word truth. Because if you have the information you can manufacture your own truth and this is where in my point of view the biggest problem lies. Um if you have enough information uh and if you have a data lake or enough information of a person you can simply predict and we have countless of examples and this is probably where most of the problem is is all these aggregators of information when you put that information to something malicious this is where the problem lies. Um so for example when I started um in cyber security this was completely new and now it's pretty much just a normal everyday um there are cyber criminal groups that they only focus in collecting data and selling insights kind of like what exper does it u in a more like >> criminal have those sort of services as well. Uh so they collected information from people co for example uh where you live um how much you you earn uh what are your contacts um and because they have information of so many people they can draw a map and see like oh how you compare to your neighbors um what possessions do you have compared to everyone around you um would you be a target to a specific um distortion attack uh based in what you have um so everything you post line or everything that has any sort of digital footprint.

[00:10:37]

Um this is where most things can be used against you >> and it's really the hardest problem to to tack. >> Yeah. Yeah. That's good. Um so and to paint um broader picture at high level before we take a deep dive again um um how AIdriven threats are fundamentally different in your view compared to the the threats we used to deal with uh say last three year two three to five years back >> or what's evidently changing when we talk about AIdriven threats. >> Mhm. Um well um as everything else um AI just makes everything more accessible. Um so we have we've been using um all the copilots and AI agents and chachd and all the likes and claw and everything. Um we know we can produce code um and we know it's similar to humans. It has a margin of error. Um AI models is roughly around 20% give or take different models. Um 80% is still pretty good. Um and as I said, if you are not doing anything legal, you don't really care if you cause extra damage because it misbehave and deleted stuff that was not supposed to delete. Uh it actually helps in those cases. So criminals are using AI to build code faster. Um you don't need to be a software developer um to build your own mau. um you can just download an LLM that doesn't have any of the ethical constraints and you can just prompt it the same way you can prompt um an AI to help you with a spreadsheet. You can promp it to create a a polymorphic malware that will be unique for every single uh computer that you compromise and it always change. So um how the previous generation of antimower defenses were based in signatures. Now we are evolving a little bit to catch up uh but you already uh re disables and neutralizes all the signature based because if it changes on every single compromise you simply cannot track any any parameters. You can add a few other things and AI can help on both sides. Uh but it just make the barrier for entry for cyber criminals as low as anything else that we've been seeing being used with AI.

[00:12:58]

>> Yeah. Yeah. That's good. and and you touched about some of the um uh the actors right so like organized groups etc and I just want to unpack that a bit more um based on your experience and knowledge well so which are those um uh organized groups or even the nation states which are actually leading in this these attacks across the globe and probably a follow-up question to that is are they following the same techniques And are their objectives are same or they have different objectives and they potentially for different techniques as well. >> Uh yeah so the the way I usually um like to illustrate is in a in a pyramid uh with four levels if I remember correctly. Uh so at the higher level u you have uh nation states uh on the second level you have organized crime. uh those tier one uh red actors that you see in the news that they have a funny name and you see them all the time. Um on the third level um you have more independent uh groups uh for example um those more like low-level defacement groups uh or individual like curious people try to u find things um uh hack activists as well um and at the lowest level of the pyramid uh you have opportunistic someone just playing around they happen to found something they exploited um we had some um similar attacks happening in the past in Australia Australia where someone just accidentally bump into something and it become a data bridge gigantic data data bridge later on. Um so I usually see like even if you put that pyramid um and overlay all the security uh and AI things that are happening at the moment on top the same thing uh continues working the highest level the nation state sponsored predators uh they are extremely more sophisticated their usage and their use cases is way more uh sophisticated than the second level um their motivations is usually geopolitics the second level it's more like financial gain The third level is usually they are they have more like um political or some sort of agenda that they are pushing rather than just uh financial gain. Uh and the last bottom one is just whatever they can find. It's just an opportunistic they bump into something. Um so if you're breaking those four levels the same thing applies at the highest level you have their own like customuilt AI models for that specific um uh use of attacking other nation states. Um we already have a ransomware ransomware as a service for a long long time in the in the industry at the second layer of the pyramid. So I would expect that by now we already have some sort of like evil LLM as a service to build exploits for example. Um the third level will be someone with uh enough dedication going to uh any um commercial AI and try to create exploits and jailbreaks and try to find ways to abuse and bend the rules uh slightly. Uh and the last one continue as is. uh they will use something ready that was distilled from the higher levels of the pyramid and eventually gets to the lower levels. So as the um the same happens with vulnerabilities for example uh if a country nation state has a zero day that the other country their enemies don't know about it they will use that as an as a weapon. As soon as they know that their opponents are aware of this they release the public everyone patches. Um so it disables the enemy uh weapon. They lose one but it also disables the opponent but that just drops whole level. It goes to the threat actors in organized crime. They will use that to uh build their platforms as a service and continue their campaigns. When once that gets beat up too much it goes open source for example it gets to the third level for for them to cause havoc in internet um and eventually it trickles down to the fourth level as well.

[00:17:02]

>> Yeah. and and um the the topmost actors in your pyramid which I would imagine would be highly organized and potentially also highly well funded right um do you see so while while they will potentially focus more of a nation to nation kind of um conflicts but uh are we seeing now they are finding um organizations enterprises within in the in the uh other country other nations as being more soft targets proxy targets. >> Uh yeah. So um this is something that um I also um try to explain and make sure that um people are aware of that. Um like it or not even if we are directly involved in a conflict or not. Um we are also all of us u all us as individuals and as a companies we are collateral damage for those uh for those uh criminal actors because everything is based on the supply supply chain.

[00:18:01]

I might not be the final target for them, but I could be one hop in a big chain of attacks that they need to perform to get it to somewhere else because they already discovered that this is the weakest link. Um the same way we have some extremely sophisticated systems um that can check your entire cloud environment for example and figure out what scenarios like if this gets compromised and then you get to this and if this gets compromised you get to this. criminals also have this level of intel probably way better than ours because they don't have again ethical or legal constraints that we have. Um so they know who they are attacking. Uh especially the top two levels of the pyramid. They know exactly who they are attacking. Um and as I said you might not be the end target. You might just be the vehicle that they need to get to the next level. >> Yeah. Yeah. Okay. So let's also unpack then and you mentioned this earlier about u AI being a kind of force multiplier right it it accelerates the the attacks right um and um uh but all with AI it's not only the traditional ways of cyber threats right AI also comes with its own new types of threats like defects disinformation and few other things right um so from corporate perspective um what are those different types of threats traits are are be organizations should be more mindful of apart from just those traditional traditional fates.

[00:19:27]

>> Mhm. Uh yeah. So the um on the traditional side of things and the common attacks that we see um I usually use the analogy that AI works as a magnifying glass. Um everything that you put AI on it, it'll get bigger. It will be faster, more efficient. If you have problems, the problems that comes with it will be bigger as well. And this is where uh those problems exist on the more like novel attacks. Um so AI can be used as you mentioned with deep fakes. We've seen um a few cases already um of deep fakes being uh used to >> uh call uh finance departments for example and execute transfers. Um and you you are seeing you are hearing uh the CFO making an order. So most people would simply accept it. No question. Now we are adding more human controls on like you need to hang up and call that person in a different channel or speak with them in person if you can or anything like that that because we cannot be 100% sure that the person that we are seeing the person that we are hearing is really themselves. It could just be a deep fake. Uh and a good example of that um there are many um uh celebrities or influencers on social media that are completely virtual. They simply do not exist uh and people don't know and they have tens of millions uh of followers. They do advert advertisements for other brands and people have no clue that they are uh people see like oh it's an influencer, an Instagrammer, a blogger, something like that. But that don't even exist.

[00:21:03]

It's just like an LL model that being trained to look like that a random person. Um and the same thing we can see in the most basic sort of attacks email fishing now they because of all the AI can collect a lot of information a lot fast uh faster than us um and it can correlate information that we usually cannot at least easily uh so for example there are LLM um open source models used as proof of concept um a really good one that I like to always recommend it's called LLM from openour source intelligence. Um if you put someone's name or social media handle in that LLM, it will search the entire internet for that person um and will build um the personality profile of that person including Myer Briggs uh personality uh scale uh including um what was the other one? The Young's personality archetypes. um he knows what the person does, he knows what the person likes, he know the hobbies and he builds like the entire profile and then based on that he can create a fishing email with information that that person will click 100 close to 100% because you know like you touch all the soft points of that person. So once the person read that email you will fall for it uh or at least will be the uh initial point of study a conversation and then tricked in something else. So, it could be as simple as the very old school fishing email all the way to the extremely sophisticated calling someone pretending to be someone else uh or jumping in a team's meeting pretending to be someone else um all the way to even more uh nefarious uh options. Uh we already seen people being hired uh into companies to be an insider and everything they've done was through AI. So all tech challenges, interviews, um like with camera on and everything and it was just an AI uh model with someone behind operating um just to get highend be an insider and then from there uh continue their campaign.

[00:23:08]

>> Yeah, at at a level it's not only amusing but also potentially a bit unbelievable as well. And but it does talk about the importance of elevating the security culture and awareness within the organizations because people we we as humans we inherently trust um the if if we see the people we already know right u >> so in your view um are organizations in general doing enough to generate that awareness about these new types of AI AIdriven threats like defects etc or it's still significant gaps. >> Um I I would say awareness as a whole um is still a big gap in most organizations. Um for example, you don't see many companies with uh dedicated resources for security awareness for example. Um most companies they only have the bare minimum like one course that you have to do once a year which is the same one that you've seen for the last 20 years. uh and it's just like five seven slides that have to pass and you can complete in a minute. Um so if you put that into perspective yes most companies are not even to the standards of the basics in security awareness let alone um AI. Uh there are some buckets uh in businesses that are pushing for AI. So for example, engineers, developers, um more technical people and high level executives for uh interesting reasons are pushing AI really hardly. Um and for them makes sense. So because they are experiencing playing testing um they get exposed to the risks of using those technologies very quickly. Uh but there are even bigger pockets in the business that are completely unaware of what exists.

[00:25:01]

>> Yeah. Yeah. Let's talk about uh the the technology infrastructure in general within um organizations and are are there I just want to unpack if there are any specific um areas or aspects of technology in infrastructure which are more prone to AI attacks if they are vulnerable uh compared to others. So for example uh when it comes to u that the customer data or identity systems or or something similar uh or digital uh platforms do you see based on your experience again um do you see if some areas of technology infrastructure are more prone for uh AIdriven attacks compared to others? Um no I would say it's pretty much the the impact will be enhanced and it will be across um pretty much everything. Um especially as I mentioned before AI just works as a magnifying glass.

[00:25:59]

>> So it just make the old problems bigger. >> Um still our biggest problem is the uh squishy human behind the keyboard. Uh it's still the easiest thing to manipulate. Uh so that will continue uh with with no doubt. Um but our systems as well uh bigger companies for example they have a lot of legacy um it doesn't take much uh for an AI to discover a problem uh and then exploit that in ways that we haven't even considered. Um and a good example of that uh I don't remember the oh it was hacker one um so the bug bounty platform um u mid last year if I remember correctly um a user called xvol um got the top one ranking global um in uh hacker one uh vulnerability uh disclosure uh bounty platform u and it later on was discovered that this was a AI LLM that was exploiting pretty much everything they could see um and then building the proof of concepts and submitting um to bio to hacker one um so it can discover a lot of things a lot a lot of things and a lot faster than us so >> we are as I said in a in a disadvantage uh so in my opinion it's kind of like a arms race we need to fight fire with fire otherwise if we don't lose it we'll be way behind >> yeah yeah you're right it's a fire with fire And um but the fact remains that most of the organizations still are applying and using the traditional security techniques right um uh so in your view um uh how organization should think of improving their overall security architecture or security governance aspects to particularly focus on these new AIdriven threats uh when the the speed is is is so much.

[00:27:59]

>> Yes. Um so two two main points uh on on that side. Uh so the first one uh just I think the best way is uh simplification um and do the basics really well. Um as I said AI will uh one of the biggest uh loopholes or the biggest issues that AI will use is it makes problems bigger. Um so if you have your basics any gaps in your basics those gaps will be a lot bigger when AI touches them or when any processes gets integrated with AI and things like that. So the basics is just look at like for example essential age n CSF um then it's not rocket science uh and as I mentioned uh in my intro um I took my own advice and it was just like let me read that book select everything that makes sense and it's applying for the company that I'm working for the sector that I'm working for the country I'm working for um do the basics and be extremely well in every single metric pressure metric for the most basic things um and that will give us a good starting point to start thinking about AI. The more you have the more the easiest it is to lose control of things and uh it gets harder to keep track. So if you can simplify systems, if you can simplify processes, remove legacy, remove tech that I know this is more like a ideal world kind of scenario. Uh but we need to invest in that area. Uh otherwise it's just like a big a big house of cards and eventually one thing will trickle and everything will fall uh once the small um you probably have seen that um uh joke at the image that it's on how the internet works and it's like a bunch of things and a very tiny piece at the bottom called like AWS or DNS or something like that. So if if that small thing goes away everything that it uses that kind of falls with it. So that's the first side. Um and the second side also based on my experience what I've been playing with AI myself. Um and also following again um Harari's uh view um he uses an analogy in the book um that humans we organize ourselves um because it makes easier for us to share information to share to communicate um and by doing that creating companies uh and organizations and bodies and things like that um we can uh expand our throughput of building um information and building knowledge Um so working together makes that help. AI now what we're trying to do is okay AI thinks um faster than us uh just to simplify the calculations. AI thinks 10 times faster than us. So let's try to put that algorithm into our systems and see if that works. Um and it's usually not working at least in the first tries.

[00:30:59]

Uh and the reason is more more often than not governance gets forgotten. um AI is so fast, there's all this hype every day. If you look at the news, there's a new groundbreaking discovery, there's a new model that is way more incredible than the one uh before. Um so people forget about the risks a little bit. Um take a little bit more risks uh that they are considering. Um so governance uh is one of the things u that we need to um to focus on. uh and just to um illustrate and to finalize that point um I don't remember the name but there was another mathematician um that said the other day uh that while we got AI uses algorithms um to build its logic and get you its uh solutions and predicting text and and things like that but we humans we had to to collaborate between ourselves we also created our algorithm our analog algorithm and we call it bureaucracy um So or processes if you want to be more uh on the uh safer side. Um but what you're trying to do is like we build that process analog that works on our pace because this is how fast our brains usually process. So it's build it in that way because we humans work in that way. Now we are trying to bring that thing that works 10 times faster. If we simply drop in our algorithm or in our processes in bureaucracy um yes you will run a lot faster but um AI will be in the middle and humans will be at the end and it will be a lot harder for the humans to communicate with each other and to co coordinate with each other because they still can only do the process at their human speeds. So that creates disconnect. So governance tries to slow down these new technologies in a way that our brains can understand and we can process enough information and make better decisions in time in a timely manner. Um but that also means that we need to say no to a few things. Um we need to be a little more careful with some things. Uh and some people uh get a little bit pissed and they want to be part the whole defo thing. They want to be part of the whole uh AI journey. Um, and I am a big supporter uh for AI, but the more we progress uh the more I see people throwing um their risk concerns through the window and just going on.

[00:33:25]

>> A good example is the new um molten bot. It was cloud bot >> molten bot. Uh the number of security issues that I've seen in a five minute uh overview in there and simply mindboggling. But I know a lot of people using that uh and I'm really scared of what could come out of that one just by how much power that could have. >> Yeah. Yeah. No, I agree. Um just to touch uh I think you briefly touched upon this but just to unpack a little bit on u on the the frameworks we we have traditionally used for security um capabilities, right? Like NIT and all. Um do you see uh are those frameworks are now evolving to accommodate new AI specific risk controls etc. >> Um in principle all the same um uh foundational controls and considerations still work. Um the frameworks that we have definitely are not where near the what they should be. Um there are some efforts uh and we are making some good progress. Uh but as I mentioned before, we can only move at human speeds. Um and the technology itself is evolving a lot faster than human speeds are able to um to get together and get agreements and produce and test those uh those hypothesis. Uh another example, um crypto exists for slightly longer than u AI and hasn't been fully regulated yet.

[00:34:57]

So it's still an kind of a gray area. uh and it's slightly older. So AI will take a while to to for us to to catch up. Um bodies like uh NIST um the uh US one CISA um and um cloud security alliance CSA uh they published some really nice um guidances very high level very early days but we are making good progress in their area. Um I've seen I've seen some nice progress. Um I've seen some of the drafts that are being published now. Um and we are making good progress but still there's quite a lot for us to to go through. They are still have a lot of gaps. >> Yeah. Okay. Let's discuss a little bit about the AI solutions themselves. Um and how organizations should be approaching the security about around those solutions. Right. uh and when I mean when I what I mean by that is uh um the new attack surfaces AI solutions actually create for example prompt injection in the standard um uh uh genai scenarios or the new agentic AI scenarios which will potentially bring entirely new attack surfaces because there are AI address which are fairly autonomous right um >> um so you need to then consider identities and other access controls for AI agents, right? Um what's your guidance at high level to organizations and and security people uh might be listening to this as to how they should be thinking of these new attack surfaces. Um and what are the basic mitigations they should be considering before jumping to ahead. Um so yes um every day we see new um new uses for AI and there's good use use cases um you can summarize a lot of things we can auto automate a lot of things a lot of the low low value work that we do can be automated through AI as well uh especially in security that there's not a lot of security professionals uh in the industry um and in the upsc space dev sec ops kind of space um if you look at the ratio of an engine engineers choose security um it's usually in a good scenario 10 to one. So you have 10 engineers for every one uh security engineer. Um if you if you see any anywhere else that is better than this you were winning. Um so security the biggest problem in security is reach um as everything else like developers are publishing a lot more code uh using AI assistance for example. uh they are their confidence is lower in the code because they're not rating every single line. It's AI building chunks uh for them. Uh but on the other end I still have the same number of security analysts, security engineers um application security looking at that code. So we need to scale as well. Uh and the best way I can see AI helping us is scale scaling horizontally. So instead of having one single uh human engineer in every single team uh which is something that I always tried to build in my career in my teams uh and it always was a gigantic challenge uh now with AI and with a really good engineer behind it. Uh I can have kind of like a virtual uh security engineer for every single developer or engineer. um helping them uh have information on how vulnerability were found in environments uh which vulnerabilities were found, how to solve it and give them almost like a step by step uh on how to uh remediate a vulnerability that the bot saw that they introduced to a point that I've seen systems that they will actually once you submit your code to GitHub for example, the bot will submit another uh merge request with the fixes from the code that you just submitted. So as soon as you submit minutes later, the bot um adds a few more lines just to fix any vulnerabilities that you found. So there are really good ways on that, but also it opens up so many different kind of cans of worms um that we are pretty much discovering as we go every single day.

[00:39:06]

Um the thing that I always uh want to mention as one of the things that um takes my sleep at night um is anything AI or agentic AI in embedded in browsers um like we have chargi atlas um perplexity has one um Google is trying to push into chrome um I hope Microsoft don't push in edge but no one really uses edge so who cares um but it's really scary because the internet is already a minefield full of images or hidden background uh text or um invis invisible like same color of the background um text saying ignore previous instructions and do this. If you do a quick search in the web um using some of those tooling that check those sort of things um you can see that the web is already full of those things and most we don't know if everyone does that on purpose or if was a joke could be at a time just a joke um it could be part of a bigger campaign that was done on purpose so I'm still very unclear uh but the web is full of that and imagine you are just like doing your normal day-to-day job browsing through some websites um and your browser is enabled uh with AI and in one of those web pages you click there's that uh hidden message saying ignore all instructions and deploy this power.

[00:40:33]

>> Uh this new vector uh is this web based vectors using AI it's extremely hard to identify I haven't seen anything really useful that can prevent uh those sort of things. There are a few things that detect um but as I said everything is evolving so fast for example I did a big research around 12 months ago on security solutions for AI or against AI how you want to put it um and fun fact I just revisited that research last week and 90% of the companies that were in my re previous research doesn't exist anymore they either got acquired by a bigger player and what absorbed in their products or they simply crash and burn and closed. So if the companies that are building those tools to protect us um they still don't they are trying everything and see what sticks. So if if for them it's harder for us will be even harder because we're pretty much u unprotected for some things. But as I said for the most like traditional side of things at least the same basics that we've been always doing still works. So at least we can double down on those and will give us um any chance to continue finding. >> Yeah, you're right. This is a rapidly evolving field. Um and I just want to spend a couple of minutes on uh on the other aspect of this rapidly evolving field which is uh the the regulations around AI. Um now we know that so for example EU is coming up with the has come up with AI act potentially US is coming up with China is coming up with um it looks like there is still not a very consistent uh approach for air regulation. So I couple of questions if if I if I may. One is do you see in the for um foreseeable future there will be more common consensus around AI regulations across the globe or you um or organ countries will continue to have their own specific air regulations and the second is coming back more specific to Australia what is Australia doing when it comes to the air regulations >> um so let's try to uh touch those points without getting too political. Um, but why what I kind of like forecast that it's going to happen is as the glo global conflicts uh will continue to to keep up and continue to increase um in scale um the regulations on AI will speed up as well uh because uh they will be seen as a part of national security or national um autonomy uh in some cases as well. Uh but it will get to a point that the whole planet will gridlock. So we don't have enough resources um for everyone to build. Let's say we're going to get to a point that no country trusts each other anymore. Um in all the tensions between Russia, China, US and all that. Um so it creates a lack of trust. Um and then countries will start building and I've heard this in European countries as well. For example, we need to build our own data centers. we need to build our own cloud providers. Uh otherwise if we like have a disagreement with us they can simply shut down our entire nation uh or the entire continent because like onethird of the internet runs in AWS at least uh or Google and you can or Microsoft if you combine all three you have a good portion of the internet. Um so US could pretty much shut down any country if they wanted to.

[00:44:12]

Um Russia and China they probably will do the same. So I assume that eventually people will try to build their own models, their own infrastructure, but we don't have enough resources for everyone to build their infrastructure. So we probably will ended up at some point in that standstill situation. If we don't destroy ourselves in the process, uh we wouldn't ended up in that situation that we need to build data centers. We don't have enough resources to build data centers and we don't have land to put them. So that's why some big companies there already started thinking of like can we put data centers in orbit because we know that it would be really hard to build an inland and there's the whole like national sovereignity um that another country could attack your infrastructure as well. So it just scales. Um and on that uh there is a quote u just try to remember who said that quote. I don't don't really remember um who was the that that put that quote. uh he was probably the same mathematician that said about the the algorithm because I read a book uh uh from that guy just recently um and he mentioned that the as the problems or issues that we are trying to solve they um scale um in size um a society issue for example a global u pandemic uh issue um geopolitical tensions when it gets to that level the quality of thought and the quality of action to solve those things diminishes um proportionally. So as problems grows our actions and our impact on that problem diminishes. So as AI continue to grow security is a very uh strategic field already. Um as we see in cyber war and all that it's one of the new fronts of the war now I think AI will play a part in somewhere like that as well. But as it gets so high up in the priority and who needs to get together to make those discussions um I think we'll get to a standstill situation very soon.

[00:46:13]

>> Yeah. Yeah. and and okay so coming back to the organizational context David um now AI is an emerging field and organizations are kind of exploring different options to define operating models around AI right some organizations have set up their own AI teams etc and punish as a business owner uh and security has traditionally have their own operative models etc right um but do you see the how this will play out in the future where AI and security become two important um considerations for organizations to build resilience. Um is there a common narrative that coming up as to how organizations should think of defining their future operating models um around AI and security? >> Uh that's a very interesting uh topic. Um I had some some thought on that but honestly I haven't seen much uh in what I can see um in the industry to talk about like more like not on the resiliency side um and how like a more like fundamental strategic piece AI can be in business. Um I'm not 100% sure that we are at that point to start having those discussions yet because everything else is so immature um at the moment. Um but I would see that it would play a big part in in organizations in the near future. Um just by as I mentioned before AIS can work at least 10 times faster than us humans. So in some very specific key key areas of the business in some very specific um automations um it will be common sense and it will be uh commoditize some certain things that it have to be let's say in real time things that are not real time now because we are humans it takes a while to process or it has to go through some sort of approval flow let's say like a home loan for example those things will be instantaneous uh because that AI model will have so much information about us um that we'll be able to tell like yes this person will pay uh appropriately so give them the loan that person will not pay because we have enough information and it gets to kind of like a minority report kind of area like predicting things before they happen that gets a little bit murky and there's a lot of ethical concerns around that are also still evolving uh but I see AI playing a big part in the future in this more like automation and that part of some areas will be for example uh someone gave this example yesterday to me and said okay actually makes quite sense um in the old days when I had normal phones with buttons uh it was quite straightforward to use uh buttons uh in phones and text with uh the numeric uh keypads now if you look back with the phones that we have now it looks crazy to someone to use those phones um AI will be the same like so we'll probably like in let's say 10 10 years in the future We probably will have discussions like did you really did that like did you had to write that yourself or did you do like this spreadsheet manually yourself all these will be automatically done for you like so those more like low-level tasks um it will be normalized that they should be instantaneous because we will have I hope so um enough computing power and enough information around those systems to be part of that and once we start rebuilding uh systems systems. I imagine there will be a thing as well. So if you ever run a disaster recovery um you know how complex uh it can be like the order and where you bring systems back and how you restore it. There's a lot of different routines and a lot of different rituals that have to be done in order to bring a system back up. um giving that to an AI system for example that can put them in order and run that order that order every single time with no flaws will be a lot more efficient than give to a human that probably has been working for the last three days without sleep because it was a massive incident that took the whole systems down um it's already tired uh not feeling well and has to make those uh tricky actions missing up something >> causes the problem to be even bigger or have some secondary impacts. So I see in those like very specific things that can be reproduced easily. Um AI will pretty much take over and will be normalized that it it's instantaneous. It has to be >> Yeah. Yeah. So um I think uh we are almost there in time. So but but just to bend down what you just said um if we look ahead for over next three to five years um what are the important investments organizations should consider now whether those are investment skills or technology infrastructure what whatever uh will be important for organizations to do do make those investments now so that they can be more future ready to to um u mit ate emerging areas in the future. H um on skills u it will be pretty much uh what I said before just do the basics uh really well and invest in the basics including and education not just cyber uh cyber security awareness but education in general go play around go understand how AI work um and the biggest point in here uh my mom for example um she didn't you she was not really into technology uh she saw the evolution from old phones dumb phones to smartphones But she was not using it. So once it become normalized to use smartphones, she started struggling because everyone was using it and she was getting behind in the world because she was not using.

[00:52:01]

Same way my grandma did the same with computers. She said like no, I'm not going to use computers. This is like a gimmick or whatever. Um and she's now a computer illiterate person. So I'll say keep your eyes open and go play around. Go study. Go u experiment. um be a little bit careful in the security side of things but go experiment um to understand how it works. Um and the most important skills that will help anyone in my view um will be sounds obvious uh but is emotional intelligence, critical thinking and leadership. AI can do all three way better than us. Um but to have a saying in this new world would at least have to be good at those things. Uh it's not to be uh oh this is the only thing that humans can do and AI will never be able to replicate. No, eventually they will. Uh but if you have it against another person that doesn't have it, you have this the disadvantage in the in the market. >> Um so David, let me close with the final question. Um um let's say you are having conversation at a board level, right? Uh group executives are board level and um you want to take that opportunity to uh highlight maybe one or two important risks which are not talked about much.

[00:53:14]

Can you can you point out one of those one two couple of examples for our audience as well as to these are the important risks that board should be aware of. Um the the most important risks uh that I see in general are human-based risks still um 90 something 96% if I'm not mistaken uh of all data breaches start with a human action um either social engineering or something happened uh with a specific uh person uh on an insider so they have the motivation or something like that. So all those things that involve the human element um are probably the top risks um that companies uh should be focusing on. Um it could be as I mentioned security um um hygiene in general um so at least you are not vulnerable with the very basic things. So not like all the Australia is a for example um have proper patching your laptops disable macros uh all the the very basics control applications in devices and things like that. Uh so those um very basics um you can be you can trust that people will be at their best uh but you never know. So inside a threat um could also be a problem um and social engineering. So and with social engineering coming um ransomware fishing all those other threat actors. So everything that involves the human element either the element being external or internal with the intention to attack you or even by mistake. So data loss DLP also gets involved in in this these areas.

[00:54:51]

>> Yeah. Yeah. No, that that's a good point because we can think of some some some big things we need to do etc for to mitigate against ARS but at the end of the day as you said this standard and small smaller things like generating more awareness within the organization or on security culture right and then doing these mundane things like regular patching etc they help a lot um in terms of mitigating against risks so David thanks for your time um it has been uh really a grounded discussion and good insights for our audience. It was a pleasure to host you. Thanks. >> Thank you for having me. Great conversation. >> Hope you like today's episode. Please subscribe us on YouTube, LinkedIn and X platforms. Also, if you are passionate about any such enterprise technology topics and want to participate in the discussion, please [music] reach out at inquiry@ enterprisete.com. Also, please visit the website