Take one small example. A research team recently spent over 100 billion tokens on a single AI model during a global cybersecurity competition run by DARPA (the US defence research agency). That's not really a story about one competition. It's a sign of how fast AI security capability is advancing, and how much serious investment is now pouring into it.
Some of the people behind that effort later joined Microsoft. What they built didn't stay in a lab. It now scans Microsoft's own software before release, and feeds into a broader security strategy the company announced this year, called Project Perception.
AI agents aren't just a productivity story anymore, drafting documents, summarising meetings, answering questions from a SharePoint site. The same technology is increasingly attacking, and defending, at a pace human teams can't match alone. That changes what "we have a security team" really means as an assurance, and it's quickly becoming a boardroom conversation, not only an IT one.
This guide looks at why that shift is happening, what it looks like in practice through a real example, and a few questions worth raising internally before your next budget cycle.
Why the Rules of AI Security Are Changing
Traditional security models were built on a simple, reasonable assumption. Attacks happen at human speed, so human-speed defence is enough. That assumption is what's shifting now.
Two things are moving in opposite directions at once. The cost of mounting an attack is falling, because AI can now generate exploits and scale a campaign with far less manual effort than before.
At the same time, the volume and complexity of what needs to be protected keeps growing. More systems, more data, more AI tools deployed inside the business itself. Microsoft frames this as a change in the "physics" of cybersecurity, meaning the fundamentals of how attacks happen have moved, not just the tools used to carry them out.
None of this is a criticism of how organisations have approached security so far. Human-speed defence was a perfectly rational model for a human-speed threat. The point worth taking to a board is narrower, and more useful. The threat itself has changed pace, and that's a strategic question, not an operational one.
AI Security in Practice: A Real Example
Abstract talk about "AI-speed threats" is easy to nod along to and hard to actually act on. So, it helps to ground this in something Microsoft has genuinely built and is running today.
The team behind that DARPA effort now leads Microsoft's Security FORGE Labs. Their focus is applying frontier AI models to find and fix software vulnerabilities before they ship.
Their system, internally called MDASH, scans a code repository, spots potential vulnerabilities, and proposes fixes. But the interesting part isn't the scanning. It's how the system decides whether a finding is real before anyone acts on it.
Rather than trusting a single AI's first answer, MDASH runs a structured internal debate. One AI persona argues against the defensive case, that a flagged issue is intended behaviour, not a real flaw. Another argues against the offensive case, that it's a genuine risk worth fixing.
The two positions get weighed against each other. Only findings that reach a confident conclusion move forward as high priority. Anything the system isn't confident about gets flagged for closer review instead of acted on straight away.
If that structure sounds familiar, it should. It's the same logic behind a four-eyes principle in financial controls, or a second opinion in clinical decision-making. Don't act on one unchallenged judgement when the cost of being wrong is high.
Microsoft applied that same instinct to AI systems, and it now runs inside their own software release process, catching vulnerabilities before code ships rather than after.
The Bigger Picture: Project Perception
MDASH is one component. Microsoft has now positioned it inside a broader strategy called Project Perception, a system built around three coordinated groups of AI agents working continuously, rather than a single tool responding to individual alerts.
| Agent group | Role |
| Red team agents | Identify potential paths an attacker could exploit, before an attacker finds them |
| Blue team agents | Investigate flagged risks and judge which ones genuinely matter |
| Green team agents | Take corrective action and strengthen defences |
These three groups form what Microsoft describes as a closed loop. They continuously discover risk, evaluate it, and improve the organisation's posture, rather than producing an ever-growing pile of alerts for a human team to triage by hand.
Project Perception entered public preview on 3 August 2026.
Microsoft has also started building purpose-made AI models for this specific job, rather than relying only on general-purpose ones. Its first, a specialised cybersecurity model called MAI-Cyber-1-Flash, was reported to score 96% on CyberGym, an industry benchmark for vulnerability detection.
This result is for MDASH's full system, not MAI-Cyber-1-Flash on its own. 'Mythos' is Anthropic's Claude Mythos 5.
Source: Microsoft Security (May, 2026)
It also reportedly cut compute costs by roughly half compared with the previous configuration. For a board, the number that matters isn't the benchmark score itself. It's the direction. Purpose-built AI security tooling is becoming both more capable and cheaper to run at the same time, which changes the economics of the decision, not just the technology.
For a board, the broader takeaway isn't the architecture itself. It's what this represents. One of the world's largest software vendors has concluded that alert volume was never the real problem, and that meaningful AI security now requires systems that can reason and act continuously, not just flag and wait.
How Trust Gets Built Into AI Security
A fair question worth asking about any AI system is how confidence in it gets earned, not just claimed. Microsoft's own approach here is a genuinely useful example for any organisation adopting AI at scale.
Alongside Project Perception, Microsoft launched an initiative called EXTRA. It funds AI safety research at 18 university labs across six continents, and builds a network of independent external specialists to help test its AI systems from the outside, not because internal work isn't rigorous, but because outside perspective strengthens it further.
The reasoning behind it is straightforward. The highest-risk failure modes in AI systems often need regional, linguistic, or domain expertise that benefits from being sourced widely, beyond what any one internal team would naturally cover on its own.
This mirrors a pattern well established in traditional cybersecurity. Coordinated vulnerability disclosure, independent audits, and external penetration testing are standard practice at well-run organisations precisely because external validation adds real value alongside strong internal review. It's the same discipline organisations already expect in financial audits and compliance work, applied here to a newer kind of system.
What This Means for AI Security in Your Organisation
None of this requires building your own version of MDASH or Project Perception. Very few organisations need that scale, and most shouldn't try.
What it does mean is that these questions belong at leadership level, alongside financial and operational risk, rather than left entirely to IT to resolve quietly in the background.
A few questions worth raising internally, not as a checklist to fail, but as a starting point for a genuinely useful conversation:
- Which AI agents and Copilot tools are actually active across our Microsoft 365 environment today, and does anyone hold a complete picture of that?
- Do our current data governance and permission structures actually determine what those agents can see and act on, or have they simply inherited whatever access already existed?
- If an AI system inside our organisation made an incorrect or harmful decision, is there a clear owner for that outcome?
- Are we relying solely on internal confidence in our AI tools, or do we have any external, independent perspective on where the risk actually sits?
Some of these questions would take businesses hours to answer properly. That's not a failure, it's just where governance sits today. Agentic AI has spread through Microsoft 365 faster than most teams have had time to keep up with.
Where Precio Fishbone Fits
Very few organisations need a Project Perception of their own. Almost every organisation running Microsoft 365 today does need a clear answer to a smaller, more immediate question: what can the AI and agents already be active in our environment see, and who's accountable for that.
We work with leadership and IT teams to build exact picture that businesses want, with data governance aligned to Microsoft Purview, clear visibility into which agents exist and what they're permitted to access, and a compliance framework that holds up against GDPR and sector-specific regulation. That's the groundwork that turns "we're using Copilot" into something a board can genuinely stand behind.
Explore our AI Security Services
Discuss with us
Frequently Asked Questions
Is this relevant if we're not a large enterprise?
Yes. The underlying shift, AI operating at machine speed on both offence and defence, affects the threat landscape generally, not just large organisations. What changes with size is the response: smaller organisations don't need to build their own detection systems, but they still need clear visibility into what AI tools are active in their environment and what those tools can access.
Do we need to understand the technical architecture to act on this?
No. The governance questions, what can our AI agents see, who's accountable, is there independent oversight, are business questions, not technical ones. The technical implementation is where a specialist partner or IT team comes in.
What is Shadow AI, and should we be worried about it?
Shadow AI refers to AI tools and agents in use across an organisation without formal approval, visibility, or governance, similar in spirit to "shadow IT." As AI tools become easier to create and deploy, this becomes a more realistic blind spot for most organisations, and one worth actively checking for rather than assuming isn't present.
What's Project Perception, in one sentence?
A Microsoft security system that uses coordinated groups of AI agents to continuously find, evaluate, and fix security risks, rather than relying on human teams to manually triage a stream of alerts.
Is Microsoft's approach relevant even though we're not using Microsoft's specific tools?
Yes, in principle rather than in product. The underlying discipline don't trust a single unchallenged AI judgement, and doesn't rely solely on internal confidence, applies regardless of which vendor's AI tools an organisation runs.
Share this on
Pär Johansson
Pär works with international business at Precio Fishbone, project delivery & digital services, helping turn complexity into progress and strategy into long-term value. With many years of experience in international business, He is known for building strong relationships and turning plans into meaningful progress. Driven by people, trust and sustainable growth.