Nathan Simmons
PortfolioAll Posts

I've Been Raving About Embracing AI Agents. Here's What I Left Out.

In January, I posted about setting up an Engineering AI Charter for my team through ownership rules, human-first problem solving, keeping fundamental skills sharp while leveraging AI to accelerate everything else. In February, I talked about building my daughter’s spelling app with Claude Code in 9 hours and how the bottleneck in software engineering has shifted from writing code to knowing what to build and why.

I meant every word of both posts. But I just finished a research project on agentic AI security for my computer science degree, and I owe you the other side of the story. The same capabilities that make these tools transformative make them genuinely dangerous if you’re not paying attention to what’s happening underneath the hood.

This isn’t a “doom” post. I’m more bullish on AI-assisted development than ever. But after weeks of digging into the research, I think most of us (myself included) have been so focused on what these tools can do that we haven’t spent enough time thinking about what they can expose.

The “Lethal Trifecta”

Security researchers have identified a convergence of three agentic capabilities that, when combined, fundamentally break the traditional security perimeter. They call it the “Lethal Trifecta”:

  1. Access to private data. For an agent to be useful, it needs to read your files, your repos, your environment variables, your credentials. That’s the whole point.
  2. Exposure to untrusted content. Agents process inputs from everywhere: emails, websites, third-party APIs, user-submitted data. Not all of it is safe.
  3. The ability to communicate externally. Agents don’t just read. They act. They call APIs, push code, send messages, execute commands.

Any one of these in isolation is manageable. But when all three converge in a single autonomous system that can read your secrets, encounter a malicious input, and then act on it externally, the established boundaries of data security aren’t just weakened. They’re gone.

Here’s what makes this worse than it sounds: unlike the chatbots of a year ago, today’s agents have persistent memory. They remember past interactions to inform future decisions. That means an attacker doesn’t need to exploit them in real time. They can plant a fragmented malicious instruction inside something innocuous, like a forwarded email, a webpage, or a document and it gets stored in the agent’s long-term memory. It sits there dormant for days or weeks until specific conditions are met, then activates. Security researchers are calling these “stateful, delayed-execution attacks,” and we have almost no mature tooling to detect them.

Think about that for a second. We’ve gone from “the AI might write buggy code” to “the AI might execute a pre-planted attack from two weeks ago using your credentials.” That’s a fundamentally different threat model.

The Numbers That Stopped Me

While researching this, a few statistics genuinely changed how I think about the current landscape:

The identity explosion is real. In the average enterprise, there are already 82 machine identities for every human employee. With agentic AI scaling the way it is, with agents spawning sub-agents and creating their own auth chains, projections have that ratio accelerating toward 500:1 by end of year, with some enterprises expected to manage up to 50 million non-human identities.

Adoption is wildly outpacing security. 80.9% of technical teams are already past the planning phase on AI agent deployment. Only 14.4% have full security approval for their agent fleets. Meanwhile, 82% of executives believe their existing policies are sufficient. That gap between confidence and reality is where breaches live.

The credential infrastructure is broken. A study of over 5,200 MCP servers (MCP being the emerging standard protocol for connecting AI agents to external tools) found that 53% relied on insecure, long-lived static secrets like API keys and personal access tokens. Only 8.5% used OAuth. We’re authenticating a machine workforce with an identity framework built for humans, and it’s a structural failure waiting to collapse.

What I’m Actually Doing About It

I don’t think the answer is to slow down. I think the answer is to be intentional about the security architecture underneath the tools we’re already using. Here are three concrete layers I’m focused on right now.

Layer 1: Know What Your Enterprise Tools Already Give You

At my work, we use GitHub Copilot through GitHub Enterprise. After digging into the security model for this research, I realized our enterprise tier already provides meaningful safeguards that a lot of teams are probably taking for granted without understanding why they matter.

Enterprise prompts and code completions aren’t used to train foundation models and your proprietary code stays inside your security boundary. There are content exclusion policies that let admins block sensitive file patterns (secrets, configs, infrastructure code) from ever being processed by Copilot. Enterprise audit logging tracks every Copilot interaction across the org, such as who used it, what was suggested, and what was accepted. And the Copilot coding agent now automatically runs CodeQL security analysis and checks against the GitHub Advisory Database for vulnerable dependencies, and uses secret scanning to detect leaked API keys. This is all before it even opens a PR.

These aren’t optional add-ons. They’re built into Enterprise. But if your org is on Enterprise Copilot and you haven’t reviewed your content exclusion policies and audit configurations, you’re leaving real safeguards on the table. That’s your lowest-hanging fruit.

Layer 2: Sandbox Your Personal Tools

For my personal projects with Claude Code, I don’t have an enterprise security team watching my back. The security posture is entirely on me.

Claude Code now has OS-level sandboxing that enforces isolation at the kernel level, not through trust or prompt engineering. Filesystem isolation restricts the agent to reading and writing only within your project directory, meaning it can’t touch your SSH keys, your shell config, or anything outside the repo. Network isolation routes all connections through a proxy that only allows domains you’ve explicitly approved, so even if a prompt injection succeeds, the agent can’t phone home to an attacker’s server.

I’m enabling sandboxing, configuring explicit deny rules for credentials and destructive commands, and keeping my permission mode set so that anything outside defined boundaries requires my explicit approval. Anthropic’s own internal data shows sandboxing reduces permission prompts by 84% while actually increasing security. Fewer prompts means less approval fatigue, which means you’re actually paying attention when something unusual comes through.

If you’re using Claude Code (or any agentic coding tool) on personal projects and you haven’t enabled sandboxing, go run /sandbox in your terminal right now. It takes about two minutes and it’s the single highest-impact security move you can make.

Layer 3: Adopt the Zero Trust Mindset

The broader principle from my research is this: we need to stop treating AI agents like trusted users and start treating them like what they are: autonomous software that should earn access through verification, not inherit it through convenience.

The industry is calling this “Zero Trust for Agents.” Under this model, every tool call and API request an agent makes is independently verified, logged, and scoped. The agent is treated as an untrusted entity by default, not a privileged user.

The full technical implementation of this involves some heavy infrastructure, specifically decentralized cryptographic identities, hardware-enforced confidential computing, zero-knowledge proofs for verifiable execution. Most of us aren’t going to build that ourselves. But the mindset applies to everyone right now:

  • Scope permissions to the minimum. Don’t give an agent read/write access to your entire system when it only needs access to one directory.
  • Verify, don’t trust. Review what your agents actually did, not just what they said they did. Check the audit logs. Read the diffs.
  • Assume compromise is possible. Design your security posture so that a compromised agent’s blast radius is contained, not catastrophic.

The AI Charter Needs a Security Chapter

In my January post, I laid out the first draft of an Engineering AI Charter: Total Ownership, Human-First Problem Solving, Active Maintenance. Looking at it now through the lens of this research, I realize it was incomplete. The charter addressed quality risks. It didn’t address security risks.

The engineers who thrive in this next phase won’t just be the ones who can design systems and direct AI agents. They’ll be the ones who understand that autonomy without accountability is just risk with extra steps.

I’m still building with these tools every day. I’m still pushing my team to adopt them. But I’m doing it with a much clearer understanding of what’s at stake underneath and I think you should be too.

If you want to go deeper, here are some of the sources that shaped my thinking:

Continue the conversation

This post also lives on LinkedIn, where the comment thread is happening.

Discussion on LinkedIn