Four teams, four different ways to run agents.

A division of labour is emerging in agent engineering.

• The model picks the next token.

• The harness (Claude Code, Codex, Opencode, Gemini CLI, pi.dev etc ) decides everything else: what context survives, what tools get exposed, when the loop terminates, where the code runs, what happens when memory overflows.

Billions have gone to the first job. Let’so look at the second part.

Vtrivedy puts it cleanly: “The Harness is a Context Manager on Behalf of the Model… What happens when the context window fills up and who decides? This decision is external to the model”.

Once you internalise this, the design space expands.

Sandbox boundaries, memory compaction, trace introspection, pause/resume protocols are runtime decisions that ship before the model ever sees a token.

From isolated tools to isolated agents

Larsen Cundric at Browser Use wrote up their sandbox evolution, sharing what gets isolated.

They started by

• isolating the tool (agent on your infra, dangerous calls hit a sandbox) and shifted to

• isolating the agent (whole agent runs in a sandbox with zero secrets, talks to a control plane that holds credentials).

That second pattern makes the agent switchable. A killed agent loses nothing, a restart costs nothing, scale-out is cheap, and there are no secrets sitting in its memory for a prompt injection to exfiltrate.

Credentials live in one place that the agent can only reach by URL.

Production runtime is Unikraft micro-VMs that boot in under a second and scale to zero when idle (remember “serverless”?)

The same Docker image runs locally for agent evaluations (a term mixing build and run time testing).

Boot times are important, but the architecture more so:

• the agent owns nothing, the control plane owns everything, and

• the protocol between them becomes the thing engineers optimise.

Memory as a layered system, not a vector pile

Tencent open-sourced TencentDB Agent Memory.

Integrated with the OpenClaw, with big purported boosts on metrics I’ve never heard of but sound important 🙂:

• 61% fewer tokens on WideSearch,

• a 51% relative pass-rate lift on the same benchmark, and

• accuracy lifted from 48% to 76%.

Most memory systems break conversation into chunks and dump them in a vector store, leaving recall to grope blindly across disconnected fragments. Classic RAG.

Tencent rejects that approach and builds two pillars instead:

• symbolic short-term memory (raw tool outputs at the bottom, step summaries in the middle, a Mermaid canvas at the top), and

• long-term memory layered into personas and scenes.

The Mermaid canvas makes a LOT of sense — I’ve personally had a lot of success with mermaid diagrams because they’re extremely efficient at communicating architectural concepts and it’s why Ceetrix uses them a lot. Using them to track state is genius.

@berryxia’s writeup calls out the same idea: structured task maps generated as Mermaid graphs, so the agent always knows which step it’s on in a 30-step workflow. Coherence across long workflows comes from a structured map of the plan, not from cramming more turns into the prompt.

Usual benchmark vendor bias of course. SWE-bench, a gold standard for choice generation quality, gains 9.93% and AA-LCR of 7.95% can be absolutely huge depending on what the model is already doing.

Separately, the token reduction is actually more valuable than you’d think because information retrieval from context is not consistent and degrades very quickly as the context gets bigger with current models.

Let the harness optimise itself

Sam Hogan’s HALO is the most contrarian piece in this batch. The premise: humans are bad harness designers, so use a reasoning model to do it.

HALO takes execution traces from a deployed agent, identifies failure modes that only show up in aggregate, suggests harness changes, and a coding agent applies them.

Reported gains include 10%+ improvements on AppWorld, TerminalBench, and Finance Bench.

The strongest result is on Terminal-Bench, where they pushed Gemini Flash toward frontier coding harness performance through harness changes alone. This is pet remarkable and I’m keen to replicate it.

Hogan says: “the harness is becoming an optimizable service layer: comparable in importance to the model itself, and increasingly measurable as its own object of engineering.”

Same way Kubernetes split schedulers, networks, and storage from application logic, the agent harness is splitting out from the model wrapper.

In theory HALO works with any harness you have access to code for, etc Opencode, pi.dev, Gemini-CLI, OpenAI Agents SDK.

Pause for a week, resume without amnesia

Google’s ADK writeup, shared by Richard Seroter, takes a different cut: durability. The walkthrough builds a new-hire onboarding agent that runs for two weeks, pauses for days while a human signs documents, then resumes without making up stuff.

Stateless chat-loop agents fail over long timeframes in three ways:

• context pollution after hundreds of turns,

• token costs ballooning on every replay, and

• the model inventing approvals that never happened when resumed cold.

Solution: explicit, durable state decoupled from raw chat history, a structural change to where the agent’s truth lives, rather than a larger window to stuff more conversation turns into.

What ties them together

Four teams, four pieces of the same machine.

• Sandboxing (Browser Use),

• memory compaction (Tencent),

• trace-driven self-optimisation (HALO),

• durable pause/resume (ADK).

They all assume the model is fixed, and the rest of the system is engineerable.

Not entirely true (models still improve), but the marginal hour of effort on a long-running agent is now better spent on runtime than on prompt tuning.

The teams above are calling that bet explicitly.

🔮 Prediction: the next wave of agent benchmark gains comes from harness changes, not model upgrades. HALO’s frame (the harness as an optimisable service layer) becomes the dominant abstraction within twelve months. The runtime designers, not the prompt engineers, will be the ones quietly moving SOTA numbers on AppWorld and Terminal-Bench.

That same week, Sakana AI dropped a paper saying the pattern itself is the wrong abstraction.

This article was reviewed and hand-edited, using curated sources I hand-picked, drafted by AI using personalised style guidance. I find it useful to keep in touch with the dozens of X.com bookmarks I come across every few days. Maybe you do too.

The default architecture, productised

What JP Morgan shipped is competent, but not groundbreaking. A supervisor decomposes a research question, fans it out to retrieval and analytics specialists, gathers the responses, passes them through a critic, then escalates to a human when the LLM-as-judge isn’t confident. This is the LangGraph-shaped, CrewAI-shaped, AutoGen-shaped default that’s settled in over the past year as the obvious way to build a serious agent product.

It works. The reason it shows up everywhere is that it’s the minimum viable structure to keep a multi-agent system from going off the rails. The supervisor handles routing, the judge handles quality, the human handles the long tail. Boring is good.

But every part of that system is hand-engineered. You wrote the supervisor’s system prompt. You decided which subagents exist. You chose the topology. You wrote the rubric the judge uses. The agent system is doing what it’s doing because you told it to.

Sakana’s Conductor: trained, not wired

The Conductor paper from Sakana takes a different swing. Train a small model (7B parameters) with reinforcement learning to do the orchestration job itself. It learns the communication topology between workers. It learns to prompt-engineer focused instructions for each one. It learns which specialist to call for which subtask, against pools of frontier LLMs from different providers.

The headline claim: that 7B Conductor, orchestrating powerful workers, hits state-of-the-art on LiveCodeBench and GPQA. Better than any single worker in the pool. Hardmaru’s framing on the launch tweet: “By training a Conductor model with RL, we found that it naturally learns to write highly effective, custom instructions for a whole pool of other models.”

Two details:

First, the Conductor was trained with randomised agent pools. So it generalises to whatever workers you give it, including closed-source models it never saw at training time. If true at scale, that’s pretty remarkable: you don’t retrain when GPT-6 shows up.

Second, the Conductor can call itself as a worker. That gives you recursive topologies and a new flavour of test-time scaling: spend more compute by letting the conductor iterate on its own outputs. It’s a clever trick. It’s also the kind of thing that loves to blow up your token budget, so I’d want to see the cost curves before getting excited.

What about Fugu?

Sakana wrapped this research direction into a commercial product called Fugu, now in beta. The pitch is straightforward: instead of you juggling API keys for OpenAI, Anthropic and Google and routing prompts manually based on which model is good at what, Fugu does the routing for you. One API call in. The Conductor figures out which workers to convene and how to instruct them. Answer out.

I’ve not been able to test it (the form’s open if you want a go), so this is a paper review and a product description, not a real-world verdict yet. The same conviction drives Sakana’s other work: evolutionary model merging, AB-MCTS, the AI Scientist. The bet is that the most capable AI systems are coordinated collections of specialists, not one ever-larger monolith.

So what do you do on Monday?

If you’re shipping a real agent product this quarter, you’re shipping something that looks like Ask David. Hand-wired supervisor, specialised subagents, judge in the loop. The Conductor is a research result with one published benchmark suite. It is not yet a thing you swap into production.

But the directional pull is clear. Two reasons.

• Hand-wiring doesn’t scale with the worker pool. Every time a new frontier model launches, you re-tune your supervisor prompt to know when to call it. A learned router amortises that work.

• The judge layer is also hand-engineered. The same RL approach that produced Conductor will eventually produce learned critics, learned planners, learned reflection steps. The whole orchestration stack is currently a stack of hand-written prompts, and that’s exactly the kind of artefact RL eats for breakfast.

🔮 Prediction

Within 18 months, the supervisor-and-subagents pattern survives, but the supervisor’s prompt does not. The orchestration framework category (LangGraph, CrewAI, AutoGen) splits in two: a structural layer for defining the worker pool and the message bus, and a learned routing layer that you bring as a model rather than as a Python file. Sakana’s Fugu, or something that looks a lot like it, will be one of the early entrants in that second layer. The hand-written supervisor prompt will start to look the way hand-written tokenisers look now: a thing we did because we hadn’t trained the better version yet.

Until next time, watch who’s writing your orchestration prompts, and ask whether that should be you or a model.

Simon Taylor flagged this alongside a broader argument: agentic commerce, as currently conceived, is dead (though he had a vested interest, given he sells a payment solution).

OpenAI supported something similar by admitting defeat and shutting down Instant Checkout in favour of merchant-handled flows. The vision of agents completing purchases inside chat windows is not working. Yet

Meanwhile, Jaya Gupta published an analysis arguing that the scarce asset in enterprise AI has shifted. Models are good enough now (mostly), and the bottleneck is permission: the right to write code, merge PRs, touch production, message customers, trigger downstream actions.

The hard problem in enterprise AI is letting agents act, and traditional RBACs and ACLs aren’t enough.

This article was reviewed and heavily hand-edited, using curated sources I hand-picked, drafted by AI using personalised style guidance. I find it useful to keep in touch with the dozens of X.com bookmarks I come across every few days. Maybe you do too.

Checkout might be the wrong metaphor.

Stripe published five levels of agentic commerce, borrowing from autonomous driving.

Level 1: human picks items, agent pays through to …

Level 5: agent anticipates needs and buys autonomously.

Every level assumes a visible checkout event. The agent grows more autonomous, but the purchase is still a discrete moment you can point to and say “here, money moved.”

Compare with more advanced levels of trust involved in Taylor’s example: a hackathon parking AI agent that just… paid.

You parked, it detected your location, and payment happened invisibly. (I hope it was the right amount, for the right duration, and wasn’t some extortionate amount!)

The Walmart checkout failed because bolting a purchase flow onto a chat interface doesn’t match how people actually transact. Maybe the real end goal of commerce is to become invisible, not agentic. And this is obvious as trust increases for any delegation, right?

The governance trap

I think Gupta’s article identifies a pattern that should be instructive for enterprise buyers.

It works like this:

1. Company introduces a capability that enterprises adopt at scale

2. That capability creates governance, security, and operational burdens

3. The same company sells the layer that manages those burdens

4. Enterprise is now locked in on both sides: capability AND governance

More reliance on the capability means more need for governance.

More reliance on governance means the capability is harder to replace.

That, my friends, is what you call a MOAT.

Google did exactly this with advertising. Built the tracking infrastructure of the web, then proposed Privacy Sandbox (its own privacy framework) when regulators arrived. The company that built the problem wrote the rules for what came after.

Gupta argues Anthropic is running the same play. Ship Claude as the capability layer. Wait for enterprises to hit the governance wall: who approved this code merge? Which agent messaged that customer? Who’s liable? Then sell the permissions layer on top. I’m pretty sure this was the playbook all along actually

No company running this pattern has ever described it honestly. The public language stays at “enablement” and “safety.”

The enterprise buyer’s dilemma

Enterprise AI buyers face a choice:

• Build your own permission layer. Define which agents can do what, with what approval flows, across which systems. Expensive. Slow. But vendor-independent so is a good strategic risk sell.

• Buy it from your model provider. Faster, better integrated, and exactly the lock-in Gupta describes. Vertical integrators will make for an easy sell, bundling things across layers in an attempt to decommercialise competitors (there’s a reason Teams grew so fast, after all).

Many will take door two because the governance burden is immediate. When your agent can merge code and message customers, “we’ll figure out permissions later” stops being an option fast.

For enterprise AI broadly, the next 18 months will be defined less by model capability and more by who controls the permission stack. I’d love to hear from readers already navigating this choice inside their organisations: are you building or buying your governance layer?

🔮 Prediction: Within two years, every major model provider will offer a full enterprise governance suite: agent permissions, audit trails, approval workflows, liability frameworks. Enterprises that adopt early will find them nearly impossible to leave. The permission layer will be stickier than the model itself. The moat is the right to let your agents act, managed by the same company that sold you the agents.

That dependency will be hard to unwind.

Not good enough they thought, and studied the remaining conversations where Billy failed.

What they revealed: customers wanted interior design help. IKEA responded by reskilling customer service staff into design advisors and spinning up an entirely new service line.

Brian Solis shared this story alongside a deeper analysis that frames what enterprise AI deployment should be: a leadership and business model test, and for many organisations, a relevance test.

The 57% Trap

That 57% number is suspect too. “Resolved” in customer service metrics is a generous term: there are 1000 ways this could be used pathologically. Did the customer actually get what they needed? Or did they just stop asking?

The IKEA story matters because it flips the question. Instead of asking

“how much can this AI handle?”

they asked

“what is this AI teaching us about our customers?”

The Organisational Work Nobody Wants to Do

Aaron Levie made a complementary point arguing that there’s a huge opportunity for resourceful talent within organisations to reimagine workflows for a world of agents.

What that requires:

• Setting up unstructured data so agents can actually access it

• Learning existing workflows and creating skills or plans for agents

• Connecting disparate systems together

• Changing the process itself to support what agents need

That last bullet is the one companies often skip apparently: bolting AI onto existing processes and wonder why adoption stalls. As we explored in our coverage of the agent-native internet, the future demands software and workflows fundamentally redesigned for agents, not human processes with AI duct-taped to the edges.

Levie calls this an opportunity for “resourceful and entrepreneurial talent”: be it led from the top,or skunkworks in the coalmine. I know which organisation I would prefer to be in!

The Maturity Scores Tell the Real Story

Solis talks about the ServiceNow AI Maturity Index. In 2025, the average score dropped to 35 out of 100. DOWN. from 44 in 2024. What the hell?!

Solis frames it charitably: companies slowed down because they finally understood the scale of what enterprise-grade adoption requires. Governance, trust, security, compliance, risk. All the boring stuff that doesn’t make LinkedIn posts go viral.

Fair enough, to a point. Some companies did hit the complexity wall and are regrouping thoughtfully. But let’s not romanticise it. Plenty of those drops represent organisations that burned budget on pilot projects, couldn’t demonstrate ROI, and quietly shelved things. The “regression for the right reasons” narrative is comforting. Reality is probably messier.

What’s undeniable is the gap Solis identifies between a small group of AI-native builders and the majority still focused on literacy and point use cases. He borrows OpenAI’s term “capability overhang”: the widening distance between what AI can do and how narrowly most people use it.

Building on what we discussed about agents becoming the primary users of all software, this overhang gets more dangerous once you factor in agent capabilities. Your competitors’ agents might be operating across systems, purchasing services, and executing complex workflows while yours might still be stuck answering FAQs.

So What Actually Works?

The IKEA case gives us a template, even if it’s just anecdata:

1. Deploy the AI. Get it into production. The chatbot, the agent, whatever fits.

2. Instrument the failures. Not just error logs. What are customers asking that the AI can’t handle?

3. Treat failures as market intelligence. Every gap is signal about unmet demand.

4. Reskill, don’t just replace. IKEA moved customer service staff into design advisory roles. The humans became more valuable, not less.

The IKEA AI showed where the workforce should actually be deployed, into higher-value roles the chatbot couldn’t touch.

This connects to something Levie is getting at, too. The people who will thrive in agent-enabled organisations aren’t prompt engineers or AI specialists, rather, they’re the ones who deeply understand the existing workflows and can redesign them. Domain expertise becomes the bottleneck, not technical capability.

🔮 Prediction: Within 18 months, the enterprises winning with AI won’t be the ones with the highest automation rates. They’ll be the ones who built systematic feedback loops between agent failures and business strategy. The 43% will matter more than the 57%. Every unresolved agent interaction is a data point about what your customers actually want. Most companies are throwing that data away.

The competitive threat isn’t AI itself. It’s what the companies who actually study their AI’s limitations will become.

That’s over $8 billion in acquisitions in the agent payments space from “legacy” financial infrastructure companies with due diligence teams paid to be skeptical. They looked at agent commerce and decided the trajectory is now worth billions.

But what exactly are they buying?

The map

Jordan Lyall got tired of tracking agent payment projects by hand, so he mapped the whole thing. The result: 162 projects across 6 layers, from settlement infrastructure up to application-level commerce.

$43M spent by agents already (USDC, a currency-backed cryptocoin). So agents use stablecoins at the moment: is that because stablecoins work better for machine-to-machine payments? Or because the only people running these experiments are crypto natives who already hold USDC? Probably a mix.

Stripe is telling you something

Stripe’s double acquisition is an agent/crypto bellwether.

Bridge gives them stablecoin rails.

Privy gives them wallet and key management.

Stripe clearly wants to be the payments layer for agents the same way it became the payments layer for SaaS. It built its empire on making payments easy for developers. Now it’s positioning to make payments easy for developers’ agents. Makes total sense.

Mastercard and Capital One are sweating: if agent commerce routes through crypto rails by default, traditional card networks get aren’t needed.

Beyond payments: the identity stack

But agents need more than money. Shiv Sakhuja’s tweet catalogued the other primitives being built:

• AgentMail: so agents can have email accounts

• AgentPhone: so agents can have phone numbers

• Kapso: so agents can have WhatsApp numbers

• Daytona / E2B: so agents can have their own computers

• Browserbase / Browser Use: so agents can browse the web

In short, an identity stack for non-human workers.

We’ve covered Aaron Levie’s thesis before: every employee will soon have multiple agents (I use upwards of 12 daily). Those agents become the primary users of all software. The infrastructure for that world is no longer theoretical.

But “fast” doesn’t mean “real.” Most of these primitives are solving problems that barely exist today. How many agents actually need a persistent email address?

Most agent workflows I’ve built use API integrations, not inboxes per se. AgentPhone has a stronger case (voice agents are getting much better), but “agent needs a phone number” feels functionally more like “Twilio with cool kid marketing.”

162 projects, or 12?

Let’s check the $43M number. Tracked across 162 projects = c.$265K / project. Early stages for sure. And most of these projects are pre-revenue, pre-product, or pre-anything really.

The 140m transactions sound better until you notice that the x402 protocol alone accounts for 75 million of them in a 30-day window. One protocol is doing more than half the volume. The “ecosystem” is really a handful of active projects surrounded by a constellation of pitch decks.

So what’s real? The six-layer stack model is useful regardless of how many projects survive. Settlement (payment completion), wallet management, routing, payment protocol, governance, application. That mirrors how traditional payment stacks evolved over decades, compressed into months.

The governance layer (Layer 4, with 32 projects) matters most here. “May this agent spend?” is a question nobody has good answers to yet. Who authorizes an agent’s purchase? What spending limits apply? We probably need a few big meaty disasters to kick in the right guards.

What happens when an agent makes a bad buy? These are unsexy problems. They’re also the ones that guide whether agent commerce scales beyond demos. Ok, when when they do. It will happen for sure.

Follow the consolidation

Last time we covered this theme, we argued that software would need to be fundamentally redesigned for agents. The more interesting development is that finance is being redesigned for agents too. Finance moves faster when incumbents feel threatened.

Stripe is assembling a full-stack agent payments product.

Capital One and Mastercard are acquiring into the space too.

The standalone primitives (AgentMail, AgentPhone, individual wallet providers) are useful today but vulnerable tomorrow.

What’s converging underneath all 162 projects is a single concept:

The agent account.

Compute, identity, payments, communication, all bundled the way AWS bundled compute, storage, and networking into something developers stopped thinking about separately.

Seasoned pros will also be asking “how is this different to a service account?” Great question.

Most of Jordan’s 162 projects won’t survive that consolidation. The problems they’ve identified (authorization, spending limits, accountability) will.

How are you using agents these days?

Token cost reduction: 98%. No site owner configuration needed. Live across the entire network.

Unsexy, essential infrastructure plumbing — deployed across millions of sites without anyone opting in. The kind of change that accumulates quietly until one day agents navigate the web better than browsers do.

Agents as the primary user of all software

Aaron Levie published a long essay this week called “Building for trillions of agents”. It’s one of the more grounded takes on where this is going from a CEO with skin in the game.

His core argument: agents now have sandboxed compute, file systems, long-term memory, and the ability to write and run code for any problem they encounter. When every employee has multiple agents working on their behalf, enterprises will have 100x to 1,000x more agents than people. Agents become the primary user of all software.

The implication is massive. Most software was designed for humans clicking buttons and reading screens. Levie’s riff on Paul Graham is apt: the new startup advice becomes “make something agents want.”

“Trillions of agents” is a big number, though. Levie runs Box, a company that will benefit enormously from agent adoption. But even if you discount the timeline by half and the scale by an order of magnitude, the directional bet is hard to argue with. Software interfaces are being renegotiated. The Cloudflare change is proof that this is already happening at the infrastructure layer.

The retrofit problem

If agents are the new users, how do you make existing software work for them? A team at Hong Kong University has one answer: CLI-Anything, an open-source tool that generates a complete command-line interface for any desktop application in a single command.

The pitch is simple. CLI is the universal interface that both humans and LLMs can use. Text in, text out. Structured, composable, self-describing through --help flags. CLI-Anything scans an application’s source code, maps GUI actions to APIs, then builds a Click-based CLI with JSON output, undo/redo, and a test suite.

A clever approach, though ultimately a band-aid.

Wrapping a GUI application in a CLI layer works for the transition period. But it inherits every assumption baked into software designed for human workflows. An agent doesn’t need a “File > Save As” equivalent. It needs an API that exposes the actual capability without the ceremony.

That said, band-aids have value during transitions. We used REST wrappers around SOAP services for years. The bridge matters while the new thing gets built.

Agents with wallets

AI bigwig François Chollet predicted that AI agents will soon become fully-fledged economic actors. They’ll buy services, compute, and data in the course of accomplishing high-level goals. His timeline: 1-2 years before we see this at scale.

If agents are buying services on behalf of users, existing business models face a brutal reckoning. Agents will optimise ruthlessly. They won’t be swayed by brand, design, or marketing copy. They’ll compare structured API responses, pricing, latency, and reliability. They’ll switch providers in milliseconds.

Chollet’s prediction connects directly to Levie’s essay. If agents are the primary software users AND autonomous economic actors, the entire SaaS pricing model starts to look fragile. Do you charge per agent? Per API call? Per outcome? The industry has no good answers yet.

Simulating the ecosystem before it exists

And then there’s the wild card. Guo Hangjiang, a 20-year-old developer in Beijing, built MiroFish — a multi-agent simulation engine. It generates thousands of autonomous digital humans, each with unique personalities, memories, and social connections, then drops them into a virtual world.

The “God’s Eye View” feature lets you inject events into the simulation and watch cascading effects. Feed it a policy draft, a financial report, or a news article. It extracts entities into a knowledge graph, spawns agents, and lets them interact. Hangjiang built it in 10 days, and a Chinese billionaire invested $4.1 million on the spot.

The underlying simulation engine (OASIS by CAMEL-AI) is a serious multi-agent framework, and the knowledge graph approach via GraphRAG is sound. But “predicting the future” through agent simulation needs years of validation, not a 10-day sprint. The impressive part is the architecture and speed of execution, not the predictive accuracy (which remains unproven).

What MiroFish does illustrate is how fast agent-to-agent interaction is becoming a first-class research problem. Thousands of agents forming emergent behaviours, group dynamics, and cascading decisions — that frontier is where the research energy is moving now.

What’s actually being built

Five layers are forming simultaneously, which is unusual for a technology transition this young:

• Infrastructure (Cloudflare): making the existing web machine-readable

• Adaptation (CLI-Anything): retrofitting human software for agent use

• Platform (Levie’s thesis): rebuilding software agent-first from scratch

• Economy (Chollet): agents as autonomous buyers and sellers

• Simulation (MiroFish): modelling what happens when all of the above collides

Each layer is at a different stage of maturity. The infrastructure layer is shipping now. The economic layer is still speculative.

🔮 Prediction: Agents that can browse, reason, and execute are already here. What’s missing is agent identity and payment infrastructure — the ability to prove who sent them, negotiate pricing, and settle payments without human approval. That layer doesn’t exist yet, and nothing else in this stack works at scale without it. Cloudflare can serve structured errors to agents all day. The hard part is what happens when those agents need to pull out a credit card.

That thesis is accelerating. And the bookmarks piling up in my feed this week tell a story that’s both obvious in hindsight and slightly unsettling: we’re not just giving agents tools anymore. We’re rebuilding entire categories of software, from scratch, for a user that isn’t human.

The Background Agent Goes Multiplayer

Let’s start with Lab13, an open-source project from @ridafkih that is, frankly, the kind of thing that makes you sit back and stare at the ceiling for a minute.

Inspired by Ramp’s internal “Inspect” tool, Lab13 spins up Docker container clusters per session, gives the agent a browser (via Vercel’s agent-browser), lets it watch its own logs and take screenshots, and loops you in through your messaging platform of choice (iMessage supported, web UI included). You describe a project, talk to “The Orchestrator,” and it spawns containers, assigns agents, and starts working. Multiple sessions can run in parallel.

Read that description again. This isn’t “a chatbot with tools.” This is a miniature DevOps platform where the primary operator happens to be an LLM.

Building on what we discussed about stereOS providing hardened agent operating systems: Lab13 takes the same philosophical stance (agents need isolated, purpose-built environments) but cranks it toward a practical, developer-facing product. The warning label is telling, though: “This repository is experimental, please proceed with caution and only deploy in a secure environment.” We’re still in the “don’t point this at production” phase. But the architecture is real, and I suspect we’ll see more projects treat agent sessions as first-class infrastructure concerns rather than afterthoughts.

Agents Get Their Own URL Protocol

Here’s where it gets properly interesting. @OnlyXuanwo dropped xURL (originally “turl,” now rebranded), and it does something deceptively simple: it gives agents a URL scheme for reading other agents’ conversations.

xurl agents://codex/019c871c-b1f9-7f60-9c4f-87ed09f13592

That’s an agent reading another agent’s thread. Let that sink in. You can query by provider, search by keyword, filter by role, discover child threads, and even start new conversations through the protocol. It supports Claude, Codex, Gemini, Pi, and OpenCode.

This is the kind of infrastructure primitive that looks boring until you realize what it enables. Right now, agents operate in isolation. Your Claude Code session doesn’t know what your Codex agent discovered last Tuesday. xURL proposes a world where agents can read each other’s work, pick up where another left off, or review a peer’s output before continuing. The agents:// URI scheme is, in effect, a shared memory protocol for multi-agent workflows.

You can install it as a Claude Code skill with a single command (npx skills add Xuanwo/xurl), which means the barrier to trying this is essentially zero. I haven’t stress-tested it yet, but the design is clean and the ambition is right.

From Tabs to Turns

Perplexity CEO @AravSrinivas shared something that sounds like a product plug but contains a genuinely important idea. He describes using the Comet browser assistant to navigate YouTube videos by asking it to jump to relevant timestamps, and then extrapolates:

“In future, I envision just users asking the AI to consume the web in the way they want to consume: not just retrieval and synthesis; but also the transformation: style, voice, modality. That will likely be the transformation from ‘tabs’ to ‘turns.’”

“Tabs to turns.” That’s a tidy phrase for a massive shift. The web was designed for humans to browse. Agents don’t browse; they query, extract, transform, and act. The entire concept of a “page” becomes irrelevant when your consumer is an LLM that wants structured data, not a layout. xURL’s agents:// protocol is already an example of this: it’s a web for agents, not a web rendered for human eyeballs.

Voice as Agent Interface

@jasonkneen is doing something with the Pi coding agent that made me genuinely jealous. He’s got a local voice bridge running under a separate agent that provides verbal progress updates while the coding agent works autonomously. The agent starts up, checks project history, reports what’s done and what’s blocking, then gets to work. A /auto command flips it into true autonomous mode where it starts tackling work proactively, running tests, creating tools and skills on its own.

The voice layer is the fascinating part. Not voice-to-text input (we’ve had that forever), but the agent narrating its own work to you while it operates independently. It’s the difference between checking a dashboard and having a colleague murmur “just finished the auth refactor, running tests now” from the next desk. That’s a fundamentally different supervision model than staring at a terminal.

The Quiet RAG Pipeline

@victor_explore shared a Google Drive to RAG vector database automation (2,200+ likes, 4,400+ bookmarks, so this clearly resonated). It watches a Drive folder, converts documents to vector embeddings, and updates the database when files are added or deleted. It’s a free, downloadable workflow.

Honestly? This is the least flashy item in this batch, and possibly the most useful for most people right now. While Lab13 and xURL are pushing the frontier of agent infrastructure, the majority of practitioners still need the basics: get my documents into a format agents can actually use, and keep them updated without me babysitting the process. The boring stuff matters.

What’s Actually Happening Here

Zoom out from the individual projects, and the pattern is stark. We’re building a parallel technology stack:

• Agents need containers → Lab13 (Docker-based session isolation)

• Agents need URLs → xURL (agents:// protocol for cross-agent communication)

• Agents need browsers → Vercel’s agent-browser, Comet

• Agents need voice → Pi’s voice bridge (agent-to-human narration)

• Agents need data pipelines → Google Drive-to-RAG automation

Every one of these categories has a human equivalent that took decades to mature. Agents are speed-running the entire stack in months. That’s simultaneously thrilling and a little terrifying, because we all remember how many security disasters, usability nightmares, and architectural dead ends the human web went through before it stabilized.

🔮 Prediction: Within 12 months, “agent infrastructure” will be a recognized category in developer tooling, with its own section on every cloud provider’s pricing page. The agents:// URI scheme (or something like it) will become a de facto standard for inter-agent communication, and the companies that build the best agent-native environments will matter more than the companies that build the best models. The plumbing always wins.

Have thoughts on any of these tools? Tried Lab13 or xURL yourself? I’d love to hear your experience. Hit reply or book a call; let’s compare notes.

Until next time, keep building. 🤘

What do you think? Should I fire Claude? I’ve added my own observations inline

I used to think one agent was enough.

I was wrong.

For months I’ve been building single-agent systems. A chatbot here, a researcher there, some document processor in between. Then I started seeing something in my bookmarks, in the tools I was testing, in what people in the community were actually shipping: teams of agents were starting to solve problems that single agents couldn’t touch.

• One agent can’t split a task across parallel reviewers.

• One agent can’t recover gracefully when it fails.

• One agent can’t hand off context to a specialist and maintain the work.

This matters because it changes what’s actually possible right now, in production, with the tools you have access to today.

I want to share what’s emerging [JH badly 😂]

The Architecture Shift

Previously I wrote about which single-agent framework wins the comparison test. That question’s becoming quaint.

The real question is: how do you orchestrate multiple agents as a unit?

Three agent teams are starting to appear in production. Some teams have six agents. I’m seeing credible system designs with ten or more autonomous agents working together on a single problem.

This isn’t complexity for its own sake. Each additional agent is being added because it solves a specific constraint that a single agent—no matter how clever you make the prompt—physically cannot handle.

Why One Agent Hits the Wall

Single agents are bottlenecks. Not in speed, necessarily. In structure.

Think about research work. An ideal research process needs someone to identify what actually matters. Someone to deep-dive the sources. Someone to fact-check claims against originals. Someone to synthesise.

Squishing all four roles into one agent prompt makes the agent either unfocused or fragile. You can hack around this with clever prompting. But you’re fighting the architecture.

Multi-agent designs let you hire specialists instead of generalists.[JH: missing point here in my experience: biased context is what really matters here]

The Claude Code agent team feature that Dhravya Shah recently highlighted does exactly this: it creates a security-focused agent team where each agent owns a specific concern. Admin hardening. XSS fixes. Rate limiting. Not because you couldn’t write a single prompt covering all three. Because specialists outperform generalists at this level of work.

The Orchestration Patterns That Work

What actually works? Three core patterns are emerging across the tools I’m testing [JH: these are not new and are intuitively obvious. Zero value.]

Sequential handoff: Agent A completes work, hands it to Agent B with full context. Agent Relay uses this heavily. You create workflows where agents pass results down a chain. Clean. Deterministic. Slower but reliable.

Parallel review: Agent A does work. Agents B, C, and D all independently review different aspects simultaneously. Then some coordinator agent synthesises the results. This is what the Khaliq Gant orchestration discussion highlighted as valuable. It’s slower in terms of wall-clock time if you’re running serially, but with concurrency it’s actually faster than waiting for one agent to do everything.

Hierarchical delegation: One agent is the boss. It breaks work into sub-tasks and delegates. Others report back. The orchestrator synthesises and decides next moves. This is messier to implement but powerful for complex problem-solving.

All three are shipping in production now. No single pattern wins universally.

What Changes for Your Work

[JH: This section entirely misses the point that you can’t automate what is not commonly happening, extremely well understood, and is stable. And it repeats very badly some of my previous observaitons on this. Fail. In a way this whole article is a good example of Agents Behaving Badly]

If you’re currently building agents, ask yourself: what roles does my single agent need to play?

If the answer is more than one or two, you probably have a multi-agent system waiting to be built.

You don’t need to go all-in on orchestration frameworks. Agent Relay offers it directly in the platform. Claude Code now explicitly supports agent team creation. The options are multiplying.

Start small. Two agents. One does something, hands off to the second. See what breaks. Iterate.

The jump from single-agent to multi-agent thinking is smaller than it looks once you actually try it.

Why This Matters Right Now

This is the inflection point. Six months ago, agent orchestration was a research problem. Now it’s a production problem. [JH: might be true but GIVE ME EVIDENCE]

We’re not waiting on new models or new frameworks. The architects building the highest-value AI systems are actively solving orchestration problems with what exists today. That tells you this is the frontier. [JH so what?]

Single-agent systems won’t disappear. They’re perfect for focused, contained problems. But the messy, high-value work is moving to teams.

That’s what I’m building next. And it’s what I think you should be experimenting with too.

Have your say: are you seeing agent teams in your work? What’s the orchestration pattern that actually wins?

Let me know.

Sources

This article is based on research from my bookmarks and hands-on testing of emerging multi-agent systems. Key references:

Source: https://www.agentrelay.io/ (Agent Relay orchestration platform)

Source: https://deepresearch.anthropic.com (Claude Deep Research with agent capabilities)

Source: https://github.com/anthropics/anthropic-sdk-python (Claude Code agent team feature)

In this video, I answer the question:

Why is this interesting?

Claude Code is probably the most popular AI Coding Agent for creating any software: in fact, Claude Code is how I make all my AI agents these days.

But it still misbehaves!

I liken it to a genius consultant with drinking problem.

Things have changed though 🥳

On 30 June Claude Code launched a feature, Hooks, which is the gateway to a much more reliable AI coding agent.

In this video, I walk through it in detail, showing how it solves a very specific problem I had.

If you enjoy it, help me make more content like this by liking, commenting and subscribing 🚀

So you want to make an agent.

How useful are agent frameworks anyway?

In a past issue of Making AI Agents, I wrote a tech writer agent without a framework.

This "no framework" agent showed how simple agents can be, yet still have quite magical properties that were sci-fi just a few years back.

To answer the framework question, I implemented the exact same tech writer agent in a selection of different frameworks, and share my insights here.

But first, how many agent maker frameworks are there?

At last count I have identified approximately:

• 133 agent maker solutions

• 46 of which are open source

• 32 of those are in python

• 15 of those are python packages (while the rest run on dedicated servers)

So far, I have rewritten the tech writer agent seven different frameworks:

• Agent Developer Toolkit (Google)

• Agno

• Atomic Agents

• Autogen (Microsoft)

• DSPy

• Langgraph

• Pydantic-AI

Why did you pick the tech writer agent for evaluation?

The tech writer agent answers plain-English questions about a github repo.

You give it a brief, and it will produce a report.

It solves the problem of how to answer a wide range of questions without needing to create different tools. E.g

• Architect: give me an overview of the code base

• New starter: give me a new starter guide

• Release engineering: what is involved in deploying this code?

These all can be answered by the same agent: in the past they'd have required quite different and complex scripts, or a lot of eyeball time.

So the tech writer agent is a great combination of being both very simple, and very useful.

What did I learn?

For a simple agent like my tech writer, the agent frameworks are useful, but not essential.

Their power may come into play with more complex scenarios such as interactive chat or multi-agent support, which I'll cover in future issues of Making AI Agents.

To explain my choices, let me go back to the essential components of an agent.

The building blocks of an agent

There are 3 essential building blocks of an agent:

• The language model

• Tools it calls

• Memory

As such, a good agent framework makes light work of these 3 things:

• Language model: the right language model depends on the use case: they are not set and forget. How easy is it to switch models?

• Tools: ideally I just point the agent at my python functions. Can I use everyday python functions with no fuss?

• Memory: unless it's unavoidable, I shouldn't have to worry about how the agent remembers its actions. Do I have to be bothered with how the agent manages its memory?

When agents answer these three questions well, they also require less code to write.

To this end, of the 7 I initially chose, standout agent makers were DSPy and Agno, both being very compact and flexible.

What did I standardise on?

A rule that in the end added a smidge of complexity to every agent was a firm rule: "use the common functions and constants wherever possible".

See, I had this idea that it would be fairer to the frameworks by having a few standard things:

• Standard prompts

• Standard tools

This turned out to be a mixed blessing:

• Input and output formats did vary a bit, so sometimes an agent would need to do a bit more wrapping than perhaps would normally be required

• It did however ensure that the complex functionality of .gitignore-enabled hierarchical file search worked consistently, and that the prompts were the same (mostly)

Also I standardised on execution: they all use uv, the new hotness when it comes to python package management. Uv was the right choice: it's ultra fast and made it easy to create isolated agent environments to avoid any potential conflicts.

How did I rank them?

All the frameworks have their pros and cons.

While I tested the frameworks with a very simple use case, it was enough to get a feel for the design philosophies each had.

On one end you have "as terse as possible", and on the other you have "as type-safe as possible". (My personal opinion is "as terse as possible" is more idiomatic of python.)

I broke out the seven I evaluated into four broad groups:

• Ultra low friction

• Low friction

• Perfectly acceptable

• Higher friction

Are any really bad? They're all mostly harmless, except I found Atomic Agents type-safe verbosity something I'd not want to repeat.

Where does the value of these frameworks come in then, for my use case?

Ultra-low friction

• DSPy 2.6.17: super concise.

• Agno 1.5.10: compact; could be even more so

Low friction

• Autogen 0.6.1: hard-coded LLM supoprt

Perfectly acceptable

• Google ADK 0.1.0: Google bias harms it

• LangGraph 0.4.8: complex tool definitions

• Pydantic-AI 0.2.16b: complex tool definitions

Higher friction

• Atomic Agents: overengineered for my use case

Google Agent Developer Kit

Browse the source code in my experimental chatbot.

At around 115 lines, this is a good, concise framework to work with:

• It's compact and optionally has server capabilities for running the agent as a server with visual tools.

• Part of a larger visual / server solution: while it can be run in isolation without a server, it's clearly intended by default to be run as a server. It is also a response to Microsoft's Autogen, offering a very powerful web-based studio tool., I expect significant work to happen on this in the coming quarters, possibly further integrated with other tools in Google's Dev ecosystem (like aistudio.google.com, jules.google, and various others).

• Note also that about 10 lines of the agent was comments I thought necessary to help with the new concepts ADK presents.

Downsides are slight:

• Annoying: it doesn't have a standard way to access language models: it has one way for Gemini and one for all others, hence my "stupid_hack_to_get_model()" function. To me this just adds unecessary friction.

• It is slightly more complex in that it needs the concept of a session, which holds memory. This is in contrast to many other agents that encapsulate it away entirely. I respect this abstraction but given for my use case there was no real need to use these concepts other than to pass them back to more ADK APIs, there could be merit in considering a higher level offering that doesn't need sessions or user ids.

Agno

Browse the source code in my experimental chatbot

Clocking in at 72 lines Agno is by far the most compact and cleanest implementation of my tech writer agent.

It could also be made even more compact if it had a universal way to instantiate a language model using the emergent format that many frameworks and tools support, specifying vendor and model id separated by a slash or colon e.g. "openai:gpt-4.1-mini" or "anthropic/claude-sonnet-4.0", such as supported by LangChain, LiteLLM, and OpenRouter (to name a few).

As it stands, Agno requires specific class usage for each vendor, and as my code has a string as input, I have a ModelFactory wrapper to do this hackery directly.

Why did Agno do this? I think there's a misunderstanding here about how language models are used.

They are absolutely not a set-and-forget solution: business-as-usual with AI engineering is to evaluate a range of models for a given use case, so having a set of different models from different vendors is absolutely normal. It's rare I find myself committing solidly to a particular language model at all – even operationally I might want to bac

Atomic Agents

Browse the source code in my experimental chatbot

This is by far the highest friction framework, clocking in at around 224 lines, which is ironic given one of its biggest selling points is being "extremely lightweight".

It leans heavily on Instructor and Pydantic, two very respectable frameworks to help type safety and data integrity, and to that end I respect their approach.

However, writing the agent, it was absolutely the highest friction approach, for instance needing separate classes for a single tool specification.

Honestly there's a point where this really starts feeling like it's moving away from Python idioms to something heavier-weight like Java.

The other big hassle was its fragmentation of prompts into separate aspects, which felt like busywork as ultimately it just stitches it all back together into one piece of text. If you want strongly-typed inputs and outputs, take a look at DSPy which does this very elegantly and compactly.

Autogen

Browse the source code in my experimental chatbot

At 124 lines, this was a middle-of the road implementation.

Autogen was actually one of the first agent frameworks, and it has a very comprehensive Autogen Studio too which I covered briefly in a past issue.

Its LLM implementation relies entirely on a vendor supporting the OpenAI API protocol (which almost all do), with an unfortunate restriction to a subset that it specifically lists.

Relying on OpenAI's API is a completely reasonable thing to do and I'd love to see it open up to be more flexible to support other models and vendors too.

Finally, its tool support is fairly lightweight too; I only had to add async wrappers to the tools, which otherwise just worked.

DSPy

Browse the source code in my experimental chatbot

I place DSPy among the top of my rankings, which is quite remarkable given it's not first and foremost a dedicated agent framework, per se.

It is unusual in that, unlike the others, it's not "just an LLM wrapper".

It has a completely novel approach to specifying prompts that later, with operational data, can be optimised using very sophisticated optimisation techniques.

Clocking in at 99 lines this is a very compact solution because:

• It can use any python function as a tool directly

• It uses LiteLLM for LLM instantiation, so it accepts "<vendor>/<model id>" combos directly

• Like Atomic Agents, it has typed input and outputs using Pydantic under the hood, but also manages to be extremely terse.

• It has a ReAct agent built-in

The only point of friction is what I consider bordering on "docstring abuse": DSPy uses docstrings as a functional source of prompts.

This might look nice, but actually a) docstrings are not functional parts of the code and really should only be used to document behaviour and b) as a result it's a bit of a hack to use external variables as prompts.

To this end, 25 lines of the file is a duplication of the prompts defined in my common utils. An alternative implementation would do this:

class.__doc__ = <TECH_WRITER_SYSTEM_PROMPT>

… but this is hacky, and if you're looking at it as a normal DSPy program, you might wonder why it has no prompt.

Langgraph

Browse the source code in my experimental chatbot

At around 155 lines, this is another decent framework that keeps things simple for the tech writer use case because:

• It has a ReAct agent built-in

• It supports vendor/model configuration strings

Tools could be lower friction though.

Honestly my python isn't good enough to understand why it's a problem, but unlike most other frameworks, Langgraph tools have extra complexity around the context in which a tool operates.

This translates to what amounts to a fairly lightweight wrapper to pass the directory being scanned.

I really tried to understand why Langgraph couldn't figure this out directly like other frameworks, but left it as-is.

Maybe a proper python practitioner or Langgraph expert can improve this.

Pydantic AI

Browse the source code in my experimental chatbot

From the creators of the amazing type safety / object-relational mapping library Pydantic comes Pydantic AI.

My attempt at writing the tech writer in Pydantic AI clocked in at the slightly heavier 123-odd lines of code.

The only reason this wasn't one of the lightest was its specific way to define tools:

• python methods require the @<agent_name>.tool annotation

• Optionally, it also requires a RunContext parameter to pass the directory of code being analysed.

Again, as for Langgraph, I don't understand python scoping rules enough to understand why this additional wrapper was required when other frameworks don't need it, but it translates to slightly higher friction and cognitive load as you have to understand what a RunContext is and why it's required.

Other python agent maker frameworks

In future I hope to cover the remaining 8 python package agent makers I've found so far:

• Ag2

• AgentStack

• BeeAI (IBM)

• Camel AI

• CrewAI

• Griptape

• Semantic Kernel (Microsoft, multilingual)

• Smolagents (HuggingFace)

Other python agent makers

In addition, there are around 15 other open source python solutions, available only, as far as I could make out, as standalone servers. These I'll also assess at some point, but many cannot easily be scripted, they will be a lot more involved to assess:

• Agent-S

• AgentVerse

• Archon

• AutoGPT

• deer-flow (ByteDance)

• dify

• julep

• Letta

• parlant

• pippin

• potpie

• pyspur (multilingual)

• rowboat

• suna

• SuperAGI

• Agent Zero

TypeScript agent makers

Outside Python the second largest set of open source agent makers are those made in TypeScript.

• BaseAI

• Flowise

• Motia

• N8n

• Open-Cuak

Other languages

There are agent frameworks available in PHP, Ruby, Golang and Rust. I'll explore those in time.

Bottom line is, I have taken a principled stance to deliver quality over regularity.

No one needs another newsletter that regularly delivers variable quality.

But EVERYONE needs another newsletter that you always look forward to regardless of when, because you know it’s always so good.

At least, that’s my personal vision (inspired by from greats like Lenny Rachitsky, Gergley Orosz, and Vicky Boykis).

But first, some AI agent news

If agents are becoming more powerful, what will remain for people to do? — an essay by renowned leader on Future of Work, Daniel Susskind.

Agent usage is expanding rapidly. Google lists 601 case studies for agents. Super comprehensive list of agent case studies but with no references, can you trust them?

Agents started with text. But where are we with talking to live, natural-sounding agents over the phone? Zendesk and others say yes:

• It’s now realistic for voice agents to handle frequent, simpler calls so people can tackle the more complex issues.

• Super cheap intelligence from voice and transcripts increase quality and better insights into what went wrong.

• Not mentioned in the article, but voice agent responsiveness and realism have really stepped up in recent months.

I harp on a bunch about AI agent evaluations all the time because if you’re not evaluating, you have no real idea what’s affecting your business pipelines. Deeplearning.ai has released a free short course on “Evaluating Agents”, including

• LLM-as-a-judge (“score this conversation against these 5 characteristics” kinda thing)

• “trajectory evaluations” that compare overall conversational flow direction. Were the right things said at the right time?

Getting started with agents and automation

Thank you to the many people who basically said “love your newsletter BUT how do I get started with agents?” 🕵🏼‍♂️

Soooo this particular issue is about nontechnical agents, in contrast to some past issues being more technical, such as:

• Implementing an agent from scratch in Python, and

• First attempt building a simple “deep research” agent using 12 different agent making tools including coding ones like crewAI and Smolagents.

But it’s all good stuff, I promise!

Recap: different kinds of agents

Let’s start with a recap from a previous issue, describing the different kinds of agent makers there (according to me 🙂).

In this issue we’re focusing the green “No Code Automators” leaf of the diagram. Automators are those that run without you needing to oversee them. We cover:

• Business process maturity and why it matters.

• Creating an AI-enabled automation using Relay.app with Google Forms, Sheets, OpenAI and LinkedIn.

It’s really useful!

BTW I will call these “Business Agents” from now on.

The different levels of automation

But aren’t all automations the same?

No: there are multiple levels. In this issue we’ll be covering the AI-assisted business agent, a powerful starting point.

• Manual: you do things step by step manually. A step may involve computers, but you decide when they’re done. No automation at all.

• Fixed-step automation: you define the steps for a computer to do, possibly with decisions, and the computer runs through with them unassisted.

• AI-assisted: still fixed-step, but some of the steps might involve some AI logic such as “do some deep research on to this topic and prepare a report on your findings”. That step in itself is likely very advanced AI agent doing lots of stuff, but you don’t care, it’s a simple fixed step in your flow. This is new with generative AI.

• Agentic automation (or just “agent”): here, the computer has “agency” — an ability to make its own plans, and learn from them. Agents are primarily you provide a plain English instructions (or prompts) along with some code to bring it all together. These are incredibly powerful and brand new since the introduction of generative AI.

Your first business agent: how do you enrich your inbound sales leads?

We’re going to pick “sales lead enrichment” because:

• Sales lead enrichment is really useful for a wide range of business types

• It’ll give you confidence to try other automations.

Now: the simplest way to get started with agents

Let's get started.

People often buy into this vision:

Vision: "AI agents will cut costs by reducing the number of people needed, and boost the productivity of existing people too"

But WHEN can you do this with AI Agents? This vision is definitely possible, and it's also really important to understand when you can use agents this way.

Spoiler alert: it's absolutely not all the time.

Trying to implement agents for every aspect of your business is doomed to fail. You have to identify where agents are the most useful for your business.

So then let's explore when agents are most useful. Hint: it comes down to business process maturity.

There are a bunch of business process maturity levels: agents are only useful for more mature processes.

“But what are my business processes?”

A lot of the time, how you run your business is in your head: not written down, and probably changes a bit each time you do a thing.

Totally legit! Sometimes, honestly, that’s the best place for them to be. Maybe forever.

But but there are some business processes which probably should be automated as business agents (eventually).

Task one: to do a stock-take.

We’ll with the the key question "what are your most important business processes?"

1. Start with writing down a simple list of business processes

Just writing down the processes is actually a really valuable starting point.

Start with a simple spreadsheet, ordering the most important ones at the top. These will be your initial list of candidates where you can make some agents. (But we’re not there yet!)

At this stage, don’t worry whether they’re an actual process (like “enrich sales leads”) or something more general (maybe “Train internal staff”).

3. Create columns for the following important questions.

We need to ask some questions of these processes. You might not have answers to these yet, but don’t worry. Having them written down in one place in a table is actually a big step towards an agent-run business.

Create columns as below:

1. Who is responsible for the outcome? (“Who”)

2. When does it happen? (e.g. new customer lead arrives in our example) (“When”)

3. How often does it happen (approximately is ok: once a day? 10x a day?) (“Frequency”)

4. What info is needed? (“Input info”)

5. What info is created? (“Output info”)

Tip: those with clear answers to these questions, and happen a lot, are promising candidates for agent work.

4. Flesh out as much info on these processes as you can.

It’s totally ok if you don’t have answers to all these things yet. It’s a normal state for many processes that this is not clear. We’re good.

This is important because we’re starting to document some information about:

• Which processes are important (only implement agents for valuable processes)

• How often they happen (the more frequent, the more likely agents can be useful)

Moving to Level 1: ad-hoc.

Having these written down is an important first step. This allows us to move to Level 1.

Level 1 looks like this:

• The business process is documented (yay! We just did some of this)

• The business owner still decides what to do each time.

• The information needs might vary each time

• There might also be different decisions being made at different times.

Let’s then imagine we have our "Sales lead enrichment" process at level 1:

1. Lead details stored in a Google doc or ideally a spreadsheet

2. Information varies: maybe just a name, sometimes an email, maybe linkedin address

3. Flesh out lead details: learn about the company by researching the web, and tayloring an offering based on what seems most appealing for that company.

4. Drafting up a personalised message based on context (do we know people in common? Have I talked to them before? etc)

5. Post it to them

6. Tracking is opportunistic and reactive: generally wait for them to respond; chasers sent when you remember

Should we use an agent for a "Level 1" maturity process?

Look at the process from the lens of:

• Decisions to make

• What knowledge is required, and

• When to make them…

…then at this early stage, it plainly becomes very difficult for a computer of any intelligence to take over.

In short: no way! Agents need much more consistency.

So what's the minimum required for an agent to step in and do its magic: to transform your business from:

just you

to

an army of agents + you?

Moving to Level 2+

The process needs to be a lot tighter. Here's what you should be experiencing with the process well before you think about agents:

• You know the process like the back of your hand

• You're bored! Because it's the same thing over and over again.

• It feels like "monkey work" where your brain could be better put to use elsewhere.

Ok so how do you get there? Let's run through a checklist for a sales lead enrichment flow as an example.

First: checklist for input info

1. Do you have the sales leads being stored in the same place each time?

2. Is it a digital format? Post-it notes won't do.

   

3. Is it a format where the key information is clearly separated (like the columns in our spreadsheet)?

• Good: spreadsheets with columns for all the information (Name, phone number, email, etc)

• Bad: dumped in Apple notes or Google Docs

Why? Having it clearly separated makes it easier for machines to understand it. Bonus: it's easier to work with too!

Second: checklist for output info

A very simple technique for storing the output info is to store it directly adjacent to the input info. In our case this would mean adding extra columns for our enrichment information we’d like.

In our sales lead enrichment flow, we are seeking senior leaders with an active interest in technology. So we might have:

• Current Position

• LinkedIn Profile

• Seniority: are they senior enough for our work?

• Recent LinkedIn activity: what have they been talking about recently?

• General interests: what topics might they engage with in an outreach?

This is tedious grunt work, clicking on links and making subjective assessments. Manually, this process could take at very best several minutes.

And we have to do this for every inbound lead, which could be dozens a week.
Prime for automation.

Let’s automate this

I’ve evaluted lots of business agent builders, and for this task, I’m choosing Relay.app because it’s incredibly simple and has LinkedIn data access built in (absolutely not the norm!).

(BTW while I go on and on about how cool Relay is, it is not a sponsored post, honest!)

Here’s what we’re going to do then:

• Create Google form to allow people to sign up their interest in an executive briefing on AI Agents.

• Link this to a Google sheet

• Look them up in LinkedIn to get all the information we know we care about

• Do some AI magic to determine the topics they might best engage in in subsequent outreach.

We’ve identified that the following knowledge is really useful:

• Seniority: we want this course for senior leaders only; if they’re not senior enough they qualify out of our process

• Themes of Interest: we want a general sense of what they’re interested in, to make subsequent outreach more relevant to them.

How do we do this in Relay.app then?

Go to https://relay.app and sign up.

It has a useful-enough free tier of 500 AI credits (around 50-60 enrichments in our case).

What you can do with the free tier:

• As you’ll see shortly, our sales enrichment flow costs about 10 “AI credits” to run so you’ll be able to enrich about 50 leads a month for free

• The pro tier is decently priced too at $19/mo (annualised)

This is exceedingly decent: when investigating LinkedIn enrichment (because it’s not built in to Zapier or Make), I saw prices of $60/month+ just for LinkedIn! (PhantomBuster).

Create the Google form with your questions, and connect it to a spreadsheet

Now we go back to Relay.app and create a new workflow, which will take you to an empty workflow page:

Choose “Add trigger” and then search for “Form”. You’ll see Google Forms: Form Submitted.

You’ll need to then click a few times to give Relay permission to access your Google Forms account.

You know it’s working when you see “Run as” with a happy little Google account icon:

Select the sheet you linked in the previous step, in my case “Making AI Agents: sign up for our AI agent executive briefing”.

Now click “Add step” and choose “Search LinkedIn profile for email address”

Choosing the input fields from the form is as easy as pie:

Next we want position, seniority and location. We do that with some plain English instructions. Check out the beautiful ease from which I was able to calculate “Seniority”:

Next we want to figure out what kinds of interest themes the lead has. Again, some very elegant use of language models: notice the “Posts” and “Profile” data sources that the plain English prompt relies on:

Testing is a marvel too, with Relay

Notice the “Test this step” option in the screenshot above? Relay has them everywhere. Why?

Because you absolutely should be testing out each step by itself first, so you know it works:

• Make a change

• Test this step

Testing gives you a whole stack of info about what was returned. Here is the result of me testing a step.

Notice we’re pulling recent posts, and then using that information to identify themes. Effortless.

Once you’ve run it, you have a full flow!

Now we need to update the sheet.

Because of how Google Forms and Sheets work, Relay (or any system for that matter) doesn’t have a link between the two to be able to say “most recent form submitted”.

So we do it by email address with a filter to find the right row to add.

Feeling lazy? Just tell Relay in plain English what you want 🤯 “The email address from the form” is what I typed, and it found it:

Now we can update the rows:

Done! We have an AI-assisted automation!

Sometimes my newsletters are issued at strange times because of th is.

I apologise for the irregularity: but I do promise every time it’s packed full of unique and valuable insights that inspire, inform and now and then entertain.

Hopefully the wait is worth it for you.

Welcome back then to my humble corner of obsession:

Making AI Agents.

The mystique is palpable. Just wave your magic wand and your little robots will do your bidding!

I'm going to build one, step by step, so you can see how the goose lays its golden eggs.

With this approach I hope to dispel any hand-wavy mystique that agents may have for you, with some very real, very specific technology.

The exciting thing is that at their core, agents are very straightforward. Sure, they use technology that 10 years ago was pure science fiction, but that’s what makes them magical.

My definitions:

• The very simplest AI agent is software that makes its own plans, and makes decisions based on those plans. 👈 this article is about this one

• The next simplest one is software that also learns from those plans to make better ones. Future articles will cover this more powerful agent type.

This issue covers the first one.

Spoiler alert, a lot of the agent magic is prompt engineering, and in this issue specifically, we cover the “ReAct” (Reason & Act) prompt pattern:

You should follow the ReAct pattern:

1. Thought: Reason about what you need to do next

2. Action: Use one of the available tools

3. Observation: Review the results of the tool

4. Repeat until you have enough information to provide a final answer

But before that, here are a few must-read articles.

You can pretty much take it as a given that every week or two there are new brains (language models) to use for our agents. It’s an exciting time for agent brains. And the past couple of weeks week is no different.

• Gemini Pro 2.5 Pro is a truly astonishing AI brain. My tests on code generation, particularly front-end visualisations were the best I’ve seen (so far). You can see it actually: I created interactive 3D visualisation of my agent in the issue, “vibe-coded”, simply by pasting the source code and having a fairly short conversation. 🤯

• CopilotKit is a way to build in-app agents into any React app. I predicted this in this past issue of MAIA actually thought can it be a prediction if it already existed and I just didn’t know about it? 🤔 A postmonition maybe (opposite of a premonition).

• Firecrawl is on fire: firecrawl is a web scraping service on steroids. It now has a new method called “deep research” which is basically an agentic web crawler as a service. See a real estate agent demo.

• Julep has launched Open Responses: Julep is one of the 100+ agent maker frameworks out there and Open Responses is an open source clone of OpenAI’s “Responses” API. Things I like Responses for is:

  • Built-in tools (web search, file search, and computer use), and very interestingly,

  • Sessions. That is, rather than sending the message structure every time (like my agent in this article does) but it manages sessions on the server (like normal software for the past 40 years).

On to my agent, with a problem to solve.

The tech writer knowledge issue

Here's my problem: both the AI coding agents and myself struggle to extract useful knowledge from a code base. For instance:

• "I want a quick way for new developers to on-board into the project. So a maintenance and new starters guide." 👶

• "I want an architecture overview in a standard way I can compare with other products we have" 🏢

• "I want a comprehensive list of all the use cases currently supported"

• "I want to know what conventions, standards and technologies are used."

• How do I improve my AI coding agent’s memory of what the code base does? (such as Windsurf, Cursor, Cline, Replit, Lovable etc).

Introducing the tech writer agent.

• You give the agent your tech writing assignment and a code base.

• It prepares a report uniquely taylored to your specific needs.

Here's a demo of the agent in action.

That's the big picture, now let's lift the hood on this thing.

My tech writer agent is single python file using no agent libraries, so if you’re impatient, here’s the full source, hand-written from scratch (by one of my hard-working AI coding agents of course 😉 — but no agent libraries I promise!)

The tech writer agent under the hood

Structurally, the tech writer agent is incredibly simple:

• Prompts: the plain English instructions we give to the language model

• Language Model (LM): the AI part.

• Tools: special functionality the language model uses to get extra information. Local files, web searches, sheep prices in New Zealand, that sort of thing.

• Programming Glue: code to to hold it all together, including ordinary Python variables for to give the agent temporary working memory.

Procedurally, it is also incredibly simple.

• Assemble the prompt

• Set up tools that can be used (in our case just three: file search, file access, and calculator)

• Initialise our short term memory (spoiler alert: arrays of strings)

• Then send a request to the language model with those bits above

• Run any tools the language model asks us to run, add the tool output to memory and repeat

• If the language model has decided it's done enough, then save the result to disk.

That's it.

Wait, a boring static picture? Naaah we can do better than that! Here’s a simulation of the agent’s workflow. Honestly it is overkill but I wanted to see how much I could get away vibe coding this baby.

It’s quite hard to follow on a small screen so save it for later when you’re on a desktop. (I hope to share the source to this but I hit some issues with Replit. I’ll let paying members know when it’s available 🙂)

Prompts: instructions to the language model.

Prompts appear innocently as “just text” but in fact, prompts are exceptionally powerful tool for communicating instructions to agents.

In my tech writer agent I’ve created clear sections for each purpose.

Honestly it’s mostly to help communicate to you, dear reader, what’s going on, but at its most basic level, it’s very useful to separate the ReAct part from the other parts (you’ll find out why in a future issue!)

Is there a best way to create prompts?

Is there… a perfect prompt? 🤨

There are certainly volumes of material on prompt engineering strategies, none of which I read 🙂. Here’s my guidance:

• Test and evaluate different prompts.

• For qualitative improvements, use more AI, repeatedly testing, evaluating and improving until the prompt doesn’t seem to change significantly. I cover this approach in a future issue actually! It’s quite remarkable and very simple. Like and subscribe, baby!

The important part of the prompt

REACT_PLANNING_STRATEGY

This is the magical part. I’m going to spend most of my time on this one. I already included it but I’ll include it again because it’s THAT important:

You should follow the ReAct pattern:

1. Thought: Reason about what you need to do next

2. Action: Use one of the available tools

3. Observation: Review the results of the tool

4. Repeat until you have enough information to provide a final answer

Is this the ultimate ReAct prompt? Actually I did do some research on this and it could potentially be made a lot simpler. Potentially down to this:

Answer the question by reasoning step by step and using tools like <tool x> or <tool y> if needed

I’ll leave that as an exercise for you to try out if you feel so inclined 😉

Oh and if you’re curious, here’s some meaty research that came up with the ReAct prompt from 2022.

And yes, there have been improvements — stay tuned for that in another issue!

Subscribe now

The other parts of the prompt

This stuff is way more arbitrary and you can just let the prompts wash over you like a warm relaxing shower 🚿, with your eyes closed even.

ROLE_AND_TASK

This really sets the solution space: This is the exact prompt:

• You are an expert tech writer that helps teams understand codebases with accurate and concise supporting analysis and documentation.

• Your task is to analyse the local filesystem to understand the structure and functionality of a codebase.

GENERAL_ANALYSIS_GUIDELINES

Follow these guidelines:

• Use the available tools to explore the filesystem, read files, and gather information.

• Make no assumptions about file types or formats - analyse each file based on its content and extension.

• Focus on providing a comprehensive, accurate, and well-structured analysis.

• Include code snippets and examples where relevant.

• Organize your response with clear headings and sections.

• Cite specific files and line numbers to support your observations.

INPUT_PROCESSING_GUIDELINES

Important guidelines:

• The user's analysis prompt will be provided in the initial message, prefixed with the base directory of the codebase (e.g., "Base directory: /path/to/codebase").

• Analyse the codebase based on the instructions in the prompt, using the base directory as the root for all relative paths.

• Make no assumptions about file types or formats - analyse each file based on its content and extension.

• Adapt your analysis approach based on the codebase and the prompt's requirements.

• Be thorough but focus on the most important aspects as specified in the prompt.

• Provide clear, structured summaries of your findings in your final response.

• Handle errors gracefully and report them clearly if they occur but don't let them halt the rest of the analysis.

CODE_ANALYSIS_STRATEGIES

When analysing code:

• Start by exploring the directory structure to understand the project organisation.

• Identify key files like README, configuration files, or main entry points.

• Ignore temporary files and directories like node_modules, .git, etc.

• Analyse relationships between components (e.g., imports, function calls).

• Look for patterns in the code organisation (e.g., line counts, TODOs).

• Summarise your findings to help someone understand the codebase quickly, tailored to the prompt.

QUALITY_REQUIREMENTS

When you've completed your analysis, provide a final answer in the form of a comprehensive Markdown document that provides a mutually exclusive and collectively exhaustive (MECE) analysis of the codebase using the user prompt.

Your analysis should be thorough, accurate, and helpful for someone trying to understand this codebase.

Language model

This is not the language model itself, but it’s basically some glue code. You could say rather grandly “orchestration” but really it’s just a loop, some variables for memory and some API calls.

The important part.

run_analysis

THIS IS IT.

This is the loop that drives the agent as we know and love.

We limit the number of steps to something sensible (it’s set to 15 in my code but 6 or 7 is probably fine) but basically it does this:

• Prompt to memory: language models have no memory by themselves so we need to keep track of all activity and repeatedly send it back to avoid instant amnesia. So we set up an a “memory” (an array, let’s be honest) with

  • All the system prompts mentioned, slammed together, then

  • Loading it up with the end user’s request

• Code base location to memory: it then also adds to the memory where the code is on disk.

• Then it loops around:

  • Give the language model its memory. It’s always thankful.

  • The language model’s response is only one of two things:

    • Final answer: we can stop and write the report which is helpfully contained entirely in the “content” part of the response.

    • Run some tools. The language model response obligingly includes parameter values too (e.g. which directory to search, which file to open).

  • When we have tools to run, the results of the tool calling is added to memory.

That’s it.

The less important stuff.

call_llm

Just a robust wrapper for calling OpenAI. Nothing to see here.

check_llm_result

This is exceptionally simple and probably badly named. If the OpenAI response has some tool calls to make, then that’s what it asks to do. If no tool calls are present, it assumes that we have a final result. Turns out this is fairly reliable (like, always, in my experience).

Tools: allowing language models to access the real world.

The most interesting tool functoin

execute_tool

The language model response is a JSON structure telling us what tool to call and with what parameters. This function figures out which tool to use (it’s really not hard, it’s a map).

Various agent frameworks have glue that do automatic mapping to python functions.

I don’t have that luxury so I do did it manually with a setup function called create_openai_tool_definitions. (Ok I lie, Windsurf + Claude 3.5 wrote all the code so it wrote this function by sniffing around in Langgraph to see how it did it and pulled bits out — all I had to do was encourage it to think like that 😉)

The workhorse tools

These are the interfaces to the rest of the world. For a code base analysis tool we really don’t need much: finding files, and opening them up. I threw in calculate too in case it needed to do some calculations.

find_all_matching_files

read_file

calculate

Why can’t the language model calculate itself?

Because it’s a token prediction system.

Yes it does billions of calculations a second to predict those tokens, but for specific calculations you need a tool.

You can certainly ask a language model to do maths, but once you go past you ten times table it gets really shakey. Because patterns (although an interesting article by Anthropic suggests it’s more intelligent than we thought).

Programming glue

Really almost nothing to see here. 🥱

analyse_codebase

• Read the prompt file

• Run the analysis

save_results

• Write a markdown file to disk, that’s it.

So that’s it.

It was that simple. Ok well maybe not super simple but in the grand scheme of things, the power agents offer compared to the code required to write them is truly, truly remarkable.

• Prompts with clever looping in them

• Tools to do the actual work

• Language model to decide which tools

• And a few lines of “glue code” bringing it all together.

What’s missing then?

Heaps. But I think that’s enough for now and I’ll leave you with some questions:

• Is ReAct the best prompt? 😯

• How do different language models affect the quality?

• Can multiple agents help? I heard about “LLM as a Judge”? Would that help?

• How can an end user improve their prompt?

• Plus a lot, lot more!

See you next time!

Recap of topics covered in previous issues

Making AI Agents is designed to offer long-lasting content. Previously on Making AI Agents we have:

Jan 28 issue:

• Checklist for making an AI Agent

• Evaluating AI Agent quality with The Agent Company’s benchmark

• Deepseek R1 language model

• The huggingface agent course

Feb 12 issue:

• Agents vs bots vs scripts

• Different ways agents can be integrated into different products

• Benchmarking agent performance with HuggingFace DABstep (450 tasks)

• Deep Research preview

• AI Agent workflow building blocks

Feb 26 issue:

• Introduction to Model Context Protocol (MCP) and why it’s taken off like wildfire

• 38 new agent makers added to my list

• Deep dive: attempting to replicate Deep Research experience using Google Gemini, ChatGPT, X.com Grok, and Perplexity, along with my experience trying to build a Deep Research agent with Relay, Make, n8n, SmythOS, Autogen and CrewAI.

The AI agent maker landscape

What kind of agent is best for you? Let’s dig into the different kinds, and how they’re useful.

At last count I know of around 100 agent maker solutions. Most will die. I see segmentation roughly as per this structure:

• Enterprise SaaS APIs. Built-into major cloud vendors, these are programming toolkits for building agents into your business to service your customers.

• No-code agentic business process automators (no code internal automation workflows)

• Turnkey agentic vertical and business functions: these work "out of the box" as it were, ready to use with no design required.

• Open source developer toolkits: mostly (but not exclusively) python. Usally requires at least being comfortable with the command line and ideally comfortable with running and writing python programs.

• General Personal Agents: ChatGPT, Claude, Perplexity, and Manus

• Agentic developer productivity: I carve this out because it’s by far the largest and most successful use of agents today.

Enterprise SaaS API : default Big Tech offerings

Enterprise SaaS APIs are managed solutions, offering support for a wide range of languages (vs Python only for many other agent makers).

They have all the quality aspects such as stable versioning, security, and a stack of value-added features like prompt asset management, tracing, etc. Amazon Bedrock Agents is an early leader in this.

🔮 Prediction: I expect enterprise customers to go with the default built-in agent solution the Big Tech companies offer.

• Amazon Bedrock Agents is a closed-source solution, deeply integrated into AWS with pretty impressive enterprise features including multi agent support. HOWEVER no access to the full gamut of LLMs (notably no access to OpenAI or Google Gemini and this is a serious problem IMHO)

• Microsoft Autogen Studio is currently open source but I expect will become a core Azure offering enterprise quality features like security, resource management, control over data residency etc.

• Google’s AgentSpace (early access as at March 2025) confuses me as it seems to be basically Deep Research For Your Docs (at least currently). That’s it for Google’s agent offering as it stands. Kinda feeble actually.

🔮 Prediction: one or two new brands will prevail in the Enterprise space such as crewAI. Some of those will end up being acquired by late adopter big tech like IBM and Oracle.

Custom agentic business process automators: internal workflow automation

These are for small groups of people to automate their work through point and click interfaces. For example:

1. N8n.io is very popular though it can’t do web search or scraping as-is without low level HTTP hackery

2. Make.com

3. Zapier agents

4. Relay.app

5. SmythOS

Note all of these have been covered in previous issues of Making AI Agents and are dozens more interesting ones to cover in future issues.

Agentic vertical solutions: prepackaged agent teams specialising for every vertical

🔮 Prediction: every vertical will have prepackaged agent solutions for their business, e.g. “as a real estate vendor I want to monitor competitor websites, identify new listings we don’t have and automatically add it to our database for record keeping and outreach”. These verticals might indeed be very niche, e.g. “electrical contract bidding for medium-sized construction business”

Agentic business functions

• Cross-vertical. eg aomni for sales lead generation, customer support, etc. This is already an enormous market and you can expect every existing business function to offer agentic support, a bit like Salesforce Agentforce

• Subsegment: in-app agents for every app (moving from from help => do). This is HUGE in the way that Intercom is huge. Intercom Fin is likely going to lead this. I cover this in more details in this previous issue

Open source toolkits

Open source is the most fertile ground for innovation. It’s a great place to do a number of things:

• Learn how agents work

• Try out the state of the art (SOTA) agent methods

• Build in agents into your own software

• Use agents for free

There are dozens and dozens of these, and new ones seem to sprout up every day.

🔮 Prediction: just like JavaScript frameworks, there will always be new ones, and there will be 2-3 that really dominate. I’ll be sharing my directory of these in time.

General Personal Agents

There are a handful of universal use cases emerging like deep research that tools like ChatGPT, Grok, Claude, Gemini, Perplexity, Manus (see below) etc will own. These are personal assistants that help individuals with their personal lives.

⚠️ Controversy: delegating too much of your own power to a personal agent can be extremely risky and there is a vocal audience that believes this is incredibly dangerous.

🎤 Opinion: there are proven, carefully managed ways to delegate privileges, so I see two simple ways:

• Anonymous web scraping

• Oauth-based explicit privilege sharing that is carefully managed and monitored by an identify provider (like Google or Microsoft).

Agentic developer productivity

• Top down / cloud first: Replit, Bolt, v0, Lovable, Databutton etc. You start with a prompt and the entire app (web or native mobile) can be created and refined with a plain-English conversation. I’ve tried them all and I personally use Replit the most as it’s the most integrated, for small tools, MVPs and prototypes.

• Bottom up / local first: Cursor, Windsurf, Cline, etc. These are developer environments that have less guidance and less automation. Where Replit will automatically create databases and offer one click deploy, these bottom-up tools provide much more control over the software engineering process. I’ve tried them all and I personally use Windsurf the most, for more serious software engineering projects that need to accommodate more robust software engineering principles like comprehensive multi-layer testing and direct support for any language.

🔮 Prediction: I expect top-down and bottom-up to converge and become much cheaper and more reliable through various means:

• Better automation of self-diagnosis (automated testing)

• Mult-agent teams to build the code for improved quality and scope (with architect, tester, manager, QA roles etc)

AI agent news worth knowing

Productivity

China’s Manus.im is a General Personal Agent that has taken the (tech) world by storm with rumours of a waitlist of over a million people.

What’s the big idea? Fundamentally it seems to be able to do everything other agents promise, but better:

MIT Technology’s article, Everyone’s talking about Manus. We put it to the test says “Promising but not perfect”. There was some observation that actually it’s “just” a clever prompt engineering on top of Claude 3.7.

Successful products create “defensible moats” around them: barriers to entry that make it hard to replicate.

In Manus’s case OpenManus was created within days of Manus’s announcement. How popular? How about 33k Github stars in 4 days (cf popular agent framework CrewAI collected 28k stars in 18 months).

But OpenManus any good? WorldOfAI says you can create pretty cool things, albeit it’s a bit slow. Well, OpenManus is precisely 4 days old, so maybe we can wait a few more days to see speed improvements 🤣

Future of work

• UK government wants to replace some civil servants with AI agents. “No person’s substantive time should be spent on a task where digital or AI can do it better, quicker and to the same high quality and standard.”

• Speaking of governments: Pentagon to give AI agents a role in decision-making "Accuracy and reliability are core design principles, and maintaining human oversight is critical,"

• Google Launches a Data Science Agent: now you can analyse data with plain-English prompts. To be clear it seems to be better thought of as a productivity tool for data scientists as the example prompts were very technical.

Language models: the brain inside AI agents.

You can use language models:

• through an API in the cloud, or

• hosted locally and privately on your own hardware by downloading the model (or “open weights”).

There were three similar-sized, open-weight model launches recently:

• Google launched Gemma 3: contrast Gemma with Google Gemini, its more powerful, API-only language models.

• Qwen China’s QwQ-32B.

• OLMo 2 32B from the Allen Institute (Ai2).

They’re all noteworthy:

• Gemma & QwQ are very good performers

• Gemma has a larger “working” memory (128k context window vs 32k for Qwen). 128k is is state of the art for open weight models (vs SOTA 2 million for Gemini)

• Gemma also score better on important benchmarks that are designed to test their ability to do maths and coding.

• OLMo 2 32B is important because it’s fully open source (data, code, and downloadable models) AND Ai2 claim it’s the first fully open source model to beat GPT 3.5 and GPT 4o mini. It’s not clear what the context window is but it’s probably similar to QwQ

🕵️‍♂️ What I’m looking out for: great quality open weight code generators. A hugely important usage of language models today is code generation, but no open weight models are good enough yet.

🔮 Prediction: when local AI coding agents work well (I expect within 12 months) this will likely significantly reduce the chance of “bill shock” where a software developer finds they burned through $10 of AI Agent credits in a few hours (happened to me! 😱)

Have your say!

Want this to be the best newsletter ever? I’d love to get your feedback.

• 1:1 call: if you’re in North America, UK or Europe, let’s chat!

• 20 second survey: what’s important to you about making AI agents?

It’s hard to think that just 3 months ago I was pooh-poohing agents. And, this week, I used an agent…
… to analyse other agents…
… to come up with a best practice agent specification…
… I then used another agent…
… to create an agent…
… using that agent spec…
…which then I now use for my own agent research 🤯

Welcome new subscribers! Such tremendous growth from just a few issues of Making AI Agents: humbly appreciated 🙏

Also thank you existing subscribers for your feedback! You are helping this become the best publication possible.

I’ve had numerous video calls and received great insights from the feedback form.

It’s not too late if you still want to provide feedback, links at the end 👇

Important news

Latest Microsoft Autogen Studio is neat

One of my favourite discoveries this week was the latest version of Microsoft Autogen Studio. Here it shows visually the execution flow of one run.

Language Model tools are becoming richer, more accurate and more powerful with Model Context Protocol (MCP)

• Language Model “Tools” are an essential way to improve the quality of language models, through fresher and more accurate information and specialist functionality.

• Anthropic’s Model Context Protocol (MCP) is a standard way to access this functionality

• For example, imagine you have specialist data, say, house insulation in the UK. Without MCP, all agent systems would need to do custom code to connect to your service. With MCP, there’s a standard way for those agent systems to connect more easily.

• It’s a new protocol and aspects such as security still need to be fleshed out (as it was originally designed as a desktop plugin solution to Anthropic Claude).

• Prediction: 🔮 I expect to see MCP to rapidly accelerate and become a standard part of the AI Engineering and AI Agents ecosystem. For example accessing proprietary data from data brokers like Redpine will become trivial to implement.

Language Models Never Sleep: Claude 3.7, Grok 3 beta and Google Coding Assistant preview launched

AI Agent intelligence relies on the language model you use. So when language models get smarter, your agent gets smarter.

Anthropic Claude 3.5 has been king on a number of benchmarks (particularly coding) for 7 months now. And the conservatively numbered Claude 3.7 has knocked the socks off that…

X.com’s Grok 3 beta also launched, and is offering some pretty attractive features like built-in Deep Research. Available — at least for now — for free.

Developers are experiencing “bill shock” when using AI coding agents intensely for software development. Google Gemini Code Assist launched a public preview, solving this by making the agent free at least for now.

38 new AI agent makers added to my list 🤯

I added 38 more agent making tools to my agent maker list since last publication 2 weeks ago — on top of the 60 I already had 🤯.

Some are not new (Rasa) and some are (Cloudflare Agents launch 25 Feb):

• Cloudflare Agents

• Typed AI

• Retool

• Factory.ai

• Airkit

• IBM BeeAI

• Langflow

• Swan

• Unitron

• Vessium

• AgentFlow

• Rasa

• AgentX

• Parlant

• Amazon Bedrock Agents

• Beamlit

• SpinAI

• Mastra

• Magma

• Autoagent

• Lindy

• Gumloop

• The Answer

• Tate a Tate

• Aident

• Relevance AI

• Fuzen.io

• CallFluent

• Relay

• Bizway

• DAGent

• Cogniflow

• Scout OS

• Superagent

• Squid

• Beam.

(I’m making a dedicated directory, which I’ll announce soon.)

Other recommended articles

These capture important sentiment around AI Agents.

• "AI agents will outmanoevre salespeople by optimizing persuasion" source

• "The next 10 years will be about the AI Agent Economy" source

• Patterns for agent collaboration: Functional, Communication, Role and Organisation. We saw some in a previous issue, here are some more

Have your say! ✍️ 💬

Want this to be the best newsletter ever? I’d love to get your feedback.

• 1:1 call: if you’re in North America, UK or Europe, let’s chat! 💬

• 20 second survey: what’s important to you about making AI agents? ✍️

And now on to the deep dive section. 🤿

What’s Deep Dive? I’m experimenting with offering two sections to my newsletter:

• Important News, and

• Deep Dive (long form)

Deep Dive is where I dig right into the weeds and share a lot more insight. I’d love to get your thoughts about the value of this structure. Feedback at the end 👇

A Deep dive into deep research agents

NOTE that this article is so long your email may truncate it. So do be sure to click “View Entire Message” or view it online inside Substack to see it all.

Deep Research is all the rage: I mentioned it briefly in the last issue and since then just a short two weeks later, we have everyone scrambling to offer “deep research”. Read on for my Deep Research experience using Google Gemini, ChatGPT, X.com Grok, and Perplexity, along with my experience trying to build a Deep Research agent with Relay, Make, n8n, SmythOS, Autogen and CrewAI.

“Deep Research” is a great example of an agentic process we can learn from:

• You give the deep research agent instruction

• It goes off, using various tools, spending several minutes or even hours researching various resources

• It then brings it together in a comprehensive multi-page report with references.

Deep research is a conversational AI agent that spends minutes or even hours scouring the web then bringing it together making a comprehensive multi-page report.

• Tired 🥱: asking an AI for an answer and getting a response quickly

• Wired ⚡️: ask an AI for a deep research answer and getting a response really slowly

This first generation of deep research agents are a big improvement over conventional AI conversations.

Below are the commercial Deep Research solutions I could find.

“Tiered” means “allow people to use Deep Research a limited number” at some level, with more access expanding the more you pay.

• Google Gemini (naff off you have to PAY)

• Perplexity (tiered from free)

• Grok 3 beta (tiered from free)

• OpenAI (tiered, from ChatGPT Plus as of 25 Feb — this is great because prior to this you had to pay $200/mo for any access to Deep Research at all)

All of them do a far far better job than a normal chat with an AI: in particular they’re much better at the accuracy of the output (though not perfect).

The results: they’re all pretty impressive

They’re all pretty good. I’d say overall, it’s a photo finish between:

• Perplexity: the most accurate and best user experience, and

• Google Gemini would probably have the most comprehensive response (one research task I did Google searched 668 websites!) … but not always.

BIG caveat: it’s a rapidly evolving space and you’ll probably do well using any of them at this point.

I used them for a range of topics over the past week or so. For this article, I chose a niche topic I know well: AI Agents. This is a great way to evaluate deep research because:

• I know the AI Agent space well already

• I have a sophisticated task specification (“prompt”) that tries to defend against problems I had in the past with things like including unrelated product types in the results.

In a bit more detail then:

• ChatGPT Deep Research: generally has some immediate follow-up questions to clarify the scope. Pretty sensible actually and will definitely end up with a better result. Didn’t really cope well with refinements: eventually it kinda gave up and stopped responding.

• Google Gemini: annoyingly doesn’t consolidate the results by default so if there are errors, you now have two separate reports, and the second one is just the changes.

• Grok 3: impressive results particularly for a “prerelease beta”. I did find it hardest to get the actual results out of Grok but I’m sure that’ll improve over time.

• Perplexity has a nice “Auto” mode where IT decides whether Deep Research is warranted. This is great because the moment you give the user a choice, they’ll get it wrong some of the time. Also some research tasks I did, I found Perplexity was better at deep-linking to precisely the relevant result. E.g. searching for “bulk cacao providers in London”, Gemini confused cacao with cocoa where Perplexity linked right to the precisely result.

They need human review.

None of these tools should be trusted to provide truth straight out the gate. They should be considered a draft that then needs to be carefully reviewed with references provided as I found a number of errors.

For example, they all snuck in products that weren’t AI Agents (e.g. infrastructure like memory or hosting or ops) despite seeing assessments to qualify them or not. Even very specific requirements. E.g.

They’re good for small research tasks (10-20 sites) but reluctant to do larger sets

Almost all my queries were “landscape” questions for competitive analysis and opportunity assessment and the first version of the report would work fine for 10-20 items but it took some coaxing to get it to increase, with varying levels of success.

How do you customise deep research then? Take a guess, go on…

These tools are really useful general purpose research tools. They all have their strengths but they all have one problem in common: they’re opaque. You cannot customise them.

For example, here are some things I’d like to do that I don’t think can be done with simple prompts:

• Connect the output automatically to another place such as a Google Spreadsheet instead of a rambling unstructured 10 page doc

• Compare against my own data provided in a spreadsheet. Even better, update it directly.

• Include searches to paid subscriptions my company has.

• Access resources that only work reliably through specialist APIs (e.g. Reddit)

• Better access control for users.

So how do you do that? Custom agents, that’s how.

Here I introduce Relay, Make, n8n, SmythOS, Microsot Autogen and CrewAI. This field is rapidly evolving, so even though I had no success with many of these tools, I will be checking regularly and seeing how they improve.

In this first batch of AI agent makers I tried to replicate Deep Research. Milestone 1: basic web search and summarisation.

Sadly, even with that the task proved challenging and I could only make even milestone 1 in two of the tools.

I expect this will change in the coming months and quarters though, so stay tuned!

Relay.app: promising but no deep research yet

Relay is probably the easiest UI to work with and I’m very much looking forward to where Relay goes with their product. HOWEVER my first take on Deep Research failed: “blocked by network security”.

Here’s the issue I encountered:

• the first node did a web search (works!), then it sent each URL to a scraper (very clear design 🥇)

• The scraper failed because Reddit is notorious for blocking scrapers. So Reddit is a special case that needs to be processed separately.

Next time I’ll try and do some kind of filtering and get special keys for Reddit.

Make.com: confusing opaque errors made it incredibly hard to figure out

First up: Make.com has a completely gorgeous interface 🤤 Just… amazing. And its integration tools are suuuper impressive too. But digging into it, I got stuck with no way out.

This is one of the biggest issues with no code tools: if you get stuck, the nice GUI protecting you from the tech complexity often hinders you from the detail needed to troubleshoot.

n8n.io: no built-in scraping.

N8n is also a really gorgeous, well-designed no code AI agent creator.

Sadly, for Deep Research specifically it didn’t get out the gate because it doesn’t seem to offer any search or scraping tools out of the box.

The only way is as a tool through an LLM (which wouldn’t work for this case).

SmythOS: got tangled up in loops

SmythOS has an agent to help making the agents (see previous issue), but I could not for the life of me figure out how to do a basic loop. Support hasn’t got back to me yet either 🤷 Totally expect this to be fixed and I’ll update you.

Microsoft Autogen Studio 0.4.7: has a Deep Research template.

Autogen Studio as it is is a much more technical solution currently.

It’s an open source project that runs on your machine locally and you need to be comfortable with command lines, API keys and that kind of thing.

I cover it because because a) I fully expect autogen to be a hosted Azure solution eventually, and b) autogen offers a few unique things of the things that make it very appealing:

• It has a visual builder but also generates code that can be deployed elsewhere (autogen agents can be written with no UI at all; the UI is a layer on top)

• It has a really nice visual trace feature for tracing task execution.

Gumloop: promising

Another product rapidly evolving. I got basic scraping working fairly easily and the Gummie agent was absolutely instrumental to this as I pretty much just asked it what to do.

There are some connectivity issues (my Google Sheets destination just failed I’m sure it’s easy to fix, and the CSV writer kept writing JSON files again likely an easy fix).

Making my own CrewAI deep search agent in python with Windsurf Cascade and Claude 3.5 Sonnet

Ok now we’re REALLY rolling up our sleeves with a hands-on attempt to use CrewAI to build the agent.

CrewAI is another AI agent biggie:

• it has a programming toolkit and command line support for running crews of agents

• It offers a GUI called CrewAI studio but as far as I can tell it can’t be run locally so I didn’t use it for this project.

Here’s how it normally goes making a CrewAI crew:

• Go through the tutorials and howtos to learn the CrewAI concepts and the command line tooling required to create and run crews.

• Write all the code

• Run it

This is what I did below. I’m so lazy 🤣

• I asked Grok Deep Research to explain to me what the “Deep Research” algorithm is

• I then asked it to “compare and contrast the various deep research open source projects out there on GitHub”. I have a list of these so I could compare.

• It missed HuggingFace’s one so I asked it to add it, which it happily obliged.

• I then asked “Ok create a state of the art deep research team using the crewAI.com framework: what the crew, agents and tasks should be to be state of the art.”

• It generated some crews.

Looking at the code it generated for crewAI I wasn’t super convinced it’d work so I thought I’d take another tack. I asked this:

using all the research so far, reconcile it to create a mutually exclusive, comprehensively exchaustive specification for what a deep research agent team be, how they interact and what they should proudice. This should not be specific to a particular agent framework, but a general one that can be customised for a specific framework by simply adding "do this for <framework x>".

I then created a markdown file that captured this and inside Whisper + Claude Sonnet 3.5

• I asked it to create a Crew based on the spec. It failed miserably and got really tangled up.

• So I asked it to create a generic crew and then modify it according to the spec, and that seemed to work.

Using Windsurf’s conversational coding agent Cascade to create a CrewAI crew using just plain English was a breakthrough for me.

It even helped me revise the output to include URLs (as v1 didn’t).

Thank you for reading this far. I’d love to get your feedback: fill in the poll below!

Your vote matters…

Have your say! ✍️ 💬

Want this to be the best newsletter ever? I’d love to get your feedback.

• 1:1 call: if you’re in North America, UK or Europe, let’s chat! 💬

• 20 second survey: what’s important to you about making AI agents? ✍️

Also, each issue is designed to have lasting value, so if you missed out on previous issues, feel free to catch up here.

Grab a coffee, settle down, and sit back. This is going to be a good one! Feedback welcome, link at end 👇

Recap: agents vs bots vs scripts: know the difference

Tired: bots 🤖 and scripts 📝
Wired: agents 🕵️‍♀️🔎 (ok not that kind of agent)

What's the real difference between bots, scripts, and agents? Fundamentally:

AI agents are better than bots and scripts because of their ability to take plain Engilsh instruction to autonomously create their own plans, act on them with tools, reflect and revise.

Just for fun though, tools that run through a fixed set of steps are also called agents. (Actually as far as I can tell, expect every marketing department to bend over backwards to present some kind of “agentic” capability, no matter what it actually does. 🤦‍♂️)

Agents can be integrated into your existing products in many ways.

I’m seeing agents being deployed at many different levels, a lot of them being an additional method of accessing existing tools.

• In-app “micro agents”: everywhere inside tools just when you have a question.

• “Product” agents: automating substantial portions of a specific product or app

• “Product bundle” agents: an agent that helps you bring together different apps in a product bundle.

• “Cross-product” agents: the most common currently: wire together inputs and outputs for different services to create your own flow.

In-app “micro agents” everywhere inside your apps to reduce customer support

This is a really powerful retrofit to ANY product you own right now. It goes like this:

• The existing UI offers a chat box users can give instructions to

• The microagent carries out the task for you.

I think this is quite a shift. For example: an “Ask AI” button to help when someone gets stuck in The Unicorn Platform below. I wanted to make sure links opened up in a new window and it made the change in the appropriate section for me:

Product agents: bootstrap instant productivity by letting the user just say what they want to do.

Another pattern emerging in popularity is a chatbot that drives the UI for you from the very beginning of the journey.

• Your users don’t need to know anything about how to use the tool, they simply say what they want in plain English.

• This means instant productivity and subsequent reduced training costs.

Example 1: Replit Agent

Example 2: Smyth OS agent creator

Smyth OS is a recent discovery for making agents visually. Even better, Smyth OS has an agent to help you create an agent 🤯

Start of flow screen shot:

Smyth OS next screen: all the boxes and lines shown here for the agent were created from my prompt in front of my very eyes. I could get more done if I liked through the chatbot.

Example 3: Microsoft Dynamics 365 built-in agents

Microsoft added an agent layer on top of very specific product areas of Dynamics 365: sales orders, supplier communications, financial reconciliaton, accounting reconciliation, and time & expenses.

These are:

• Super niche agents

• Incredibly boring for people who don’t do these tasks daily 🥱

• Incredibly valuable for those who DO these tasks daily 🥳💰

Cross-product agents

This is the big one 🧨: cross-product are those that automate business workflows across multiple products from different vendors. Smyth OS is one I showed already that is a promising agent maker tool. Others include n8n.io, Zapier Agents, and Make.com, and there are many others. I’m building a directory of them all: stay tuned!

Future: product bundle agents will increase bundle value

This is speculation 🔮: I expect a standard part of any bundle of products to offer tightly integration agent workflows spanning across the products for customised yet highly valuable workflows, which could become a big selling point for buying the bundle over individual solutions.

DABstep: helping make higher quality agents by finding the right language model

So you want to make an agent. How do you know if it’s any good? The promise of agents is to:

• Take plain English instructions from people, to

• Do useful things with information they provide, by

• Planning, acting on the plan and

• Revising it until it's good enough.

Agents do this by using:

• A language model (such as GPT-4, Claude 3.5, Llama 3.3), combined with

• Tools such as file access, web search, sending emails, voice calls with customers, etc. These are called variously "integrations" or "agent tools".

But language models are hard to trust: they:

• Are very expensive

• Are evolving rapidly

• Can be quite tricky to get consistent quality results from

• Can take hours and hours of experimentation to get a “feel” for.

Benchmarks with leaderboards can help accelerate confidence in a language model for your agent:

• They offer a broad baseline of real world use cases with reasonable ways to measure the results

• They provide clear measures of how one language model performs against the others

• They can be extended easily to support your own usage data.

The previous issue of Making AI Agents covered an evaluation benchmark called The Agent Company with 175 agent use cases over a range of every day organisational roles.

In this issue I dig into another agent benchmark called DABStep by HuggingFace & data specialist partner that is compelling in a number of ways.

DABstep is appealing because:

• It has 450 realistic use cases from real world commercial and financial customer queries

• It's easy run evaluations (for a programmer)

• Extra bonus: to avoid bias, the leaderboard position determined by how the language model performs against unknown questions too

I'm an AI strategist: how can I use DABstep?

DABStep can give you a signal for which language models work best with financial and commercial plain English use cases, and natural language data queries in general. For strategy and planning the simplest way is to bookmark their leaderboard and when looking to build an agent, check out which ones are ranked the best. That'll give you a shortlist of which models to start experimenting with in your agent design.

Also:

• Every use case is a plain English question (typically shorter than the long paragraph briefings of The Agent Company) designed to produce a "factoid"

• Agent planning required: the steps needed to do the work are left to the agent, and the nature of the questions require the agent to make plans of at least two steps. ("multi-hop" is the term used for this)

• Heterogeneous data types: the data is a mix of an operations manual, a very complex set of business rules, and some tabular data that the agents have access to, to figure out how to answer each of the 450 questions

I'm an AI Engineer: gimme the details.

• It uses the HuggingFace Smolagents AI agent developer toolkit to create the agents and run the benchmarks

• SmolAgents is a bit of a brain-bender:

  • It's written in python (so far so good)

  • uses LLM prompts to create plain English plans plans (yep, with you)

  • … that generates MORE python (oh wait what?)

  • … which is then executed (yes, no human eval, you have to trust the code is ok)

  • The result is then evaluated: It could outright fail to run, or the result might not meet the plain English evaluation criteria

  • Rinse and repeat: feed all that back into the generation loop and try again (with a hard limit on numbers of tries)

• Using Smolagents and its novel on-the-fly code generation comes with risks. I'd DEFINITELY not recommend using this on external-facing products, although Python can be sandboxed fairly well these days (see Pyodide for instance)

• The DABstep repo contains a Colab Notebook you can use to explore and the code. However it relies on HuggingFace's inference services which were down when I tried them (less 🤗 more like sad ☹️), so I created a pure python version of DABStep that you can use here that only relies on OpenAI. You need HuggingFace API keys (free) and OpenAI API keys (preload with a few bucks).

• In DAPstep-py I also tried using various other open weight models through ollama but had to settle at least for now using 4o-mini. I'd have prefered to use Google Gemini but SmolAgents didn't yet have a Gemini connector (though it's really trivial to add).

Recommended non-hypey AI Agent articles

“Deep research” is the new hotness

Deep Research is at its core an sophisticated language model + web search tool agent that have a bunch of steps to provide more intelligent, yet factually-accurate research reports on a topic. Both Google and OpenAI have “Deep Research” products but are behind expensive paywalls ($20 and $200😱 respectively).

As the basic mechanism is fairly well-known, there was an immediate explosion of “Deep Research” open source “clones” just days later. Here are the open source Deep Research clones I found recently:

• Open Deep Research (bhatir)

• AI Researcher (mshumer) (one of the original ones)

• Deep Research (dzhng) & python port (epuerta9)

• Open Deep Research (nicksamara)

• Open Source DeepResearch (huggingface) / live demo

• Replit Deep Research + Browser Use source (gdntn)

• Open Deep Research (fdarkaou)

More on this in a future issue as it deserves a deep dive. (Note: only one of them uses an actual agent framework 🤔 and guess what: it was Smolagents!.)

AI Agent workflow building blocks

Useful resource looking at reusable AI Agent design workflow building blocks like orchestrator, parallelization, routing, and chain.

Have your say!

Want this to be the best newsletter ever? I’d love to get your feedback.

• 1:1 call: if you’re in North America, UK or Europe, let’s chat!

• 20 second survey: what’s important to you about making AI agents?

Towards a must-read newsletter on making AI agents: what’s important to you?

Welcome! A huge thanks to everyone’s feedback from the last newsletter! 🙏 It helped a whole bunch! For your feedback, the invite is still open to have your say. Links at the end… ⬇️

Wait what’s an agent again? 🤔

Here’s my latest attempt at the 🎺 World’s Most Useful Definition of Agent 🎺:

• Agents are traditionally a way of automating a business workflow with a set of tools. They’ve been around a very long time (decades), and can be created with visual drag-and-drop lines and boxes, or programmed by a developer.

• The first breakthrough is that some agents can now do work solely from plain text instructions. They can create a plan, figure out which tools to use, review the results and adjust the plan. E.g. “Do a competitive analysis of Tesla vs its top 5 competitors and why everyone should boycott Tesla” (or something like that 😉).

• The second breakthrough is that agent tools themselves can use AI. This means workflows can intelligently process and generating text, images, video and audio. For example, summarisation was almost impossible before 2018, and now it’s basically free. Also, realistic image generation was basically impossible until a few years ago too.

How real is the magnetic allure of a “low cost 24/7 virtual workforce under your control”?

There are millions of people with a business idea, held back by lack of funds, expertise and time to hire a team to build it. Maybe you’re one of them?

That’s the promise of agents in 2025. But is it that simple though: substitute an agent for a person? A team of people with a team of agents? Well, no, but sometimes yes. More on this in future: in the mean time here’s something to reflect on: no matter how good an agent is, they’re not accountable for SQUAT in a way that people can be.

What’s your experience building agents personally? I’d love to hear from you. There are “free booking” buttons sprinkled all over this newsletter so I won’t add another one 🙂. I do want to learn from you if you are on this journey. Thanks in advance 🙏

Building your first agent: a limited-time invite

“I want to make my first agent. Where do I start?” This is an incredibly common question at the moment. And let’s start why do you think you need an agent? Maybe you don’t need one… yet.

Starter questions:

• What’s the business workflow you have and the steps required (e.g. “research trending AI agent news and create an inspirational and insighful newsletter article that inspires everyone to sign up to my paid subscription” 😉)

• How often does it happen?

• How often does it change?

• What tools does it use?

✅ You have a good candidate for an agent if:

• You know the steps

• They happen a lot

• They don’t change much, and

• the steps mostly involve digital tools (eg web crawling) vs real world (e.g. rubbish sorting)

❌ Agents are probably less useful when

• The workflow doesn’t happen very often

• Seems to vary from situation to situation

• Involves lots of people, or

• Has many steps in the physical world (e.g. “install puppet dictator”).

What next? You have to choose from the 50+ agent making tools out there 😵‍💫. Let me help: book a free 25 minute call

Book 25 minutes with me for free

Reader feedback: but first, AI Readiness.

One signal that came clear from you, dear readers, is that before people embark on their AI journey, it’s important to be able to walk before you can run, (“AI Readiness”). This includes:

• Ensuring all stakeholders impacted by an agent are all up to speed on AI technology,

• How humans and AI tools can coexist effectively and

• How to manage new factors such as hallucinations, jailbreaking and new operational costs.

Here are some questions that people should be comfortable answering before embarking on their AI agent journey:

• What is the promise of a large language model (LLM), and what makes it different from other technology?

• What is the promise of prompt engineering, and what are some best practices?

• How can these concepts help us solve problems better than before, and how do they translate to positive benefits?

“We’re implementing AI but our competitors are doing it faster than us” — says one client of mine. If you are interested in understanding ways accelerate AI readieness in your org, I have years of experience with AI and for a short time I’m offering a free call.

Book 25 minutes with me for free

How do you know if an agent is any good?

Agents are red hot and everyone’s yapping like young pups with endless examples of how to make them. But how do you know if an agent is any good? In other words, how do you evaluate agents?

The Agent Company is an open source project that is trying to fix this. It provides 175 agents (!) and, critically, a way to evaluate how well the agents do their jobs.

Here’s the gist:

• The agent use cases cover software engineers, product managers, data scientists, human resources, financial staff and office administration (meeting bookings etc).

• An agent is given a textual briefing only. No steps.

• The agent is expected to do all the work to figure out what steps to take, and what tools to use (albeit from a fixed set provided for a given task)

• Stay tuned to see how this evolves: ideally it will support any agent from any agent maker tool but we’ll see…

The AI agent tools keep getting better and cheaper.

The key enabler of AI agents is the intelligence of large language model (LLM). There are literally millions of LLMs and their “fine-tuned variations” to choose from, but in the spotlight recently was the recent news about China’s Deepseek R1. It’s all good news for agent makers: Deepseek is one step closer to faster, cheaper and better quality LLMs, and because of how much they’ve shared, there will be a fast following of copycats. HuggingFace for example is already promising to fully replicate R1 with their “Open R1” project, but be fully open source.

Wait but I thought R1 was open source?

R1 is “open weights”: they share the file that can be used to do prediction (the model file). This is a huge step forwards because OpenAI (and others) is locked behind an API. HOWEVER, the code and datasets used for training need also to be shared for R1 to be “open SOURCE”.

• Open weights: model file only (for predictions)

• Open source: model file, source code and data used to train the model

Other news I think is worth appreciating

HuggingFace offered a free AI Agent course for programmers. THIRTY FIVE THOUSAND people signed up. Overhyped, much? 🤯

Running agents on your desktop automation is in vogue. OpenAI launched Operator, and if you’re in the US, for $200/mo you can give OpenAI full control of your computer (!) to accidentally do illegal, expensive and silly things completely on your behalf faster than ever before.

(This isn’t new: Claude has had one, and there are dozens of other competitors in this space, many open source.)

Now, back to your feedback on the newsletter…

Have your say! I want this newsletter to be one of those that you stop everything for, and take time to digest, because it’s that good. Not sure it’s there yet… but you can help make it that!

• 1:1 call: if you’re in North America, UK or Europe, let’s chat!

• Survey: what’s important to you about making AI agents? Just a few questions

Welcome! This is my first issue of Making AI Agents. I’m still figuring out the best way to share my knowledge and experience. Have a read of this one, then I’d love to hear what you think. Let’s make it better together!

• 1:1 call: if you’re in North America, UK or Europe, let’s chat!

• Survey: what’s important to you about making AI agents? Super short: complete in 20s or less.

Things I Know To Be True about AI Agents.

TRUTH! In a space of ultra-hype, I’m obsessing about integrity and truth, so you can trust what I say is grounded in reality and not some venture capitalist fever dream driven by FOMO.

Here are some things I know to be true.

“Agents” are not new. But “AI Agent” is being used to describe a very … generous… range of offerings. Some academics are up in arms about this “abuse” of their beloved term, but marketing appears to have won the fight.

When someone says “agent” in 2025, they’re probably referring to one of these:

• fixed workflows, that can’t change, probably with AI steps, and

• agents that decide for themselves what to do (autonomous and reflective planning), again likely with AI steps.

ALSO there are two ways you can make agents as well, listed below.

No-code point and click: visual business workflows with AI steps such as text and real time voice chat. Mainstays like Zapier and Make.com have been in this game for years and offer lots of AI tools in their workflows. Another very interesting product is n8n.io (pronounced “n eight n”, not “neight-n” as I first thought).

AI agent programmer toolkits: things that help software developers make agents more easily (mostly in Python). This includes Microsoft Autogen, crewAI, and recently an intriguing offering from HuggingFace called “SmolAgents”:

There are a lot of solutions out there! I’ve found at least 50 AI agent maker solutions so far, for making your own custom agent workflows (most are programmer toolkits). You can rest assured though: I’ll be your guide to make sure you pick the right one.

Prebuilt agent solutions exist too

Separate from this is a quickly growing market selling fully-baked vertical agent solutions such as aomni for Sales Research. These need no workflow design and are designed to work right out of the box.

Must-read articles

• Future of Work: SalesForce “seriously debating” software engineer hires due to the incredible productivity gains of agentic AIs: “Salesforce CEO Marc Benioff says the company might not hire software engineers in 2025, while Microsoft CEO Satya Nadella foresees a "swarm of AI agents" complementing workers soon.”

• 4 ways agents will be even better in 2025 “We believe that, in 2025, we may see the first AI agents ‘join the workforce’ and materially change the output of companies”— Sam Altman, OpenAI

• Nvidia announces tools to advance agentic AI: “Leading AI agent platform providers including SAP and ServiceNow are expected to be among the first to use the new Llama Nemotron models.”

Now, back to feedback!

Have your say!

• 1:1 call: if you’re in North America, UK or Europe, let’s chat!

• Survey: what’s important to you about making AI agents? Just a few questions

• No one can agree what an agent actually is

• Billions will be invested in making them anyway

The questions many have are:

• Where does this leave businesses?

• How do they filter the hype from real solutions people can move with confidence on?

I feel I'm uniquely qualified to help guide people and businesses through these because:

• I have been creating internet software for more than 30 years, the last 7 in AI tech

• I obsess about about keeping up to date with the latest new shiny stuff

• I work with AI agent tech hands-on every day, and from this, talk from direct experience, be it good or bad!

• I have no qualms popping hype bubbles: you can rely on me to call out vacuous promises, VC-driven product launches not based on any practical use cases, and exciting stories that sound great if only they were grounded in reality

But mostly you can rely on me to ask…

Subscribe now

Subscribe now