What Are AI Agents? A Practical Guide for Product and Engineering Teams

“AI agent” has become one of the most stretched terms in software. It is used for chatbots, for scripted automations with a model inside, and for systems that plan and execute multi-step work on their own. The confusion matters, because each of those has a different cost, risk profile and engineering effort.

This guide gives a working definition, explains how agents actually run, and sets out the decisions a product team has to make before an agent is allowed near customers, money or production data. If you are comparing agents with other uses of language models, our guide to agentic AI versus generative AI covers that distinction in more depth.

A working definition

An AI agent is software that uses a language model to decide its own next step towards a goal, take actions through tools, and adjust based on what happens, with limited human supervision.

Three properties separate an agent from other AI features:

• It chooses the path. A workflow follows steps a developer defined. An agent decides which step to take next at run time.
• It acts. It does more than produce text: it searches, queries, updates records, sends messages or runs code through tools it has been given.
• It observes and adapts. It reads the result of each action and uses it to decide what to do next, including retrying or changing approach.

Anthropic’s engineering team draws a useful line here. Workflows are systems where models and tools are orchestrated through predefined code paths; agents are systems where the model dynamically directs its own process and tool usage. Both are valuable. The mistake is building an agent when a workflow would do.

How an agent runs: the loop

Under the hood, almost every agent is a loop:

1. Receive a goal — for example, “find out why order 48213 has not shipped and tell the customer what will happen next”.
2. Look at the current state — the goal, the conversation, any results so far and the tools available.
3. Decide the next action — the model chooses a tool and its arguments, or decides it is finished.
4. Execute the action — your code, not the model, calls the tool and enforces permissions.
5. Observe the result — the tool output is added to the agent’s state.
6. Repeat or stop — the loop continues until the goal is met, the agent asks for help, or a limit is reached.

In the order example, the agent might look up the order, see that it is waiting on stock, check the warehouse system for the restock date, check the shipping policy, and draft a reply with an honest estimate. A workflow could do the same if every order problem looked alike. The agent earns its place when the cause varies — payment hold, address problem, stock, carrier delay — and the right sequence of checks depends on what it finds.

The important design point is step 4. The model only *proposes* an action. Your application decides whether that action is allowed, executes it and returns the result. That is where security, cost limits and audit live.

The five components every agent needs

Teams usually spend most of their time on the model and instructions. In our experience, tool design and controls decide whether an agent is trustworthy. A tool called update_order that can change any field on any order is a liability; tools called add_order_note and request_address_change, each scoped to the current customer, are much easier to reason about.

Types of agents you will hear about

Labels vary between vendors, but most agents fall into a few practical shapes:

• Conversational agents handle a dialogue with a user and take actions on their behalf — support, booking, account changes.
• Task agents receive a goal and work through it with little interaction — research a supplier, reconcile two reports, triage a backlog of tickets.
• Coding agents read a codebase, make changes and run tests in a sandbox. They are among the most mature agent uses because the environment gives fast, objective feedback.
• Browser or computer-use agents operate interfaces built for people. They are flexible but slower and more fragile than API tools; use them when no API exists.
• Multi-agent systems split work between specialised agents, such as a planner and several workers, coordinated by an orchestrator. They can help with large tasks but multiply cost, latency and failure modes. Start with one agent and split only when you can show a single agent is the bottleneck.

Older textbooks classify agents as simple reflex, model-based, goal-based, utility-based and learning agents. That taxonomy is still useful for understanding the theory, but product teams get more from asking how much autonomy the agent has and what it is allowed to touch.

Autonomy is a dial

The most useful question is not “is this an agent?” but “how much can it do without a person?”

Most of the business value we see comes from levels 1 to 3. Level 3 is powerful when the limits are enforced by the application — the refund API itself rejects amounts over the limit — rather than by an instruction the model is asked to follow.

Where agents fit and where they do not

Agents are a good fit when:

• the steps cannot be scripted in advance because they depend on what is discovered along the way;
• the work spans several systems that each need to be queried or updated;
• mistakes are detectable and reversible, or a person reviews the outcome;
• the volume of work justifies the engineering, evaluation and monitoring effort.

They are a poor fit when:

• the process is stable and can be drawn as a flowchart — use a workflow or rules;
• every step must be explainable to a regulator in deterministic terms;
• an error is costly and cannot be caught before it reaches a customer;
• latency must be low and predictable, since each loop iteration adds model calls.

A practical test: write down the steps a skilled employee follows. If the list is the same every time, build a workflow and use a model only for the steps that need language understanding. If the list changes with every case, an agent may be justified. Our guide to choosing between AI and rules for workflow automation walks through this decision.

Common use cases by business area

Risks you must design for

Agents inherit every risk of language models and add new ones because they act. The OWASP Top 10 for LLM Applications (2025) is a good checklist. Four items matter most for agents:

• Prompt injection (LLM01). Text in an email, web page or document can contain instructions the agent may follow. Treat all retrieved and user-provided content as untrusted data, never as instructions.
• Excessive agency (LLM06). An agent with broad tools and permissions can do broad damage. Limit functionality, permissions and autonomy to what the task needs.
• Improper output handling (LLM05). Model output passed directly into a database query, shell or web page can cause injection. Validate tool arguments like any other untrusted input.
• Unbounded consumption (LLM10). A loop that never converges can run up cost. Budget steps, tokens and time per task.

There are also business risks: confident but wrong answers given to customers, actions that conflict with policy, and difficulty explaining why an agent did what it did. These are solved with design and process, not with a better prompt.

Controls to put in place before launch

1. Start read-only. Give the first version tools that only look things up. Add write tools one at a time once read behaviour is reliable.
2. Apply least privilege per tool. Scope each tool to the current user, tenant and task. Use separate service credentials with narrow permissions.
3. Enforce limits in code. Amount caps, rate limits and allowed values belong in the tool implementation, where the model cannot talk its way past them.
4. Require approval for irreversible actions. Payments, deletions, external messages and contract changes go through a human until you have evidence to relax that.
5. Budget every run. Set maximum steps, tokens, wall-clock time and cost per task, and define what the agent does when it hits a limit — usually hand over with a summary.
6. Log everything. Record the goal, each model decision, each tool call with arguments and results, and the final outcome. You will need this for debugging, audits and evaluation.
7. Evaluate before and after release. Build a set of realistic cases with expected outcomes, including adversarial ones, and run it on every change to prompts, tools or models.
8. Provide a kill switch. Be able to disable the agent, or individual tools, instantly without a deployment.

How to start

A sensible first project is narrow, valuable and reversible. Pick one workflow where staff already spend time gathering information from several systems. Build the agent at level 1, measure how often its suggestions are accepted unchanged, and only then let it act with approval. This produces evidence for each increase in autonomy and gives the team time to learn how the agent fails.

Before choosing a framework, decide the autonomy level, the tools and the controls. The framework matters less than those decisions; our comparison of agentic AI frameworks covers the main options once you get there.

Frequently asked questions

Is a chatbot an AI agent?

Only if it chooses and takes actions through tools. A chatbot that answers from a knowledge base is an assistant. One that can look up an order, change a delivery address and decide which of those to do is an agent.

Do agents learn from experience?

Not automatically. Most production agents do not change their model weights. They can store notes or preferences in memory, and teams improve them by updating instructions, tools and evaluation sets based on logged failures.

How much does an AI agent cost to run?

It depends on the number of loop steps, the model used at each step and the size of the context. Agents typically use several model calls per task, so measure cost per completed task rather than cost per call.

Do we need a multi-agent system?

Rarely at the start. One well-designed agent with good tools is easier to build, test and debug. Split into multiple agents only when you can show that one agent’s context or responsibilities have become the bottleneck.

Can an agent be compliant in a regulated industry?

It can be part of a compliant process when actions are logged, permissions are narrow, high-risk steps are approved by a person and the system is evaluated and monitored. Check the specific rules for your sector with your compliance team.