Skip to content

Blog

How we think about AI agents in a tracker

Agents in Deckbell are assigned like teammates, reviewed like teammates and billed per run. Here is the thinking behind the roles, the run page and the budgets.

5 min readDeckbell team

An AI agent that writes code is only useful if you can hand it work, check what it did and know what it cost. Those three questions shaped how agents work in Deckbell. This post explains the thinking. The details are in the AI agents docs.

Agents are assigned, not prompted

The first decision was where an agent should live. We did not want a separate chat window where you paste in an issue and copy out a diff. We wanted the agent to be a teammate on the board.

So an agent is started the same ways you would hand work to a person:

  • Assign an issue to it.
  • @mention it in an issue comment: "@Coder can you take this?"
  • @mention it in a channel with an issue key: "@Coder pick up HRB-142".
  • @mention it in a document, in the page or in a comment.
  • Press Run on the Agents tab.

The agent answers where it was asked, with something like "On it. Working in harbor/billing on branch coder/hrb-142." It reads the issue's title, description, status, priority and labels, its parent and sub-issues, linked issues with a short excerpt, the last 30 comments, and what you asked. That is the same context a new teammate would read before starting.

Agents never take a seat. There is no per-agent price; you pay for the runs.

Roles decide what an agent may change

Not every job should be allowed to push code. Every agent has a role:

RoleWhat it doesChanges code
CoderImplements features and fixes, then a pull request is openedYes
ReviewerReviews a pull request for correctness and qualityNo
SecurityAudits changes for secrets, injection, auth and dependency risksNo
TesterWrites and fixes automated tests, then a pull request is openedYes
TriageInvestigates an issue: likely cause, what's affected, a plan and a sizeNo

Coder, Reviewer and Security are on every plan. Tester and Triage come with Pro. On Business you can write custom roles with their own name, instructions and read-only flag.

The read-only roles matter as much as the coding ones. A Reviewer or Security agent can read every new pull request in its repositories automatically and post its findings on the pull request and the linked issue. That gives you a second pair of eyes on every change, including changes a Coder agent made.

Every run ends in a pull request you can review

An agent does not merge anything. A coding agent works on one branch per issue, pushes it, and Deckbell opens the pull request. A PR card is posted on the issue, and in the channel when the agent was asked there, with the lines added and removed, the files changed, how long the run took and what it cost.

We say this plainly in the docs: agents are new, so review every pull request they open before you merge it, just as you would a teammate's. The product is built around that review, not around skipping it.

If the agent did not push any changes, there is no pull request and the result says so. When a run cannot start or fails, the agent says why.

You do not pick a model

Agents on Deckbell run on Auto. You do not choose a model. Deckbell sizes the task from the issue and its discussion, then picks an approach:

  • Small tasks go straight to coding.
  • Normal and hard tasks get three steps: a plan, the coding, and a review posted on the pull request. Hard tasks use a stronger frontier model for the plan and the review.
  • If the coding step fails or pushes nothing, Deckbell retries it once on a stronger model.

Auto also learns. Once enough runs of a size have finished, it uses the cheapest approach that succeeds at least 80% of the time, and moves up when the default keeps failing. The run page has a Routing card that says which approach was used and why. Owners and admins can turn learning off under Settings → AI budget & routing.

If you would rather control the model, run agents on your own Anthropic key. Then you pick one model for every run and Anthropic bills you directly.

Cost is on the run, not in a spreadsheet

Every run has a page with a Steps table: each step, the model tier it used, its input, cached and output tokens, and its cost. The run's cost is the sum of its steps. The PR card shows the total, and the Runs list on an agent's card shows every run with its status, issue, who started it, the pull request, duration and cost.

Each run also has a cost limit. Before each step, Deckbell checks what the run has spent. If the next step would go past the limit, the run stops and says which step was skipped. A pull request that was already opened stays open.

Across the workspace, the Agent Dashboard lists every agent with its role, project, run count and last run. On Business, View costs opens the cost dashboard: totals, daily spend, spend by model tier, by person, by agent, by project and by role, and how Auto's routing performed by task size.

Budgets that actually stop things

Visibility is not enough if a scheduled agent can run up a bill overnight. Under Settings → AI budget & routing, owners and admins set a monthly budget for AI credits. With Stop hosted runs at the budget on, new runs on Deckbell are refused once it is used, until next month or until the budget is raised. Runs already going finish. Scheduled runs, @mentions and automatic reviews are skipped the same way.

Each agent can have its own monthly cap, and alert thresholds notify owners and admins as spend crosses them. See Monthly AI budgets.

The short version

Hand an agent an issue the way you would hand it to a person. Let a role decide whether it may change code. Review its pull request before you merge. Read the cost on the run, and set a budget that stops runs rather than just reporting them.

Start on the Agents tab of any project with New agent. The AI agents page walks through the rest.

Call the crew. Ship the watch.

Create a workspace in under a minute. Free for up to 5 members, agents included.