Factory AI review: where Droid wins and what to test

Verdict by buyer

For an individual developer, Droid is a capable paid terminal agent with unusually broad model choice and a path from local pairing to remote background work. Pro at $20 is competitive with the entry subscriptions for Claude Code, Codex, Cursor, and Devin. The decision turns on whether Factory’s harness and interfaces produce better results for your repository than the agent already included with another subscription.

For a platform team, Factory is more differentiated. Model gateways, centralized policy, organization settings, audit, telemetry, SSO, provisioning, zero data retention, dedicated compute, and hybrid or on-premises options solve adoption problems that a personal CLI leaves to the buyer. Persistent computers and Missions give the control plane real execution scope rather than an administrative dashboard alone.

Agent quality and benchmark claims

Factory has invested in the Droid harness as a product independent of any one model. Its published Terminal-Bench work describes hierarchical prompting, model-specific adaptations, a small tool repertoire, environment bootstrapping, shorter default timeouts, explicit planning, controlled background processes, and runtime awareness. Those are sensible agent engineering choices. They address failure modes that remain even when two products call the same foundation model.

Factory reported a leading 58.75 percent score on Terminal-Bench v0.1.1 in September 2025 and compared Droid favorably with other harnesses on the same model families. The historical number should be read with its date, task set, harness configuration, and permission setting. Agent leaderboards move as models, prompts, tools, and benchmark versions change. Factory itself now publishes results across newer suites and routing experiments.

• What the result supports: harness design has measurable value and Droid was highly competitive under the disclosed benchmark conditions.
• What it cannot establish: completion quality in your languages, monorepo, test environment, infrastructure, security policy, and task distribution.
• What to reproduce: run 15 to 30 representative tickets with fixed acceptance tests, record interventions and failures, and compare another agent on matched tasks.
• What to inspect: unsuccessful trajectories. A tool that fails legibly and recovers cheaply may be more useful than one with a slightly higher headline pass rate.

Droid’s model independence is a practical hedge. A user can choose models explicitly, and Factory is developing routing that selects models by task. Enterprise configurations can connect direct providers, Bedrock, Vertex, Azure OpenAI, gateways, or approved internal endpoints. The value comes from avoiding a harness migration when the best model changes. The corresponding risk is that routing policy and model availability add another layer to debug.

Droid CLI as a daily tool

The CLI has the features expected from a current terminal agent. Running droid starts an interactive full-screen session. droid exec performs a non-interactive task for scripts and CI. A session can be resumed by ID. Prompts can come from an argument, file, or stdin. Output can be text, JSON, streaming JSON, or JSON-RPC. Model choice, autonomy, tool enablement, tool disablement, reasoning effort, and specification behavior are exposed as flags.

droid

droid "explain the authentication flow"

droid exec --auto low "fix the failing unit tests"

droid exec -f .factory/prompts/review.md -o json

git diff | droid exec "write a release note"

droid exec -s session-123 "continue from the failed check"

droid exec --list-tools

The command design works well for a Unix pipeline. Exit codes distinguish success, runtime failure, and invalid arguments. An API key supports unattended authentication. MCP commands, project settings, custom Droids, skills, hooks, and plugins give the CLI extensibility. The growing surface also creates configuration risk: model policy, organization settings, project settings, user settings, tool policy, and execution environment can each change behavior.

Specification Mode and the approval model

Specification Mode is Factory’s most useful control for unfamiliar or consequential work. It directs Droid to inspect the repository and produce a detailed proposal before editing code. The plan includes acceptance criteria, implementation steps, a file-level change outline, tests, and relevant security or compliance considerations. The developer can request changes until the plan expresses the intended result.

Approval then selects an execution posture. Manual execution keeps command decisions visible. Low, Medium, and High autonomy progressively allow broader actions without repeated confirmation. Organization policy, deny lists, tool policy, and sandbox checks can constrain those choices. The separation between interaction mode and autonomy level is sound: planning versus doing is one decision, and command risk tolerance is another.

A plan is useful evidence only when it reflects what the code actually does. Reviewers should check named interfaces, data migrations, backward compatibility, failure behavior, and the exact verification commands. A polished generic plan can still authorize the wrong change. The fastest correction happens before the first edit.

Local, cloud, and persistent computer execution

Factory supports local background agents, cloud execution, managed Droid Computers, and customer-owned computers. This is broader than products that choose one ephemeral cloud sandbox or one local process. It allows a developer to pair in a local repository, send long work to a persistent remote machine, or keep a prepared enterprise environment available for repeated sessions.

Droid Computers preserve packages, repositories, services, credentials, and filesystem state across sessions. That is excellent for expensive setups and long-running integration work. It also means each session inherits yesterday’s mutations. A pilot should test environment recovery, branch cleanup, service collisions, secret exposure, resource contention, and the time required to reproduce a completed run on a clean machine.

Factory’s ability to move across these targets is a genuine platform advantage. It also makes architecture review important. Ask where prompts, source excerpts, transcripts, tool results, model calls, telemetry, and secrets travel for each mode. The answer can differ between local SaaS, customer gateway, hybrid, and on-premises configurations.

Missions: powerful and expensive to review

Missions are the clearest reason to choose Factory over a single terminal process. A larger goal is decomposed into milestones, specialist workers execute parts, and validators assess results. Factory supports worker and validator model settings and headless Mission runs. The pattern can compress migration, refactor, test, and documentation programs that have repeated units of work.

The coordination problem remains real. Workers can share wrong assumptions, touch overlapping contracts, pass narrow checks, or produce a combined diff too large for responsible review. A Mission needs a branch and worktree strategy, interface ownership, integration order, validation artifacts, a resource ceiling, and stop conditions. High autonomy is a prerequisite for Mission orchestration in current documentation unless the unsafe permission bypass is used inside a genuinely isolated sandbox.

• Good Mission: migrate 30 independent packages to one documented API, with a per-package test and a final integration suite.
• Weak Mission: improve the architecture of the platform. The objective has no acceptance boundary and decomposition will encode unreviewed taste.
• Good validator: compile, run focused tests, compare generated contracts, and check a deterministic migration inventory.
• Weak validator: ask another model whether the code looks good. Agreement between agents is not executable evidence.

Missions draw from regular rolling limits and require Extra Usage to be enabled. That rule prevents silent abandonment when Standard Usage ends, yet it can convert a large plan into prepaid variable spend. Use Droid Core where appropriate, stage the work, and record cost per accepted milestone.

Enterprise controls are substantive

Factory’s enterprise story is more than seat administration. Current documentation covers SSO, SAML and SCIM provisioning, organization-level model and autonomy controls, deny lists, audit trails, OpenTelemetry, zero data retention, dedicated compute, encryption and network policy, data residency, session retention, on-premises options, hybrid configurations, and customer model gateways. Factory lists SOC 2, ISO 27001, and ISO 42001 in its compliance material.

Those controls matter because an engineering agent combines sensitive code, tool execution, developer identity, provider traffic, and autonomous action. A standard rollout needs a model allowlist, context rules, permission defaults, secret handling, repository risk tiers, audit ownership, incident response, and a human merge policy. Central configuration can make that baseline enforceable rather than advisory.

1. Map every data path. Document source, prompt, tool output, transcript, telemetry, and model provider handling for each deployment mode.
2. Start read-only. Establish search and review value before granting source writes, cloud credentials, or production tools.
3. Separate development and production identity. A coding agent rarely needs the same credentials as a deployment operator.
4. Export evidence. Send audit and OTEL events to systems the security and platform teams already monitor.
5. Keep merge human-owned. Automated checks can narrow review; they do not carry business accountability.

Factory wins against a personal CLI when these are purchase requirements. The buyer should still verify exact contract language, regional availability, retention settings, subprocessor handling, and whether a control applies to every interface. Marketing-level certification names do not substitute for the trust center and architecture review.

Pricing and quota tradeoffs

The $20 Pro entry price is reasonable for a serious coding agent. Plus at $100 and Max at $200 follow the same individual price ladder used by other premium agent subscriptions. Factory differentiates Plus by adding managed Droid Computers and approximately five times Pro Standard Usage. Max advertises approximately ten times Pro usage and early features.

The hard review point is quota transparency. Factory uses 5-hour, weekly, and monthly rolling limits, while the public documentation describes relative multipliers instead of fixed token quantities. Token and computer compute both count. After Standard Usage, Droid Core has a separate included pool. Extra Usage is prepaid with a $10 minimum and applies at Factory’s token rates. BYOK also has a plan-specific free allowance before charges described by the plan.

Run the same weekly task mix on Pro before extrapolating to seats. Record each limit hit, fallback choice, paused Mission, Extra Usage draw, and completed task. For Teams and Enterprise, request a modeled invoice for realistic usage plus deployment, support, and compute. Compare it against model subscriptions or API spend already in the company.

The limitations that matter

Factory documentation is broad and unusually detailed, which is a strength. The pace of product change is also high. Pricing rules, models, flags, permissions, and computer behavior can move faster than internal platform documentation. Treat the vendor changelog as an input to rollout management and re-run a small acceptance suite after material updates.

The deepest limitation is shared with every coding agent: a plausible, tested patch can implement the wrong intent. A Droid can explore and execute more of the lifecycle, which increases leverage and the radius of a mistaken specification. Small milestones, explicit invariants, clean branches, deterministic checks, and human acceptance remain the controls that hold.

A fair two-week evaluation

Questions people ask

Keep reading