Why Your "AI-First" Strategy Is Probably Wrong为什么你的"AI 优先"战略可能大错特错？

Today I came across the article "Why Your 'AI-First' Strategy Is Probably Wrong" a few times (the original translation is below), and I want to say something different. Rather than AI First, it should be Software Engineering First.

This article looks like it's talking about AI, but underneath it's all about software engineering.

Setting aside the parts about organization and people, here's a simple summary of the key points from the first half of the original:

In the AI era, people have become the bottleneck. PMs spend weeks on requirements, AI can implement them in two hours—PMs become the bottleneck. QA tests for three days, AI writes code in just two hours—QA becomes the bottleneck. A team of 25 people versus a competitor's few hundred—headcount is also a bottleneck.

What to do? Take people out of the chain. AI writes the code, AI reviews the code, AI runs the tests, AI deploys to production, AI monitors production status, and automatically rolls back when something goes wrong. Every day it scans logs on schedule, automatically discovers issues, assigns tasks, and tracks fixes. The whole pipeline runs on its own, with people only needed to make judgment calls at key junctures.

As for the unified codebase mentioned in the article, that's a nice-to-have, not really central to AI First. It's better to have it, but there are plenty of alternatives if you don't.

Taken as a whole, the approach is internally consistent and produces impressive results: deploying multiple times a day, shipping and pulling features the same day, letting the data decide.

First, check yourself against five things

But before rushing to copy this, think through a few things about your own situation first:

First, automated testing. After AI modifies code, you need a way to confirm it hasn't broken anything else. If test coverage isn't sufficient, you'll have to manually regression-test every AI commit, and there's no way things can move fast that way.

Second, the CI/CD process. From code commit to production deployment, are the testing, review, release, and rollback steps all fully automated? If this pipeline isn't solid, no matter how fast AI writes code, it'll just pile up waiting for someone to handle it manually.

Third, A/B testing and production monitoring. Whether a new feature works well after launch needs to be backed by data, and if it doesn't work you need to be able to turn it off at any time. Without this mechanism, if AI ships five features a day, you won't even know which ones to keep and which to cut.

Fourth, task management. Tasks need to be broken down to the right granularity, and their lifecycle needs to be tracked. Handing AI a big, vague task is more than it can currently handle. When multiple agents are working at once, you need somewhere to manage who's doing what, what's the priority, and how far along things are.

Fifth, system architecture. Code with a messy architecture—or no architecture at all—is just as much of a headache for AI to maintain as it is for a human. Even with the context window full, it still can't figure out where the boundaries are, and fixing one thing breaks three others.

If you can't check off any of these boxes, humans will have to fill the gap. If you can't fill it, AI First is just a slogan.

What scenarios fit, and what don't

But suppose you've got all of this covered—does that mean you can go AI First?

Still no. This approach only fits certain scenarios.

Scenarios where it fits: products that are mostly backend logic without complex interfaces, like API services, data processing platforms, internal tools. Whether a feature works can be verified just by running the data, without needing a human to scrutinize every pixel. The example in the original article is an agent platform, essentially a backend-driven product, so this playbook can work there.

Another fit: early-stage products doing rapid experimentation, where features that don't work get pulled, user expectations aren't that high to begin with, and AI's speed advantage can be fully leveraged.

Scenarios where it doesn't fit:

UI-heavy products. Social media loves to declare that frontend is dead every day, but try having AI build a complex interface—it can't handle the various usability issues, interaction details, or pixel-perfect visual fidelity. Otherwise Elon Musk would've already had AI redesign X who knows how many times by now.

Products where feature quality is sensitive. Do you think Anthropic and OpenAI don't know about AI First? Would they dare do this with Claude Code and Codex? Let AI fully autonomously iterate on their own core products, and users would tear them apart.

Scenarios that demand high security. Banking systems, online trading platforms—if AI code makes a mistake there, a rollback isn't going to fix it.

The real endpoint of AI First

The direction of AI First isn't wrong—it represents a shift in mindset: with every decision you make, ask yourself whether this could be done by AI, and if not, what's missing, and how to fill that gap.

But putting this mindset into practice takes more than buying a few AI tool subscriptions—it also requires building a solid foundation. Get testing, CI/CD, monitoring, architecture, and task management solidly in place, and AI's capabilities will naturally be unleashed. Get it wrong, and adding more AI on top is just building a house on sand.

From this angle, the real endpoint of AI First may not be having AI do all the work, but rather using this momentum to actually push forward the engineering improvements you've always wanted to make but never had the drive to.

It's good to look up at the stars, but you also need to keep your feet on the ground.

Why Your "AI-First" Strategy Is Probably Wrong [Translation]

By Peter Pang. Original: Why Your "AI-First" Strategy Is Probably Wrong

99% of our production code is written by AI. Last Tuesday at 10am, we shipped a new feature, ran an A/B test on it at noon, and killed it by 3pm because the numbers were bad. By 5pm we'd shipped an improved version. Three months ago, a full iteration cycle like that would have taken at least six weeks.

We didn't get here just by installing a Copilot plugin in our code editor. We completely broke down our existing engineering workflow and rebuilt it around AI from the ground up. We changed how we plan, how we write code, how we test, how we deploy, and how we organize the team. We even redefined everyone's role at the company.

CREAO is an AI agent platform. The company has 25 employees, 10 of whom are engineers. We started building agents in November 2025, and just two months ago I rebuilt our entire product architecture and engineering workflow from scratch.

In February 2026, OpenAI introduced a new concept that perfectly sums up what we'd been doing all along. They called it Harness Engineering: the core job of the engineering team is no longer writing code, but empowering agents to do valuable work. When the system fails, the answer is never "try again" or "try harder." The real approach is to ask: what capability is the AI missing? How do we make that capability clearly visible to the agent, and enforce it?

We arrived at this conclusion ourselves, just without a ready-made term for it at the time.

"AI-first" isn't the same as "using AI"

Most companies just force AI into their existing workflows. Engineers open Cursor to help write code, product managers use ChatGPT to help draft requirements docs, QA teams try using AI to generate test cases. The workflow itself stays exactly the same. Efficiency does go up 10% to 20%, but the underlying structure hasn't changed at all.

At best, that's "AI-assisted."

Truly being "AI-first" means completely redesigning your processes, architecture, and organization around the core assumption that AI is the primary builder. You stop asking "how can AI help our engineers?" and start asking "how do we rebuild everything so that AI does the building, while engineers just set direction and judge quality?"

The gap between these two approaches is exponential.

I've seen a lot of teams call themselves "AI-first" while still running the same old agile sprint cycles, the same Jira task boards, the same weekly standups, and the same QA sign-off process. They've just forced AI into the existing loop instead of redesigning the loop.

A typical manifestation of this is what's now commonly called "vibe coding." Open Cursor, keep tweaking prompts until the code runs, commit, repeat. This approach only works for prototyping. A system that's actually meant for production needs to be stable, reliable, and secure. When AI is writing the code, you need to build a system that guarantees those qualities as a backstop. What you need to build are systems—the prompts are disposable.

Why we had to change

Last year, I took a close look at how the team worked and found three bottlenecks that nearly killed us.

The product management bottleneck

In the past, our product managers would spend weeks researching, designing, and drafting detailed feature specs. That's how product management has worked for decades. But an AI agent can implement a feature in just two hours. When development time shrinks from months down to hours, that multi-week planning cycle becomes the biggest drag.

It made no sense to spend months conceiving an idea and then build it in just two hours.

Product managers have to evolve into architects with product thinking, working at the pace of rapid iteration, or they get pushed out of the development loop. Product design has to happen through a "rapid prototype - ship - test - iterate" cycle, not through committee reviews of lengthy requirements documents.

The QA bottleneck

Same story. After an AI agent ships a feature in two hours, our QA team would spend several days testing edge cases and extreme scenarios. Two hours to build, three days to test.

So we replaced manual QA with an AI-built automated testing platform, using AI to test the code AI wrote. Verification speed has to keep up with development speed. Otherwise you're just building a new bottleneck ten feet away from the old one.

The headcount bottleneck

Our competitors have 100 times or more people doing the same work, while we only have 25. There's no way we could out-hire them. Our only path was to redesign our way through.

We needed to embed AI deeply into three systems: how we design products, how we build products, and how we test products. If any one of these still relies purely on humans, it drags down the entire pipeline.

A bold decision: unifying the architecture

I had to start by tackling the codebase.

In the past, our architecture was scattered across multiple independent systems. Changing one feature might require touching three or four repositories at once. From a human engineer's perspective, that was barely manageable. But from an AI agent's perspective, it was a black box. The agent couldn't see the whole picture, couldn't reason about cross-service ripple effects, and couldn't run integration tests locally.

I had to consolidate everything into a single large monorepo. There was only one reason: to let AI see the whole picture.

This is Harness Engineering in practice. The more of the system you turn into something AI can inspect, verify, and modify, the more leverage you get. A fragmented codebase is invisible to AI, while a unified one is clear and legible to it.

I spent a week designing the new system: planning phase, implementation phase, testing phase, integration testing phase. Then I spent another week using agents to help refactor the entire codebase.

CREAO itself is an agent platform. We used our own agents to rebuild the platform that runs agents. If a product can build itself, that proves the approach works.

Our tech stack

Here's our tech stack, and what each piece does.

Underlying infrastructure: AWS

We run on AWS, using auto-scaling container services and a circuit-breaker rollback mechanism. If monitoring metrics degrade after a deployment, the system automatically rolls back to the last safe version.

CloudWatch is the central nervous system of the whole setup. Every service has structured logging, more than 25 automated alerts are configured, and automated workflows query custom metrics daily. Every piece of infrastructure exposes structured, queryable signals. If AI can't read the logs, it can't diagnose problems.

CI/CD: GitHub Actions

Every code change goes through an uncompromising six-stage pipeline:

Validate CI → Build and deploy to dev → Test dev → Deploy to production → Test production → Official release

Gates on every pull request (PR) enforce type checking, lint checks, unit and integration tests, Docker builds, Playwright-based end-to-end tests, and environment consistency checks. No stage can be skipped. No one can force a manual green light. The whole pipeline is completely deterministic, so AI can predict outcomes and reason about failure causes.

AI code review: Claude

Every PR triggers three parallel rounds of AI review from Claude Opus 4.6:

Code quality: checks for logic errors, performance issues, maintainability.

Security: vulnerability scanning, auth boundary checks, injection risks.

Dependency scanning: supply chain risk, version conflicts, open-source license issues.

These are hard gates that must be passed, not just suggestions. They run in parallel with human review, catching in bulk the errors humans tend to miss. When you're deploying 8 times a day, no human engineer can stay laser-focused on every single PR.

Engineers can also tag @claude in any GitHub Issue or PR to get an implementation plan, kick off a debugging session, or request code analysis. The AI agent can see the entire monorepo. Context carries seamlessly across different conversations.

Self-healing feedback loop

This is the soul of the whole system.

Every morning at 9:00 UTC, the automated health-check workflow kicks off right on schedule. Claude Sonnet 4.6 queries CloudWatch, analyzes error patterns across all services, and generates a system health executive summary that gets sent to the team's chat. No one needs to lift a finger to make this happen.

An hour later, the triage engine kicks in. It clusters production errors into categories, scores each issue's severity across nine dimensions, and automatically opens investigation tickets in the task management system. Each ticket thoughtfully comes with log samples, affected users, affected endpoints, and suggested directions for investigation.

The system also deduplicates automatically. If an existing ticket already covers the same class of error, it updates that ticket. If a previously resolved issue resurfaces, it sharply detects the regression and reopens the ticket.

When an engineer submits a fix, the same pipeline takes over. Claude runs three rounds of review, CI validates it. A six-stage deployment pipeline pushes it through the environments and tests it. Once deployed, the triage engine checks the monitoring data again. If the original error is resolved, the ticket closes automatically.

Each tool is responsible for exactly one stage. No single tool tries to do everything. This daily loop creates a "self-healing closed loop": detecting, triaging, fixing, and verifying errors with minimal human intervention.

I told a Business Insider reporter: "AI handles the writing and submitting of code; humans only need to review for strategic risk."

Feature flags and the supporting tech stack

We use Statsig to manage feature flags (a technique for controlling whether features are enabled in code, allowing features to be toggled at any time without redeploying code). Every new feature launches hidden behind a flag. The rollout pattern is very solid: first open it internally to the team, then release it gradually by percentage, and finally either open it up fully or kill it outright. The so-called "one-click kill switch" can instantly disable a feature — no redeployment needed at all. If a feature makes the metrics worse, we can pull it within hours. Bad features die on the same day they launch. A/B testing runs on this same system too.

Graphite manages our branches: the merge queue reruns validation, and only an all-green run gets merged into main. This lets us commit code at high frequency while still reviewing it in an orderly way.

Sentry reports structured exceptions across all services, and the triage engine then combines them with monitoring data. Linear is the human-facing interface: it automatically creates tickets with severity scores and investigation suggestions, then closes them automatically once verification passes.

How a feature goes from idea to production

New feature development path

The architect defines the task as a structured prompt, including codebase context, goals, and constraints.

The agent breaks down the task, plans the implementation, writes the code, and automatically generates accompanying tests.

A PR is opened. Claude runs three rounds of review. Human reviewers only check for high-level risk, rather than getting bogged down line by line.

Pipeline validation: type checking, linting, unit tests, integration tests, end-to-end tests.

Queue, revalidate, merge once it passes.

A six-stage deployment pipeline pushes it through the different environments, with testing at every stage.

The feature flag is opened internally for the team. Rollout proceeds gradually. Metrics are watched closely.

If the metrics deteriorate, it can be killed with one click at any time. Serious issues automatically trigger a circuit breaker and rollback.

Bug fix path

The monitoring system detects an error.

The Claude triage engine assesses severity and automatically opens a ticket with full investigation context.

An engineer steps in to investigate. By this point the AI has already finished the diagnostic work. The engineer just needs to verify the conclusion and submit the fix.

It goes through the same rigorous pipeline of code review, validation, deployment, and monitoring.

The triage engine re-verifies. If confirmed resolved, the ticket closes automatically.

These two paths run on exactly the same pipeline. Same system, same standard.

How are the results?

Over the past 14 days, we've averaged 3 to 8 production deployments per day. Under the old model, we couldn't have shipped even a single release in that same two-week span.

Bad features get killed the same day they launch. New features go live the same day they're conceived. A/B tests validate business impact in real time.

Many people assume we're trading quality for speed. It's the opposite — user engagement is up, and paid conversion is up too. We're building a better product than before, because the feedback loop has become incredibly short. Shipping once a day teaches you far more than shipping once a month ever could.

A brand-new engineering org structure

In the future, there will only be two types of engineers.

Architects

Only one or two people. They design the standard operating procedures and teach the AI how to work. They build the test scaffolding, the integration systems, and the triage network. They call the shots on system architecture and boundaries. They're the ones who define what "good" means in the eyes of the agents.

This role demands extremely deep critical thinking. Your job is to poke holes in the AI, not to blindly trust it. When an agent proposes a solution, the architect needs to be sharp enough to spot the gaps: which failure modes did it miss? Which safety boundaries did it cross? What technical debt is it accumulating?

I hold a PhD in physics. The most useful thing I learned during my doctorate was how to question assumptions, stress-test arguments, and hunt for logical gaps. In the future, the ability to critique AI will be worth more than the ability to write code.

Naturally, this is also the hardest role to hire for.

Operators

Everyone else. The work still matters, but the structure has changed.

Now it's the AI that assigns tasks to humans. The triage system finds a bug, opens a ticket, surfaces the diagnosis, and assigns it to the right person. The human investigates, verifies, and approves the fix. AI submits the code; humans review for risk.

This work still demands a very high level of skill and focus, but it no longer requires the kind of from-scratch systems-architecture reasoning that the old model demanded.

Who adapts fastest?

I've observed a surprising phenomenon: junior engineers adapt faster than senior engineers.

Junior engineers, without entrenched traditional habits of mind, feel like they've been given superpowers. They've picked up tools that can amplify their impact enormously, and they don't have a decade of old habits to unlearn.

Senior engineers with rich traditional experience, on the other hand, have struggled the hardest. Work that used to take them two grueling months, AI now finishes in an hour. For people who spent years honing a scarce skill, that's a brutal blow to absorb.

I'm not passing judgment on who's right or wrong — I'm just describing what I've observed. In this transformation, adaptability matters far more than accumulated past skill.

The human side of things

The death of management

Two months ago, I spent 60% of my time on people management. Aligning priorities, meetings, giving feedback, coaching engineers.

Today: under 10%.

The traditional CTO model tells you to empower the team to handle architecture, train them, and hand off the work. But if the system only needs one or two architects, then I have to be the one who builds it myself first. I went from being a "manager" back to being a "builder." These days I write code roughly from 9am to 3am. I design the underlying logic and architecture of the system, and maintain the scaffolding for the whole infrastructure.

The pressure is greater. But I love the pure joy of "building," much more than the daily grind of "aligning" with people.

Fewer arguments, better relationships

My relationship with my co-founders and engineers has actually gotten better than before.

Before the transition, most of my interactions with the team happened in meetings. Debating tradeoffs, arguing over priorities, going red in the face over technical decisions. Under the old model, these conversations were necessary, but they were also exhausting.

I still talk with the team now. We talk about things outside of work, chat casually, or do team-building activities to unwind. We get along better, because we no longer argue over work that the system can now handle entirely on its own.

The anxiety is real

I don't want to pretend everyone is happy.

When I stopped checking in with everyone every day, some team members felt unsettled. What does it mean if the CTO isn't talking to me? Where's my value in this new world? These worries are entirely reasonable.

Some people spend more time arguing in chat about "whether AI can really replace my job" than they spend actually working. A transition period inevitably brings anxiety. I don't have some perfect line to make it go away.

But I do have one principle: we don't fire an engineer just because they shipped a bug in production. We improve the review process, strengthen testing, add more guardrails. We treat AI the same way. If the AI makes a mistake, we build better verification mechanisms, clearer constraints, and stronger system observability.

Beyond engineering

I've seen companies go "AI-first" in engineering and R&D while every other department still runs entirely by hand.

If engineers can ship a feature in hours, but marketing takes a week to put out an announcement, then marketing becomes the new bottleneck. If the product team is still planning on a monthly cadence, then product planning becomes the bottleneck.

At CREAO, we've pushed the AI-native way of working into every single function:

Product release notes: automatically generated by AI from code change logs and feature descriptions.

Feature demo videos: automatically generated by AI with dynamic walkthroughs.

Daily social media posts: curated and posted automatically by AI.

Health reports and data analysis: generated by AI, pulled from monitoring and production databases.

Engineering, product, marketing, and growth all run on the same AI-native workflow. If one department moves at agent speed while another still crawls at human speed, the human-speed department drags the whole company down.

What this means

For engineers

Your core value is shifting from "volume of code written" to "quality of decisions made." The ability to bang out code fast loses value every month. The ability to evaluate, critique, and guide AI is gaining value fast.

Product sense and taste are critical. Can you glance at an AI-generated UI and intuitively sense something's off before users even complain? Can you look at an architecture proposal and instantly spot the systemic risk the AI missed?

I always tell our 19-year-old intern: deliberately practice critical thinking. Learn to evaluate arguments, hunt for logical gaps, and question assumptions people take for granted. Learn what good design looks like. These skills compound.

For CTOs and founders

If your team spends more time planning a product feature than actually writing the code for it, start cutting there first.

Before rolling out AI agents at scale, build your test scaffolding first. Fast AI without fast verification behind it just produces a rapidly accumulating technical disaster.

Start with a single architect. Find someone who can build this system and prove it works. Once the system is running, bring others into "operator" roles.

Force AI-native practices into every single function.

Be mentally prepared — you will absolutely run into resistance and pushback.

For the industry as a whole

OpenAI, Anthropic, and many independent teams are converging on the same principles: structured context, specialized agents, persistent memory, and closed execution loops. Scaffolding engineering is becoming the industry standard.

The engine driving all of this is the evolution of model capability. I credit every qualitative leap CREAO has made recently to just the last two months. Things Claude Opus 4.5 couldn't do, Opus 4.6 already can. The next generation of models will only make this transformation come faster and harder.

I believe "one-person companies" are going to become very common. If one architect with a fleet of agents can do the work of 100 people, a lot of companies simply won't need to hire a second employee.

This is only the beginning

Most of the founders and engineers I've talked to are still running on the old model. Some are starting to consider making the shift, but very few have actually taken the leap.

A journalist friend told me she'd interviewed about five people on this topic. She said we were further ahead than anyone else: "I don't think anyone else has rebuilt their entire workflow as completely as you all have."

Any team could do this with the tools that already exist. Nothing in our stack is proprietary or secret.

The real competitive edge is having the resolve to rebuild everything around these tools, and the willingness to bear the enormous cost that comes with it. That cost is real, and it's painful: employees' confusion and anxiety, the CTO's grinding 18-hour days, senior engineers doubting their own worth, and the suffocating two-week vacuum where the old system has been torn down and the new one isn't running yet.

We bore those costs. Two months later, the numbers speak for themselves.

We built an agent platform. And that platform is exactly what we used agents to build.