Building a product in the age of AI

I hope you have read Alex Brandes’s latest post, Software is writing itself. He writes:

“Software stops being a finished object and becomes a starting point.”

I agree with him completely. Open Poker is what that idea looks like in my own work. It is an arena where developers connect AI bots to six-player No-Limit Texas Hold’em games. The bots play through two-week seasons and compete on a public leaderboard. Every release gives people and their bots new ways to use it. Production then shows me what the next version needs.

I built it after work. On a normal weekday I may have one or two hours. I use some of my weekends too. There is no chance I could have built the current product in that time without AI. Open Poker crosses areas that would each take years to master. I could not become a database expert before touching the backend, then stop again to become an infrastructure engineer before deploying it. AI agents let me use good tools in each part of the product without becoming a specialist in all of them first.

That has changed my view of software development. Writing every line is no longer the useful measure of who built a product. The direction matters. The way the system is organized matters. What matters most is whether the product works and whether the process around the agents can keep it working.

I own the direction

The ideas and the overall plan are mine. I decide what Open Poker should do and what should be left out. For a larger feature, the development agent reads the repository and works out an implementation plan. I check whether that plan fits the product I am trying to build. I want to understand where the feature will live and which libraries it depends on. I also want to know how we will notice a problem after it ships.

I am not sitting beside the agent approving every step. Routine work can move from investigation into implementation and testing without me. I come back when there is a result to review or when the agent reaches a decision it cannot make from the code. I spend more time on changes that affect stored data or security. I also step in when a proposal changes the architecture or adds far more than I asked for.

I do not need to memorize every function to make those decisions. I do need a working understanding of how the system is put together. If an agent introduces a new dependency, I want to know why. If it changes a boundary in the application, I want to know what now depends on that boundary. A plan can be technically sound and still be wrong for the product as a whole.

The interface gets more attention from me than most code. Agents can produce a screen that passes its tests and still feels wrong. A button may exist but be difficult to find. A status may be accurate but confusing. I do important interface changes myself because the last part is judgment, not element detection.

Most of the code, if not all of it, is written by agents in ChatGPT and Claude. I switch models depending on the work. Both my development and production agents reach them through Constellation Gate. I first put it there for the caching and compression, which cut the token usage during long sessions with a large repository.

The token savings are useful. The production side is why I would keep it. Bug reports are untrusted text and can contain prompt-injection attempts, so they are screened before reaching the model. A policy can block suspicious input instead of allowing a report to steer the production agent. I do not treat that as a promise that every attack will be caught. It is one more control outside the agent, which is where I want it.

Development and production see different things

I keep two separate environments. The development agent has the repository in a local setup that behaves like production. It has its own database and browser. It can change code, run the application and test a fix without touching users. The production agent sees what is actually running. It can inspect live logs and runtime state. It also knows which version is deployed. Production credentials stay in that environment.

Most bug reports begin in a Google Form. I read the report and send it to the production agent when it needs investigation. Then I leave the investigation with the agent. It starts from the time of the report and reconstructs what happened from live evidence. It checks the deployed version, follows the event through the logs and compares that with the database state.

When it is done, I get an explanation of the failure and what should be true after a repair. If the analysis makes sense, I tell the production agent to create an internal GitHub issue. That issue is the handoff to development. The development agent investigates again from the source code. It does not inherit the production explanation as fact. It has to reproduce the behavior or show that the explanation fits the implementation before it repairs anything.

For a material change, I review the development plan before the agent executes it. Once the repair passes the normal checks, it is merged into main. The production agent then reviews the full implementation against the live environment. It looks for assumptions that were safe locally but dangerous with current production state. It handles the deployment and checks the original bug again.

The repeated investigation is intentional. No agent carries its own theory from the first report all the way to production. Each environment supplies different evidence. GitHub keeps that evidence after an agent session ends and after I have forgotten the details. The issue holds the original symptom next to the proposed repair, the test result and the later production check.

The three-hour test

I do not approve code on faith. I put most of the weight on what the whole system does. My production and development agents built an end-to-end release rehearsal that runs for about three hours. It builds the production application and applies the migrations. Then it signs up 1,000 distinct bots and puts them into real games. Some win. Some lose. The run continues long enough for problems to appear outside a single happy path.

The browser is part of the same rehearsal. It uses the interface while the bots exercise the service underneath it. That matters because a feature can be present in the HTML and still be unusable. When a test fails, the run keeps evidence for an agent to investigate instead of leaving only a red status.

The rehearsal tells me more than scrolling through a large diff and convincing myself that each line looks reasonable. It still is not production. Live data has a history. Processes may have been running for weeks. A deployment may begin while poker hands are in progress. The production agent checks those conditions before it deploys, then verifies the result in the environment that produced the original problem.

The factory comparison is the closest one I know. A manager does not stand at the end of an automated line and inspect every item personally. The machines do the repeated work. The manager decides what should be produced and puts controls around the process. Software is moving in the same direction.

For Open Poker, those controls are specific. Chips cannot disappear. A season transition must be safe to retry. A private competition cannot leak into public play. A deployment should not interrupt a hand already in progress. If a production change exists outside version control, it has to come back. Those are better quality checks than asking whether I personally read every generated line.

The agents still make mistakes

This setup is not magic. Agents overbuild. A small request can come back with new abstractions and files I never asked for. They can become so focused on completing one task that they miss what the change does to the rest of the product. Interface work often needs another pass from me because technically correct is not the same as clear.

The failures keep moving. A new model or a better agent tool makes work that was difficult a few months ago routine. Then I ask for something more ambitious and find a different edge. I do not expect that to stop. Every improvement in the tools changes what I am willing to build with them.

I am also experimenting with AI analysis of how Pro users play. A user could ask for a review of a hand and receive feedback from one of the models behind the same gateway. This is still an experiment. Open Poker is not tied to one model provider, so I can change the model as the feature develops while keeping the same security and usage controls in front of it.

The workflow has to survive those changes. Models will be replaced. The separation between development and production remains. So do the tests and the written record in GitHub. I remain responsible for the direction of Open Poker and for decisions that create real risk. The agents handle most of the work between those points.

No one knows where this lands

Some developers will say that if I do not understand every line, I am not really building the product. I think that definition has already changed. Reading every generated line would consume the time AI gave back to me while creating a false sense of control. A line can look fine on its own and still belong to a bad design. I would rather spend that time deciding what to build, checking how a feature fits the system and using the product when it is ready.

Software development is shifting too quickly for anyone to know what the job will look like in a few years. The models are changing. The tools around them are changing. So is the point where a person needs to step in. Some work that needed close supervision a few months ago is now routine, while every improvement gives us the confidence to try something more ambitious.

What will not change are the fundamentals. The system still has to be secure. You still need to see what it is doing in production and understand why it failed. Cost still matters, especially when agents can work for hours and call several models along the way. Security, observability and cost are not side concerns that we can add after the agents finish. They are part of the environment the agents need in order to do useful work safely.

That is why Alex’s description feels right to me. Open Poker is not a finished object that I occasionally patch. It keeps supplying new facts about itself, and those facts shape what gets built next. I do not know exactly where this way of working will lead. I do know that it is the reason I can keep building a product that would otherwise have remained an idea.