Methodology: Harness Engineering and Scaling the IRify R&D Practice

In 2026, large-model capabilities are widely seen as having entered a plateau: the brute-force aesthetics of simply piling on more compute now yields sharply diminishing marginal returns, and high-quality human training corpora are nearing exhaustion. Confronted with this bottleneck, the industry's focus has begun to shift. There is a growing consensus that the era of competing purely on models is fading, and an era of competing on Harness Engineering has arrived. It is much like building a car: once you have squeezed the engine's horsepower to its physical limit, further gains in overall vehicle performance can no longer come from staring only at the engine — your attention must turn to the systems that actually harness that engine, such as the drivetrain and the steering wheel.

Against this backdrop, the term "Harness Engineering" has started flooding tech media, packaged as yet another cutting-edge AI breakthrough. Yet this grand narrative feels deeply disconnected from reality: for the vast majority of ordinary developers who do not build low-level Agents, "Harness" sounds more like an irrelevant pseudo-concept. Beyond stoking the anxiety that comes with rapid technological iteration, it seems to have little tangible impact on day-to-day development work.

Today's AI industry has settled into a pattern where a handful of big tech firms lead the way while everyone else follows anxiously in their wake. In truth, the excellent AI coding tools that ordinary developers rely on every day — Claude Code, Codex, Cursor, and the like — are themselves the products of Harness Engineering best practices. Yet when we use these tools, we often fall into an exhausting loop: day after day, we spend enormous amounts of time in chat windows feeding requirements to the AI, watching it write new features line by line, and then manually switching back to the system to review and verify the results. In this "AI writes the code, human runs the errands" mode, I sometimes cannot help but wonder: should I be happy about the work getting done efficiently, or should I be saddened by scurrying around doing low-value labor?

After this comedy of frustration, a question kept turning over in my head: we are all caught in the same wave of Harness Engineering, so why do the big tech firms see it as the core of how large models deliver productivity — while we ordinary developers, clearly enjoying its best practices every day and seeing our efficiency rise substantially, feel our sense of self-worth steadily eroding with each round of AI conversation?

The Scarcity of Human Attention

I think that to answer this question, we can go back to the original article OpenAI published on Harness Engineering and examine what problem Harness was originally meant to solve. That article devotes considerable space to the details of its engineering practice — designing environments, feedback loops, and control systems — but I believe its core point, the one the author flags right at the start, is this: the problem that human time and energy are scarce.

The article describes how OpenAI had an internal million-line-scale project that needed AI involvement and had to be delivered within weeks. Under the traditional model of human-AI collaboration, the Agent produces pull requests and a human serves as the quality gatekeeper — but this runs into a severe problem: AI generates code far faster than humans can verify it. The human becomes the bottleneck of the whole development flow. Worse still, if a feature the AI implements does not meet expectations, you have to talk it through, wait for a fix, and verify the result all over again. If the AI does not fix it correctly on the first attempt, you are trapped in this loop, burning through your energy until you are exhausted. This development flow is like a giant funnel, and the human is its narrowest opening. To widen that narrowest opening, OpenAI continuously optimized the large model's execution environment, and named this process Harness Engineering.

So it is not hard to see that the core pain point Harness thinking addresses is not only the stability of AI's sustained work, but even more so the "scarcity of human attention" in the AI age. Looking back over the evolution of programming, you can trace how the focus of human attention has shifted. In the pre-AI era, people poured 100% of their attention into writing code and implementing features. In the era of AI-assisted coding (IDE plugins), AI became a code-completion tool, yet human attention remained tethered to the concrete steps of implementation. Now we have entered the era of autonomous Agent programming, where models possess the ability to act on their own. At this point, if humans cling to old habits, they fall into an attention-devouring dead loop: the AI executes tasks in a chat window while the human manually switches to the system to click, compile, and run acceptance tests.

To break this vicious, attention-draining loop, you must ensure that the AI runs autonomously and reliably, and reduce the need for human involvement. Building a highly observable, auto-verifiable execution environment for the AI is merely the concrete engineering step. In my view, the deeper essence of Harness is that it reminds us we should "reflect on and manage our attention." The specific transformation techniques do not matter; what matters is that in your daily work you must keenly identify exactly which "blocker" is ruthlessly occupying and consuming the largest share of your attention. Once you find that blocker, you should take the initiative: set the rules for it, build the right environment, and offload that verification work to the AI. That is the Harness that belongs to every ordinary developer.

On the question of how to "actively manage attention," I had firsthand experience while pushing IRify toward scale recently. In this project I went through a real transformation — from "the more I use AI, the more exhausted I get" to "building a Harness workflow for it."

Harness Practice in IRify R&D

Over the past few months, my task has been to upgrade IRify from client-side code scanning to distributed scanning. This involves not only complex concurrency schemes and state transitions, but also a large amount of SaaS-feature grunt work. To save time, I spun up multiple AI Agent terminals and developed in parallel. AI certainly writes code fast, but trouble followed: code conflicts became frequent; and because this is a SaaS system with a frontend, unit tests alone could hardly guarantee that features actually worked. The result was that every time the AI spat out a feature, I had to manually launch the page, log in, click around, and hunt for errors. The faster the undisciplined AI ran, the more of my attention was drained as a human test runner.

To turn this reactive situation around, I began experimenting with building a Harness execution environment for the AI. First, I added constraints to the AI's global prompt: it was forced to use an isolated worktree for development, physically preventing code conflicts. Second, every time the AI implemented a feature, it had to land tests alongside it — for hard-to-test points it could write E2E test scripts or even do log instrumentation for testing. Finally, to make frontend acceptance easier, I had it add a Chrome DevTool so it could take screenshots of the frontend once a task was done.

If you have read OpenAI's original article, these measures may look very familiar. That is right — I essentially fed the Harness source material directly to the AI and asked it to retrofit itself according to that set of best practices. Looking back on the process, I realized that ordinary people practicing Harness is not something esoteric; its essence is simply "consciously making the AI's running environment better, with less friction." All I did was proactively feed it the industry's best practices and let the AI, following the current, smooth the road itself and reshape its own dedicated execution environment. After this overhaul, I could finally dispatch a large batch of tasks at once: the AI writes the code, auto-verifies it, and if it finds errors it goes back to the furnace and reworks them, while I only need to sit securely in the "final acceptance" step.

However, as more and more AI workflows ran automatically end-to-end, I spotted a new blocker eating at my attention: during acceptance I had to spend a lot of time reviewing the AI's summary text and screenshots. Because an AI running in a terminal cannot display images directly, I was confronted every day with a pile of scattered text and screenshot paths it had thrown at me, and I had to manually open the folder and click through them one by one. When many tasks ran in parallel, just "matching screenshots to tasks" consumed an enormous amount of my mental energy. To reclaim my attention completely, I realized: Harness is not only about making the AI do the work automatically — it also has to make the AI report back in the way that imposes the "lowest cognitive load." So I put in place the final piece of the Harness puzzle: I forced the AI, after completing each task, to lay out all verification results and screenshots strictly in a "newspaper layout" format. Because newspaper-style mixed text-and-image layout is the medium that, over centuries, has proven best suited to how humans visually scan and absorb information. Now I only need to glance over these richly illustrated summaries, like reading the morning paper, to sign off on every feature in one sweep — colloquially, "cyber morning paper"~

"What step it has reached, and what is still unread,

IRify, the full topic path marked completed"

"Cyber morning paper" illustration

AI auto-screenshot illustration

In fact, there are many tools we can lean on to consciously clear away the AI's running friction. You can write Python or Shell scripts to have the AI perform a fixed operation, or you can pin down prompts to make it precisely understand a task's intent. When "scripts" and "prompts" are combined, you get a concept that has been hugely popular over the last couple of years: Skills. We all go hunting for popular, useful Skills to give our Agents wings. But in my view, genuine Harness thinking asks us to do more: do not be mere consumers of Skills — actively "distill" your own dedicated Skills out of complex business workflows.

This kind of "distillation" packs astonishing power when handling long-chain, complex tasks. Take the BenchMark validation I recently did on IRify: I needed the AI to scan a specific project, find false negatives and false positives, fix rules, and finally add tests to lock the content down. Under a traditional AI-conversation mode, this process harbors three major "attention black holes": first, the task chain is extremely long and forces constant context-switching; second, the AI does not understand the specific rule syntax (SyntaxFlow), so every pass it wastes a lot of time referencing old rules; third, the project has a large number of test suites, and the AI often buzzes around like a headless fly unable to find the right place to add tests. In the old conversational way of driving the AI, not only was it inefficient, but when it erred I had to keep chatting to nudge it, which would wear me down. But now, after patiently running this workflow — covering environment configuration, background knowledge, and execution logic — end to end in a single conversation, I simply "distilled" it into a dedicated Skill. Facing the same requirement later, I just trigger it with one click and the AI executes automatically under minimal friction, completely spared the torment of repeated conversation and coaxing.

The Skill distilled from IRify's business

A Working Philosophy

Having been through these real battles, I am increasingly certain of one thing: Harness is by no means just a few lines of script or a few sets of prompts — at its core it is an engineering idea, and even more so a working philosophy / methodology. To push it further, it is not even something new — it is old wine in a new bottle, its core being the engineering thinking of DevOps, process engineering, and automation, which only looks like a fresh revolution under the halo of the AI context. But it should also be admitted that it does genuinely remind us to shift how we think about work. Granted, the industry already has excellent best-practice products like Claude Code, but they are by no means a one-size-fits-all panacea. In real-world daily projects that vary wildly and are full of "grunt work," no off-the-shelf tool can perfectly fit everyone's workflow. This is exactly why every ordinary developer must keep this Harness awareness alive at all times in daily work.

I believe this working philosophy can be applied along two axes in daily development. For ourselves, we should keenly identify where our precious "attention" is leaking away, and actively seize it back; for the AI, we should change roles and, as a "facilitator," observe where its "blockers" are when executing tasks, and proactively build bridges and pave roads for it and optimize its running environment.

Of course, holding this Harness philosophy matters not only from the perspective of an "Agent user"; I believe "Agent developers" need it even more. I often hear friends doing Agent development say they are building wheels just like Claude Code or Codex. But I think they are more like assembling various wheels together to fit the vertical business of a specific industry. There is no such thing as a "general Agent" that fits every scenario perfectly; sometimes we find a particular Agent powerful largely because the underlying large model is itself strong. But once model capability development hits a certain ceiling, identifying the blockers in how the model runs in a specific scenario and reshaping its running environment is, in my view, still extremely valuable.

The recently released product Memfit, I think, illustrates this philosophy well. Memfit is an Agent designed for the cybersecurity domain, capable of handling non-deterministic tasks like penetration testing. Unlike programming tasks with clear requirements, penetration testing is highly "divergent" and non-deterministic: the AI can only grope for a test path while gathering information, and its course can be rerouted at any moment by newly discovered intelligence. This uncertainty dooms Memfit to be unable to simply copy Claude Code's design philosophy. Take the core "Plan mode" as an example: a conventional programming Agent executes a set Plan strictly and rigidly, whereas Memfit's Plan "grows" in real time — it dynamically revises and rewrites the subsequent test plan based on new information collected during execution.

Refusing to fetishize a generic, rigid process, and instead tailoring a self-correcting running environment for uncertainty — that, I believe, is what Harness truly is. Whether facing the clear-requirement, conventional development of IRify, or the unknown-filled penetration testing of Memfit, the kernel of Harness never actually changes. It asks us to step outside the frame of "passive execution," gain insight into the real friction in a specific business scenario, and then tailor the most fitting running rules for the AI. Genuinely useful productivity does not depend purely on an almighty large model — it depends on a Harness execution environment that fits the specific business.

Conclusion

I feel deeply how much AI has reshaped programming over the past two years. Thinking back two years ago, I still had to spend whole days poring over low-level SSA code, constantly tuning how a specific function should be written; a year ago, it was endless conversations with Cursor to refine the details of a specific feature; and now, what I need to do is build my own automated execution environment, tell it where to go, and let it find its own way. When we stop being a passive, worn-out "code review machine" and instead personally clear the obstacles for the AI, we transform into the true masters who actually steer complex systems. Only then does that long-lost joy of self-actualization truly return to our hands.

References: OpenAI. (2026). Engineering Technology: Harnessing Codex in an Agent-First World (Harness Engineering). Retrieved from: https://openai.com/index/harness-engineering/ Dowson Tang. (2026). AI Officially Enters the Harness Era. Tencent CSIG. Huang Jia. (2026). From GoF to Agent: 30 Years of Design Pattern Evolution and the Harness Revolution. V1ll4n. (2026). Memfit AI: A Production-Grade AI Agent for Continuous Penetration Testing Without Losing Its Way. Yak Project. The Architect of Connectivity: A Comprehensive Technical Analysis of Modern Harness Engineering. (2026). Datawhale. (2026). Latest! A 10,000-Word Survey of the Harness Revolution!

---

This article was first published on the Yak Project WeChat official account. Read the original.