Role And Org-Structure Rebundling Around Agents
Sources: 1 • Confidence: Medium • Updated: 2026-03-25 17:51
Key takeaways
- As coding becomes easier to automate, the software engineering role shifts from writing code toward iterative verification and alignment with business and user goals.
- Tasks tied to Knightian uncertainty and unknown unknowns remain hard to automate because future states are not measurable enough to assign reliable probabilities.
- AI agents can produce large volumes of work quickly, but the output is often subtly flawed in ways that are easy to miss without careful review.
- A durable moat in AI businesses can come from proprietary interaction and failure data that enables machine-scale verification and superior risk underwriting, lowering cost while improving safety.
- Extremely low automation costs increase systemic risk because AI-generated code and content is rationally shipped with unverified errors that humans cannot fully review at scale.
Sections
Role And Org-Structure Rebundling Around Agents
- As coding becomes easier to automate, the software engineering role shifts from writing code toward iterative verification and alignment with business and user goals.
- A proposed organizational structure for AI-heavy firms is a small group of human directors setting intent, agents executing, and a bottom layer of top verifiers ensuring outputs match intent at scale.
- Creating labels, evaluations, and training data for verification can accelerate automation in that domain and reduce future demand for similar verification labor.
- Top verifiers must continually move up the value stack to stay ahead as automation expands.
- AI agents have recently improved such that they can complete longer-running tasks with much less step-by-step guidance than before.
- A hollow-economy risk arises when firms do more with fewer people (especially fewer juniors) while verification demands rise and incentives favor deploying unverified AI, potentially shrinking the future verifier class.
Measurement Limits And Where Automation Stalls
- Tasks tied to Knightian uncertainty and unknown unknowns remain hard to automate because future states are not measurable enough to assign reliable probabilities.
- Some tasks are not improvable via measurement because they are social constructs or status games where groups coordinate on shared meaning rather than objective correctness.
- A key automation boundary is whether what matters is measured outside a human brain versus inside an individual's lived experience.
- As people carry devices that capture richer audio, video, and other signals, the barrier to externalizing and measuring individual experience will decrease.
- Seasoned engineers have lived experience that yields internal models which weight rare or hard-earned examples differently than systems that only ingest code.
- Machines can be average or above average on standard components of professions like law, engineering, and strategy because they have ingested sufficient examples and materials.
Verification As The Binding Constraint
- AI agents can produce large volumes of work quickly, but the output is often subtly flawed in ways that are easy to miss without careful review.
- As coding becomes easier to automate, the software engineering role shifts from writing code toward iterative verification and alignment with business and user goals.
- Extremely low automation costs increase systemic risk because AI-generated code and content is rationally shipped with unverified errors that humans cannot fully review at scale.
- Firms face a near-term incentive conflict where investing in verification tooling and cryptographic primitives is expensive and slows shipping while the benefits accrue later.
- Automation tends to improve as tasks become measurable, while verification is the residual work of ensuring outputs match intent under real-world nuance and exceptions.
Trust, Provenance, And Crypto As Verification Infrastructure
- A durable moat in AI businesses can come from proprietary interaction and failure data that enables machine-scale verification and superior risk underwriting, lowering cost while improving safety.
- Deterministic cryptographic provenance and identity can reduce verification costs and restore trust in digital information, making crypto complementary to AI.
- A high-trust future with cheap automation requires a strong verification stack to maintain ground truth and resist fake identities and coordinated manipulation such as Sybil attacks.
- In a reported case, switching from legacy payments to stablecoin rails made an agentic commerce system more reliable because transaction signals were fully on-chain rather than behind brittle APIs and intermediaries.
Risk Pricing Via Liability And Insurance
- Extremely low automation costs increase systemic risk because AI-generated code and content is rationally shipped with unverified errors that humans cannot fully review at scale.
- A durable moat in AI businesses can come from proprietary interaction and failure data that enables machine-scale verification and superior risk underwriting, lowering cost while improving safety.
- Firms face a near-term incentive conflict where investing in verification tooling and cryptographic primitives is expensive and slows shipping while the benefits accrue later.
- Liability and insurance will become increasingly central as agents are deployed as labor-as-software, including productized insurance for autonomous systems in production.
Unknowns
- What are the measurable, end-to-end productivity and quality deltas (cycle time, defect rates, incident rates, security findings) for AI-agent workflows versus non-agent baselines in real production settings?
- How quickly are organizations actually reallocating engineering time from implementation to verification and intent alignment, and what new roles are being formalized (if any)?
- Does verification labor meaningfully self-erode in practice as evaluation work becomes training signal, and if so, at what pace and in which domains?
- What concrete evidence exists that professional-standard tasks are now handled at average or above-average levels by machines across law, engineering, and strategy, and under what constraints and failure modes?
- Will richer data capture actually externalize lived experience in ways that are usable and permissible for automation, and what constraints (privacy, policy, operational) limit this trend?