Reborn from Failure: A Real-World Retrospective on Landing a Frontend AI Agent

Today at FEDay, I shared a case study on implementing a Frontend Agent. The core narrative wasn't about a victory lap; it was the story of how a team went from "Technical Success" to "Product Failure," and how that failure led to a crucial upgrade in our cognitive framework.

The value of this story lies not in a methodology for success, but in the pitfalls we encountered and the evolution of our thinking.

2025 is being hailed as the "Year of the Agent." With the release of Deep Research, Manus, and Claude Code, the tech community is buzzing.

Many teams are asking the same question: "Should we build an Agent?"

Before we dive in, let me clarify my definition of an AI Agent:

> AI Agent: A Large Language Model (LLM) that loops through tool calls to achieve a specific goal.

- Tools in a loop: Model calls tool -> Gets result -> Continues reasoning.
- Clear Endpoint: It works to achieve a goal, not to loop infinitely.
- Flexible Goal Source: The goal can come from a user or another LLM.
- Basic Memory: Maintains context through conversation history.

The Challenge: Private Design Systems

A friend's team was facing a genuine enterprise pain point: The company had a complete internal Design System and a private frontend framework. However, because this code was private, public AI models had never been trained on it. General-purpose models simply could not generate code that adhered to their internal specifications.

The goal seemed clear: Build a "Lovable-like" tool, but powered by their own Design System. Users would upload a Figma design or a screenshot, and the Agent would automatically generate frontend code compliant with internal standards.

Sounds perfect, right?

The Reality Check The challenges were substantial:

1. Building a complete Agent system is harder than it looks (User interaction, context engineering, etc.).

2. The model has to understand and use private components it has never seen.

3. We needed real-time browser previews of the generated code.

4. We needed auto-repair capabilities if the code failed.

The "Technical Success": How We Built It

As a technical consultant, my first piece of advice was pragmatic: "Get it running before you optimize." Building the Agent isn't the hardest part; completing the full execution loop is.

1. The Foundation: Claude Agent SDK

Instead of reinventing the wheel, we built upon the Claude Agent SDK.

- Proven: Claude Code proved this architecture works.
- Ready-to-use: Built-in tools cover 90% of scenarios.
- Extensible: Supports custom tools, MCP (Model Context Protocol), and custom Skills.

(You can find some of the prototype code open-sourced here: https://github.com/JimLiu/claude-agent-kit)

2. The Preview Solution: Local File System

We initially tried Sandpack (browser-based sandbox) for code previews, but it failed with complex private components.The Pivot: We gave the Agent a Local File System (a VM or directory per session). This allowed the Agent to freely read, write, modify, and compile code.

Giving the Agent a local file system is the only way to maximize its capabilities.

3. Solving the "Unknown Component" Issue

How do you teach AI to use a component library it has never seen?

Treat it like a new employee. We converted the Design System specs, component lists, and API docs into Markdown.

No complex RAG needed: We simply allowed the Agent to perform file retrieval on local documentation and "high-quality reference code."

4. Quality Assurance: The Verification Loop

To ensure the code actually worked, we built an automated loop: Generation -> Verification -> Repair

- Tools: Static Linting, Compilation checks, and Visual Diffing (using Chrome DevTool MCP).

- Optimization: We placed verification tools into a Skill or SubAgent to avoid polluting the main Agent's context window.

The "Product Failure": The Silence After Launch

The system worked. The demo was stunning. We launched it... and almost no one used it.

After the initial novelty wore off, abandonment rates skyrocketed. We conducted a deep post-mortem and user interviews, realizing the problem wasn't the technology—it was a misalignment between Product Logic and User Habits.

Why It Failed

1. Habit Resistance: Designers and PMs live in Figma, not chat windows. Moving from their comfort zone to a conversational interface was a massive friction point. Most didn't even know what to type.

2. The 80/20 Bottleneck: The Agent did 80% of the work perfectly. But the final 20% required manual modification, which was incredibly high-effort. Often, that 20% determined whether the code was usable at all.

3. Workflow Fragmentation: The generation environment was disconnected from the real development environment. Developers had to manually copy-paste code, making the process tedious.

The Cognitive Upgrade: Reframing the Problem

We realized we asked the wrong question: "How do we build a Design System AI Agent?" This made the Agent the goal, rather than the means.

The Right Question was: "What is the ultimate purpose of our Design System?"

1. Unified design specifications across the enterprise.

2. Increased development efficiency.

Shift 1: Design for AI, Not Humans

Current workflows are human-centric: Manual communication, iterative modification, manual confirmation.Future workflows must be AI-Centric: Input -> AI Agent -> Output.

New Design Principles:

- AI-Friendly: Choose tech stacks that LLMs understand easily.

- Lightweight: Keep only Design Tokens. Extend upon AI-friendly open-source systems (like shadcn/ui) rather than maintaining a massive private library.

Shift 2: From "Agent" to "Skill"

The most critical pivot was abandoning the "Independent Agent Platform."

Old Model (Island): A standalone Agent isolated from the developer, causing friction.

New Model (Integration): Turn the Design System into a Skill that can be embedded into existing AI development environments (like Cursor or Claude Code).

What is a Skill?

It is simply Markdown Documentation (for the AI to read) + Automation Scripts (to initialize projects and install the system).

Now, a developer works in their familiar environment. When they need the Design System, the generic Agent calls this "Skill," and the generated code goes directly into the project repository.

(Reference approach: anthropics/skills/tree/main/skills/web-artifacts-builder)

Deep Insights: 4 Key Takeaways

1. Technical Success != Product Success

Many engineers (myself included) fall into the trap of thinking "It works, therefore it is successful." Users don't care about your tech stack; they care if it solves their problem without breaking their flow.

2. Design "AI-Centric" Workflows

We talk about "User-Centric," but we must add a layer: "AI-Centric." Don't make AI mimic human workflows. Redesign the workflow so the AI can operate at peak efficiency, then let the human enjoy the result.

3. Skill > Agent

Independent Agent platforms face high adoption barriers. Encapsulating capabilities as a Skill that plugs into the existing ecosystem is a far more pragmatic path.

4. Action

Even though the initial product "failed," the cognitive upgrade was priceless. You cannot learn to shift from "Human Workflow" to "AI Workflow" without getting your hands dirty.

Just Build It!

Failure is acceptable. It is infinitely better than doing nothing at all.