Agents Still Fail
A modern agent works in a loop: it reads the current state, proposes a next action, executes it, observes the result, and continues.
Throughout this post, we use agent to mean the full system interacting with the task environment, and policy to mean the underlying language model that proposes each next action. A trajectory is the resulting sequence of states, proposed actions, executions, and observations produced as the agent works toward a goal.
The reasoning can be fluent. The tools can work. The format can be correct.
And the agent can still fail.
These failures are often surprisingly repetitive (Cemri et al., 2026). The policy answers before verifying, commits to an early mistake and never revisits it, or repeats the same unproductive search. Across trajectories, the surface details change — but the failure patterns often do not.
The problem is often not whether the agent can solve the task. It is which step the policy chooses next, and whether anything catches that decision before it is executed.
That repetition creates an opportunity. If a failure recurs, we can name it. If we can name it, we can specify when it happens. And if we can specify when it happens, we can intervene.
What should a reusable skill look like if we want it to do more than give advice?
The full story is also available as a short video — from the gap in the agent loop, to executable skills, to three concrete interventions and results across nine benchmarks: watch the 2.5-minute demo.
- Cemri, M., Pan, M. Z., Yang, S., Agrawal, L. A., Chopra, B., Tiwari, R., Keutzer, K., Parameswaran, A., Klein, D., Ramchandran, K., Zaharia, M., Gonzalez, J. E., & Stoica, I. (2026). Why Do Multi-Agent LLM Systems Fail? The Thirty-Ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track. https://openreview.net/forum?id=fAjbYBmonr
Program Functions: Executable Skills
The obvious way to fix a recurring failure is to give the agent a reusable skill.
Today, that skill is overwhelmingly represented as text. Reflexion (Shinn et al., 2023), ExpeL (Zhao et al., 2024), and Voyager (Wang et al., 2023) store reusable experience as verbal lessons; more recent systems such as MemSkill (Zhang et al., 2026), SkillRL (Xia et al., 2026), and SAGE (J. Wang et al., 2026) study how those memories can be selected and evolved.
But however the skill is stored, it usually reaches the policy in the same way: pasted into the prompt, retrieved as advice, or folded into a learning signal.
That makes the skill flexible.
It also makes the skill passive.
A rule such as “avoid repeated searches” expresses the right idea, but does not say exactly when it should fire or how it should change the action the policy just proposed. The policy still has to decide whether to follow the advice — so the skill can be ignored, forgotten, or applied at the wrong moment.
HASP makes one change:
Compile the skill into an executable program.
From advice to intervention
HASP — Harnessing LLM Agents with Skill Programs — is a runtime harness around the policy. It does not replace the policy or generate the entire trajectory itself. Instead, it inspects the policy’s proposed action before execution and can selectively repair that proposal.
HASP represents each reusable skill as a Program Function (PF).
A Program Function has two jobs:
Detect. should_activate(...) decides whether the current state and proposed action match the failure pattern encoded by the skill.
Repair. intervene(...) decides what should happen instead. It returns a typed intervention, such as modifying the proposed action, using INJECT_CONTEXT to provide corrective context, or abstaining when no repair is appropriate.
The interface is deliberately small:
detect the failure state → repair the next decision
Two modules, and nothing else
At runtime, HASP maintains a PF library: the collection of executable skills currently available to the agent. The policy first proposes the next action. HASP retrieves candidate PFs from this library, evaluates their activation conditions, and intervenes only when one fires. If no PF activates, the original action executes unchanged.
Some variants additionally use an optional teacher model when several PFs are plausible. The teacher helps select among candidate interventions; it does not replace the PF that actually performs the repair. We use PF-only to refer to the same runtime mechanism without this teacher.
This creates a simple separation of responsibilities:
The policy proposes. The skill decides whether that proposal needs repair.
And because HASP records both the policy’s original proposal and the repaired action,
every intervention is explicit and auditable. That same before-and-after record will later become useful for training and skill evolution.
This places HASP between skill-based agents and learning-based agents. A PF can simultaneously serve as reusable memory, runtime control, a structured learning signal, and an object that can itself evolve.
How reusable experience is represented
The comparison below separates five capabilities:
- Skill / memory form: whether reusable experience is represented explicitly.
- Runtime control: whether that experience can directly affect the agent’s next decision at inference time.
- Learning signal: whether interaction produces structured supervision that can later support learning.
- Policy training: whether those signals can be used to update the policy’s model weights.
- Skill evolution: whether the external skill collection itself can be revised or expanded.
The distinction is therefore not simply that HASP stores skills.
The stored skill becomes an executable part of the agent loop.
- Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. https://arxiv.org/abs/2303.11366
- Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2023). Voyager: An Open-Ended Embodied Agent with Large Language Models. https://arxiv.org/abs/2305.16291
- Wang, J., Yan, Q., Wang, Y., Tian, Y., Mishra, S. S., Xu, Z., Gandhi, M., Xu, P., & Cheong, L. L. (2026). Reinforcement Learning for Self-Improving Agent with Skill Library. https://arxiv.org/abs/2512.17102
- Xia, P., Chen, J., Wang, H., Liu, J., Zeng, K., Wang, Y., Han, S., Zhou, Y., Zhao, X., Chen, H., Zheng, Z., Xie, C., & Yao, H. (2026). SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning. https://arxiv.org/abs/2602.08234
- Zhang, H., Long, Q., Bao, J., Feng, T., Zhang, W., Yue, H., & Wang, W. (2026). MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents. https://arxiv.org/abs/2602.02474
- Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.-J., & Huang, G. (2024). ExpeL: LLM Agents Are Experiential Learners. https://arxiv.org/abs/2308.10144
Three Domains, One Pattern
What that means in practice is easiest to see through three trajectories.
All three use the same Qwen2.5-7B-Instruct (Qwen et al., 2025) policy with unchanged model weights. The difference between the failing trajectory and the corrected one is simply that a Program Function is allowed to fire.
Web Search — read before answering
The question asks which Walton died after John Walton was married. The correct page is already in the search results, but the policy never opens it and proposes FINAL too early.
The insufficient_exploration PF recognizes that the policy is about to answer without reading the available evidence and changes the next action:
FINAL → READ
The page opens, the policy finds the evidence, and the final answer becomes Sam Walton.
(FINAL, answer) → (READ, relevant result).Mathematical Reasoning — count the right object
The problem asks how many coefficient pairs produce three distinct integer roots. The policy instead counts orderings of the roots and gets nine.
The arithmetic is not the real problem — the object being counted is.
The PF redirects the intermediate reasoning target:
root triples → coefficient pairs
The policy then continues its own reasoning and reaches the correct answer: five.
Coding — change the strategy
The task asks for a root of an arbitrary polynomial. The policy reaches for the quadratic formula and fails on the first public example.
Here the PF changes the approach itself:
quadratic formula → bracket a sign change → bisect
Every example passes.
In this case, the PF uses INJECT_CONTEXT: it supplies the corrected strategy as context while leaving source-code generation to the policy itself.
INJECT_CONTEXT, leaving source-code generation to the policy itself.A premature answer, a wrong counting target, and a wrong algorithm are very different failures.
But the repair follows the same pattern:
detect → intervene → correct → continue
Three examples show what the mechanism looks like.
The next question is whether it scales beyond individual cases.
- Qwen, :, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., … Qiu, Z. (2025). Qwen2.5 Technical Report. https://arxiv.org/abs/2412.15115
Nine Benchmarks, No Training
Before introducing post-training, we first ask what executable skills can do with the policy itself completely frozen.
Our baseline is a multi-loop ReAct agent (Yao et al., 2023): the policy repeatedly alternates between reasoning, tool actions, and resulting observations until it produces a final answer.
We evaluate HASP across web-search reasoning, mathematical reasoning, and coding using the same Qwen2.5-7B-Instruct (Qwen et al., 2025) policy.
The model weights remain frozen: there is no parameter update.
Across the reported benchmarks and coding variants, allowing Program Functions to intervene improves the original multi-loop ReAct agent. The gains are especially large in web search and coding, while mathematical reasoning improves more modestly — consistent with PFs being strongest at repairing recurring failure patterns rather than discovering entirely new reasoning strategies.
Even without the optional teacher model, PF-only intervention places the 7B agent 8.5 points above GPT-4o on web-search reasoning.
Web-search and mathematical reasoning follow the training and evaluation protocol of AgentFlow (Li et al., 2025); coding follows the same training-data principle.
Evaluation covers nine benchmarks: HotpotQA (Yang et al., 2018), 2Wiki (Ho et al., 2020) and MuSiQue (Trivedi et al., 2022); AIME24, AMC23 and GameOf24; HumanEval (Chen et al., 2021), MBPP (Austin et al., 2021) and BigCodeBench (Zhuo et al., 2024), the last three reported with their extended variants.
- Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., & others. (2021). Program Synthesis with Large Language Models. arXiv Preprint arXiv:2108.07732.
- Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., … Zaremba, W. (2021). Evaluating Large Language Models Trained on Code.
- Ho, X., Duong Nguyen, A.-K., Sugawara, S., & Aizawa, A. (2020). Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps. Proceedings of the 28th International Conference on Computational Linguistics, 6609–6625. https://www.aclweb.org/anthology/2020.coling-main.580
- Li, Z., Zhang, H., Han, S., Liu, S., Xie, J., Zhang, Y., Choi, Y., Zou, J., & Lu, P. (2025). In-the-Flow Agentic System Optimization for Effective Planning and Tool Use. https://arxiv.org/abs/2510.05592
- Qwen, :, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., … Qiu, Z. (2025). Qwen2.5 Technical Report. https://arxiv.org/abs/2412.15115
- Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022). MuSiQue: Multihop Questions via Single-hop Question Composition. https://arxiv.org/abs/2108.00573
- Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., & Manning, C. D. (2018). HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. https://arxiv.org/abs/1809.09600
- Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. https://arxiv.org/abs/2210.03629
- Zhuo, T. Y., Vu, M. C., Chim, J., Hu, H., Yu, W., Widyasari, R., Yusuf, I. N. B., Zhan, H., He, J., Paul, I., & others. (2024). BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions. arXiv Preprint arXiv:2406.15877.
Beyond Intervention: Teach & Evolve
Making a skill executable changes more than inference.
A textual skill is placed in context and consumed. A Program Function leaves a structured record every time it runs: when it fired, what the policy wanted to do, what it did instead, and whether the repair helped.
That turns one skill interface into a full learning loop:
act → teach → evolve
Act
At inference time, PFs detect recurring failure states and repair the next decision without changing model weights.
The teacher model introduced earlier is optional. When used, it can help select among multiple plausible PFs; the PF itself still performs the intervention.
Teach
Every activation gives HASP a natural before-and-after pair:
The system therefore observes exactly where the policy’s original proposal was replaced and what corrected action continued the trajectory.
These PF-corrected trajectories can then be used for post-training. We study three training recipes:
- Supervised fine-tuning (SFT): directly train the policy on corrected trajectories.
- Rejection sampling (RS): retain higher-quality corrected trajectories and train on the selected examples.
- On-policy distillation (OPD): train using corrections collected from trajectories generated by the policy being updated.
Together, these recipes test whether behavior that first exists externally in PFs can later be internalized into the policy — that is, transferred into the model weights so that the policy can reproduce more of the corrected behavior on its own.
Each intervention is evaluated along four dimensions:
| Signal | What it asks | Weight | Cost if removed |
|---|---|---|---|
| Timing | Did we intervene at the right moment? | 0.15 | −7.8 |
| Mode | Did we intervene in the right way? | 0.10 | −15.5 |
| Correctness | Was the repair itself valid? | 0.25 | −12.1 |
| Outcome | Did the intervention actually help? | 0.50 | −12.8 |
A trajectory is therefore selected not simply because it eventually reaches the right answer, but because the intervention occurred at an appropriate state, used an appropriate repair mode, produced a valid correction, and improved the outcome.
Evolve
Training does not eliminate every failure.
After updating the policy, HASP revisits its residual failures — failures that still occur under the updated model. It identifies recurring failure–repair patterns, compiles them into new candidate PFs, validates those candidates, and adds accepted skills back into the PF library.
The loop closes:
A failure becomes a skill.
The skill repairs behavior.
The repair becomes supervision.
The remaining failures become new skills.
Fixed and evolving skill libraries
We study the three training recipes under two library settings.
A fixed library keeps the original PF collection unchanged while the policy is trained.
An evolving library allows newly generated and validated PFs to be added as training proceeds.
Combining the three training recipes with these two library settings gives six experimental configurations, E1–E6. Throughout, inference-time HASP refers to the frozen-policy intervention setting before any post-training, with a web-search reasoning average of 56.2.
The difference between the settings is revealing.
OPD performs best when the skill library is fixed, but becomes unstable when both the policy and the library change simultaneously. In that regime, the policy is learning while the external source of corrections is also changing, creating a moving target for optimization.
Rejection sampling better absorbs this additional variance, making HASP-Evolve + RS our main closed-loop setting: policy updates change the remaining failures, those failures generate new PFs, and the evolving PF library in turn changes the supervision available to the next policy update.
Evolution also needs filtering
Generating a candidate PF does not mean it should automatically enter the library.
Before a new PF is accepted, HASP applies evolution filtering: validation steps designed to reject skills that are invalid, overly specific, or harmful. The complete pipeline includes two complementary checks:
- Executable validation: checks whether the generated PF can be executed as a valid intervention.
- Teacher review: uses a teacher model to review the quality of the proposed skill, including its concept, trigger, intervention, executability, and validation behavior.
Filtering matters dramatically. The complete evolving system reaches 60.3 on web-search reasoning. Removing evolution filtering collapses performance to 36.3, worse than not evolving at all. Executable validation alone reaches 48.8, while teacher review alone reaches 47.2.
A self-improving loop must also know when not to learn.
What kinds of skills survive evolution?
What enters the PF library matters as much as how large the library becomes.
We group related PFs into skill families: sets of skills that target the same recurring type of failure and repair. An evolution round is one cycle in which residual failures are collected, candidate PFs are generated and validated, and accepted PFs are added to the library.
Useful families such as reasoning_hallucination repeatedly repair high-frequency failures. Overly specific skills contribute little, and some generated PFs are actively harmful — pf_override_harmful averages −0.8.
More skills are not automatically better skills.
Some skills migrate into the policy. Others should stay outside it.
Training also changes which skills remain necessary.
Some PFs correct persistent habits of the policy. For example, retrieval_failure and multi_hop_reasoning_failure eventually stop firing because the policy has internalized the corrected behavior: after post-training, it performs more of that behavior directly from its own model weights.
Other PFs, such as decompose_complex_question, remain useful because their activation depends on the structure of the incoming task rather than on a model habit that can simply be trained away.
This suggests two complementary forms of reusable skill:
behavior-correcting skills can migrate into the model; input-dependent skills can remain online as external control.
And the effect is broader than simply eliminating PF activations.
Only 8–38% of the final accuracy gain is directly explained by PFs no longer needing to fire. The larger effect is a shift in the policy’s overall behavior: it searches more effectively, reads more evidence, and adopts stronger reasoning patterns.
Executable skills therefore do more than repair individual actions.
They can teach the policy how to behave differently.
Conclusion & Limitations
HASP starts from a simple observation: agents often fail in recurring ways, but the skills we give them are usually passive text.
We instead represent reusable experience as Program Functions — executable state-to-intervention skills that know when to activate and how to repair the policy’s next decision.
That single change gives the same skill object three roles:
Act. Repair the agent online.
Teach. Turn interventions into structured supervision.
Evolve. Compile residual failures into new validated skills.
Across web-search reasoning, mathematical reasoning, and coding, this improves a frozen 7B policy at inference time, supports selective internalization through post-training, and enables controlled growth of an external PF library.
The broader idea is not that Program Functions replace policy optimization — training methods that improve the underlying model weights.
It is that agents can improve along a second axis.
Instead of putting every improvement into model weights, we can make reusable skills explicit, executable, and selectively internalizable.
HASP turns a skill from something an agent reads into something the system can run.
HASP — Skills you can execute.
There are also clear limits.
Teachers are optional at inference time, but stronger variants still use them for PF selection, evolution validation, or distillation. PFs are better at recognizing and repairing recurring strategies than discovering fundamentally new ones, which is most visible on AIME24.
Our current domains also provide relatively clean verification signals, so transfer to open-ended or weakly verifiable environments remains untested.
Finally, the intervention vocabulary is intentionally small, and evolving PF libraries can drift or accumulate harmful skills without strong validation.