Identity Drift in Long-Running AI Agents

TL;DR

• Long-running AI agents accumulate contradictory user understanding over time, creating “identity drift” that degrades output quality without clear error signals
• Drift is caused by the absence of structured contradiction resolution, agents append new context without reconciling it against existing understanding
• Self-models prevent drift by maintaining an authoritative, continuously validated representation of user identity

Identity drift in long-running AI agents is the gradual accumulation of contradictory user understanding that degrades output quality without triggering any error signals. Unlike forgetting, where the agent loses information, drift means the agent retains everything but loses coherence, because append-only memory systems store new observations without reconciling them against existing beliefs. This post covers the four stages of identity drift, why standard mitigation approaches fail, and how self-models act as identity anchors that maintain coherent user understanding over months of interaction.

Anatomy of Drift

Identity drift develops through four stages, each building on the previous:

Stage 1: Accumulation (Months 1-2). The agent builds a growing pool of user context. Early interactions establish baseline understanding. New observations are appended to the context store without conflict because the user’s behavior is relatively consistent.

Stage 2: Divergence (Months 2-4). The user’s needs, preferences, or circumstances begin to evolve, as they naturally do over time. New observations start to conflict with earlier ones. The agent stores both without resolution. The context now contains valid-but-outdated information alongside current information, with no mechanism to distinguish between them.

Stage 3: Interference (Months 4-6). The contradictory context begins to affect output quality. When the agent retrieves historical context, it pulls a mix of current and outdated information. Its responses become hedged, inconsistent, or inappropriately calibrated. The user notices that the agent “does not quite get them” anymore.

Stage 4: Incoherence (Months 6+). The accumulated contradictions reach a threshold where the agent’s behavior becomes noticeably erratic. It oscillates between treating the user as a beginner and an expert. It references completed projects as if they are ongoing. It contradicts its own previous recommendations. Trust erodes rapidly.

The Root Cause: Append-Only Memory

Identity drift has a precise technical cause: most agent memory systems are append-only. New information is added to the context store without reconciling it against existing information. The system accumulates assertions without maintaining a coherent model.

In database terms, agent memory systems have inserts but no updates. When a user’s risk tolerance changes from conservative to moderate, the system does not update the existing record: it adds a new record alongside the old one. Both records persist. Both are retrievable. The retrieval system has no mechanism to determine which one reflects the user’s current state.

This is not a bug in any specific agent framework. It is a design choice that optimizes for simplicity and write speed at the cost of coherence. For short-lived agents handling single-session interactions, append-only memory works fine. For long-running agents maintaining relationships over months, it guarantees drift.

The fix requires fundamentally different memory semantics: an upsert model rather than an insert model, where new observations either confirm existing understanding (increasing confidence) or explicitly supersede it (recording an evolution). This is what self-models provide.

Why Standard Approaches Fail

The typical response to identity drift is one of three interventions, and all three are insufficient:

Context window pruning. Remove old context to prevent contradictions. This prevents drift but introduces forgetting, you are trading one problem for another. The agent loses valuable historical understanding to maintain consistency.

Recency weighting. Prioritize recent interactions over older ones. This helps for simple preference changes but fails for complex identity evolution. A user’s core values do not change just because they were expressed months ago. Recency weighting treats all old information as less reliable, which is often wrong.

Periodic re-summarization. Periodically compress the full context into a fresh summary. This forces contradiction resolution through the summarization process, but the resolution is implicit and uncontrolled. The summarizer may arbitrarily choose which side of a contradiction to preserve based on surface-level cues rather than meaningful analysis.

Self-Models as Identity Anchors

A self-model prevents identity drift by maintaining a single, authoritative representation of user identity that is continuously validated against new observations. The key mechanisms are:

Explicit contradiction resolution. When a new observation conflicts with an existing belief, the self-model does not simply store both. It evaluates the evidence for each, considers temporal context, and either updates the existing belief (with confidence adjustment) or records an explicit evolution, “User’s risk tolerance changed from conservative to moderate in October 2025.”

Confidence decay. Beliefs that are not confirmed by recent observations gradually lose confidence. This is not the same as forgetting: the belief remains in the model, but the system’s certainty decreases. When an agent encounters a low-confidence belief, it can choose to verify rather than assume.

Trajectory tracking. Instead of treating each observation as independent, the self-model tracks trajectories: how beliefs, goals, and preferences evolve over time. This enables the agent to understand not just where the user is but where they are heading, and to distinguish between temporary fluctuations and genuine identity shifts.

1// Month 1: Initial belief recorded← Baseline identity
2await clarity.observe(userId, {
3  belief: 'Conservative approach to technology adoption',
4  confidence: 0.78,
5  context: 'adoption_style'
6});
7
8// Month 4: New observation conflicts← Drift prevention
9await clarity.observe(userId, {
10  belief: 'Open to experimental technology in specific domains',
11  confidence: 0.72,
12  context: 'adoption_style'
13});
14// Self-model resolves: adoption_style evolved from
15//   'broadly conservative' → 'conservative with domain-specific openness'
16//   Evidence: 3 conservative observations, 2 experimental observations
17//   Trajectory: gradually opening in high-ROI domains

Detecting Drift in Existing Systems

Before implementing self-models, you can measure whether your existing agents suffer from identity drift. There are three diagnostic approaches:

Consistency probing. Ask the agent the same question about a user at different times and compare the responses. If the agent gives inconsistent answers about a user’s fundamental preferences or goals, drift is present. Automate this as a periodic quality check.

Contradiction counting. Analyze the context store for direct contradictions, statements that assert incompatible things about the same user attribute. A healthy context store has few contradictions. A drifting one accumulates them linearly with interaction count.

User feedback correlation. Track the timestamp of negative user feedback (corrections, repeated instructions, expressed frustration) and correlate with agent tenure. If negative feedback increases with time rather than decreasing, drift is likely the cause.

Measuring Drift Quantitatively

Identity drift is measurable once you know what to look for. We recommend three quantitative metrics:

Contradiction density. The ratio of contradictory beliefs to total beliefs in the agent’s user context. Compute this by extracting all assertions the agent holds about a user, identifying pairs that are logically incompatible, and dividing by the total assertion count. Healthy systems maintain contradiction density below 5 percent. Systems experiencing active drift typically show 15-25 percent.

Temporal consistency score. Ask the agent the same question about a user at different times (without intervening user interactions) and measure response similarity. A temporally consistent agent gives the same answer to the same question. An agent experiencing drift gives different answers because different context fragments dominate at different times.

User correction rate. Track how often users explicitly or implicitly correct the agent. Implicit corrections include re-stating preferences, contradicting agent assumptions, or abandoning recommendations. A rising correction rate over time is a strong signal of drift: the agent’s model is diverging from the user’s actual state.

These metrics can be computed automatically and monitored continuously. They provide early warning of drift well before it reaches the incoherence stage, enabling intervention while the problem is still manageable.

The Enterprise Cost of Drift

For enterprise deployments, identity drift has consequences beyond user experience:

Decision quality. Enterprise agents that inform business decisions, financial analysis, strategic planning, risk assessment, cannot afford contradictory user models. A drifted agent might simultaneously treat a stakeholder as risk-averse and risk-tolerant, producing recommendations that are internally inconsistent.

Audit failure. Regulated industries require that AI-assisted decisions be explainable. When an agent’s understanding of a user is contradictory, the decision rationale becomes contradictory too. “We recommended this because the user prefers X” falls apart when the system also believes the user prefers not-X.

Relationship damage. Enterprise users interact with AI agents in high-stakes contexts. An agent that contradicts itself or forgets a user’s stated position damages professional trust in ways that are difficult to recover.

Compounding cost. Unlike bugs that occur consistently and can be found through testing, identity drift is stochastic: it manifests differently for each user depending on their specific interaction history. Traditional QA cannot catch it because each instance is unique. The cost compounds because the longer drift goes undetected, the more contradictions accumulate, and the harder it becomes to restore a coherent model. Early detection and prevention through self-models are far cheaper than retroactive correction.

Trade-offs and Limitations

Self-model identity anchoring is not a perfect solution, and introduces its own considerations.

Rigidity risk. An identity anchor that is too rigid can prevent the model from adapting to genuine user changes. The confidence decay and trajectory tracking mechanisms mitigate this, but they need careful tuning. Too fast and the model is volatile. Too slow and it ignores real changes.

Observation extraction quality. The self-model is only as good as the observations fed into it. If the extraction pipeline misinterprets user statements or misses implied beliefs, the model will drift in a different way : not from contradiction accumulation, but from inaccurate foundations.

Computational overhead. Contradiction resolution and trajectory tracking require more computation than simple context appending. For agents handling high-volume interactions, this overhead needs to be managed through async processing and batched updates.

False stability. Users who change dramatically in a short period (career change, crisis, strategic pivot) may temporarily appear to be “drifting” when they are genuinely transforming. The system needs mechanisms to detect and accommodate rapid legitimate change versus slow contradictory drift.

What to Do Next

1. Run a drift audit. Select 10 long-running user sessions (more than 50 interactions) and manually review the agent’s context for contradictions. Count the contradictions, categorize them (preference conflicts, goal conflicts, factual conflicts), and estimate the impact on output quality. This gives you a baseline drift score.

2. Implement consistency probes. Add automated consistency checks that periodically ask your agent to summarize a user’s key attributes and compare the summaries over time. Flag sessions where summaries change significantly without corresponding user input. This is your early warning system for drift.

3. Pilot an identity anchor. Choose your most complex, longest-running agent workflow and implement a self-model as the identity anchor. Route context queries through the self-model rather than the raw context store. Measure consistency, user satisfaction, and correction frequency before and after. See self-models in action to understand the architecture.

Your agent remembers everything. It understands less every day. Self-models anchor identity against drift. Build agents that stay coherent.

References

1. New America analysis of AI agents and memory
2. none include user modeling as a built-in primitive
3. multi-agent research system
4. context engineering
5. memory vs. retrieval augmented generation

Related

• Why Every AI Agent Needs a User Model
• Belief State vs Session State: What Your Agent Should Track
• Agent Memory: Why Chat Logs Are Not Enough