Skip to content

Why don't user profile preferences decay? I currently have 100% satisfaction for each one #115

@haisto

Description

@haisto

opencode-mem: User Profile Confidence Decay Analysis

Plugin: opencode-mem v2.14.2
Analyzed: May 28, 2026


1. Overview

opencode-mem builds a User Profile by periodically analyzing user prompts with an AI model (Claude/GPT). The profile includes preferences (code style, communication style, tool preferences), each with a confidence score (0–1) meant to represent how certain the system is about that preference.

However, due to a chain of implementation issues, confidence never decays — it only increases until reaching 100%, where it stays forever.


2. Core Types

interface UserProfilePreference {
    category: string;       // e.g. "Code Style", "Communication"
    description: string;    // e.g. "Prefers concise code without comments"
    confidence: number;     // 0–1, meant to represent certainty
    evidence: string[];     // Example prompts supporting this preference
    lastUpdated: number;    // Timestamp of last confirmation
}

interface UserProfileData {
    preferences: UserProfilePreference[];
    patterns: UserProfilePattern[];
    workflows: UserProfileWorkflow[];
}

3. The Flow

User sends prompts
        │
        ▼
┌─────────────────────────────┐
│  UserPromptManager          │  saves every prompt to DB
│  (user-prompt-manager.js)   │
└─────────────┬───────────────┘
              │
              ▼  (every N prompts where N = userProfileAnalysisInterval, default 10)
┌─────────────────────────────┐
│  performUserProfileLearning │  calls AI model to extract profile data
│  (user-memory-learning.js)  │
└─────────────┬───────────────┘
              │
              ▼
┌─────────────────────────────┐
│  mergeProfileData()         │  merges AI output with existing profile
│  (user-profile-manager.js)  │
└─────────────┬───────────────┘
              │
              ▼
     ┌ ─ ─ ─ ─ ─ ─ ─ ┐
     │ applyConfidenceDecay() │  ← DEAD CODE, never called
     └ ─ ─ ─ ─ ─ ─ ─ ┘

4. The Three Broken Links

Link ① — Merge Only Increases, Never Decreases

File: user-profile-manager.js, line 218

// Inside mergeProfileData(), when a matching preference already exists:
merged.preferences[existingIndex] = {
    ...newPref,
    confidence: Math.min(1, (existingItem.confidence || 0) + 0.1),
    //                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    //                     Always adds +0.1, capped at 1.0.
    //                     No branch for decreasing confidence.
    evidence: [
        ...new Set([
            ...this.ensureArray(existingItem.evidence),
            ...this.ensureArray(newPref.evidence),
        ]),
    ].slice(0, 5),
    lastUpdated: Date.now(),
};

Effect: Every time the same preference is extracted again by the AI, its confidence gets +0.1. After 2–3 analysis cycles, it hits 1.0 and stays there.

Link ② — Decay Function Exists But Is Never Called

File: user-profile-manager.js, lines 138–160

applyConfidenceDecay(profileId) {
    const profile = this.getProfileById(profileId);
    if (!profile) return;

    const profileData = JSON.parse(profile.profileData);
    const now = Date.now();
    const decayThreshold = CONFIG.userProfileConfidenceDecayDays * 24 * 60 * 60 * 1000;
    //                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    //                       Default: 30 days

    profileData.preferences = profileData.preferences
        .map((pref) => {
            const age = now - pref.lastUpdated;
            if (age > decayThreshold) {
                const decayFactor = Math.max(0.5, 1 - (age - decayThreshold) / decayThreshold);
                return { ...pref, confidence: pref.confidence * decayFactor };
            }
            return pref;
        })
        .filter((pref) => pref.confidence >= 0.3);
    // ...
}

A grep for applyConfidenceDecay across the entire codebase yields only one match — the definition itself. No caller exists anywhere.

Search scope Matches
Definition in user-profile-manager.js ✅ 1 (line 138)
Callers in any .js file ❌ 0

Link ③ — Refresh API Is a Shell

File: api-handlers.js, lines 767–790

export async function handleRefreshProfile(userId) {
    // Step 1: Count unanalyzed prompts
    const unanalyzedCount = userPromptManager.countUnanalyzedForUserLearning();

    // Step 2: Return "queued" — does nothing else
    return {
        success: true,
        data: {
            message: "Profile refresh queued",
            unanalyzedPrompts: unanalyzedCount,
            note: "Profile will be updated when threshold is reached",
            //       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            //       No actual refresh happens. No decay. No re-analysis.
        },
    };
}

The endpoint POST /api/user-profile/refresh is misleading — it only reports how many prompts are pending. It should, at minimum, call applyConfidenceDecay before returning.

Relevant config (opencode-mem.jsonc):

"userProfileConfidenceDecayDays": 30,
"userProfileAnalysisInterval": 10,
"userProfileMaxPreferences": 20,

Despite the config existing, userProfileConfidenceDecayDays is effectively useless because the only function that reads it (applyConfidenceDecay) is never invoked.


5. Root Cause Summary

# Issue Location Impact
mergeProfileData() hardcodes +0.1 on every merge user-profile-manager.js:218 Confidence can only increase
applyConfidenceDecay() is dead code — defined but never called user-profile-manager.js:138 Decay mechanism is non-functional
POST /api/user-profile/refresh is a no-op shell api-handlers.js:767 No manual trigger for decay exists

6. Ideal Fix (Reference)

An upstream fix would need to address all three links:

  1. Add a configurable merge strategy (e.g., confidenceMergeStrategy: "max" | "increment" | "ai-score")
  2. Hook applyConfidenceDecay() into the profile analysis pipeline
  3. Make the refresh API actually trigger decay + re-analysis

Until then, confidence will always reach and stay at 100% for any preference that appears more than once.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions