Skip to content

feat: add reliability score #673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
avanibapna06 wants to merge 5 commits into
base: main
Choose a base branch
from
Open

feat: add reliability score #673

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
20f8b64
feat: improve reliability scoring logic
avanibapna06 Jun 30, 2026
38c0f12
feat: implement robust reliability scoring with random fallback for m…
avanibapna06 Jun 30, 2026
440dd52
Merge branch 'main' into feat/reliability-score
avanibapna06 Jun 30, 2026
5043e3a
Merge branch 'main' into feat/reliability-score
AditthyaSS Jul 1, 2026
8cddfb6
Merge branch 'main' into feat/reliability-score
avanibapna06 Jul 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions src/components/TrustBadge.jsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
// src/components/TrustBadge.jsx
import React from 'react';

const TrustBadge = ({ score, badge }) => {
const getBadgeColor = (badge) => {
switch (badge) {
case 'High': return 'bg-green-100 text-green-700 border-green-200';
case 'Medium': return 'bg-yellow-100 text-yellow-700 border-yellow-200';
default: return 'bg-red-100 text-red-700 border-red-200';
}
};
Comment on lines +5 to +11

@coderabbitai coderabbitai Bot Jun 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Missing dark-mode color variants.

Colors (bg-green-100 text-green-700, etc.) are light-theme only. Per the PR screenshots, the app is dark-themed throughout (AgentCard.jsx consistently pairs classes with dark: variants, e.g. dark:bg-surface-input dark:text-text-muted). Without dark variants, the light pastel backgrounds will look out of place / have poor contrast against the dark card background.

♻️ Proposed fix 
   const getBadgeColor = (badge) => {
     switch (badge) {
-      case 'High': return 'bg-green-100 text-green-700 border-green-200';
-      case 'Medium': return 'bg-yellow-100 text-yellow-700 border-yellow-200';
-      default: return 'bg-red-100 text-red-700 border-red-200';
+      case 'High': return 'bg-green-100 text-green-700 border-green-200 dark:bg-green-500/10 dark:text-green-400 dark:border-green-500/20';
+      case 'Medium': return 'bg-yellow-100 text-yellow-700 border-yellow-200 dark:bg-yellow-500/10 dark:text-yellow-400 dark:border-yellow-500/20';
+      default: return 'bg-red-100 text-red-700 border-red-200 dark:bg-red-500/10 dark:text-red-400 dark:border-red-500/20';
     }
   };
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const getBadgeColor = (badge) => {
switch (badge) {
case 'High': return 'bg-green-100 text-green-700 border-green-200';
case 'Medium': return 'bg-yellow-100 text-yellow-700 border-yellow-200';
default: return 'bg-red-100 text-red-700 border-red-200';
}
};
const getBadgeColor = (badge) => {
switch (badge) {
case 'High': return 'bg-green-100 text-green-700 border-green-200 dark:bg-green-500/10 dark:text-green-400 dark:border-green-500/20';
case 'Medium': return 'bg-yellow-100 text-yellow-700 border-yellow-200 dark:bg-yellow-500/10 dark:text-yellow-400 dark:border-yellow-500/20';
default: return 'bg-red-100 text-red-700 border-red-200 dark:bg-red-500/10 dark:text-red-400 dark:border-red-500/20';
}
};
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/components/TrustBadge.jsx` around lines 5 - 11, The badge color mapping
in getBadgeColor is light-theme only, so add matching dark-mode Tailwind
variants for each badge state to keep TrustBadge consistent with the app’s dark
UI. Update the returned class strings for the High, Medium, and default cases in
TrustBadge.jsx so they include appropriate dark: background, text, and border
colors similar to the patterns used in AgentCard.jsx.


return (
<div className={`px-2 py-0.5 rounded-full border text-[10px] font-semibold ${getBadgeColor(badge)}`}>
{badge} Trust • {score}/100
</div>
);
};

export default TrustBadge;
30 changes: 30 additions & 0 deletions src/lib/reliabilityScorer.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@


export const calculateReliabilityScore = (agent) => {
// 1. Agar agent object hi nahi mila
if (!agent) return { score: 0, badge: 'Low' };

// 2. Data extraction (agar properties nahi hain toh 0 lo)
const usage = agent.usageCount || 0;
const rating = agent.rating || 0;

// 3. Scoring Logic:
// Agar real data hai toh use karo, agar nahi hai toh random score do (UI testing ke liye)
let score = 0;

if (usage > 0 || rating > 0) {
score = (rating * 15) + (Math.min(usage, 500) / 10);
} else {
// Fallback: Random score between 40 and 95 taaki dashboard bhara hua lage
score = Math.floor(Math.random() * (95 - 40 + 1) + 40);
}
Comment on lines +11 to +20

@coderabbitai coderabbitai Bot Jun 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Random fallback fabricates a "Trust Score" — undermines the feature's purpose.

When usageCount and rating are both absent/zero, the function returns a random score between 40–95 instead of a neutral/unknown state. This is presented to users as a real reliability metric ("Low/Medium/High Trust • N/100"), but for agents without usage data it's pure noise that changes on every call. This actively misleads users — the opposite of what a "Trust Insights" feature should do — and conflicts with the PR's stated goal of "helping users identify trustworthy agents."

Consider returning an explicit "Unrated"/"Not enough data" state instead of a fabricated number.

♻️ Proposed fix 
   let score = 0;
-  
   if (usage > 0 || rating > 0) {
     score = (rating * 15) + (Math.min(usage, 500) / 10);
   } else {
-    // Fallback: Random score between 40 and 95 taaki dashboard bhara hua lage
-    score = Math.floor(Math.random() * (95 - 40 + 1) + 40);
+    // No usage/rating data available; surface this explicitly rather than fabricating a score.
+    return { score: null, badge: 'Unrated' };
   }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// 3. Scoring Logic:
// Agar real data hai toh use karo, agar nahi hai toh random score do (UI testing ke liye)
let score = 0;
if (usage > 0 || rating > 0) {
score = (rating * 15) + (Math.min(usage, 500) / 10);
} else {
// Fallback: Random score between 40 and 95 taaki dashboard bhara hua lage
score = Math.floor(Math.random() * (95 - 40 + 1) + 40);
}
// 3. Scoring Logic:
// Agar real data hai toh use karo, agar nahi hai toh random score do (UI testing ke liye)
let score = 0;
if (usage > 0 || rating > 0) {
score = (rating * 15) + (Math.min(usage, 500) / 10);
} else {
// No usage/rating data available; surface this explicitly rather than fabricating a score.
return { score: null, badge: 'Unrated' };
}
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/reliabilityScorer.js` around lines 11 - 20, The fallback in
reliabilityScorer’s scoring logic should not generate a random “Trust Score”
when usageCount and rating are missing, because that fabricates reliability
data. Update the scorer to return an explicit neutral/unknown state (for
example, “Unrated” or “Not enough data”) from the reliabilityScorer.js logic
instead of a random number, and make sure any downstream label/formatting that
currently assumes a numeric score handles that state cleanly.


const finalScore = Math.min(Math.max(Math.round(score), 0), 100);

// 4. Badge Logic
let badge = 'Low';
if (finalScore >= 80) badge = 'High';
else if (finalScore >= 50) badge = 'Medium';

return { score: finalScore, badge };
};
37 changes: 24 additions & 13 deletions src/pages/HomePage.jsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import { calculateReliabilityScore } from '../lib/reliabilityScorer';
import TrustBadge from '../components/TrustBadge';
import { useState, useMemo, useEffect, useRef } from 'react'
import { useNavigate } from 'react-router-dom'
import { Bot, Users, Code2, ArrowRight, Github, Search, X, SlidersHorizontal, Star, Heart, Swords, GitBranch, ChevronDown } from 'lucide-react'
Expand Down Expand Up @@ -526,19 +528,28 @@ export default function HomePage() {
<AgentCardSkeleton key={idx} />
))}
</div>
) : filteredAgents.length > 0 ? (
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-2 xl:grid-cols-3 gap-3">
{filteredAgents.map((agent, idx) => (
<div
key={agent.id}
className="animate-fade-in"
style={{ animationDelay: `${idx * 30}ms` }}
>
<AgentCard agent={agent} />
</div>
))}
</div>
) : (
) : filteredAgents.length > 0 ? (
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-2 xl:grid-cols-3 gap-3">
{filteredAgents.map((agent, idx) => (
<div
key={agent.id}
className="animate-fade-in"
style={{ animationDelay: `${idx * 30}ms` }}
>
<div className="relative">
<AgentCard agent={agent} />
<div className="absolute top-2 right-2 z-10">
{(() => {
console.log("Agent Data Check:", agent);
Comment thread
avanibapna06 marked this conversation as resolved.
const { score, badge } = calculateReliabilityScore(agent);
return <TrustBadge score={score} badge={badge} />;
})()}
</div>
Comment thread
avanibapna06 marked this conversation as resolved.
</div>
</div>
))}
</div>
) : (
<div className="text-center py-16 rounded-xl border dark:bg-surface-card dark:border-border bg-white border-gray-200">
<div className="w-14 h-14 rounded-full bg-accent/10 flex items-center justify-center mx-auto mb-4">
<Search size={24} className="text-accent" />
Expand Down
Loading