-
Notifications
You must be signed in to change notification settings - Fork 0
Add Weighted Multi-Constraint Scoring Engine #53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # CuraFrame Weighted Multi-Constraint Scoring Engine (v3.0.0) | ||
|
|
||
| ## Overview | ||
|
|
||
| The Weighted Multi-Constraint Scoring Engine is a subsystem added in v3.0.0 that sits above the existing constraint bundles. It takes the output (signals, falsifications, and narrative summaries) from all bundles and distills them into a unified, modular, and configurable **Composite Stability Score**. | ||
|
|
||
| This system does not alter or modify the underlying deterministic bundle evaluation logic. Instead, it aggregates results and applies domain-specific weighting, penalties, and bonuses to produce a nuanced report of therapeutic candidate viability. | ||
|
|
||
| ## Architecture | ||
|
|
||
| 1. **WeightProfile (`scoring/WeightProfile.hpp`)**: An interface (with default implementations) that defines the weights, penalties, multipliers, and scaling factors for different domains and specific signals. | ||
| 2. **WeightedScoringEngine (`scoring/WeightedScoringEngine.hpp/.cpp`)**: The core engine that ingests the outputs from all bundles (`EvaluationReport`), applies the provided `WeightProfile`, computes the final score, and generates a structured summary. | ||
| 3. **ScoringPipeline (`scoring/ScoringPipeline.hpp/.cpp`)**: A wrapper pipeline managing the execution of the scoring engine with a specified weight profile. | ||
| 4. **ScoringReport (`scoring/ScoringReport.hpp`)**: The structured output structure, detailing penalty breakdowns, bonus breakdowns, falsification impacts, and a narrative summary. | ||
|
|
||
| ## Composite Stability Score | ||
|
|
||
| The Composite Stability Score is a numerical representation of candidate stability scaled between 0 and 100: | ||
| - **100**: Perfect theoretical safety baseline, zero constraint violations or penalized signals. | ||
| - **< 100**: Progressive deterioration of the score based on domain-weighted penalties, amplified by specific severity levels, or falsification flags. | ||
| - **Falsification impact**: Candidate falsification triggers massive penalty deductions (configurable per weight profile), severely lowering the final score. | ||
|
|
||
| Higher scores indicate more stable candidates, and lower scores indicate more risk-burdened candidates. | ||
|
|
||
| ## Example Report Execution | ||
|
|
||
| ```cpp | ||
| MultiBundleEvaluator evaluator; | ||
| EvaluationReport eval_report = evaluator.evaluate(candidate); | ||
|
|
||
| // Generate standard composite scoring | ||
| ScoringReport score = evaluator.score(eval_report); | ||
|
|
||
| // Generate with an aggressive safety profile | ||
| ScoringReport safety_score = evaluator.score_with_profile(eval_report, std::make_shared<HighSafetyProfile>()); | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| # Weight Profiles (v3.0.0) | ||
|
|
||
| ## Design and Intent | ||
|
|
||
| Weight Profiles allow the CuraFrame Scoring Engine to view candidates under different lenses of scrutiny. Instead of hardcoded aggregation metrics, `WeightProfile` provides a configurable approach to amplifying certain domains (e.g., Cardiac or CNS safety) based on the context of the evaluation. | ||
|
|
||
| ## Built-in Profiles | ||
|
|
||
| ### 1. `DefaultResearchProfile` | ||
| - A balanced, general-purpose profile. | ||
| - Mildly emphasizes safety parameters but treats most domains with an equal base weight. | ||
|
|
||
| ### 2. `HighSafetyProfile` | ||
| - Designed for late-stage conservative screening. | ||
| - Applies heavy multipliers to `Safety`, `Cardiac`, `CNS`, and `SystemicExposure` domains. | ||
| - Heavily amplifies specific toxicity signals. | ||
| - Punishes falsification flags almost entirely zeroing out candidate viability. | ||
|
|
||
| ## Extending Weight Profiles | ||
|
|
||
| To create a new weight profile, inherit from the `WeightProfile` abstract base class and implement the necessary configuration hooks. | ||
|
|
||
| ```cpp | ||
| class PediatricSafetyProfile : public WeightProfile { | ||
| public: | ||
| std::string name() const override { return "PediatricSafetyProfile"; } | ||
|
|
||
| double bundle_weight(const std::string& bundle_name) const override { | ||
| // Significantly amplify systemic exposure and safety for pediatric focus | ||
| if (bundle_name == "SystemicExposure" || bundle_name == "Safety") return 3.0; | ||
| return 1.0; | ||
| } | ||
|
|
||
| double signal_weight(const std::string& bundle_name, const std::string& signal_name) const override { | ||
| if (signal_name.find("clearance") != std::string::npos) return 2.0; | ||
| return 1.0; | ||
| } | ||
|
|
||
| double falsification_penalty() const override { return 100.0; } | ||
| }; | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| #include "ScoringPipeline.hpp" | ||
|
|
||
| ScoringPipeline::ScoringPipeline(std::shared_ptr<WeightProfile> profile) | ||
| : engine_(std::move(profile)) {} | ||
|
|
||
| ScoringReport ScoringPipeline::execute(const EvaluationReport& eval_report) const { | ||
| // In a more complex pipeline, normalization or pre-processing could happen here | ||
| // Currently, the WeightedScoringEngine handles everything deterministically | ||
| return engine_.score(eval_report); | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| #ifndef CURAFRAME_SCORING_PIPELINE_HPP | ||
| #define CURAFRAME_SCORING_PIPELINE_HPP | ||
|
|
||
| #include "../constraint_core/EvaluationReport.hpp" | ||
| #include "ScoringReport.hpp" | ||
| #include "WeightedScoringEngine.hpp" | ||
| #include "WeightProfile.hpp" | ||
| #include <memory> | ||
|
|
||
| class ScoringPipeline { | ||
| public: | ||
| ScoringPipeline(std::shared_ptr<WeightProfile> profile); | ||
|
|
||
| ScoringReport execute(const EvaluationReport& eval_report) const; | ||
|
|
||
| private: | ||
| WeightedScoringEngine engine_; | ||
| }; | ||
|
|
||
| #endif // CURAFRAME_SCORING_PIPELINE_HPP |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| #ifndef CURAFRAME_SCORING_REPORT_HPP | ||
| #define CURAFRAME_SCORING_REPORT_HPP | ||
|
|
||
| #include <string> | ||
| #include <vector> | ||
| #include <map> | ||
|
|
||
| struct ScoringReport { | ||
| double composite_score = 0.0; // 0 to 100 | ||
|
|
||
| // Per-bundle weighted contributions | ||
| std::map<std::string, double> bundle_contributions; | ||
|
|
||
| // Breakdowns | ||
| std::map<std::string, double> penalty_breakdown; | ||
| std::map<std::string, double> bonus_breakdown; | ||
|
|
||
| // Falsification | ||
| std::vector<std::string> falsification_flags; | ||
| double falsification_impact = 0.0; | ||
|
|
||
| // Metadata | ||
| std::string weight_profile_name; | ||
|
|
||
| // Narrative summary | ||
| std::string narrative_summary; | ||
| }; | ||
|
|
||
| #endif // CURAFRAME_SCORING_REPORT_HPP |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| #include "WeightProfile.hpp" | ||
|
|
||
| // Intentionally left blank or simple implementations if needed. | ||
| // Weight profiles are currently fully implemented inline in the header for simplicity and polymorphism. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| #ifndef CURAFRAME_WEIGHT_PROFILE_HPP | ||
| #define CURAFRAME_WEIGHT_PROFILE_HPP | ||
|
|
||
| #include <string> | ||
| #include <map> | ||
| #include <memory> | ||
|
|
||
| class WeightProfile { | ||
| public: | ||
| virtual ~WeightProfile() = default; | ||
|
|
||
| virtual std::string name() const = 0; | ||
|
|
||
| // Default fallback weight if a bundle or signal is not explicitly defined | ||
| virtual double default_weight() const { return 1.0; } | ||
|
|
||
| // Per-bundle weights | ||
| virtual double bundle_weight(const std::string& bundle_name) const = 0; | ||
|
|
||
| // Per-signal weights (can override bundle weights for specific signals) | ||
| virtual double signal_weight(const std::string& bundle_name, const std::string& signal_name) const = 0; | ||
|
|
||
|
Comment on lines
+14
to
+22
|
||
| // Penalty and Bonus multipliers | ||
| virtual double penalty_multiplier() const { return 1.0; } | ||
| virtual double bonus_multiplier() const { return 1.0; } | ||
|
|
||
| // Global scaling factor | ||
| virtual double global_scaling_factor() const { return 1.0; } | ||
|
|
||
| // Falsification penalty | ||
| virtual double falsification_penalty() const { return 50.0; } // Heavy penalty for falsification | ||
| }; | ||
|
|
||
| class DefaultResearchProfile : public WeightProfile { | ||
| public: | ||
| std::string name() const override { return "DefaultResearchProfile"; } | ||
|
|
||
| double bundle_weight(const std::string& bundle_name) const override { | ||
| if (bundle_name == "Safety") return 1.5; | ||
| return 1.0; | ||
| } | ||
|
|
||
| double signal_weight(const std::string& bundle_name, const std::string& signal_name) const override { | ||
| return 1.0; | ||
| } | ||
|
Comment on lines
+43
to
+45
|
||
| }; | ||
|
|
||
| class HighSafetyProfile : public WeightProfile { | ||
| public: | ||
| std::string name() const override { return "HighSafetyProfile"; } | ||
|
|
||
| double bundle_weight(const std::string& bundle_name) const override { | ||
| if (bundle_name == "Safety" || bundle_name == "Cardiac" || bundle_name == "CNS" || bundle_name == "SystemicExposure") { | ||
| return 2.0; | ||
| } | ||
| return 1.0; | ||
| } | ||
|
|
||
| double signal_weight(const std::string& bundle_name, const std::string& signal_name) const override { | ||
| if (signal_name.find("toxicity") != std::string::npos || signal_name.find("risk") != std::string::npos) { | ||
| return 1.5; | ||
| } | ||
| return 1.0; | ||
|
Comment on lines
+59
to
+63
|
||
| } | ||
|
|
||
| double penalty_multiplier() const override { return 1.5; } | ||
| double bonus_multiplier() const override { return 0.5; } // Conservative | ||
| double falsification_penalty() const override { return 100.0; } // Immediate zero or near-zero | ||
| }; | ||
|
|
||
| #endif // CURAFRAME_WEIGHT_PROFILE_HPP | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
score_with_profileaccepts astd::shared_ptr<WeightProfile>that may be null; downstream code dereferences the profile without checks, which can crash. Consider validatingprofile(throw/assert) and/or falling back toDefaultResearchProfilewhenprofile == nullptr.