An Explainable AI Clinical Decision Support System for Chronic Kidney Disease (CKD) that combines machine learning prediction, SHAP explainability, clinical evidence retrieval, and LLM-based reasoning to provide transparent and reliable clinical insights.
The system integrates ML predictions, explainability, retrieval-augmented reasoning, safety guardrails, and monitoring to support clinicians with interpretable decision support.
This project is a production-grade architecture demonstration demonstrating the architecture of an explainable AI clinical decision support system.
It focuses on system design, explainability, and AI reasoning pipelines rather than production deployment.
The implementation is intended for educational and research purposes.
Chronic Kidney Disease (CKD) requires early risk detection and transparent clinical decision support. Traditional machine learning models often provide predictions without clear reasoning, limiting their usefulness in real-world clinical settings.
This project implements a layered AI clinical decision support system that:
- Predicts CKD risk using machine learning models
- Explains predictions using SHAP feature attribution
- Retrieves supporting clinical guideline evidence
- Generates evidence-grounded reasoning using a large language model
- Applies safety guardrails before presenting outputs
- Supports human-in-the-loop clinician review
- Monitors system reliability and data drift
The system design emphasizes interpretability, reliability, and responsible AI deployment.
| Section | Description |
|---|---|
| Overview | Project goals and background |
| Project Highlights | High-level system components |
| Key Features | Core capabilities of the system |
| Architecture | High-level system architecture |
| System Pipeline | End-to-end decision pipeline |
| Project Structure | Repository organization |
| Tech Stack | Technologies used |
| Documentation | Detailed technical documents |
| Running the System | Setup and execution instructions |
| Responsible AI | Safety and governance principles |
| Future Improvements | Potential system extensions |
| Component | Description |
|---|---|
| CKD Risk Prediction | Machine learning model predicts CKD risk from patient clinical features |
| SHAP Explainability | Identifies which features contribute most to the prediction |
| Clinical Evidence Retrieval | Retrieves relevant clinical guidelines using vector similarity search |
| LLM Clinical Reasoning | Generates evidence-grounded explanations using SHAP insights and retrieved clinical knowledge |
| AI Safety Guardrails | Validates generated explanations using safety filters and consistency checks |
| Human-in-the-Loop Review | Enables clinicians to review model explanations before final decisions |
| Monitoring & Governance | Tracks prediction confidence, system reliability, and data drift |
Machine learning model predicts CKD risk from patient clinical features.
SHAP feature attribution explains which variables contribute to model predictions.
Clinical guideline evidence is retrieved using vector similarity search.
An LLM synthesizes explanations using both SHAP insights and retrieved clinical evidence.
Safety filters, clinical disclaimers, and reasoning validation ensure responsible outputs.
Clinicians can review explanations and override system decisions when necessary.
Prediction confidence tracking, drift detection, and system observability ensure reliability.
| Layer | Responsibility |
|---|---|
| Presentation Layer | Streamlit dashboard for clinician interaction |
| API Layer | FastAPI gateway handling request routing |
| Orchestration Layer | AI pipeline controller managing prediction workflow |
| AI Intelligence Layer | CKD prediction model, SHAP explainability, RAG retrieval, and LLM reasoning |
| Data & Knowledge Layer | Model registry, SHAP store, vector index, and clinical knowledge base |
| Monitoring & Governance | Model monitoring, data drift detection, and system observability |
The architecture prioritizes interpretability, reliability, and clinical safety.
- System Architecture Overview
docs/architecture/architecture_overview.md
The orchestration controller executes the following pipeline:
Patient Input → Risk Prediction → SHAP Explainability → Evidence Retrieval → LLM Clinical Reasoning → Guardrail Validation → Clinician Review
Each stage ensures predictions remain interpretable, evidence-grounded, and safety validated before reaching clinicians.
This system integrates multiple AI capabilities to provide reliable and interpretable clinical decision support.
The CKD prediction model provides interpretable outputs using SHAP feature attribution, allowing clinicians to understand which clinical variables influence predictions.
Relevant clinical guideline evidence is retrieved using vector similarity search, enabling evidence-grounded reasoning.
A large language model synthesizes explanations using both SHAP insights and retrieved clinical evidence.
Generated outputs are validated through safety filters, reasoning consistency checks, and clinical disclaimers.
Clinicians can review system explanations and override system decisions when necessary.
The system tracks prediction confidence, monitors calibration stability, and detects data drift to maintain reliability.
clinical-ai-system/
│
├── api/ # FastAPI endpoints and API gateway
├── configs/ # Configuration files
├── data/ # Dataset and preprocessing artifacts
│
├── genai/ # Generative AI pipeline
│ ├── controller/ # Pipeline orchestration
│ ├── explainability/ # SHAP explanation logic
│ ├── retrieval/ # Clinical evidence retrieval (RAG)
│ ├── llm/ # LLM reasoning components
│ ├── guardrails/ # Safety validation
│ ├── evaluation/ # Model evaluation
│ └── prompts/ # Prompt templates
│
├── models/ # Machine learning models
├── services/ # Core prediction services
├── tests/ # System tests
│
├── docs/
│ ├── architecture/
│ │ ├── architecture_overview.md
│ │ └── System_architecture.png
│ │
| ├── failure_modes.md
| ├── deployment_staretgy.md
│ ├── model_card.md
│ └── data_shift.md
│
├── system_design.md # Detailed system design
├── requirements.txt
├── Dockerfile
└── README.md
- Python
- Scikit-learn
- SHAP
- Large Language Models (LLMs)
- Retrieval-Augmented Generation (RAG)
- FastAPI
- Streamlit
- Vector similarity search for clinical evidence retrieval
- Monitoring and observability components
This project follows several design principles commonly used in production AI systems.
Clinical AI systems require interpretability.
The model integrates SHAP feature attribution so clinicians can understand the factors influencing predictions.
LLM reasoning is combined with retrieved clinical guideline evidence, reducing hallucination risk and improving reliability.
The architecture separates prediction, explainability, retrieval, reasoning, and guardrails into independent services, enabling easier maintenance and scalability.
Clinical decisions require expert oversight.
The system allows clinicians to review explanations and override system outputs.
AI-generated explanations pass through validation layers to ensure safety, consistency, and appropriate disclaimers.
Model predictions and system performance are monitored through:
- Prediction confidence tracking
- Data drift detection
- System observability
These components help ensure long-term reliability of the AI system.
Detailed documentation for the system is available in the docs directory.
| Document | Description |
|---|---|
| Architecture Overview | High-level system architecture |
| Model Card | Model behavior, evaluation, and limitations |
| Failure Modes | Identified system failure scenarios and mitigation strategies |
| Deployment Strategy | Design-level deployment architecture, services, and scaling considerations |
| Data Shift | Data drift monitoring and mitigation |
| System Design | Detailed design decisions and implementation |
Links:
-
System Architecture Overview
→ Open -
Model Card
→ Open -
Data Drift Monitoring
→ Open -
Detailed System Design
→ Open -
Failure Modes and Safety
→ Open -
Deployment Strategy
→ Open
Follow these steps to run the Clinical AI Decision Support System locally.
pip install -r requirements.txtCopy the example environment file and update it with your API key:
cp .env.example .env
```env
GROQ_API_KEY=your_api_key_hereuvicorn api.main:app --reloadThis starts the FastAPI server responsible for routing requests to the AI services.
streamlit run app.pyThe Streamlit dashboard provides the interface for interacting with the CKD risk prediction system.
Responsible AI Considerations
This project incorporates several responsible AI principles to ensure safe and interpretable clinical decision support.
All predictions include SHAP explanations so clinicians can understand which clinical features influence the model output.
LLM explanations are supported by retrieved clinical guideline evidence, helping reduce hallucinations and improve reliability.
Clinicians remain responsible for reviewing predictions and explanations through a human-in-the-loop decision process.
Generated reasoning passes through validation layers that enforce safety checks and clinical disclaimers.
The system monitors:
- Prediction confidence
- Model calibration
- Data drift
- Pipeline reliability
These mechanisms help maintain long-term stability of the AI system.
Future Improvements
Potential future extensions include:
- Integration with real Electronic Health Record (EHR) systems
- Continuous model retraining pipelines
- Federated learning across hospitals
- Real-time clinical alert systems
This project is a research and educational implementation of an AI-powered clinical decision support system.
It is not intended for real medical diagnosis or treatment decisions.