feat: v0.1.01 - Critical Metric Formatting Fixes & Documentation Restructuring

[laplaque]

Description

This patch release resolves critical metric formatting issues discovered during testing, ensuring proper display of metrics in Instana UI with correct decimal precision and clean metric names.

🔧 Primary Metric Formatting Fixes:

• Metric Names: Removed trailing "{}" from metric names in Instana UI display
• Integer Decimals: Fixed integer values displaying with unnecessary decimals (15.000 → 15)
• Percentage Display: Corrected percentage metrics to show proper decimal precision (42.678912 → 42.68)

📊 Before/After Examples:

Before v0.1.01:
- cpu_usage{}: 42.678912 (excessive decimals)
- process_count{}: 15.000 (unnecessary decimals)
- memory_usage{}: 85.234567 (excessive decimals)

After v0.1.01:
- cpu_usage: 42.68 (proper 2-decimal percentage)
- process_count: 15 (clean integer)
- memory_usage: 85.23 (proper 2-decimal percentage)

🔧 Technical Changes:

• Enhanced common/otel_connector.py with dynamic reading of decimals field from manifest.toml
• Implemented proper formatting based on metric configuration (percentage vs integer vs float)
• Fixed metric name cleaning to remove trailing braces using metadata store functionality
• Applied manifest-specified decimal places instead of hardcoded 3-decimal formatting
• Maintained backward compatibility for existing installations

📚 Documentation Restructuring (Per .clinerules):

• Moved all documentation to organized /docs structure
• Updated README.md in sync with new capabilities
• Maintained changes in RELEASE_NOTES.md (now docs/releases/)
• Restructured: DEVELOPER_GUIDE.md → docs/development/
• Restructured: SECURITY_SETUP.md → docs/security/
• Restructured: All TAG_*.md files → docs/releases/

📋 Changed Files:

• ✅ 30 files changed, 591 insertions(+), 147 deletions(-)
• ✅ Core metric formatting logic in common/otel_connector.py and common/metadata_store.py
• ✅ Version updated to 0.1.01 in common/manifest.toml
• ✅ Enhanced test suite to validate formatting fixes
• ✅ Complete documentation reorganization

Checklist before requesting a review

• I have performed a self-review of my code
• If this is a merge to master, I will create a version tag (v*..)
• The version tag will be higher than the previous version

Tag Information

Planned tag: v0.1.01

Ready for immediate merge and deployment. Zero downtime deployment with immediate formatting improvements.

[laplaque]

PR Comments Review Summary

I've reviewed the PR comments from Copilot. Here's a summary of the suggested improvements and their status:

1. Database Connection Management

• ✅ Implemented: Using context managers for SQLite connections
• ✅ Expanded: Implemented a comprehensive centralized database connection management pattern throughout the entire MetadataStore class
• 📝 Documentation: Created ADR-001 for the centralized database connection management pattern

2. Schema Caching & Performance

• ✅ Implemented: Caching the metrics table schema to avoid repeated PRAGMA calls
• ✅ Implemented: Added diagnostic logging for schema inconsistencies and transitions

3. Code Structure & Error Handling

• ✅ Implemented: Error handling in _parse_range to manage unexpected range_spec formats
• ✅ Implemented: Consolidated error messages in otel_connector.py
• ✅ Implemented: Refactored duplicate code in database operations

4. Additional Improvements

• ✅ Implemented: Enhanced schema validation with diagnostic logging
• ✅ Enhanced: Applied diagnostic capabilities beyond the original requests

All suggested improvements have been implemented, often with enhancements that go beyond the original suggestions. The code is now more robust, maintainable, and follows better practices for resource management and error handling.

[laplaque]

I've addressed the potential AttributeError issue in the InstanaOTelConnector class in common/otel_connector.py. The issue could occur if an exception happened after creating the metadata store but before setting self.display_name, and then the code tried to access self.display_name in the attributes dictionary.

The fix adds a robust fallback using Python's getattr() function:

"service.name": getattr(self, 'display_name', service_name),  # Use self.display_name with fallback to self.service_name

This ensures that even if display_name isn't set for any reason, the code will gracefully fall back to using the original service_name parameter instead of raising an AttributeError.

I've verified the fix works by running the unit tests, which all passed successfully. I've also updated the RELEASE_NOTES.md to document this enhancement.

[Copilot]

Pull Request Overview

This release implements dynamic metric formatting based on manifest.toml definitions, cleans up metric names for Instana UI, and reorganizes all documentation under a unified docs/ directory.

• Load default metrics and formatting rules from manifest.toml, applying per-metric decimal precision and removing trailing braces.
• Refactor database connection handling in common/metadata_store.py to use a context manager and cache schema, and bump schema version.
• Move and restructure all project documentation into docs/ and update README links.

Reviewed Changes

Copilot reviewed 33 out of 33 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
docs/releases/RELEASE_NOTES.md	Added v0.1.01 release notes covering formatting and DB changes
docs/adr/001-centralized-database-connection-management.md	New ADR for centralized DB connection management
docs/README.md	Top-level documentation overview and directory structure
common/toml_utils.py	Added TOML-based metric helpers: get_default_metrics, expand_metric_patterns, get_expanded_metrics
common/otel_connector.py	Refactored metric registration to consume TOML definitions, apply dynamic decimals, and clean names
common/metadata_store.py	Introduced _get_db_connection(), schema caching, and dynamic query builder with version bump to 2.0
common/manifest.toml	Bumped version and metadata_schema_version, added default_metrics definitions
README.md	Updated links to point at new docs/ paths

Comments suppressed due to low confidence (2)

common/toml_utils.py:157

• Consider adding unit tests for get_default_metrics and expand_metric_patterns, including edge cases (e.g., unsupported pattern_source, 'auto' ranges) to ensure pattern expansion works correctly.

def get_default_metrics() -> List[Dict[str, Any]]:

common/metadata_store.py:27

• You’re calling logger.error before logger is defined (it’s declared on line 29). Move the logger = logging.getLogger(__name__) assignment above this try/except block or use a fallback print in this exception.

    logger.error("TOML utilities not available. Metric definitions cannot be loaded. Please ensure common/toml_utils.py exists and get_expanded_metrics is available.")