Implement FML Execution Validation Test Suite with FHIR IG-based TestPlan

README.md

@@ -184,6 +184,9 @@ The project includes comprehensive test coverage across platforms:
 - **Execution Tests**: Transformation logic and cross-platform behavior
 - **Terminology Tests**: ConceptMap, ValueSet, CodeSystem operations
 - **Platform Tests**: JVM and JavaScript specific functionality
+- **FML Execution Validation Test Suite**: FHIR IG-based test plan with real-world test cases
+
+### Core Testing
 
 Run specific test suites:
 ```bash
@@ -203,6 +206,29 @@ gradle test --info
 gradle jvmTest --tests "FmlRunnerTest"
 ```
 
+### FML Execution Validation Test Suite
+
+The project includes a comprehensive FHIR IG-based validation test suite located in `input/`:
+
+```bash
+# Import test cases from community FML projects
+npm run import:test-data
+
+# Explore available test repositories
+npm run explore:test-repos
+```
+
+The test suite includes:
+- **TestPlan Definition**: `input/fsh/tests/FMLExecutionValidationTestPlan.fsh`
+- **Test Data**: `input/testdata/` with properly attributed test cases from:
+  - ahdis/matchbox (Apache 2.0 license)
+  - FHIR/fhir-test-cases (HL7 copyright/CC0 license)
+  - Local examples (MIT license)
+
+**Published Test Documentation**: https://litlfred.github.io/fmlrunner/TestPlan/FMLExecutionValidationTestPlan.html
+
+For detailed information, see [`input/README.md`](input/README.md).
+
 ## API Reference
 
 ### Core FmlRunner API

input/README.md

@@ -0,0 +1,101 @@
+# FML Execution Validation Test Suite
+
+This directory contains a comprehensive FHIR IG-based validation test suite for FML (FHIR Mapping Language) using FHIR TestPlan resources.
+
+## Overview
+
+The test suite validates FML execution capabilities using real-world test cases sourced from community FML projects, with proper license compliance and attribution.
+
+## Directory Structure
+
+```
+input/
+├── fsh/
+│   └── tests/
+│       └── FMLExecutionValidationTestPlan.fsh    # FSH TestPlan definition
+└── testdata/                                     # All test data with license attribution
+    ├── examples/                                 # Local example test cases
+    │   ├── patient-transform.map                 # Example FML mapping
+    │   ├── patient-input.json                    # Example input data
+    │   └── patient-output.json                   # Expected output data
+    ├── fhir-test-cases/                         # Test cases from FHIR/fhir-test-cases
+    └── matchbox/                                 # Test cases from ahdis/matchbox
+```
+
+## Test Data Sources
+
+### Local Examples (`input/testdata/examples/`)
+- **License**: MIT (local examples)
+- **Purpose**: Immediate testing and development
+- **Files**: Basic patient transformation examples
+
+### FHIR Test Cases (`input/testdata/fhir-test-cases/`)
+- **Source**: [FHIR/fhir-test-cases/r5/structure-mapping](https://github.com/FHIR/fhir-test-cases/tree/main/r5/structure-mapping)
+- **License**: HL7 FHIR License (CC0 "No Rights Reserved")
+- **Attribution**: HL7 FHIR trademark acknowledgment included
+
+### Matchbox Test Cases (`input/testdata/matchbox/`)
+- **Source**: [ahdis/matchbox test resources](https://github.com/ahdis/matchbox/tree/main/matchbox-server/src/test/resources)
+- **License**: Apache License 2.0
+- **Attribution**: Apache 2.0 license header included
+
+## Test File Mapping
+
+Test cases follow these naming conventions:
+- `*-map.txt` or `*.map` — FML mapping specification
+- `*-input.json` or `*-input.xml` — FHIR resource to be mapped
+- `*-output.json` or `*-output.xml` — Expected output after applying the map
+
+Files are paired by base name (e.g., `patient-transform.map`, `patient-input.json`, `patient-output.json`).
+
+## Usage
+
+### Import Test Data
+
+```bash
+# Import test cases from external repositories
+npm run import:test-data
+
+# Explore available test files in repositories
+npm run explore:test-repos
+```
+
+### TestPlan Structure
+
+The `FMLExecutionValidationTestPlan` defines:
+- **Test Cases**: Each mapping scenario with input/output validation
+- **Test Data**: References to map files, input data, and expected outputs
+- **Dependencies**: FHIR R5 StructureMap specification requirements
+- **Validation**: Test execution expectations and requirements
+
+## License Compliance
+
+All imported files include proper license attribution:
+
+- **FHIR Test Cases**: HL7 copyright and CC0 license
+- **Matchbox Test Cases**: Apache 2.0 license 
+- **Local Examples**: MIT license
+
+License headers are automatically added by the import script to ensure compliance with original contribution requirements.
+
+## Development
+
+### Adding New Test Cases
+
+1. Place test files in appropriate subdirectory under `input/testdata/`
+2. Ensure proper license attribution headers
+3. Update `FMLExecutionValidationTestPlan.fsh` with new test case definitions
+
+### Test Execution
+
+Test execution will be available through:
+- FHIR IG publisher validation
+- FML Runner library test suites
+- Continuous integration workflows
+
+## Published Documentation
+
+When published, the test plan documentation will be available at:
+**https://litlfred.github.io/fmlrunner/TestPlan/FMLExecutionValidationTestPlan.html**
+
+This provides comprehensive documentation of all test cases, validation requirements, and execution results for the FML execution validation test suite.

input/fsh/tests/FMLExecutionValidationTestPlan.fsh

@@ -0,0 +1,104 @@
+// FML Execution Validation Test Plan
+// Defines a comprehensive test suite for validating FML (FHIR Mapping Language) execution
+
+Instance: FMLExecutionValidationTestPlan
+InstanceOf: TestPlan
+Usage: #example
+Title: "FML Execution Validation Test Plan"
+Description: "Comprehensive test plan for validating FML execution with real-world test cases from community FML projects"
+* meta.versionId = "1.0.0"
+* url = "http://litlfred.github.io/fmlrunner/TestPlan/FMLExecutionValidationTestPlan"
+* identifier.value = "fml-execution-validation-test-plan"
+* version = "1.0.0"
+* name = "FMLExecutionValidationTestPlan"
+* title = "FML Execution Validation Test Plan"
+* status = #active
+* experimental = false
+* publisher = "FML Runner Project"
+* contact.name = "FML Runner Development Team"
+* contact.telecom.system = #url
+* contact.telecom.value = "https://github.com/litlfred/fmlrunner"
+* description = "This test plan validates FML execution capabilities using test cases sourced from community FML projects including ahdis/matchbox and FHIR/fhir-test-cases repositories. Each test case validates the complete FML transformation pipeline from input data through map execution to expected output validation."
+
+* purpose = "Ensure FML execution engine correctly transforms input data according to mapping specifications and produces expected outputs that conform to FHIR resource definitions."
+
+* scope.artifact = "http://hl7.org/fhir/StructureDefinition/StructureMap"
+* scope.conformance.requirements = "Validate that FML execution produces outputs that match expected results for known test cases"
+
+// Test Case 1: Basic Patient Transform
+* testCase[0].id = "patient-basic-transform"
+* testCase[0].sequence = 1
+* testCase[0].scope.artifact = "http://example.org/StructureMap/PatientTransform"
+* testCase[0].scope.phase = #unit-test
+* testCase[0].requirement.linkId = "REQ-001"
+* testCase[0].requirement.description = "Basic patient data transformation using FML mapping"
+
+* testCase[0].testRun[0].id = "patient-basic-transform-run"
+* testCase[0].testRun[0].description = "Execute patient transform with valid input and validate output"
+
+// Test Data References
+* testCase[0].testData[0].id = "patient-transform-map"
+* testCase[0].testData[0].description = "Patient transformation mapping file"
+* testCase[0].testData[0].type = #test-data
+* testCase[0].testData[0].content.sourceAttachment.url = "testdata/examples/patient-transform.map"
+* testCase[0].testData[0].content.sourceAttachment.contentType = "text/plain"
+
+* testCase[0].testData[1].id = "patient-input-data"
+* testCase[0].testData[1].description = "Input patient data for transformation"
+* testCase[0].testData[1].type = #test-data
+* testCase[0].testData[1].content.sourceAttachment.url = "testdata/examples/patient-input.json"
+* testCase[0].testData[1].content.sourceAttachment.contentType = "application/fhir+json"
+
+* testCase[0].testData[2].id = "patient-expected-output"
+* testCase[0].testData[2].description = "Expected output after patient transformation"
+* testCase[0].testData[2].type = #test-data
+* testCase[0].testData[2].content.sourceAttachment.url = "testdata/examples/patient-output.json"
+* testCase[0].testData[2].content.sourceAttachment.contentType = "application/fhir+json"
+
+// Additional test cases can be added here for:
+// - Complex transformations with nested data
+// - Error handling scenarios
+// - Performance testing with large datasets
+// - Terminology-aware transformations using ConceptMaps
+// - Test cases from imported external repositories
+
+// Test Case 2: Basic Observation Transform
+* testCase[1].id = "observation-basic-transform"
+* testCase[1].sequence = 2
+* testCase[1].scope.artifact = "http://example.org/StructureMap/ObservationTransform"
+* testCase[1].scope.phase = #unit-test
+* testCase[1].requirement.linkId = "REQ-002"
+* testCase[1].requirement.description = "Basic observation data transformation using FML mapping"
+
+* testCase[1].testRun[0].id = "observation-basic-transform-run"
+* testCase[1].testRun[0].description = "Execute observation transform with valid input and validate output"
+
+// Test Data References for Observation
+* testCase[1].testData[0].id = "observation-transform-map"
+* testCase[1].testData[0].description = "Observation transformation mapping file"
+* testCase[1].testData[0].type = #test-data
+* testCase[1].testData[0].content.sourceAttachment.url = "testdata/examples/observation-transform.map"
+* testCase[1].testData[0].content.sourceAttachment.contentType = "text/plain"
+
+* testCase[1].testData[1].id = "observation-input-data"
+* testCase[1].testData[1].description = "Input observation data for transformation"
+* testCase[1].testData[1].type = #test-data
+* testCase[1].testData[1].content.sourceAttachment.url = "testdata/examples/observation-input.json"
+* testCase[1].testData[1].content.sourceAttachment.contentType = "application/fhir+json"
+
+* testCase[1].testData[2].id = "observation-expected-output"
+* testCase[1].testData[2].description = "Expected output after observation transformation"
+* testCase[1].testData[2].type = #test-data
+* testCase[1].testData[2].content.sourceAttachment.url = "testdata/examples/observation-output.json"
+* testCase[1].testData[2].content.sourceAttachment.contentType = "application/fhir+json"
+
+// Additional test cases can be added here for:
+// - Complex transformations with nested data
+// - Error handling scenarios
+// - Performance testing with large datasets
+// - Terminology-aware transformations using ConceptMaps
+// - Test cases from imported external repositories
+
+// Dependency on FHIR R5 StructureMap specification
+* dependency[0].description = "FHIR R5 StructureMap Resource"
+* dependency[0].predecessor = "http://hl7.org/fhir/5.0.0/StructureDefinition/StructureMap"

input/testdata/examples/LICENSE.md

@@ -0,0 +1,23 @@
+# Test Data License Attribution
+
+## Examples Directory (`examples/`)
+
+All files in this directory are local examples created for the FML Runner project.
+
+**License**: MIT License  
+**Copyright**: 2025 Carl Leitner  
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+## Files Covered
+
+- `patient-transform.map` - Example patient transformation mapping
+- `patient-input.json` - Example patient input data  
+- `patient-output.json` - Example expected patient output
+- `observation-transform.map` - Example observation transformation mapping
+- `observation-input.json` - Example observation input data
+- `observation-output.json` - Example expected observation output

input/testdata/examples/observation-input.json

@@ -0,0 +1,23 @@
+{
+  "resourceType": "Observation",
+  "status": "final",
+  "code": {
+    "coding": [
+      {
+        "system": "http://loinc.org",
+        "code": "29463-7",
+        "display": "Body Weight"
+      }
+    ]
+  },
+  "subject": {
+    "reference": "Patient/example"
+  },
+  "effectiveDateTime": "2023-01-15",
+  "valueQuantity": {
+    "value": 70.5,
+    "unit": "kg",
+    "system": "http://unitsofmeasure.org",
+    "code": "kg"
+  }
+}

input/testdata/examples/observation-output.json

@@ -0,0 +1,23 @@
+{
+  "resourceType": "Observation",
+  "status": "final",
+  "code": {
+    "coding": [
+      {
+        "system": "http://loinc.org",
+        "code": "29463-7",
+        "display": "Body Weight"
+      }
+    ]
+  },
+  "subject": {
+    "reference": "Patient/example"
+  },
+  "effectiveDateTime": "2023-01-15",
+  "valueQuantity": {
+    "value": 70.5,
+    "unit": "kg",
+    "system": "http://unitsofmeasure.org",
+    "code": "kg"
+  }
+}

input/testdata/examples/observation-transform.map

@@ -0,0 +1,14 @@
+/*
+ * Example Observation Transform Map  
+ * License: MIT (local example)
+ */
+
+map "http://example.org/StructureMap/ObservationTransform" = "ObservationTransform"
+
+group main(source src, target tgt) {
+  src.status -> tgt.status;
+  src.code -> tgt.code;
+  src.subject -> tgt.subject;
+  src.valueQuantity -> tgt.valueQuantity;
+  src.effectiveDateTime -> tgt.effectiveDateTime;
+}

input/testdata/examples/patient-input.json

@@ -0,0 +1,11 @@
+{
+  "resourceType": "Patient",
+  "name": [
+    {
+      "family": "Doe",
+      "given": ["John"]
+    }
+  ],
+  "active": true,
+  "gender": "male"
+}

input/testdata/examples/patient-output.json

@@ -0,0 +1,11 @@
+{
+  "resourceType": "Patient",
+  "name": [
+    {
+      "family": "Doe", 
+      "given": ["John"]
+    }
+  ],
+  "active": true,
+  "gender": "male"
+}

input/testdata/examples/patient-transform.map

@@ -0,0 +1,12 @@
+/*
+ * Example Patient Transform Map
+ * License: MIT (local example)
+ */
+
+map "http://example.org/StructureMap/PatientTransform" = "PatientTransform"
+
+group main(source src, target tgt) {
+  src.name -> tgt.name;
+  src.active -> tgt.active;
+  src.gender -> tgt.gender;
+}