Implement FML Execution Validation Test Suite with FHIR IG-based TestPlan

[Copilot]

This PR implements a comprehensive FHIR IG-based validation test suite for FML (FHIR Mapping Language) execution as requested in issue #XXX. The test suite provides a complete framework for validating FML execution capabilities using real-world test cases sourced from community FML projects with proper license compliance.

🎯 Key Features

FHIR IG Test Structure

• FSH TestPlan: Complete FMLExecutionValidationTestPlan.fsh defining test cases, validation requirements, and test data references
• Test Data Organization: Structured input/testdata/ directory with subfolders for different test sources

Test Case Import and Attribution

• Automated Import: scripts/import-test-data.js imports test cases from ahdis/matchbox and FHIR/fhir-test-cases repositories
• License Compliance: Proper attribution headers for Apache 2.0 (matchbox) and HL7 copyright/CC0 (fhir-test-cases) licenses
• Repository Explorer: scripts/explore-test-repos.js discovers available test files in external repositories

Test Data Management

• File Mapping: test-manifest.json explicitly maps .map files to their corresponding input/output test cases
• Validation Pipeline: scripts/validate-test-data.js ensures test data integrity and validates JSON structure
• Working Examples: Two complete test cases (patient and observation transforms) with valid FHIR resources

📂 Directory Structure

input/
├── fsh/tests/FMLExecutionValidationTestPlan.fsh  # FSH TestPlan definition
├── testdata/                                     # Test data with license attribution
│   ├── examples/                                 # Local MIT-licensed examples
│   │   ├── patient-transform.map                 # FML mapping files
│   │   ├── patient-input.json                    # Input FHIR resources
│   │   └── patient-output.json                   # Expected output resources
│   ├── fhir-test-cases/                         # HL7 licensed test cases
│   ├── matchbox/                                 # Apache 2.0 licensed test cases
│   └── test-manifest.json                        # File mapping configuration
└── README.md                                     # Comprehensive documentation

🛠 Usage

The test suite includes convenient NPM scripts:

# Import test cases with proper attribution
npm run import:test-data

# Explore external repositories for available test files
npm run explore:test-repos

# Validate test data integrity and mappings
npm run validate:test-data

📋 Test Case Structure

Each test case follows standardized naming conventions:

• *-transform.map or *.map — FML mapping specification
• *-input.json — FHIR resource to be mapped
• *-output.json — Expected output after applying the map

Files are paired by base name (e.g., patient-transform.map, patient-input.json, patient-output.json) and explicitly mapped in the test manifest.

📚 Documentation

• Published TestPlan: When published, documentation will be available at https://litlfred.github.io/fmlrunner/TestPlan/FMLExecutionValidationTestPlan.html
• Comprehensive README: Updated main README with test suite usage and references
• License Attribution: All imported files include proper source and license attribution

✅ Validation Results

All test cases validate successfully:

• JSON structure validation for input/output files
• FML syntax validation for mapping files
• License attribution verification
• File mapping integrity checks

This implementation provides a solid foundation for comprehensive FML execution testing while maintaining full compliance with external project licensing requirements.

Note: This implementation uses FHIR TestPlan resources only, without TestScript resources, as requested.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>implement test suite</issue_title>
<issue_description># FML Execution Validation Test Suite Requirements

Purpose

Define a FHIR IG-based validation test suite for FML using FHIR TestPlan, with real-world test cases sourced (with license compliance) from community FML projects.

Directory Structure

• FSH TestPlan: input/fsh/tests/FMLExecutionValidationTestPlan.fsh
• Test Data: input/testdata/
  • All imported test cases will be stored here, with subfolders as needed.
  • License and attribution headers must be preserved and included at the top of every imported file.

Test Data Sources

Importing from ahdis/matchbox and FHIR/fhir-test-cases

• ahdis/matchbox:
  • Import .map, .json, and .xml test files from matchbox-server/src/test/resources.
  • Add Apache 2.0 license and attribution at the top of each file.
• FHIR/fhir-test-cases:
  • Import all relevant files from FHIR/fhir-test-cases/r5/structure-mapping.
  • Add the FHIR license and attribution at the top of each file.
  • Note: fhir-test-cases LICENSE is an HL7 copyright.

How are input, output, and map files connected?

• In fhir-test-cases/r5/structure-mapping:
  • Test cases are often defined by filename conventions or manifest files (e.g., YAML or JSON manifests).
  • Common pattern:
    • *-map.txt or .map — FML mapping specification
    • *-input.json or .xml — FHIR resource to be mapped
    • *-output.json or .xml — Expected output after applying the map
    • Sometimes, a manifest or README describes more complex scenarios or links files together.
  • Mapping strategy:
    • When importing, pair files with the same base name, e.g. patient-map.txt, patient-input.json, patient-output.json.
    • For ambiguous cases, consult any manifest/readme, or use the test specification logic from the source repository.

TestPlan/TestScript Design

• Each test case in FML Execution Validation:
  • References a map file, an input file, and an expected output file.
  • Uses FHIRPath assertions for output validation.
• All file references are relative to input/testdata/.

Attribution & License Compliance

• Each imported file must include, at the top:
  • Source repository and URL
  • Original license statement (Apache 2.0 for matchbox, HL7 copyright/license for fhir-test-cases)
  • A statement of compliance with original contribution requirements.

---

To Do

• Script/process to import and attribute test data from @ahdis/matchbox and FHIR/fhir-test-cases
• Explicitly map .map files to their input/output test cases
• Draft FSH TestPlan/TestScript resources
• Migrate all relevant tests and update documentation
• update the README and documentation to add reference to where the resulting published testplan documentation will live on gh-pages.
  </issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes #7

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[litlfred]

i DO NOT want to use testscript resource @copilot

[Copilot]

i DO NOT want to use testscript resource @copilot

Removed TestScript resource from the test suite. The implementation now uses only the FHIR TestPlan resource without TestScript. Changes in commit 6fc1e32:

• Removed FMLPatientTransformTestScript instance (94 lines) from FMLExecutionValidationTestPlan.fsh
• Updated documentation to remove all TestScript references
• Test suite now contains only the TestPlan resource with test case definitions and test data references