Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
name: Bug report
about: File a bug report to help us improve PromptDrifter GitHub Action
title: "[BUG]: "
labels: bug, to triage
assignees: ''
type: bug

---

<!--
Thank you for using PromptDrifter GitHub Action! 🎉

Before filing a bug report, please check:
- Have a usage question? Ask in Discussions: https://github.com/Code-and-Sorts/PromptDrifter-action/discussions
- Have an idea for a feature? Use the feature request template instead
- Search existing issues to avoid duplicates
- Check the main PromptDrifter CLI repo for CLI-specific issues: https://github.com/Code-and-Sorts/PromptDrifter

Please provide the information below to help us understand and fix the bug.
-->

## Bug Description
<!-- A clear and concise description of what the bug is -->

## Steps to Reproduce
1.
2.
3.

## Expected Behavior
<!-- What you expected to happen -->

## Actual Behavior
<!-- What actually happened -->

## Minimal Reproduction
<!-- Please provide the minimal workflow configuration that reproduces the issue -->

### GitHub Workflow
```yaml
# Your GitHub workflow using the action
name: Test PromptDrifter
on: [push]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: Code-and-Sorts/[email protected]
with:
command: validate # or other command
files: your-config.yaml
```

### PromptDrifter Configuration
```yaml
# Your promptdrifter.yaml or test configuration
```

## Environment
- **Action version**: <!-- e.g., v0.0.1, @main, specific commit -->
- **GitHub Runner**: <!-- e.g., ubuntu-latest, windows-latest, macos-latest -->
- **PromptDrifter CLI version**: <!-- If known, from action logs -->
- **Command used**: <!-- validate, run, init, etc. -->

## Additional Context
<!-- Add any other context, screenshots, or logs about the problem -->
59 changes: 59 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
name: Feature request
about: Suggest an idea or enhancement for PromptDrifter GitHub Action
title: "[FEATURE]: "
labels: enhancement
assignees: ''
type: feature

---

<!--
Thank you for using PromptDrifter GitHub Action! 🚀

Before filing a feature request, please check:
- Have a usage question? Ask in Discussions: https://github.com/Code-and-Sorts/PromptDrifter-action/discussions
- Experiencing a bug? Use the bug report template instead
- Search existing issues to see if someone already requested this
- For PromptDrifter CLI features, file issues in the main repo: https://github.com/Code-and-Sorts/PromptDrifter

Please provide the information below to help us understand your feature request.
-->

## Problem Statement
<!-- Is your feature request related to a problem? Describe what the problem is -->

## Proposed Solution
<!-- Describe the solution you'd like to see -->

## Use Case
<!-- Describe how this feature would be used in GitHub workflows. Include specific examples if possible -->

### Example Workflow Usage
```yaml
# How you would like to use this feature in a GitHub workflow
name: Example
on: [push]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: Code-and-Sorts/[email protected]
with:
# New feature usage here
```

## Alternatives Considered
<!-- Describe any alternative solutions or features you've considered -->

## Additional Context
<!-- Add any other context, mockups, or examples about the feature request -->

## Implementation Notes
<!-- Optional: If you have ideas about how this could be implemented in the GitHub Action, share them here. Consider:
- New action inputs
- Changes to entrypoint.sh
- Docker image modifications
- Integration with PromptDrifter CLI features
-->
220 changes: 220 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,220 @@
name: Test PromptDrifter Action

on:
pull_request:
branches: [ main ]

jobs:
test-validate:
runs-on: ubuntu-latest
name: Test Validate Command
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Create test config
run: |
mkdir -p tests
cat > tests/sample.yaml << 'EOF'
version: "0.1"
adapters:
- id: test-validation
prompt: "Say hello"
expect_substring: "hello"
adapter:
- type: openai
model: gpt-4o-mini
max_tokens: 50
temperature: 0.7
tags:
- test
EOF

- name: Test validate command
uses: ./
with:
command: 'validate'
files: 'tests/sample.yaml'

test-init:
runs-on: ubuntu-latest
name: Test Init Command
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Test init command
uses: ./
with:
command: 'init'
directory: 'test-init-dir'

- name: Verify init created files
run: |
ls -la test-init-dir/
test -f test-init-dir/promptdrifter.yaml || echo "Config file not found"

test-input-validation:
runs-on: ubuntu-latest
name: Test Input Validation
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Create test config
run: |
mkdir -p tests
cat > tests/minimal.yaml << 'EOF'
version: "0.1"
adapters:
- id: minimal-test
prompt: "Test"
expect_substring: "test"
adapter:
- type: openai
model: gpt-4o-mini
max_tokens: 10
tags:
- minimal
EOF

- name: Test with fake API keys (should fail gracefully)
uses: ./
continue-on-error: true
with:
command: 'run'
files: 'tests/minimal.yaml'
openai-api-key: 'fake-key-for-testing'
max-concurrent: '2'
no-cache: 'true'

- name: Test missing files input (should fail)
uses: ./
continue-on-error: true
with:
command: 'run'
openai-api-key: 'fake-key'

test-multiple-commands:
runs-on: ubuntu-latest
name: Test Multiple Commands
strategy:
matrix:
test-case:
- command: 'validate'
files: 'tests/config1.yaml tests/config2.yaml'
- command: 'test-drift-type'
drift-type: 'semantic'
expected: 'Hello'
actual: 'Hi'
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Create test configs
run: |
mkdir -p tests
cat > tests/config1.yaml << 'EOF'
version: "0.1"
adapters:
- id: config1-test
prompt: "Test 1"
expect_substring: "test"
adapter:
- type: openai
model: gpt-4o-mini
max_tokens: 10
tags:
- test1
EOF
cat > tests/config2.yaml << 'EOF'
version: "0.1"
adapters:
- id: config2-test
prompt: "Test 2"
expect_substring: "test"
adapter:
- type: openai
model: gpt-4o-mini
max_tokens: 10
tags:
- test2
EOF

- name: Test command
uses: ./
continue-on-error: true
with:
command: ${{ matrix.test-case.command }}
files: ${{ matrix.test-case.files }}
drift-type: ${{ matrix.test-case.drift-type }}
expected: ${{ matrix.test-case.expected }}
actual: ${{ matrix.test-case.actual }}

test-api-key-handling:
runs-on: ubuntu-latest
name: Test API Key Environment Variables
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Create test config
run: |
mkdir -p tests
cat > tests/multi-provider.yaml << 'EOF'
version: "0.1"
adapters:
- id: openai-test
prompt: "Say hello"
expect_substring: "hello"
adapter:
- type: openai
model: gpt-4o-mini
max_tokens: 50
temperature: 0.7
tags:
- test
- id: claude-test
prompt: "Say hello"
expect_substring: "hello"
adapter:
- type: claude
model: claude-3-haiku
max_tokens: 50
temperature: 0.7
tags:
- test
EOF

- name: Test with multiple fake API keys
uses: ./
continue-on-error: true
with:
command: 'run'
files: 'tests/multi-provider.yaml'
openai-api-key: 'fake-openai-key'
claude-api-key: 'fake-claude-key'
gemini-api-key: 'fake-gemini-key'
no-cache: 'true'

test-error-handling:
runs-on: ubuntu-latest
name: Test Error Handling
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Test invalid command
uses: ./
continue-on-error: true
with:
command: 'invalid-command'
files: 'nonexistent.yaml'

- name: Test missing migrate parameters
uses: ./
continue-on-error: true
with:
command: 'migrate'
migrate-input: 'input.yaml'
# missing migrate-output should cause error
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
CLAUDE.md
17 changes: 17 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
FROM python:3.11-slim

LABEL "com.github.actions.name"="PromptDrifter Action"
LABEL "com.github.actions.description"="Run PromptDrifter CLI tests in GitHub Actions"
LABEL "com.github.actions.icon"="check-circle"
LABEL "com.github.actions.color"="blue"

RUN apt-get update && apt-get install -y \
git \
&& rm -rf /var/lib/apt/lists/*

RUN pip install --no-cache-dir promptdrifter==0.0.2

COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

ENTRYPOINT ["/entrypoint.sh"]
Loading