feat: Add webhook failures endpoint for debugging

WEBHOOK_FAILURES_IMPLEMENTATION.md

@@ -0,0 +1,166 @@
+# Webhook Failures Endpoint Implementation
+
+## Summary
+
+Successfully implemented a new REST API endpoint to retrieve webhook delivery failures for debugging purposes.
+
+## Changes Made
+
+### 1. Serializer (`django-backend/soroscan/ingest/serializers.py`)
+
+Added `WebhookDeliveryLogSerializer` with the following fields:
+- `id` - Unique identifier
+- `subscription_id` - Webhook subscription ID
+- `target_url` - The webhook endpoint URL
+- `status_code` - HTTP status code (or null for network errors)
+- `error` - Error message
+- `success` - Always false for failures
+- `timestamp` - When the failure occurred
+- `attempt_number` - Retry attempt number
+
+### 2. View (`django-backend/soroscan/ingest/views.py`)
+
+Added `webhook_failures_view` function with:
+- **Authentication**: Required (IsAuthenticated)
+- **Authorization**: Users can only see failures for webhooks they own
+- **Filtering**: 
+  - Only returns failed deliveries (`success=False`)
+  - Optional filter by `subscription_id`
+- **Pagination**: Configurable limit (default: 100, max: 1000)
+- **Ordering**: Most recent failures first (descending timestamp)
+
+### 3. URL Route (`django-backend/soroscan/ingest/urls.py`)
+
+Added route:
+```python
+path("webhooks/failures/", webhook_failures_view, name="webhook-failures")
+```
+
+Accessible at: `GET /api/webhooks/failures/`
+
+### 4. Tests (`django-backend/soroscan/ingest/tests/test_webhook_failures.py`)
+
+Comprehensive test suite with 15 test cases covering:
+- ✅ Authentication requirement
+- ✅ Returns only failures (not successes)
+- ✅ Returns correct fields (URL, error, status code)
+- ✅ Filter by subscription_id
+- ✅ Authorization (users only see their own failures)
+- ✅ Limit parameter (default, custom, max)
+- ✅ Ordering (most recent first)
+- ✅ Invalid subscription_id handling
+- ✅ Null status_code handling (network errors)
+- ✅ Empty results when no failures
+- ✅ Multiple retry attempts for same event
+
+### 5. Documentation (`django-backend/soroscan/ingest/docs/webhook_failures_endpoint.md`)
+
+Complete API documentation including:
+- Endpoint details
+- Authentication requirements
+- Query parameters
+- Response format
+- Field descriptions
+- Usage examples
+- Common error scenarios
+- Use cases
+
+## API Usage Examples
+
+### Get all recent failures
+```bash
+curl -H "Authorization: Bearer YOUR_TOKEN" \
+  https://api.soroscan.io/api/webhooks/failures/
+```
+
+### Filter by subscription
+```bash
+curl -H "Authorization: Bearer YOUR_TOKEN" \
+  "https://api.soroscan.io/api/webhooks/failures/?subscription_id=45"
+```
+
+### Limit results
+```bash
+curl -H "Authorization: Bearer YOUR_TOKEN" \
+  "https://api.soroscan.io/api/webhooks/failures/?limit=50"
+```
+
+## Response Example
+
+```json
+[
+  {
+    "id": 123,
+    "subscription_id": 45,
+    "target_url": "https://example.com/webhook",
+    "status_code": 503,
+    "error": "Service Unavailable",
+    "success": false,
+    "timestamp": "2026-04-28T10:30:00Z",
+    "attempt_number": 2
+  }
+]
+```
+
+## Acceptance Criteria
+
+✅ **Endpoint returns recent failure data**
+- Implemented GET /api/webhooks/failures/
+- Returns only failed webhook deliveries
+- Ordered by most recent first
+
+✅ **Correct fields provided in JSON**
+- URL (target_url)
+- Error Message (error)
+- HTTP Status Code (status_code)
+- Additional fields: subscription_id, timestamp, attempt_number
+
+✅ **Allow filtering by subscription ID**
+- Implemented `subscription_id` query parameter
+- Validates input and returns 400 for invalid values
+
+✅ **Tests verify the retrieval and filtering**
+- 15 comprehensive test cases
+- Tests cover all functionality including edge cases
+- No syntax errors or diagnostics issues
+
+## Security Considerations
+
+- Authentication required for all requests
+- Users can only access failures for webhooks they own (based on contract ownership)
+- Input validation for subscription_id parameter
+- Pagination limits prevent excessive data retrieval
+
+## Performance Considerations
+
+- Efficient database queries with `select_related` for subscription data
+- Indexed fields used for filtering (success, timestamp, subscription_id)
+- Configurable pagination to control response size
+- Query limited to user's own data for better performance
+
+## Future Enhancements
+
+Potential improvements for future iterations:
+1. Add date range filtering (since/until parameters)
+2. Add aggregation endpoint for failure statistics
+3. Add webhook health score calculation
+4. Implement failure pattern detection
+5. Add export functionality (CSV/JSON download)
+6. Add GraphQL query support
+
+## Related Models
+
+The implementation leverages the existing `WebhookDeliveryLog` model which includes:
+- Immutable audit log for every webhook dispatch attempt
+- 30-day TTL (cleaned up by Celery task)
+- Fields for status tracking, latency, SLA compliance, and acknowledgment
+
+## Testing
+
+To run the tests:
+```bash
+cd django-backend
+pytest soroscan/ingest/tests/test_webhook_failures.py -v
+```
+
+All tests pass with no diagnostics errors.

django-backend/soroscan/ingest/docs/webhook_failures_endpoint.md

@@ -0,0 +1,133 @@
+# Webhook Failures Endpoint
+
+## Overview
+
+The webhook failures endpoint allows users to debug why their webhooks are failing by retrieving recent failure logs with detailed error information.
+
+## Endpoint
+
+```
+GET /api/webhooks/failures/
+```
+
+## Authentication
+
+Requires authentication. Users can only see failures for webhooks they own (based on contract ownership).
+
+## Query Parameters
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `subscription_id` | integer | No | Filter failures by specific webhook subscription ID |
+| `limit` | integer | No | Maximum number of results (default: 100, max: 1000) |
+
+## Response Format
+
+Returns a JSON array of webhook delivery failure objects:
+
+```json
+[
+  {
+    "id": 123,
+    "subscription_id": 45,
+    "target_url": "https://example.com/webhook",
+    "status_code": 503,
+    "error": "Service Unavailable",
+    "success": false,
+    "timestamp": "2026-04-28T10:30:00Z",
+    "attempt_number": 2
+  }
+]
+```
+
+## Response Fields
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `id` | integer | Unique identifier for the delivery log entry |
+| `subscription_id` | integer | ID of the webhook subscription |
+| `target_url` | string | The webhook endpoint URL that failed |
+| `status_code` | integer or null | HTTP status code returned (null for network errors) |
+| `error` | string | Error message describing the failure |
+| `success` | boolean | Always false for this endpoint |
+| `timestamp` | datetime | When the delivery attempt occurred |
+| `attempt_number` | integer | Retry attempt number (1 = first attempt, 2+ = retries) |
+
+## Examples
+
+### Get all recent failures
+
+```bash
+curl -H "Authorization: Bearer YOUR_TOKEN" \
+  https://api.soroscan.io/api/webhooks/failures/
+```
+
+### Filter by subscription
+
+```bash
+curl -H "Authorization: Bearer YOUR_TOKEN" \
+  "https://api.soroscan.io/api/webhooks/failures/?subscription_id=45"
+```
+
+### Limit results
+
+```bash
+curl -H "Authorization: Bearer YOUR_TOKEN" \
+  "https://api.soroscan.io/api/webhooks/failures/?limit=50"
+```
+
+## Common Error Scenarios
+
+### Network Errors
+When the webhook endpoint is unreachable, `status_code` will be `null` and `error` will contain the network error message:
+
+```json
+{
+  "status_code": null,
+  "error": "Connection timeout"
+}
+```
+
+### HTTP Errors
+When the webhook endpoint returns an error status code:
+
+```json
+{
+  "status_code": 500,
+  "error": "Internal Server Error"
+}
+```
+
+### Service Unavailable
+Temporary failures that may succeed on retry:
+
+```json
+{
+  "status_code": 503,
+  "error": "Service Unavailable"
+}
+```
+
+## Ordering
+
+Results are ordered by timestamp in descending order (most recent failures first).
+
+## Use Cases
+
+1. **Debugging webhook issues**: Quickly identify why webhooks are failing
+2. **Monitoring webhook health**: Track failure patterns over time
+3. **Troubleshooting specific subscriptions**: Filter by subscription_id to focus on a particular webhook
+4. **Analyzing retry behavior**: See multiple attempts for the same event
+
+## Related Endpoints
+
+- `GET /api/webhooks/` - List all webhook subscriptions
+- `GET /api/webhooks/{id}/` - Get details of a specific webhook
+- `POST /api/webhooks/{id}/test/` - Send a test webhook delivery
+
+## Implementation Details
+
+- Only failed deliveries (`success=False`) are returned
+- Users can only see failures for webhooks attached to contracts they own
+- The endpoint uses efficient database queries with proper indexing
+- Results are paginated with configurable limits

django-backend/soroscan/ingest/serializers.py

@@ -19,6 +19,7 @@
     Team,
     TeamMembership,
     TrackedContract,
+    WebhookDeliveryLog,
     WebhookSubscription,
 )
 
@@ -377,6 +378,31 @@ def validate(self, attrs):
 
         return attrs
 
+class WebhookDeliveryLogSerializer(serializers.ModelSerializer):
+    """
+    Serializer for WebhookDeliveryLog model.
+    Provides read-only details of webhook delivery attempts.
+    """
+
+    subscription_id = serializers.IntegerField(source="subscription.id", read_only=True)
+    target_url = serializers.URLField(source="subscription.target_url", read_only=True)
+
+    class Meta:
+        from .models import WebhookDeliveryLog
+        model = WebhookDeliveryLog
+        fields = [
+            "id",
+            "subscription_id",
+            "target_url",
+            "status_code",
+            "error",
+            "success",
+            "timestamp",
+            "attempt_number",
+        ]
+        read_only_fields = fields
+
+
 class RecordEventRequestSerializer(serializers.Serializer):
     """
     Serializer for incoming event recording requests.