Skip to content

Feature/server side url matching - #209

Merged
jez500 merged 16 commits into
mainfrom
feature/server-side-url-matching
Aug 7, 2026
Merged

Feature/server side url matching#209
jez500 merged 16 commits into
mainfrom
feature/server-side-url-matching

Conversation

@jez500

@jez500 jez500 commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Summary by CodeRabbit

  • New Features

    • Added normalized URL matching for product lookups, including tracking-parameter handling and current listing indicators.
    • Added exact store filtering by domain.
    • Added a protected client configuration endpoint with capability discovery, version information, caching, and conditional responses.
    • Added a command to rebuild URL matching data after configuration changes.
  • Documentation

    • Documented URL matching, API filters, client configuration, and renormalization procedures.
  • Bug Fixes

    • Improved handling of malformed, oversized, comma-containing, and affiliate URLs while preventing cross-account data exposure.

jez500 added 14 commits August 7, 2026 20:31
Addresses code review findings on task 1:
- test_env_extra_appends_rather_than_replaces now sets both _EXTRA env
  vars to real values and re-requires the config file, so a regression
  that replaces the defaults instead of appending is actually caught.
- config/url_matching.php filters empty/whitespace-only entries out of
  the exploded env value before merging, so the arrays contain exactly
  the documented defaults when no _EXTRA env var is set.
- filter[url] and filter[domain]: Spatie splits filter values on commas,
  turning a comma-bearing value into an array before the callback runs.
  (string) on an array raised an uncaught ErrorException -> HTTP 500.
  Rejoin array values with ',' before normalising in both handlers.
- filter[url]=: ConvertEmptyStringsToNull turns it into null, and Spatie
  skips non-nullable callback filters on null, so the filter silently
  vanished and returned the caller's whole product list. Add ->nullable(),
  matching the existing filter[domain] pattern.
- ProductTransformer::toArray(): price_cache decoration ran unconditionally,
  so a sparse fieldset excluding price_cache got a phantom price_cache: []
  key back, misreporting an untracked product. Only decorate when the key
  is actually present. Also drop null product ids (from fields[products]
  without id) before the current_url whereIn lookup in PaginationHandler.
@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b770f88e-ca26-4136-b56d-98d53967f4cc

📥 Commits

Reviewing files that changed from the base of the PR and between 2cac0b5 and 14c84f8.

📒 Files selected for processing (5)
  • app/Filament/Resources/ProductResource/Api/Handlers/PaginationHandler.php
  • app/Filament/Resources/StoreResource/Api/Handlers/PaginationHandler.php
  • app/Filament/Traits/ApiHelperTrait.php
  • tests/Feature/Api/ProductApiTest.php
  • tests/Feature/Api/StoreApiTest.php
🚧 Files skipped from review as they are similar to previous changes (3)
  • tests/Feature/Api/StoreApiTest.php
  • app/Filament/Resources/ProductResource/Api/Handlers/PaginationHandler.php
  • tests/Feature/Api/ProductApiTest.php

📝 Walkthrough

Walkthrough

Changes

URL matching and API filtering

Layer / File(s) Summary
URL normalization and backfill
app/Models/Url.php, config/url_matching.php, database/migrations/..., app/Console/Commands/RenormalizeUrls.php, tests/Unit/Models/*, tests/Feature/Console/*, docs/docs/advanced.md, docs/docs/installation.md
URLs now store normalized match keys. Normalization removes configured tracking parameters, canonicalizes URLs, and supports bulk rebuilding.
Product URL filtering and current listings
app/Filament/Resources/ProductResource/Api/..., app/Filament/Traits/ApiHelperTrait.php, tests/Feature/Api/ProductApiTest.php, docs/docs/api.md
Product endpoints support normalized URL filtering and transient is_current flags for matching price-cache entries.
Exact store domain filtering
app/Filament/Resources/StoreResource/Api/Handlers/PaginationHandler.php, tests/Feature/Api/StoreApiTest.php, docs/docs/api.md
Store pagination supports exact normalized domain matching while preserving partial domain filtering.

Client configuration endpoint

Layer / File(s) Summary
Protected client configuration flow
app/Enums/ApiAbility.php, app/Http/Controllers/Api/ClientConfigController.php, routes/api.php, tests/Feature/Api/ClientConfigApiTest.php, docs/docs/api.md, .ai/guidelines/general-preferences.blade.php
The authenticated /api/client-config endpoint returns capabilities and application version with private caching, ETags, conditional requests, and Scramble documentation requirements.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Sanctum
  participant ClientConfigController
  Client->>Sanctum: GET /api/client-config with token
  Sanctum->>ClientConfigController: authorize ClientConfigRead
  ClientConfigController-->>Client: version, capabilities, ETag, cache headers
  Client->>ClientConfigController: If-None-Match request
  ClientConfigController-->>Client: 304 Not Modified
Loading
sequenceDiagram
  participant ProductAPI
  participant UrlModel
  participant ProductTransformer
  ProductAPI->>UrlModel: resolve normalized current_url matches
  UrlModel-->>ProductAPI: matching URL IDs
  ProductAPI->>ProductTransformer: configure current URL and matching IDs
  ProductTransformer-->>ProductAPI: price_cache entries with is_current
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 6.06% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: server-side URL matching.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/server-side-url-matching

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (1)
tests/Feature/Api/ProductApiTest.php (1)

826-826: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Remove unused fixture variables.

Both assignments trigger PHPMD unused-variable warnings.

  • tests/Feature/Api/ProductApiTest.php#L826-L826: Call productWithTrackedUrl() without assigning $product.
  • tests/Feature/Api/StoreApiTest.php#L459-L459: Create the fixture without assigning $store.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/Feature/Api/ProductApiTest.php` at line 826, Remove the unused fixture
assignments: in tests/Feature/Api/ProductApiTest.php lines 826-826, call
productWithTrackedUrl() without assigning $product; in
tests/Feature/Api/StoreApiTest.php lines 459-459, create the fixture without
assigning $store.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@config/url_matching.php`:
- Around line 20-25: Update the tracking-parameter environment value handling in
the tracking_params configuration and the corresponding additional matching
values around the related entries to trim each exploded value before filtering
empty strings and merging. Preserve the existing exclusion of blank values while
ensuring inputs such as “foo, bar” produce normalized parameter names without
leading or trailing whitespace.

In `@docs/docs/api.md`:
- Around line 136-140: Add the text language identifier to the fenced code block
containing the example URLs in the API documentation, leaving the URL examples
unchanged.

In `@tests/Feature/Api/ClientConfigApiTest.php`:
- Around line 30-36: Update test_readable_with_a_minimal_ability_token to
authenticate with only ApiAbility::ClientConfigRead->value, removing
ApiAbility::UserDetail and ApiAbility::MetaExtractionExtract while preserving
the rest of the test.

In `@tests/Unit/Models/UrlNormalizeForMatchTest.php`:
- Around line 90-117: Update test_env_extra_appends_rather_than_replaces() to
snapshot the existing getenv(), $_ENV, and $_SERVER values for both environment
keys before overwriting them, then restore those prior states in finally instead
of always removing the entries. Preserve whether each variable was previously
absent or present, including its original value.

---

Nitpick comments:
In `@tests/Feature/Api/ProductApiTest.php`:
- Line 826: Remove the unused fixture assignments: in
tests/Feature/Api/ProductApiTest.php lines 826-826, call productWithTrackedUrl()
without assigning $product; in tests/Feature/Api/StoreApiTest.php lines 459-459,
create the fixture without assigning $store.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 4b2fbaaf-f051-4edf-97af-357c2d20e2e8

📥 Commits

Reviewing files that changed from the base of the PR and between ed901c7 and 07c6862.

📒 Files selected for processing (21)
  • .ai/guidelines/general-preferences.blade.php
  • app/Console/Commands/RenormalizeUrls.php
  • app/Enums/ApiAbility.php
  • app/Filament/Resources/ProductResource/Api/Handlers/DetailHandler.php
  • app/Filament/Resources/ProductResource/Api/Handlers/PaginationHandler.php
  • app/Filament/Resources/ProductResource/Api/Transformers/ProductTransformer.php
  • app/Filament/Resources/StoreResource/Api/Handlers/PaginationHandler.php
  • app/Http/Controllers/Api/ClientConfigController.php
  • app/Models/Url.php
  • config/url_matching.php
  • database/migrations/2026_08_07_000000_add_url_normalized_to_urls_table.php
  • docs/docs/advanced.md
  • docs/docs/api.md
  • docs/docs/installation.md
  • routes/api.php
  • tests/Feature/Api/ClientConfigApiTest.php
  • tests/Feature/Api/ProductApiTest.php
  • tests/Feature/Api/StoreApiTest.php
  • tests/Feature/Console/RenormalizeUrlsTest.php
  • tests/Unit/Models/UrlNormalizeForMatchTest.php
  • tests/Unit/Models/UrlNormalizedColumnTest.php

Comment on lines +130 to +138
$normalized = Url::normalizeForMatch($currentUrl);

if ($normalized === '') {
return [];
}

return $this->resource->urls()
->where('url_normalized', $normalized)
->pluck('id')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Remove final path slashes during URL normalization.

Url::normalizeForMatch() preserves the final slash in a non-root path. Therefore, https://shop.com/p/x and https://shop.com/p/x/ resolve to different keys.

This breaks the trailing-slash contract used by current_url and filter[url]. Normalize the path without final slashes before building url_normalized.

Proposed upstream fix in app/Models/Url.php
-        $path = $uri->path();
-        $path = $path === '/' ? '' : '/'.Str::lower($path);
+        $path = rtrim((string) $uri->path(), '/');
+        $path = $path === '' ? '' : '/'.Str::lower(ltrim($path, '/'));

Comment thread config/url_matching.php Outdated
Comment thread docs/docs/api.md Outdated
Comment thread tests/Feature/Api/ClientConfigApiTest.php Outdated
Comment thread tests/Unit/Models/UrlNormalizeForMatchTest.php Outdated
jez500 added 2 commits August 7, 2026 23:08
- config/url_matching.php: trim each exploded env value before filtering
  empties, so "foo, bar" yields normalised names without whitespace
- ClientConfigApiTest: authenticate the minimal-token test with only
  client-config:read, proving the endpoint is not gated on other abilities
- UrlNormalizeForMatchTest: snapshot and restore the env vars instead of
  unconditionally unsetting them, and pin the trimming behaviour
- api.md: tag the URL example fence as text
- drop two unused fixture assignments
filter[url][]= and filter[domain][]= produce a nested array once Spatie
splits each element on the comma delimiter, which reached implode() and
raised "Array to string conversion" -> ErrorException -> HTTP 500.

Adds ApiHelperTrait::filterValueToString(), which flattens before
rejoining with ',' so the original value is reconstructed exactly, and
uses it from both filter callbacks in place of the duplicated inline cast.
@jez500
jez500 merged commit 5cc7d8f into main Aug 7, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant