[Fixes #13653] Remote WMS cannot match existing harvestable resource if the remote URL contains query parameters

[mattiagiupponi]

Checklist

Reviewing is a process done by project maintainers, mostly on a volunteer basis. We try to keep the overhead as small as possible and appreciate if you help us to do so by completing the following items. Feel free to ask in a comment if you have troubles with any of them.

For all pull requests:

• Confirm you have read the contribution guidelines
• You have sent a Contribution Licence Agreement (CLA) as necessary (not required for small changes, e.g., fixing typos in the documentation)
• Make sure the first PR targets the master branch, eventual backports will be managed later. This can be ignored if the PR is fixing an issue that only happens in a specific branch, but not in newer ones.

The following are required only for core and extension modules (they are welcomed, but not required, for contrib modules):

• There is a ticket in https://github.com/GeoNode/geonode/issues describing the issue/improvement/feature (a notable exemption is, changes not visible to end-users)
• The issue connected to the PR must have Labels and Milestone assigned
• PR for bug fixes and small new features are presented as a single commit
• PR title must be in the form "[Fixes #<issue_number>] Title of the PR"
• New unit tests have been added covering the changes, unless there is an explanation on why the tests are not necessary/implemented

Submitting the PR does not require you to check all items, but by the time it gets merged, they should be either satisfied or inapplicable.

[gemini-code-assist]

Summary of Changes

Hello @mattiagiupponi, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a critical issue where GeoNode was unable to correctly identify and match existing harvestable resources if the remote WMS service URL contained query parameters. The core of the fix involves making the Harvester lookup more flexible by using a substring match for the remote URL. Additionally, it refines how WMS URLs are retrieved and includes an updated end-to-end test to ensure the corrected behavior is properly validated, enhancing the system's ability to manage remote WMS resources reliably.

Highlights

• Harvester URL Matching Logic: The lookup for existing Harvester objects has been updated to use a contains operator for the remote_url field. This change allows WMS service URLs that include query parameters to be successfully matched to their corresponding harvesters, resolving an issue where exact URL matches would fail.
• WMS URL Parameter Retrieval: The WMS handler now correctly retrieves the OWS (Open Geospatial Consortium Web Services) URL from input parameters using the more specific ows_url key instead of parsed_url. This ensures that the correct and complete URL is used when attempting to associate a WMS resource with a harvester.
• End-to-End Test Refinement: An existing end-to-end test for importing WMS harvestable resources has been refactored. The test now accurately simulates the scenario where a Harvester is created with a remote_url containing query parameters, thereby validating the fix and ensuring that harvestable resources are correctly created and linked under these conditions.
• Test Skipping: A specific test, test_get_arcgis_alternative_structure, has been temporarily skipped with a note indicating it needs to be revisioned. This change is unrelated to the core bug fix but was included in the pull request.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request addresses an issue where a remote WMS harvester cannot be matched if its URL contains query parameters. The fix involves using the full OWS URL, including query parameters, when searching for a harvester and changing the database query from an exact match to a __contains lookup. The test suite is also updated to cover this scenario. My review focuses on the potential risks of using __contains for URL matching and suggests a safer alternative to prevent incorrect matches.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.97%. Comparing base (591f4c6) to head (07c81fe).
⚠️ Report is 10 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #13654       +/-   ##
===========================================
+ Coverage   24.03%   73.97%   +49.94%     
===========================================
  Files         931      931               
  Lines       55216    55218        +2     
  Branches     7437     7439        +2     
===========================================
+ Hits        13269    40846    +27577     
+ Misses      41545    12752    -28793     
- Partials      402     1620     +1218     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.