Skip to content

Search Solr (DS 7.X) with title containing colon doesn't return results except with quotes or escaping or having a stop word after the colon #9670

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
fribeiro-fccn opened this issue Jun 27, 2024 · 3 comments · May be fixed by DSpace/dspace-angular#4207
Open

Search Solr (DS 7.X) with title containing colon doesn't return results except with quotes or escaping or having a stop word after the colon #9670

fribeiro-fccn opened this issue Jun 27, 2024 · 3 comments · May be fixed by DSpace/dspace-angular#4207
Labels
affects: 7.x Issue impacts 7.x releases affects: 8.x Issue impacts 8.x releases bug component: Discovery Related to Discovery search or browse system needs discussion Ticket or PR needs discussion before it can be moved forward.

Comments

@fribeiro-fccn
Copy link

fribeiro-fccn commented Jun 27, 2024 •
edited
Loading

When searching for an item having ": "in the title doesn't return any values. The exceptions are:
1 - Quoting all the title "TITLE"
2 - Escaping the : with the backslash Example: FTG-Net-E: A hierarchical ensemble graph neural network for DDoS attack detection
3 - Having a stopword after the : lie on, or, at, in....

And it doesn't work either with a substring of the search expression.

To Reproduce
Using the demo.dspace.org with an actual title containing : FTG-Net-E: A hierarchical ensemble graph neural network for DDoS attack detection

  1. Insert FTG-Net-E: A hierarchical ensemble graph neural network for DDoS attack detection in the search box
  2. Search doesn't retrieve any results

To get reslts:
1 - Use quotes "FTG-Net-E: A hierarchical ensemble graph neural network for DDoS attack detection"
2 - Escape : : FTG-Net-E: A hierarchical ensemble graph neural network for DDoS attack detection
3 - Use a stopword: FTG-Net-E: An hierarchical ensemble graph neural network for DDoS attack detection

Note: I din't edit the title to add the the stop word, only inserted "an" instead of "a" and it works

Expected behavior
The title should be retrieved in search without the workaround
@fribeiro-fccn fribeiro-fccn added bug needs triage New issue needs triage and/or scheduling labels Jun 27, 2024
@github-project-automation github-project-automation bot moved this to 🆕 Triage in DSpace Backlog Jun 27, 2024
@fribeiro-fccn fribeiro-fccn changed the title Search Solr (DS 7.6.2) with title containing : doesn't return results except with "" or escaping or having a stop word after the : Search Solr (DS 7.X) with title containing : doesn't return results except with "" or escaping or having a stop word after the : Jun 27, 2024
@tdonohue
Copy link
Member

@fribeiro-fccn : The behavior you are seeing is because ":" (colon) is a special character in Solr, and it allows you to perform more advanced searches, like those described at https://wiki.lyrasis.org/display/DSDOC7x/Search+-+Advanced

So, there are two main options here:

  1. Always escape a ":" in search string
    • This seems to be the behavior you are looking for, as it avoids any workarounds (i.e. having to quote the string or escape the colon manually)
    • Unfortunately, this means that searching withing specific fields & other advanced search techniques will no longer work
  2. Never escape a ":" in a search string
    • This is the current behavior. It allows for advanced search techniques.
    • But means that users need to surround strings containing special characters with quotes, or else sometimes no results will be returned.

The only other option I can think of is to provide both options and let the user (or configuration) select which they want.

This ticket is also related to / duplicate of DSpace/dspace-angular#1996

Needs discussion, as there are two approaches here and different sites seem to favor different approaches to this searchbox behavior.

@tdonohue tdonohue changed the title Search Solr (DS 7.X) with title containing : doesn't return results except with "" or escaping or having a stop word after the : Search Solr (DS 7.X) with title containing colon doesn't return results except with quotes or escaping or having a stop word after the colon Jun 27, 2024
@tdonohue tdonohue added affects: 7.x Issue impacts 7.x releases affects: 8.x Issue impacts 8.x releases needs discussion Ticket or PR needs discussion before it can be moved forward. and removed needs triage New issue needs triage and/or scheduling labels Jun 27, 2024
@tdonohue tdonohue added the component: Discovery Related to Discovery search or browse system label Jun 27, 2024
@fribeiro-fccn
Copy link
Author

@tdonohue Thanks Tim.

I opened this bug because in DSpace 5 when searching with : it automatically escapes the special characters as you can see in the example below.
image

I agree that needs discussion.
For now I'm telling users to use quotes in the whole search string!

@ineiti
Copy link

ineiti commented Apr 16, 2025

Personally I vote for adding a checkbox or button advanced search, and only then interpret the : for advanced search.

If the checkbox is not ticked, escape :.

ineiti added a commit to c4dt/dspace-angular that referenced this issue Apr 16, 2025
@ineiti
Escaping ':' in search queries, except if advanced=true
a8e8917 
Following up on the issue DSpace/DSpace#9670 - here is a first
proposal.

Per default, this sets the 'advanced' flag to false and will escape ':' characters in the search string.
Only with 'advanced = true' is the search string passed as-is.

TODO:
- add tests
- make sure this looks good
@ineiti ineiti linked a pull request Apr 16, 2025 that will close this issue
Open
15 tasks
ineiti added a commit to c4dt/dspace-angular that referenced this issue Apr 16, 2025
@ineiti
Escaping ':' in search queries, except if advanced=true
ecb9e2f 
Following up on the issue DSpace/DSpace#9670 - here is a first
proposal.

Per default, this sets the 'advanced' flag to false and will escape ':' characters in the search string.
Only with 'advanced = true' is the search string passed as-is.

TODO:
- add tests
- make sure this looks good
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
affects: 7.x Issue impacts 7.x releases affects: 8.x Issue impacts 8.x releases bug component: Discovery Related to Discovery search or browse system needs discussion Ticket or PR needs discussion before it can be moved forward.
Projects
DSpace 8.x and 7.6.x Maintenance
Status: 📋 To Do
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Escaping ':' in search queries, except if advanced=true c4dt/dspace-angular
3 participants
@tdonohue @ineiti @fribeiro-fccn