Add trip ID cleanup regex + early route/stop ID cleanup

[mmathieum]

Requires to:

• solve ID unicity constraint while reading GTFS > ignore same 1 real route/stop twice with different IDs in GTFS
  • also merge route long name if only difference (ignore different URL)
• keep trip original route ID for STL Laval (2 routes in GTFS = 1 route w/ 2 directions in reality)

[gemini-code-assist]

Summary of Changes

Hello @mmathieum, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new configurable mechanism for cleaning trip IDs using regular expressions. It also refactors the GTFS data parsing logic to apply ID cleanup for routes, stops, and trips much earlier in the process, specifically when reading data from GTFS files. This change ensures that all relevant IDs are consistently cleaned before they are used to construct internal data models, leading to more robust and predictable data processing.

Highlights

• Trip ID Cleanup: A new configurable regex (trip_id_cleanup_regex) is introduced to allow custom cleaning of trip IDs.
• Early ID Cleanup: The parsing of GTFS files (stop_times.txt, frequencies.txt, directions.txt, trips.txt, stops.txt, routes.txt) now applies ID cleanup for routes, stops, and trips immediately upon reading the data.
• Refactored ID Handling: The DefaultAgencyTools class is updated to support the new trip ID cleanup and to remove redundant cleanup calls for route and stop IDs, as these are now handled earlier.
• Interface Extension: The GAgencyTools interface is extended to include methods for managing trip ID cleanup regex and patterns.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a cleanup mechanism for trip IDs using a regex, similar to what's already in place for route and stop IDs. It also refactors the route and stop ID cleaning to happen earlier in the parsing process, which is a good improvement for consistency and maintainability. The changes are well-structured and follow existing patterns. I've identified a potential thread-safety issue and several places where string trimming is applied inconsistently to IDs, which could lead to bugs. Addressing these points will make the implementation more robust.

[mmathieum]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a feature to clean trip IDs using a regex, similar to how route and stop IDs are handled. It also refactors the ID cleaning for routes and stops to occur earlier in the parsing process, which enables deduplication of routes and stops with different IDs but same content. The logic for merging routes that only differ in their long name is also a great addition. The changes are well-implemented and consistent with the existing codebase. I've found one potential issue regarding agency ID comparison when checking for route equality, which could lead to incorrect data merging in multi-agency GTFS files.