Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Multithreaded processing #364

Open
Oleg4260 opened this issue Feb 4, 2025 · 1 comment
Open

[FEATURE] Multithreaded processing #364

Oleg4260 opened this issue Feb 4, 2025 · 1 comment
Labels
feature Feature request

Comments

@Oleg4260
Copy link
Contributor

Oleg4260 commented Feb 4, 2025

Is your feature request related to a problem? Please describe.
The program uses only 1 thread while processing data and generating the map, so it's much slower than it could be on modern CPUs

Describe the solution you'd like
I understand that this isn't easy to code but it would be very good for generating large maps since modern CPUs have 4, 8, 12 or more threads, I think it's possible to run the program in multiple threads by invoking a few instances of the same function for each thread with different coordinates given for each function.

Image

@Oleg4260 Oleg4260 added the feature Feature request label Feb 4, 2025
@Oleg4260 Oleg4260 changed the title [FEATURE] Implement multithreaded processing [FEATURE] Multithreaded processing Feb 4, 2025
@benjamin051000
Copy link
Contributor

benjamin051000 commented Feb 6, 2025

See #16 never mind, that's from back when the codebase was python

Excerpt from Louis in #300:

Arnis is indeed processing only a single element at a time, since elements can stretch over the entire map (e.g. streets). In combination with the element type processing order and many potentially overlapping block read and write calls at the same time, I failed on implementing multithreading on my first try. Only the saving step makes use of multithreading in the upcoming release since there are no "overlapping" write calls involved.

This is 100% on the to-do list. Right now I've been focusing my efforts on cleaning up a lot of the codebase (disclaimer: I am not a maintainer of this repo).

After that's done and the codebase is in a more maintainable state, we can look into performance optimizations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Feature request
Projects
None yet
Development

No branches or pull requests

2 participants