Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize heatmap query #33853

Merged
merged 7 commits into from
Mar 20, 2025
Merged

Conversation

lunny
Copy link
Member

@lunny lunny commented Mar 11, 2025

When there are over 5M records on action table, the heatmap on dashboard is very slow as below SQL.

database duration=1.8881s db.sql="SELECT created_unix DIV 900 * 900 AS timestamp, count(user_id) as contributions FROM `action` WHERE user_id=? AND act_user_id=? AND (created_unix > ?) GROUP BY timestamp ORDER BY timestamp"

This PR add a new index for action table with columns user_id, act_user_id and created_unix so that this query will become about 6 times faster than before.

@lunny lunny added the performance/speed performance issues with slow downs label Mar 11, 2025
@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Mar 11, 2025
@pull-request-size pull-request-size bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Mar 11, 2025
@github-actions github-actions bot added modifies/go Pull requests that update Go code modifies/migrations labels Mar 11, 2025
@lunny lunny added this to the 1.24.0 milestone Mar 11, 2025
@silverwind
Copy link
Member

are those index names consistent with previous indexes? maybe better to spell out the column names wholly.

@lunny
Copy link
Member Author

lunny commented Mar 14, 2025

are those index names consistent with previous indexes? maybe better to spell out the column names wholly.

ce57b3b will follow previous index naming pattern.

@silverwind
Copy link
Member

I mean why not use act_user_id_created_unix_user_id instead of au_c_u? Note I don't know who other indexes were named.

@lunny
Copy link
Member Author

lunny commented Mar 14, 2025

I mean why not use act_user_id_created_unix_user_id instead of au_c_u? Note I don't know who other indexes were named.

All other indexes in this table uses a first character of the columns in this index. au == act_user_id, c == created_unix and u == user_id.

@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Mar 15, 2025
@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Mar 18, 2025
@lunny lunny merged commit 4a7ab0a into go-gitea:main Mar 20, 2025
26 checks passed
@lunny lunny deleted the lunny/optimaze_dashboard_heatmap branch March 20, 2025 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. modifies/go Pull requests that update Go code modifies/migrations performance/speed performance issues with slow downs size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants