Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove rest of page limit #5429

Draft
wants to merge 136 commits into
base: main
Choose a base branch
from
Draft

Remove rest of page limit #5429

wants to merge 136 commits into from

Conversation

dessalines
Copy link
Member

@dessalines dessalines commented Feb 14, 2025

WIP

#4517

@dullbananas
Copy link
Collaborator

I released i-love-jesus version 0.2.0. To migrate the lemmy code, replace then_desc with then_order_by, and add the new constructor argument (see docs below). In some places, the argument should be something like asc_if(o.sort == Old). If a query doesn't need both asc and desc, then it no longer requires ReverseTimestampSort. Don't forget to update or delete the corresponding index when removing a use of ReverseTimestampSort. ID tie-breakers don't need a particular sort direction, so query.then_desc(ReverseTimestampSort(key::published)).then_desc(key::id) can become query.then_order_by(key::published).then_order_by(key::id) with direction set to ascending. This might solve the ascending text sort problem.

https://docs.rs/i-love-jesus/0.2.0/i_love_jesus/struct.PaginatedQueryBuilder.html#method.new

@dessalines
Copy link
Member Author

Sweet thx, I'll give it a try now.

@dessalines
Copy link
Member Author

When should I be using page_before_or_equal ?

For a simple example, check this block of the person_view list banned code.

Currently I have cursor_data, and page_back for the request, and next_page, prev_page for the response.

Right now I'm just passing None.

@dullbananas
Copy link
Collaborator

Right now I'm just passing None.

That is correct. The parameter is only used for the optimization that post view does.

ActiveMonthly => pq.then_order_by(key::users_active_month),
ActiveWeekly => pq.then_order_by(key::users_active_week),
ActiveDaily => pq.then_order_by(key::users_active_day),
// TODO the lower function doesn't work for these
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Create a LowerKey struct, exactly the same as ReverseTimestampKey other than what SQL function it calls.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thx.

@@ -256,6 +271,7 @@ impl CommentQuery<'_> {
// + !post_id isn't used anyways (afaik)
if o.post_id.is_some() || o.parent_path.is_some() {
// Always order by the parent path first
// TODO not sure if this initial then_order_by will work correctly
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing any sorting before creating the paginated query builder is always incorrect.

Copy link
Member Author

@dessalines dessalines Mar 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one I couldn't figure out and I'll need your help with. Problem is here, trying do a subpath sort:

pq = pq.then_order_by(subpath(key::path, 0, -1));

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That can be done using a new struct similar to ReverseTimestampKey but it would call subpath(x, 0, -1) instead of reverse_timestamp_key(x). If it needs to be in a different direction than the direction that PaginatedQueryBuilder is initialized with, then something more complicated is needed, which I could try to figure out when I have time.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, it doesn't matter if the subpath sort is ascending or descending, just like the id tie-breaker at the end

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, to prevent the confusion that I had, add a comment saying that putting comments underneath their parents is done by the frontend instead of in the db query


// Only sort by ascending for Old
let sort = o.sort.unwrap_or(CommentSortType::Hot);
let sort_direction = asc_if(sort == CommentSortType::Old);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will cause distinguished comments to go to the bottom when sorting by old. Either use ReverseTimestampKey here again, conditionally invert the distinguished key (using a new wrapper similar to ReverseTimestampKey), or don't sort by distinguished at all when using the Old sort type (I think this is the best solution, and maybe it also should be done for New sort for consistency). Update indexes to match.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

K I got this one handled (indexes I'll go over shortly.)

// |pool| PersonActions::read_blocks_for_person(pool, person_id),
// |pool| CommunityModeratorView::for_person(pool, person_id, Some(&local_user_view.local_user)),
// |pool| LocalUserLanguage::read(pool, local_user_id)
// ))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do all of these functions return the same error type?


for comment_id in &comment_ids {
Comment::update(&mut conn.into(), *comment_id, form).await?;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This query is extremely inefficient if there are more than a dozen comments. It would be much more efficient with update(comment::table).filter(comment_id.eq_any(comment_ids)).set(form)

@dessalines
Copy link
Member Author

I'm going to wait for #5515 to be merged first since there are going to be a ton of conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants