Skip to content

feat: add searches and aggregation to express backend #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 7, 2025
+1,777 −155
Merged

feat: add searches and aggregation to express backend #28

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2755c63
feat: add searches and aggregation to express backend
jordan-smith721 Nov 6, 2025
3ee207f
Merge remote-tracking branch 'upstream/development' into express-updates
jordan-smith721 Nov 6, 2025
dbb1dbf
fix vector search implementation
jordan-smith721 Nov 6, 2025
24b407a
Dachary feedback
jordan-smith721 Nov 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion client/app/lib/api.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -410,7 +410,7 @@ export async function fetchMoviesWithComments(
const queryParams = new URLSearchParams();
queryParams.append('limit', limit.toString());
if (movieId) {
queryParams.append('movie_id', movieId);
queryParams.append('movieId', movieId);
}

console.log(`Fetching comments from: ${API_BASE_URL}/api/movies/aggregations/reportingByComments?${queryParams}`);
Expand Down
1 change: 1 addition & 0 deletions client/app/lib/constants.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ export const APP_CONFIG = {
description: 'Browse movies from the sample MFlix database',
defaultMovieLimit: 20,
maxMovieLimit: 100,
vectorSearchPageSize: 20, // Fixed page size for vector search results display
imageFormats: ['image/avif', 'image/webp'],
} as const;

Expand Down
120 changes: 64 additions & 56 deletions client/app/movies/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,16 +239,18 @@ export default function Movies() {
let result;

if (searchParams.searchType === 'vector-search') {
// Vector Search: Fetch all results and implement client-side pagination
// Vector Search: Use the limit from search params as the fetch limit
const vectorSearchParams = {
q: searchParams.q!,
limit: searchParams.limit || 50, // Get more results for better pagination experience
limit: searchParams.limit || 50, // This is how many results to fetch from backend
};

result = await vectorSearchMovies(vectorSearchParams);

if (result.success) {
const allResults = result.movies || [];
const pageSize = searchParams.limit || 20;
const pageSize = APP_CONFIG.vectorSearchPageSize; // Fixed page size for UI display

setAllVectorSearchResults(allResults);
setVectorSearchPage(1);
setVectorSearchPageSize(pageSize);
Expand All @@ -257,10 +259,7 @@ export default function Movies() {
const firstPageResults = allResults.slice(0, pageSize);
setSearchResults(firstPageResults);

// Set pagination state based on total results
const totalPages = Math.ceil(allResults.length / pageSize);
setSearchHasNextPage(totalPages > 1);
setSearchHasPrevPage(false);
// Set pagination state for vector search
setSearchTotalCount(allResults.length);
}
} else {
Expand Down Expand Up @@ -352,15 +351,19 @@ export default function Movies() {
return { paginatedResults: [], hasNext: false, hasPrev: false, totalPages: 0 };
}

const totalResults = allVectorSearchResults.length;
const totalPages = Math.ceil(totalResults / vectorSearchPageSize);
const hasNext = vectorSearchPage < totalPages;
const hasPrev = vectorSearchPage > 1;

const startIndex = (vectorSearchPage - 1) * vectorSearchPageSize;
const endIndex = startIndex + vectorSearchPageSize;
const paginatedResults = allVectorSearchResults.slice(startIndex, endIndex);
const totalPages = Math.ceil(allVectorSearchResults.length / vectorSearchPageSize);

return {
paginatedResults,
hasNext: vectorSearchPage < totalPages,
hasPrev: vectorSearchPage > 1,
hasNext,
hasPrev,
totalPages
};
};
Expand Down Expand Up @@ -653,55 +656,60 @@ export default function Movies() {
/* Vector search results with client-side pagination */
(() => {
const { hasNext, hasPrev, totalPages } = getVectorSearchPageData();
return totalPages > 1 ? (
<nav className={movieStyles.pagination} aria-label="Vector search results pagination">
<div className={movieStyles.paginationContainer}>
{/* Previous Button */}
{hasPrev ? (
<button
onClick={() => handleVectorSearchPageChange(vectorSearchPage - 1)}
className={movieStyles.pageButton}
aria-label="Go to previous page"
>
← Previous
</button>
) : (
<span className={`${movieStyles.pageButton} ${movieStyles.disabled}`}>
← Previous
</span>
)}

{/* Current Page Info */}
<div className={movieStyles.pageInfo}>
Page {vectorSearchPage} of {totalPages}

if (totalPages > 1) {
return (
<nav className={movieStyles.pagination} aria-label="Vector search results pagination">
<div className={movieStyles.paginationContainer}>
{/* Previous Button */}
{hasPrev ? (
<button
onClick={() => handleVectorSearchPageChange(vectorSearchPage - 1)}
className={movieStyles.pageButton}
aria-label="Go to previous page"
>
← Previous
</button>
) : (
<span className={`${movieStyles.pageButton} ${movieStyles.disabled}`}>
← Previous
</span>
)}

{/* Current Page Info */}
<div className={movieStyles.pageInfo}>
Page {vectorSearchPage} of {totalPages}
</div>

{/* Next Button */}
{hasNext ? (
<button
onClick={() => handleVectorSearchPageChange(vectorSearchPage + 1)}
className={movieStyles.pageButton}
aria-label="Go to next page"
>
Next →
</button>
) : (
<span className={`${movieStyles.pageButton} ${movieStyles.disabled}`}>
Next →
</span>
)}
</div>

{/* Next Button */}
{hasNext ? (
<button
onClick={() => handleVectorSearchPageChange(vectorSearchPage + 1)}
className={movieStyles.pageButton}
aria-label="Go to next page"
>
Next →
</button>
) : (
<span className={`${movieStyles.pageButton} ${movieStyles.disabled}`}>
Next →
</span>
)}
</div>

{/* Additional Info */}
<div className={movieStyles.additionalInfo}>
{vectorSearchPageSize} movies per page • {allVectorSearchResults.length} total results
{/* Additional Info */}
<div className={movieStyles.additionalInfo}>
{vectorSearchPageSize} movies per page • {allVectorSearchResults.length} total results
</div>
</nav>
);
} else {
return (
<div className={movieStyles.searchInfo}>
Showing {allVectorSearchResults.length} results (vector search)
</div>
</nav>
) : (
<div className={movieStyles.searchInfo}>
Showing {allVectorSearchResults.length} results (vector search)
</div>
);
);
}
})()
)
) : (
Expand Down
2 changes: 1 addition & 1 deletion client/tsconfig.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"moduleResolution": "bundler",
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "react-jsx",
"jsx": "preserve",
Copy link
Collaborator

@dacharyc dacharyc Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I run the client, it's automatically switching this value back to "react-jsx" with this message:

   We detected TypeScript in your project and reconfigured your tsconfig.json file for you.
   The following mandatory changes were made to your tsconfig.json:

        - jsx was set to react-jsx (next.js uses the React automatic runtime)

Curious why you're changing it here, and if we need it to be "preserve", what we might need to change in the client to force it to respect this config field?

Copy link
Collaborator Author

@jordan-smith721 jordan-smith721 Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's interesting, every time I run it it's been automatically changing it to preserve. I'm not sure what's causing the change but I'll look into it. For now I'll revert to react-jsx since that's what I originally had in there before it started changing it

Copy link
Collaborator Author

@jordan-smith721 jordan-smith721 Nov 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interestingly it's almost the same message too:

We detected TypeScript in your project and reconfigured your tsconfig.json file for you.
The following mandatory changes were made to your tsconfig.json:

     - jsx was set to preserve (next.js implements its own optimized jsx transform)

Copy link
Collaborator Author

@jordan-smith721 jordan-smith721 Nov 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh I see, it's a versioning issue. From Claude:

The Problem
The issue occurs because Next.js automatically modifies your tsconfig.json file when it starts up. Different versions of Next.js have different preferences for the jsx setting:

Your current setup: jsx: "preserve" (which Next.js 15+ prefers)
Others' setup: jsx: "react-jsx" (which older versions or different configurations prefer)
Why This Happens
Version Differences: Different Next.js versions have different default TSConfig preferences
Auto-Configuration: Next.js automatically "fixes" what it thinks are incorrect settings
Lock File Issues: Different package-lock.json or yarn.lock states can cause different dependency resolutions

Since this is using the latest version of Next, I think we should actually stick with preserve since later versions prefer it

Copy link
Collaborator

@dacharyc dacharyc Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That doesn't track with the usage, though. I ran npm install on the client in the last few days so it's pulling in the most recent version of Next.js. (Actually I checked and I've got 16.0.0 and it looks like Next released 16.0.1 recently.)

It may be a versioning difference issue, but that doesn't really hold up if we're both on the latest version of Next. So I think it's more likely it's some sort of configuration difference, or maybe a Node or other version issue, and we should probably try to get to the bottom of this as a fast follow.

Copy link
Collaborator Author

@jordan-smith721 jordan-smith721 Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. Okay yeah I'll add it to the list of fast follows and try and get to the bottom of what's going on here

"incremental": true,
"plugins": [
{
Expand Down
5 changes: 4 additions & 1 deletion server/express/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,10 @@
"dev": "ts-node src/app.ts",
"test": "jest",
"test:watch": "jest --watch",
"test:coverage": "jest --coverage"
"test:coverage": "jest --coverage",
"test:unit": "jest tests/controllers",
"test:verbose": "jest --verbose",
"test:silent": "jest --silent"
},
"dependencies": {
"cors": "^2.8.5",
Expand Down
Loading
Loading