Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update DB ingestion for Frames #1522

Open
Tracked by #1508
manasaV3 opened this issue Jan 24, 2025 · 0 comments
Open
Tracked by #1508

Update DB ingestion for Frames #1522

manasaV3 opened this issue Jan 24, 2025 · 0 comments
Assignees
Labels
backend P0 Must have

Comments

@manasaV3
Copy link
Contributor

manasaV3 commented Jan 24, 2025

Blocked by: #1055

Task

This import step should be executed before the import of PerSectionParameter to ensure all the records are created for related frames.

This import step currently finds the files to populate the db entry. We should instead use the frames_metadata.json for this.

DB schema

Name Status Data type Nullable Defaults
acquisition_order   int false  
dose Delete float false  
accumulated_dose Add float false  
exposure_dose Add float false  
id   int false Auto increment
is_gain_corrected   bool true  
https_frame_path   string true  
raw_angle Delete float false  
s3_frame_path   string true  

Metadata mapping

Name source
acquisition_order  metadata: frames[*]. acquisition_order
accumulated_dose metadata: frames[*]. accumulated_dose
exposure_dose metadata: frames[*]. exposure_dose
is_gain_corrected metadata: is_gain_corrected
https_frame_path http path generation from frames[*].path
s3_frame_path s3 path generation from frames[*].path

What done looks like?

The api has been updated to reflect the frames data, and data can be populated for all the frames field.

Notes

  1. https://github.com/chanzuckerberg/cryoet-data-portal-backend/tree/main/apiv2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend P0 Must have
Projects
None yet
Development

No branches or pull requests

2 participants