Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a hubTemplate section that describes how to access hub data from S3 #22

Closed
1 task done
bsweger opened this issue Jan 28, 2025 · 5 comments · Fixed by #23
Closed
1 task done

Add a hubTemplate section that describes how to access hub data from S3 #22

bsweger opened this issue Jan 28, 2025 · 5 comments · Fixed by #23
Assignees
Labels
documentation Improvements or additions to documentation higher priority work to prioritize in the near future

Comments

@bsweger
Copy link
Contributor

bsweger commented Jan 28, 2025

Background

@nickreich noted on Slack that Hubverse hubs that sync their data to AWS:

  • don't indicate anywhere that their model-output and target data are available outside the hub repo, in public S3 buckets
  • don't have any documentation about how to access the files on S3

The Hubverse should do a better job of advertising data available on S3.

Definition of done

  • README.md in hubTemplate has a section about data access

Once we agree on the wording and location for this info in the above templates, the follow-up work will be submitting PRs to existing hubs with data on S3.

@bsweger bsweger added documentation Improvements or additions to documentation higher priority work to prioritize in the near future labels Jan 28, 2025
@bsweger bsweger moved this from Todo to Up Next in hubverse Development overview Feb 4, 2025
@bsweger
Copy link
Contributor Author

bsweger commented Feb 4, 2025

I'm wondering how best to organize the cloud documention, given that we have a related open issue about expanding cloud documentation for hub admins: hubverse-org/hubDocs#92

Option 1

Add a separate Cloud section to hubDocs, which would document all aspects of Hubverse-hosted clouds. Something like:

hubverse.io/
├─ About/
├─ Quickstart - hub administration/
├─ User Guide/
├─ Developer Guide/
├─ Hubverse Cloud Hosting/
│  ├─ overview
│  ├─ onboarding to cloud 
│  ├─ accessing cloud data
│  ├─ developer-related cloud stuff
├─ Code of Conduct/
  • Consolidates cloud topic into a single place so it doesn't clutter the "core" hubverse documetnation
  • Deviates from our current pattern of organizing topics by persona

Option 2

Add cloud topics to existing documentation structure, e.g.,

hubverse.io/
├─ About/
├─ Quickstart - hub administration/
│  ├─ onboarding to cloud 
├─ User Guide/
│  ├─ accessing cloud data
├─ Developer Guide/
│  ├─ developer-related cloud stuff
├─ Code of Conduct/
  • Retains existing pattern of organizing docs by persona
  • Cloud-hosted hubs can be conceptually hard to grok, and current "core" hub documentation is already pretty dense

@micokoch
Copy link

micokoch commented Feb 4, 2025

I like option two because it integrates cloud hosting into the other topics so that one can decide whether to pursue the cloud path in any given approach.

@annakrystalli
Copy link
Member

Also I think accessing cloud data should fall under a more general topic of accessing data. The mechanics are the same between local and cloud, you just point to a different location

@bsweger
Copy link
Contributor Author

bsweger commented Feb 10, 2025

Thanks for weighing in! Per discussion at this morning's discussion, I'm moving this out of "in progress" state, pending the reorganization of the Hubverse website.

Wrong issue! For this issue, we decided to limit the scope of the data access documentation to the READMEs of individual hubs and hub dashboards (starting with the templates, and then submitting PRs to hubs that are hosted on the cloud).

This will help surface the data access information where people are looking for it (we can add to the revised Hubverse website when the time comes).

@bsweger bsweger moved this from Up Next to Todo in hubverse Development overview Feb 10, 2025
@bsweger bsweger moved this from Todo to Up Next in hubverse Development overview Feb 10, 2025
@bsweger bsweger moved this from Up Next to In Progress in hubverse Development overview Feb 10, 2025
@bsweger bsweger self-assigned this Feb 10, 2025
@bsweger bsweger changed the title Add a hubDoc section that describes how to access hub data from S3 Add a hubTemplate section that describes how to access hub data from S3 Feb 10, 2025
@bsweger bsweger transferred this issue from hubverse-org/hubDocs Feb 10, 2025
@bsweger bsweger moved this from In Progress to Ready for Review in hubverse Development overview Feb 12, 2025
@bsweger bsweger moved this from Ready for Review to Reviewed/Ready to Merge in hubverse Development overview Feb 12, 2025
@bsweger bsweger moved this from Reviewed/Ready to Merge to Ready for Review in hubverse Development overview Feb 12, 2025
@bsweger
Copy link
Contributor Author

bsweger commented Feb 12, 2025

Moved the dashboard-template part of this work to its own issue: hubverse-org/hub-dashboard-template#4

@github-project-automation github-project-automation bot moved this from Ready for Review to Done in hubverse Development overview Feb 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation higher priority work to prioritize in the near future
Projects
Development

Successfully merging a pull request may close this issue.

3 participants