Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable upload of experiment configs #406

Open
1 of 2 tasks
cjqian opened this issue Jan 16, 2025 · 5 comments
Open
1 of 2 tasks

Enable upload of experiment configs #406

cjqian opened this issue Jan 16, 2025 · 5 comments
Assignees

Comments

@cjqian
Copy link
Member

cjqian commented Jan 16, 2025

CUJs:

  • Experimenters should be able to download an experiment from a .json file (Already implemented)
  • Experimenters should be able to load an experiment from a .json file (Upload button). This should be the reverse of the download in that the parsed JSON becomes ExperimentDownload object. Then, the experiment config from that object can be loaded into the experiment editor
@vivtsai
Copy link
Collaborator

vivtsai commented Jan 16, 2025

We already have download experiment JSON (https://github.com/PAIR-code/deliberate-lab/blob/main/utils/src/data.ts#L25), so we just need an upload button!

@vivtsai vivtsai added this to the ❄️ January 2025 milestone Jan 16, 2025
@vivtsai vivtsai changed the title Enable upload / download of experiment stages Enable upload of experiment configs Jan 16, 2025
@vivtsai
Copy link
Collaborator

vivtsai commented Jan 16, 2025

Migrate lost_at_sea.ts and chip_negotiation.ts to .json. This will simplify working with experimenters on iterating on these in the future: They edit in UI, download, push it to the repository -> Now the game stage is updated for everyone.

@cjqian instead of doing this, what do you think about replacing games with ExperimentTemplates? This is what is supposed to be used for experimenters uploading/updating their own experiment configs that can be reused. We initially had "games" because they had features that couldn't be edited in UI,* but once we fix that (which we will need to if we want experimenters to be able to config via UI anyway), we don't really need games.

*We have a backlog of adding experiment editor UI support for chat threads, chip negotiation, and maybe some tiny things - so if we want this, we'll have to do those first.

@cjqian
Copy link
Member Author

cjqian commented Jan 17, 2025

What’s the difference between “games” and “experiment templates”?

@vivtsai
Copy link
Collaborator

vivtsai commented Jan 17, 2025

Games are supposed to be a temporary workaround (hardcoded and configurable only by us. For example, the current negotiation game must be hardcoded and can't be edited via the UI because we didn't build the negotiation stage editor yet).

"Experiment templates" are for experimenters to configure in the UI (notably, they are stored in firebase) and otherwise work like games (in that you select them and they are loaded into the editor for you to optionally edit + save.

We already have backend support experiment templates (set up when the platform was set up), so it sounds like this is a good point to move over (and if we have no more hardcoded examples, then we can just get rid of games - which would also prevent them from being publicly visible in the repo).

I'm going to move this bug into a new "Add experiment templates" milestone to help illustrate what's involved here (all of this would be blocking the ability for experimenters to edit current games anyway). Let me know what you think - we can always cancel the milestone and/or adjust scope?

In the meantime, let's consider this bug scoped to "upload button that loads an experiment json correctly into the experiment editor."

@dimits-ts fyi

@cjqian
Copy link
Member Author

cjqian commented Jan 17, 2025

Great, thanks for the clarification!

@vivtsai vivtsai self-assigned this Feb 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants