Skip to content

Update the library installation script for EC2 machines to install optional dependencies like RealTabFormer #388

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
R-Palazzo opened this issue Mar 21, 2025 · 0 comments · May be fixed by #389
Open

Update the library installation script for EC2 machines to install optional dependencies like RealTabFormer #388

R-Palazzo opened this issue Mar 21, 2025 · 0 comments · May be fixed by #389
Assignees
@R-Palazzo
Labels
feature request Request for a new feature internal The issue doesn't change the API or functionality maintenance Tasks related to infrastructure & dependencies
Milestone
0.10.1

Comments

@R-Palazzo
Copy link
Contributor

R-Palazzo commented Mar 21, 2025 •
edited by amontanez24
Loading

Problem Description

In our goal of including more synthesizers and datasets in our sdgym monthly run, we should ensure that any new synthesizer can be run on EC2 machines. Currently, this is not the case since new synthesizers are optional dependencies of the library that we do not install on the EC2 machines.

Expected behavior

Update the installation script on EC2 machines to check for any optional dependencies needed to run the external synthesizers and install the appropriate sdgym accordingly.

Additional context

Below is the code that we run on the ec2 instance. We can update it to install the optional dependencies as well.

SDGym/sdgym/benchmark.py

Lines 674 to 692 in 4032757

user_data_script = f"""#!/bin/bash
sudo apt update -y
sudo apt install python3-pip -y
echo "======== Install Dependencies ============"
sudo pip3 install sdgym
sudo pip3 install anyio
pip3 list
sudo apt install awscli -y
aws configure set aws_access_key_id {credentials.access_key}
aws configure set aws_secret_access_key {credentials.secret_key}
aws configure set region {session.region_name}
echo "======== Write Script ==========="
sudo touch ~/sdgym_script.py
echo "{script_content}" > ~/sdgym_script.py
echo "======== Run Script ==========="
sudo python3 ~/sdgym_script.py
echo "======== Complete ==========="
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
aws ec2 terminate-instances --instance-ids $INSTANCE_ID

We could either always install the optional dependencies, or pass an argument with the synthesizers that require optional dependencies to be installed, and only install them if the argument is provided.
@R-Palazzo R-Palazzo added feature request Request for a new feature internal The issue doesn't change the API or functionality maintenance Tasks related to infrastructure & dependencies labels Mar 21, 2025
@amontanez24 amontanez24 changed the title Update the library installation script for EC2 mahcines to install optional dependencies like RealTabFormer Update the library installation script for EC2 machines to install optional dependencies like RealTabFormer Mar 21, 2025
@R-Palazzo R-Palazzo self-assigned this Apr 4, 2025
@R-Palazzo R-Palazzo added this to the 0.10.1 milestone Apr 4, 2025
@R-Palazzo R-Palazzo linked a pull request Apr 4, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@R-Palazzo R-Palazzo

Labels
feature request Request for a new feature internal The issue doesn't change the API or functionality maintenance Tasks related to infrastructure & dependencies
Projects
None yet
Milestone
0.10.1
1 participant
@R-Palazzo