Trajectory Generation for SWE Smith

[wyettzeng]

Describe the issue

Hello, so I was able to use mini-swe-agent to generate trajectories with OpenAI Codex models. However, when I try to evaluate it with the swe-smith eval harness, all of them errored out when trying to apply patches. Could it be that when trying to generate trajectories, mini-swe-agent did not properly initialize the containers?

---

Check (logs/run_evaluation/ram/oauthlib__oauthlib.1fd52536.combine_file__09vlzwgc/run_instance.log) for more information.
Error validating oauthlib__oauthlib.1fd52536.combine_file__0hkl0pea: oauthlib__oauthlib.1fd52536.combine_file__0hkl0pea: >>>>> Patch Apply Failed:
patching file oauthlib/openid/connect/core/grant_types/base.py
Hunk #1 FAILED at 20.
1 out of 1 hunk FAILED -- saving rejects to file oauthlib/openid/connect/core/grant_types/base.py.rej

Error validating pandas-dev__pandas.95280573.combine_file__8xdtc4hn: pandas-dev__pandas.95280573.combine_file__8xdtc4hn: >>>>> Patch Apply Failed:
patching file pandas/core/methods/describe.py
Reversed (or previously applied) patch detected! Assuming -R.
Hunk #4 FAILED at 215.
Hunk #5 FAILED at 288.
2 out of 5 hunks FAILED -- saving rejects to file pandas/core/methods/describe.py.rej

I am using all official commands (note that I have to use swesmith harness as the swebench harness does not natively support swe smith dataset):
python swesmith/harness/eval.py
--predictions_path /n/home08/hzeng/mini-swe-agent/runs/smith_oai_10/preds.json
--run_id ram
--workers 5

mini-extra swebench
--subset smith
--split train
--slice 0:5
--workers 5
--environment-class singularity
--output "$RUN_DIR"