Skip to content

Commit cd2cece

Browse files
authored
refactor(benchmarks): evaluation pipeline (#71)
1 parent 90b0e66 commit cd2cece

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

65 files changed

+2784
-1600
lines changed

.gitignore

+3-3
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ licenses.txt
5555
**/dist/
5656
**/checkpoints/
5757
**/outputs/
58+
**/multirun/
5859

5960
# Other env files
6061
.python-version
@@ -74,7 +75,6 @@ coverage.xml
7475

7576
# dotenv
7677
.env
77-
src/dbally_benchmark/.env
7878

7979
# coverage and pytest reports
8080
coverage.xml
@@ -87,8 +87,8 @@ cmake-build-*/
8787
**/.terraform.lock.hcl
8888
**/.terraform
8989

90-
# experiments results
91-
experiments/
90+
# benchmarks
91+
benchmarks/sql/data/
9292

9393
# mkdocs generated files
9494
site/

benchmark/dbally_benchmark/config.py

-22
This file was deleted.

benchmark/dbally_benchmark/constants.py

-27
This file was deleted.

benchmark/dbally_benchmark/dataset/bird_dataset.py

-63
This file was deleted.

benchmark/dbally_benchmark/e2e_benchmark.py

-157
This file was deleted.

benchmark/dbally_benchmark/evaluate.py

-41
This file was deleted.

benchmark/dbally_benchmark/experiment_config/config.yaml

-10
This file was deleted.

benchmark/dbally_benchmark/experiment_config/e2e/superhero.yaml

-6
This file was deleted.

benchmark/dbally_benchmark/experiment_config/iql/superhero.yaml

-6
This file was deleted.

benchmark/dbally_benchmark/experiment_config/text2sql/superhero.yaml

-5
This file was deleted.

benchmark/dbally_benchmark/iql/iql_result.py

-13
This file was deleted.

0 commit comments

Comments
 (0)