Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark metrics to simulator #823

Open
wants to merge 38 commits into
base: og-develop
Choose a base branch
from
Open
Changes from 2 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
457a8be
add metrics to simulator
steven200796 Jul 5, 2023
1910c1b
build-push action for docker
steven200796 Jul 19, 2023
374f842
merged with og-develop
Jul 29, 2024
b536a4c
action primitives initial commit
Jul 30, 2024
067cded
Delete .github/workflows/ci.yml
cgokmen Jul 30, 2024
a14f99d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 30, 2024
b58f38c
Delete scripts/download_datasets.py
cgokmen Jul 30, 2024
30de844
metric refactor
Jul 30, 2024
e32ebb4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 30, 2024
a15102a
updated energy metric comment
Jul 30, 2024
fb3402f
updated energy metric comment
Jul 30, 2024
420edd2
setting metric to 0
Jul 30, 2024
ba7e992
metrics change
Jul 31, 2024
ae74fc5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 31, 2024
328e772
fixed base robot fix
Aug 2, 2024
aa2f4c1
Merge branch 'metrics' of https://github.com/StanfordVL/OmniGibson in…
Aug 2, 2024
0b3b1d1
object initialization bugfix
Aug 5, 2024
628490b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 5, 2024
dbec742
metric tests for work added
Aug 6, 2024
f2a9997
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 6, 2024
d267a51
pending for _dummy
Aug 12, 2024
75aa0ed
pending for _dummy
Aug 12, 2024
b6fc0ea
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 12, 2024
b20bc4a
metric refactoring
Aug 13, 2024
f018043
refactoring
Aug 13, 2024
a60f36c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 13, 2024
c7eed0e
refactoring
Aug 13, 2024
4773e07
work energy metric
Aug 13, 2024
88c2105
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 13, 2024
a9ef20e
corrected rotational inertia calculation
Aug 15, 2024
efd3353
rotational work fix
Aug 15, 2024
9f51314
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 15, 2024
85dabe9
metric work energy finished
Aug 16, 2024
5ecf8cf
work energy metric complete
Aug 16, 2024
0227d71
metric functions
Sep 10, 2024
539da44
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 10, 2024
1446b9b
styling fix
Sep 10, 2024
7806a71
Merge branch 'metrics' of https://github.com/StanfordVL/OmniGibson in…
Sep 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion omnigibson/tasks/behavior_task.py
Original file line number Diff line number Diff line change
@@ -596,5 +596,5 @@ def default_metric_config(cls):
"work": 0.2,
"energy": 0.2,
"rotation": 1,
"translation": 1
"translation": 1,
}
7 changes: 3 additions & 4 deletions omnigibson/tasks/task_base.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import time
from abc import ABCMeta, abstractmethod
from copy import deepcopy

import time
import numpy as np

from omnigibson.utils.gym_utils import GymObservable
@@ -441,7 +441,7 @@ def info(self):
"""
assert self._info is not None, "At least one step() must occur before info can be calculated!"
return self._info

@property
def last_step_wall_time(self):
"""
@@ -452,13 +452,12 @@ def last_step_wall_time(self):
# return 0 if the simulation has not started yet
if not self._prev_sim_end_ts or not self._cur_sim_start_ts:
return 0

assert (
self._prev_sim_end_ts < self._cur_sim_start_ts
), "end time from the previous iteration must be less than the start time of the current iteration"
return self._cur_sim_start_ts - self._prev_sim_end_ts


@classproperty
def valid_scene_types(cls):
"""
3 changes: 0 additions & 3 deletions tests/test_metric_functions.py
Original file line number Diff line number Diff line change
@@ -151,6 +151,3 @@ def test_behavior_task_object_addition_removal():
assert np.allclose((add_energy - remove_energy) / robot_mass, 0, atol=1e-1)

env.reset()

if __name__ == "__main__":
test_behavior_task_work_metric()