Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Add EnvBase.all_actions and impl for ChessEnv #2780

Open
wants to merge 1 commit into
base: gh/kurtamohler/3/base
Choose a base branch
from
Open

[Feature] Add EnvBase.all_actions and impl for ChessEnv #2780

wants to merge 1 commit into from

Conversation

kurtamohler
Copy link
Collaborator

@kurtamohler kurtamohler commented Feb 12, 2025 •
edited
Loading

@kurtamohler
Update
1ac847d 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 12, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2780

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 2 Unrelated Failures

As of commit 1ac847d with merge base f5445a4 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

kurtamohler added a commit that referenced this pull request Feb 12, 2025
@kurtamohler
[Feature] Add EnvBase.all_actions and impl for ChessEnv
29b4dbc 
ghstack-source-id: 9ee21835f35437e856b0726019114eb81a1115bc
Pull Request resolved: #2780
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 12, 2025
@kurtamohler kurtamohler requested a review from vmoens February 12, 2025 00:56
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}15$. Worsened: $\large\color{#d91a1a}13$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.5762s 0.4912s 2.0360 Ops/s 2.0137 Ops/s $\color{#35bf28}+1.11\%$
test_transformed 1.0193s 0.9392s 1.0647 Ops/s 1.0158 Ops/s $\color{#35bf28}+4.82\%$
test_serial 1.5763s 1.4779s 0.6766 Ops/s 0.6590 Ops/s $\color{#35bf28}+2.68\%$
test_parallel 1.3847s 1.2829s 0.7795 Ops/s 0.7756 Ops/s $\color{#35bf28}+0.49\%$
test_step_mdp_speed[True-True-True-True-True] 0.1666ms 30.6270μs 32.6509 KOps/s 33.0305 KOps/s $\color{#d91a1a}-1.15\%$
test_step_mdp_speed[True-True-True-True-False] 57.8290μs 17.9104μs 55.8336 KOps/s 56.5026 KOps/s $\color{#d91a1a}-1.18\%$
test_step_mdp_speed[True-True-True-False-True] 59.9430μs 17.1763μs 58.2196 KOps/s 57.7037 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[True-True-True-False-False] 54.7330μs 10.0563μs 99.4402 KOps/s 99.9279 KOps/s $\color{#d91a1a}-0.49\%$
test_step_mdp_speed[True-True-False-True-True] 0.6570ms 32.4685μs 30.7991 KOps/s 30.7611 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[True-True-False-True-False] 78.6880μs 19.7293μs 50.6860 KOps/s 50.6498 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[True-True-False-False-True] 48.8220μs 19.0901μs 52.3832 KOps/s 45.9728 KOps/s $\textbf{\color{#35bf28}+13.94\%}$
test_step_mdp_speed[True-True-False-False-False] 55.6940μs 11.8914μs 84.0944 KOps/s 84.8159 KOps/s $\color{#d91a1a}-0.85\%$
test_step_mdp_speed[True-False-True-True-True] 77.6960μs 34.4922μs 28.9921 KOps/s 29.2400 KOps/s $\color{#d91a1a}-0.85\%$
test_step_mdp_speed[True-False-True-True-False] 64.3210μs 21.5667μs 46.3677 KOps/s 46.5068 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-False-True-False-True] 68.9500μs 19.0213μs 52.5726 KOps/s 52.1537 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[True-False-True-False-False] 58.0890μs 11.8397μs 84.4616 KOps/s 85.4414 KOps/s $\color{#d91a1a}-1.15\%$
test_step_mdp_speed[True-False-False-True-True] 0.1055ms 36.4325μs 27.4481 KOps/s 27.8647 KOps/s $\color{#d91a1a}-1.50\%$
test_step_mdp_speed[True-False-False-True-False] 64.6520μs 23.5651μs 42.4356 KOps/s 42.9809 KOps/s $\color{#d91a1a}-1.27\%$
test_step_mdp_speed[True-False-False-False-True] 70.6230μs 20.7943μs 48.0901 KOps/s 47.7300 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[True-False-False-False-False] 57.4880μs 13.6863μs 73.0659 KOps/s 73.8996 KOps/s $\color{#d91a1a}-1.13\%$
test_step_mdp_speed[False-True-True-True-True] 98.0340μs 34.3325μs 29.1269 KOps/s 29.1907 KOps/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[False-True-True-True-False] 72.9070μs 21.6584μs 46.1715 KOps/s 46.5724 KOps/s $\color{#d91a1a}-0.86\%$
test_step_mdp_speed[False-True-True-False-True] 66.9160μs 21.7247μs 46.0306 KOps/s 46.0369 KOps/s $\color{#d91a1a}-0.01\%$
test_step_mdp_speed[False-True-True-False-False] 76.1260μs 13.3074μs 75.1464 KOps/s 75.4490 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-True-False-True-True] 84.4490μs 36.2395μs 27.5942 KOps/s 27.7466 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[False-True-False-True-False] 0.5401ms 23.4729μs 42.6023 KOps/s 42.8913 KOps/s $\color{#d91a1a}-0.67\%$
test_step_mdp_speed[False-True-False-False-True] 2.8159ms 23.7940μs 42.0274 KOps/s 42.3996 KOps/s $\color{#d91a1a}-0.88\%$
test_step_mdp_speed[False-True-False-False-False] 58.5100μs 14.9858μs 66.7299 KOps/s 66.3268 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[False-False-True-True-True] 79.8100μs 38.4323μs 26.0198 KOps/s 26.2700 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-False-True-True-False] 66.4250μs 25.5333μs 39.1646 KOps/s 39.0688 KOps/s $\color{#35bf28}+0.25\%$
test_step_mdp_speed[False-False-True-False-True] 69.7210μs 23.6856μs 42.2198 KOps/s 41.8864 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-False-True-False-False] 56.7570μs 15.0431μs 66.4758 KOps/s 66.1436 KOps/s $\color{#35bf28}+0.50\%$
test_step_mdp_speed[False-False-False-True-True] 0.2831ms 39.8102μs 25.1192 KOps/s 25.1674 KOps/s $\color{#d91a1a}-0.19\%$
test_step_mdp_speed[False-False-False-True-False] 71.8750μs 26.9289μs 37.1348 KOps/s 36.9058 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[False-False-False-False-True] 54.7030μs 25.0926μs 39.8524 KOps/s 39.8413 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[False-False-False-False-False] 50.6850μs 16.6109μs 60.2014 KOps/s 60.5929 KOps/s $\color{#d91a1a}-0.65\%$
test_values[generalized_advantage_estimate-True-True] 10.3449ms 9.9651ms 100.3500 Ops/s 99.2101 Ops/s $\color{#35bf28}+1.15\%$
test_values[vec_generalized_advantage_estimate-True-True] 28.4594ms 26.7056ms 37.4453 Ops/s 41.0073 Ops/s $\textbf{\color{#d91a1a}-8.69\%}$
test_values[td0_return_estimate-False-False] 0.2372ms 0.1945ms 5.1409 KOps/s 5.1689 KOps/s $\color{#d91a1a}-0.54\%$
test_values[td1_return_estimate-False-False] 28.2084ms 24.7464ms 40.4100 Ops/s 41.3140 Ops/s $\color{#d91a1a}-2.19\%$
test_values[vec_td1_return_estimate-False-False] 33.8582ms 26.6471ms 37.5276 Ops/s 40.7824 Ops/s $\textbf{\color{#d91a1a}-7.98\%}$
test_values[td_lambda_return_estimate-True-False] 35.9359ms 35.3364ms 28.2995 Ops/s 28.9359 Ops/s $\color{#d91a1a}-2.20\%$
test_values[vec_td_lambda_return_estimate-True-False] 27.7515ms 26.0903ms 38.3285 Ops/s 41.2308 Ops/s $\textbf{\color{#d91a1a}-7.04\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.6207ms 8.4797ms 117.9289 Ops/s 118.6926 Ops/s $\color{#d91a1a}-0.64\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.4081ms 1.9244ms 519.6498 Ops/s 529.7084 Ops/s $\color{#d91a1a}-1.90\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6000ms 0.3637ms 2.7496 KOps/s 2.7365 KOps/s $\color{#35bf28}+0.48\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.8732ms 45.3528ms 22.0493 Ops/s 23.5379 Ops/s $\textbf{\color{#d91a1a}-6.32\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.8318ms 3.4656ms 288.5537 Ops/s 288.3371 Ops/s $\color{#35bf28}+0.08\%$
test_dqn_speed[False-None] 5.4417ms 1.4032ms 712.6666 Ops/s 701.9365 Ops/s $\color{#35bf28}+1.53\%$
test_dqn_speed[False-backward] 1.9917ms 1.8899ms 529.1276 Ops/s 505.0689 Ops/s $\color{#35bf28}+4.76\%$
test_dqn_speed[True-None] 0.6347ms 0.4870ms 2.0534 KOps/s 2.0288 KOps/s $\color{#35bf28}+1.22\%$
test_dqn_speed[True-backward] 1.0075ms 0.9229ms 1.0835 KOps/s 1.0835 KOps/s $-0.00\%$
test_dqn_speed[reduce-overhead-None] 0.7186ms 0.4889ms 2.0454 KOps/s 2.0442 KOps/s $\color{#35bf28}+0.06\%$
test_dqn_speed[reduce-overhead-backward] 1.5877ms 1.1534ms 867.0310 Ops/s 1.0957 KOps/s $\textbf{\color{#d91a1a}-20.87\%}$
test_ddpg_speed[False-None] 3.3250ms 2.8985ms 345.0073 Ops/s 347.4655 Ops/s $\color{#d91a1a}-0.71\%$
test_ddpg_speed[False-backward] 4.1948ms 4.0877ms 244.6371 Ops/s 247.7810 Ops/s $\color{#d91a1a}-1.27\%$
test_ddpg_speed[True-None] 1.5797ms 1.2433ms 804.2886 Ops/s 812.5290 Ops/s $\color{#d91a1a}-1.01\%$
test_ddpg_speed[True-backward] 2.2779ms 2.1514ms 464.8155 Ops/s 448.3389 Ops/s $\color{#35bf28}+3.68\%$
test_ddpg_speed[reduce-overhead-None] 1.4933ms 1.2390ms 807.0753 Ops/s 806.8352 Ops/s $\color{#35bf28}+0.03\%$
test_ddpg_speed[reduce-overhead-backward] 2.2059ms 2.1378ms 467.7691 Ops/s 465.5418 Ops/s $\color{#35bf28}+0.48\%$
test_sac_speed[False-None] 8.4901ms 7.9699ms 125.4726 Ops/s 122.3787 Ops/s $\color{#35bf28}+2.53\%$
test_sac_speed[False-backward] 12.7658ms 10.9091ms 91.6665 Ops/s 90.9161 Ops/s $\color{#35bf28}+0.83\%$
test_sac_speed[True-None] 2.6953ms 2.1206ms 471.5746 Ops/s 463.0098 Ops/s $\color{#35bf28}+1.85\%$
test_sac_speed[True-backward] 4.0412ms 3.9452ms 253.4733 Ops/s 263.7067 Ops/s $\color{#d91a1a}-3.88\%$
test_sac_speed[reduce-overhead-None] 5.7214ms 2.2154ms 451.3786 Ops/s 463.7547 Ops/s $\color{#d91a1a}-2.67\%$
test_sac_speed[reduce-overhead-backward] 3.8805ms 3.7920ms 263.7148 Ops/s 257.9136 Ops/s $\color{#35bf28}+2.25\%$
test_redq_speed[False-None] 20.0264ms 13.8159ms 72.3806 Ops/s 52.2559 Ops/s $\textbf{\color{#35bf28}+38.51\%}$
test_redq_speed[False-backward] 24.2395ms 22.9916ms 43.4942 Ops/s 43.8890 Ops/s $\color{#d91a1a}-0.90\%$
test_redq_speed[True-None] 7.8156ms 5.2906ms 189.0155 Ops/s 192.5673 Ops/s $\color{#d91a1a}-1.84\%$
test_redq_speed[True-backward] 15.8757ms 13.7217ms 72.8775 Ops/s 79.9215 Ops/s $\textbf{\color{#d91a1a}-8.81\%}$
test_redq_speed[reduce-overhead-None] 6.5004ms 5.9837ms 167.1206 Ops/s 178.5597 Ops/s $\textbf{\color{#d91a1a}-6.41\%}$
test_redq_speed[reduce-overhead-backward] 13.6920ms 13.0395ms 76.6900 Ops/s 77.3910 Ops/s $\color{#d91a1a}-0.91\%$
test_redq_deprec_speed[False-None] 14.6563ms 13.2621ms 75.4026 Ops/s 76.6571 Ops/s $\color{#d91a1a}-1.64\%$
test_redq_deprec_speed[False-backward] 20.4215ms 19.4029ms 51.5387 Ops/s 53.5908 Ops/s $\color{#d91a1a}-3.83\%$
test_redq_deprec_speed[True-None] 5.0207ms 4.0404ms 247.5003 Ops/s 247.7985 Ops/s $\color{#d91a1a}-0.12\%$
test_redq_deprec_speed[True-backward] 9.8497ms 9.4166ms 106.1953 Ops/s 118.8644 Ops/s $\textbf{\color{#d91a1a}-10.66\%}$
test_redq_deprec_speed[reduce-overhead-None] 4.7067ms 3.9401ms 253.8030 Ops/s 236.3990 Ops/s $\textbf{\color{#35bf28}+7.36\%}$
test_redq_deprec_speed[reduce-overhead-backward] 9.1390ms 8.7633ms 114.1116 Ops/s 117.9395 Ops/s $\color{#d91a1a}-3.25\%$
test_td3_speed[False-None] 8.9727ms 8.1219ms 123.1235 Ops/s 122.5303 Ops/s $\color{#35bf28}+0.48\%$
test_td3_speed[False-backward] 12.9736ms 10.7524ms 93.0029 Ops/s 94.5752 Ops/s $\color{#d91a1a}-1.66\%$
test_td3_speed[True-None] 1.9585ms 1.8193ms 549.6756 Ops/s 547.8101 Ops/s $\color{#35bf28}+0.34\%$
test_td3_speed[True-backward] 4.0029ms 3.4707ms 288.1282 Ops/s 289.0349 Ops/s $\color{#d91a1a}-0.31\%$
test_td3_speed[reduce-overhead-None] 2.3698ms 1.8079ms 553.1392 Ops/s 543.3685 Ops/s $\color{#35bf28}+1.80\%$
test_td3_speed[reduce-overhead-backward] 3.5827ms 3.4484ms 289.9859 Ops/s 285.9790 Ops/s $\color{#35bf28}+1.40\%$
test_cql_speed[False-None] 39.1541ms 36.3535ms 27.5076 Ops/s 26.1255 Ops/s $\textbf{\color{#35bf28}+5.29\%}$
test_cql_speed[False-backward] 51.9290ms 47.2625ms 21.1584 Ops/s 20.4755 Ops/s $\color{#35bf28}+3.34\%$
test_cql_speed[True-None] 17.5910ms 16.1227ms 62.0243 Ops/s 60.4855 Ops/s $\color{#35bf28}+2.54\%$
test_cql_speed[True-backward] 24.9283ms 23.9831ms 41.6960 Ops/s 43.5943 Ops/s $\color{#d91a1a}-4.35\%$
test_cql_speed[reduce-overhead-None] 17.8837ms 16.3610ms 61.1208 Ops/s 60.3134 Ops/s $\color{#35bf28}+1.34\%$
test_cql_speed[reduce-overhead-backward] 24.6226ms 23.3392ms 42.8463 Ops/s 42.8731 Ops/s $\color{#d91a1a}-0.06\%$
test_a2c_speed[False-None] 8.6238ms 7.2882ms 137.2083 Ops/s 135.3994 Ops/s $\color{#35bf28}+1.34\%$
test_a2c_speed[False-backward] 16.0921ms 14.8112ms 67.5165 Ops/s 67.7246 Ops/s $\color{#d91a1a}-0.31\%$
test_a2c_speed[True-None] 4.1254ms 3.7856ms 264.1581 Ops/s 231.9478 Ops/s $\textbf{\color{#35bf28}+13.89\%}$
test_a2c_speed[True-backward] 11.0351ms 10.4694ms 95.5165 Ops/s 95.8934 Ops/s $\color{#d91a1a}-0.39\%$
test_a2c_speed[reduce-overhead-None] 4.0794ms 3.7609ms 265.8953 Ops/s 262.7819 Ops/s $\color{#35bf28}+1.18\%$
test_a2c_speed[reduce-overhead-backward] 11.1456ms 10.2829ms 97.2484 Ops/s 97.7037 Ops/s $\color{#d91a1a}-0.47\%$
test_ppo_speed[False-None] 8.7176ms 7.4796ms 133.6964 Ops/s 132.1036 Ops/s $\color{#35bf28}+1.21\%$
test_ppo_speed[False-backward] 15.5233ms 15.1597ms 65.9642 Ops/s 67.3958 Ops/s $\color{#d91a1a}-2.12\%$
test_ppo_speed[True-None] 5.1811ms 4.1803ms 239.2150 Ops/s 239.6860 Ops/s $\color{#d91a1a}-0.20\%$
test_ppo_speed[True-backward] 10.5678ms 10.0693ms 99.3120 Ops/s 98.9852 Ops/s $\color{#35bf28}+0.33\%$
test_ppo_speed[reduce-overhead-None] 5.0157ms 4.2073ms 237.6846 Ops/s 242.5892 Ops/s $\color{#d91a1a}-2.02\%$
test_ppo_speed[reduce-overhead-backward] 11.0482ms 10.3116ms 96.9778 Ops/s 94.1315 Ops/s $\color{#35bf28}+3.02\%$
test_reinforce_speed[False-None] 7.9971ms 6.5751ms 152.0879 Ops/s 145.1558 Ops/s $\color{#35bf28}+4.78\%$
test_reinforce_speed[False-backward] 10.0447ms 9.8680ms 101.3380 Ops/s 95.6555 Ops/s $\textbf{\color{#35bf28}+5.94\%}$
test_reinforce_speed[True-None] 3.5089ms 3.1174ms 320.7814 Ops/s 304.2433 Ops/s $\textbf{\color{#35bf28}+5.44\%}$
test_reinforce_speed[True-backward] 9.9270ms 9.2590ms 108.0035 Ops/s 100.4221 Ops/s $\textbf{\color{#35bf28}+7.55\%}$
test_reinforce_speed[reduce-overhead-None] 3.6052ms 3.2606ms 306.6943 Ops/s 312.8681 Ops/s $\color{#d91a1a}-1.97\%$
test_reinforce_speed[reduce-overhead-backward] 9.4306ms 8.9840ms 111.3086 Ops/s 101.6946 Ops/s $\textbf{\color{#35bf28}+9.45\%}$
test_iql_speed[False-None] 38.1266ms 32.4966ms 30.7724 Ops/s 29.9073 Ops/s $\color{#35bf28}+2.89\%$
test_iql_speed[False-backward] 49.0509ms 45.2177ms 22.1152 Ops/s 21.1841 Ops/s $\color{#35bf28}+4.40\%$
test_iql_speed[True-None] 11.9625ms 11.0646ms 90.3787 Ops/s 84.6827 Ops/s $\textbf{\color{#35bf28}+6.73\%}$
test_iql_speed[True-backward] 24.0913ms 22.2342ms 44.9758 Ops/s 42.6174 Ops/s $\textbf{\color{#35bf28}+5.53\%}$
test_iql_speed[reduce-overhead-None] 13.0374ms 11.5600ms 86.5055 Ops/s 89.5878 Ops/s $\color{#d91a1a}-3.44\%$
test_iql_speed[reduce-overhead-backward] 23.9865ms 22.9998ms 43.4786 Ops/s 44.7151 Ops/s $\color{#d91a1a}-2.77\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 0.4920s 7.6355ms 130.9673 Ops/s 206.0462 Ops/s $\textbf{\color{#d91a1a}-36.44\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8225ms 0.5147ms 1.9428 KOps/s 1.9406 KOps/s $\color{#35bf28}+0.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8419ms 0.4875ms 2.0515 KOps/s 2.0262 KOps/s $\color{#35bf28}+1.25\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.7602ms 4.8175ms 207.5769 Ops/s 218.5437 Ops/s $\textbf{\color{#d91a1a}-5.02\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.5836ms 0.5012ms 1.9952 KOps/s 1.9455 KOps/s $\color{#35bf28}+2.56\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9019ms 0.4939ms 2.0249 KOps/s 2.0801 KOps/s $\color{#d91a1a}-2.66\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.2634ms 1.6556ms 604.0166 Ops/s 593.4014 Ops/s $\color{#35bf28}+1.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.3414ms 1.5707ms 636.6728 Ops/s 628.2961 Ops/s $\color{#35bf28}+1.33\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.5154ms 4.9867ms 200.5334 Ops/s 212.3585 Ops/s $\textbf{\color{#d91a1a}-5.57\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.3974ms 0.6436ms 1.5537 KOps/s 1.5211 KOps/s $\color{#35bf28}+2.14\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8843ms 0.6186ms 1.6165 KOps/s 1.5532 KOps/s $\color{#35bf28}+4.08\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.0830ms 4.8550ms 205.9742 Ops/s 205.6054 Ops/s $\color{#35bf28}+0.18\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0416ms 0.5145ms 1.9438 KOps/s 1.9093 KOps/s $\color{#35bf28}+1.81\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7037ms 0.4919ms 2.0328 KOps/s 1.9475 KOps/s $\color{#35bf28}+4.38\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8754ms 4.8572ms 205.8818 Ops/s 201.7486 Ops/s $\color{#35bf28}+2.05\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.2632ms 0.5051ms 1.9798 KOps/s 1.9692 KOps/s $\color{#35bf28}+0.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7714ms 0.4905ms 2.0386 KOps/s 2.0664 KOps/s $\color{#d91a1a}-1.34\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.8775ms 5.0848ms 196.6659 Ops/s 202.9514 Ops/s $\color{#d91a1a}-3.10\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1877ms 0.6477ms 1.5440 KOps/s 1.5102 KOps/s $\color{#35bf28}+2.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8283ms 0.6179ms 1.6184 KOps/s 1.5885 KOps/s $\color{#35bf28}+1.88\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.4898s 14.0892ms 70.9766 Ops/s 231.6850 Ops/s $\textbf{\color{#d91a1a}-69.37\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 7.4539ms 2.3300ms 429.1902 Ops/s 418.2084 Ops/s $\color{#35bf28}+2.63\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.0782ms 1.3484ms 741.6315 Ops/s 748.6671 Ops/s $\color{#d91a1a}-0.94\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.3874ms 4.3849ms 228.0568 Ops/s 227.8756 Ops/s $\color{#35bf28}+0.08\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 9.2160ms 2.2604ms 442.3968 Ops/s 416.3258 Ops/s $\textbf{\color{#35bf28}+6.26\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.8711ms 1.2687ms 788.2083 Ops/s 697.6004 Ops/s $\textbf{\color{#35bf28}+12.99\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.4201s 12.8741ms 77.6755 Ops/s 219.6675 Ops/s $\textbf{\color{#d91a1a}-64.64\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 7.1094ms 2.4440ms 409.1708 Ops/s 338.6799 Ops/s $\textbf{\color{#35bf28}+20.81\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.1574ms 1.4879ms 672.0961 Ops/s 655.0960 Ops/s $\color{#35bf28}+2.60\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 12.1702ms 11.9203ms 83.8905 Ops/s 78.0867 Ops/s $\textbf{\color{#35bf28}+7.43\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 15.6809ms 14.6397ms 68.3075 Ops/s 69.5121 Ops/s $\color{#d91a1a}-1.73\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 21.1683ms 20.6172ms 48.5031 Ops/s 46.6134 Ops/s $\color{#35bf28}+4.05\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 16.7550ms 14.5833ms 68.5716 Ops/s 68.3191 Ops/s $\color{#35bf28}+0.37\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 23.6315ms 20.7891ms 48.1021 Ops/s 47.4949 Ops/s $\color{#35bf28}+1.28\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 17.1178ms 15.7580ms 63.4599 Ops/s 64.0116 Ops/s $\color{#d91a1a}-0.86\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.8805s 0.7952s 1.2576 Ops/s 1.2589 Ops/s $\color{#d91a1a}-0.11\%$
test_transformed 1.4636s 1.3764s 0.7265 Ops/s 0.7251 Ops/s $\color{#35bf28}+0.20\%$
test_serial 2.3873s 2.2948s 0.4358 Ops/s 0.4362 Ops/s $\color{#d91a1a}-0.10\%$
test_parallel 1.9476s 1.8552s 0.5390 Ops/s 0.5300 Ops/s $\color{#35bf28}+1.71\%$
test_step_mdp_speed[True-True-True-True-True] 0.2304ms 39.7999μs 25.1257 KOps/s 25.1901 KOps/s $\color{#d91a1a}-0.26\%$
test_step_mdp_speed[True-True-True-True-False] 52.3800μs 23.7058μs 42.1838 KOps/s 42.3239 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[True-True-True-False-True] 51.6500μs 22.3341μs 44.7747 KOps/s 43.7096 KOps/s $\color{#35bf28}+2.44\%$
test_step_mdp_speed[True-True-True-False-False] 44.2400μs 13.0806μs 76.4493 KOps/s 75.4676 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[True-True-False-True-True] 79.3010μs 42.8783μs 23.3218 KOps/s 23.0008 KOps/s $\color{#35bf28}+1.40\%$
test_step_mdp_speed[True-True-False-True-False] 62.0710μs 25.9040μs 38.6040 KOps/s 37.9478 KOps/s $\color{#35bf28}+1.73\%$
test_step_mdp_speed[True-True-False-False-True] 55.9310μs 24.9290μs 40.1139 KOps/s 39.8295 KOps/s $\color{#35bf28}+0.71\%$
test_step_mdp_speed[True-True-False-False-False] 51.1110μs 15.6912μs 63.7300 KOps/s 63.0520 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[True-False-True-True-True] 81.9110μs 44.8361μs 22.3035 KOps/s 21.6805 KOps/s $\color{#35bf28}+2.87\%$
test_step_mdp_speed[True-False-True-True-False] 60.0010μs 28.3907μs 35.2227 KOps/s 34.9321 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[True-False-True-False-True] 58.3310μs 24.9615μs 40.0616 KOps/s 39.4900 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[True-False-True-False-False] 43.0310μs 15.5036μs 64.5012 KOps/s 63.9775 KOps/s $\color{#35bf28}+0.82\%$
test_step_mdp_speed[True-False-False-True-True] 81.1410μs 47.3091μs 21.1376 KOps/s 21.3795 KOps/s $\color{#d91a1a}-1.13\%$
test_step_mdp_speed[True-False-False-True-False] 59.0300μs 30.8711μs 32.3928 KOps/s 32.5026 KOps/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[True-False-False-False-True] 61.4400μs 26.6412μs 37.5359 KOps/s 37.1432 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-False-False-False-False] 48.9000μs 17.7505μs 56.3363 KOps/s 55.5326 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[False-True-True-True-True] 72.6610μs 44.8040μs 22.3194 KOps/s 21.6686 KOps/s $\color{#35bf28}+3.00\%$
test_step_mdp_speed[False-True-True-True-False] 62.0710μs 28.3189μs 35.3121 KOps/s 34.7693 KOps/s $\color{#35bf28}+1.56\%$
test_step_mdp_speed[False-True-True-False-True] 63.9010μs 28.5777μs 34.9923 KOps/s 34.3231 KOps/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[False-True-True-False-False] 44.9210μs 17.3582μs 57.6098 KOps/s 58.0257 KOps/s $\color{#d91a1a}-0.72\%$
test_step_mdp_speed[False-True-False-True-True] 93.0810μs 47.1007μs 21.2311 KOps/s 20.6874 KOps/s $\color{#35bf28}+2.63\%$
test_step_mdp_speed[False-True-False-True-False] 70.3410μs 30.5764μs 32.7050 KOps/s 32.4141 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[False-True-False-False-True] 3.1355ms 30.8094μs 32.4576 KOps/s 31.6657 KOps/s $\color{#35bf28}+2.50\%$
test_step_mdp_speed[False-True-False-False-False] 45.9900μs 19.6873μs 50.7941 KOps/s 50.8512 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[False-False-True-True-True] 77.5510μs 49.8691μs 20.0525 KOps/s 19.5642 KOps/s $\color{#35bf28}+2.50\%$
test_step_mdp_speed[False-False-True-True-False] 61.1310μs 32.9268μs 30.3704 KOps/s 30.0802 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[False-False-True-False-True] 62.1910μs 31.1729μs 32.0791 KOps/s 32.0523 KOps/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-False-True-False-False] 56.5300μs 19.4846μs 51.3225 KOps/s 50.8730 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-False-False-True-True] 89.7510μs 50.4892μs 19.8062 KOps/s 19.0744 KOps/s $\color{#35bf28}+3.84\%$
test_step_mdp_speed[False-False-False-True-False] 63.1310μs 35.4320μs 28.2231 KOps/s 28.2112 KOps/s $\color{#35bf28}+0.04\%$
test_step_mdp_speed[False-False-False-False-True] 62.3210μs 33.0732μs 30.2360 KOps/s 29.6567 KOps/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[False-False-False-False-False] 53.8510μs 21.7403μs 45.9975 KOps/s 44.9663 KOps/s $\color{#35bf28}+2.29\%$
test_values[generalized_advantage_estimate-True-True] 25.1975ms 24.8042ms 40.3158 Ops/s 40.9673 Ops/s $\color{#d91a1a}-1.59\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1122s 3.1370ms 318.7718 Ops/s 337.1923 Ops/s $\textbf{\color{#d91a1a}-5.46\%}$
test_values[td0_return_estimate-False-False] 0.1051ms 80.5412μs 12.4160 KOps/s 12.4015 KOps/s $\color{#35bf28}+0.12\%$
test_values[td1_return_estimate-False-False] 56.1719ms 55.5425ms 18.0042 Ops/s 18.4625 Ops/s $\color{#d91a1a}-2.48\%$
test_values[vec_td1_return_estimate-False-False] 1.3296ms 1.0882ms 918.9254 Ops/s 935.9835 Ops/s $\color{#d91a1a}-1.82\%$
test_values[td_lambda_return_estimate-True-False] 88.6791ms 87.7778ms 11.3924 Ops/s 11.6586 Ops/s $\color{#d91a1a}-2.28\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3001ms 1.0829ms 923.4402 Ops/s 929.1144 Ops/s $\color{#d91a1a}-0.61\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.2651ms 24.7873ms 40.3432 Ops/s 40.7652 Ops/s $\color{#d91a1a}-1.04\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0275ms 0.7526ms 1.3287 KOps/s 1.3343 KOps/s $\color{#d91a1a}-0.41\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7410ms 0.6678ms 1.4975 KOps/s 1.4962 KOps/s $\color{#35bf28}+0.08\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5580ms 1.4879ms 672.0997 Ops/s 677.3715 Ops/s $\color{#d91a1a}-0.78\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7405ms 0.6891ms 1.4511 KOps/s 1.4633 KOps/s $\color{#d91a1a}-0.83\%$
test_dqn_speed[False-None] 6.8750ms 1.5349ms 651.5001 Ops/s 653.5954 Ops/s $\color{#d91a1a}-0.32\%$
test_dqn_speed[False-backward] 2.1991ms 2.1486ms 465.4125 Ops/s 467.3349 Ops/s $\color{#d91a1a}-0.41\%$
test_dqn_speed[True-None] 0.7035ms 0.5631ms 1.7758 KOps/s 1.6910 KOps/s $\textbf{\color{#35bf28}+5.01\%}$
test_dqn_speed[True-backward] 1.1870ms 1.1304ms 884.6495 Ops/s 880.8501 Ops/s $\color{#35bf28}+0.43\%$
test_dqn_speed[reduce-overhead-None] 0.6816ms 0.5742ms 1.7417 KOps/s 1.6686 KOps/s $\color{#35bf28}+4.38\%$
test_dqn_speed[reduce-overhead-backward] 1.0111ms 0.9728ms 1.0280 KOps/s 1.0179 KOps/s $\color{#35bf28}+0.99\%$
test_ddpg_speed[False-None] 3.2326ms 2.8958ms 345.3292 Ops/s 345.3774 Ops/s $\color{#d91a1a}-0.01\%$
test_ddpg_speed[False-backward] 4.3789ms 4.1688ms 239.8781 Ops/s 240.5129 Ops/s $\color{#d91a1a}-0.26\%$
test_ddpg_speed[True-None] 1.6806ms 1.3795ms 724.8764 Ops/s 738.4561 Ops/s $\color{#d91a1a}-1.84\%$
test_ddpg_speed[True-backward] 2.8231ms 2.4908ms 401.4830 Ops/s 399.7822 Ops/s $\color{#35bf28}+0.43\%$
test_ddpg_speed[reduce-overhead-None] 1.8206ms 1.3897ms 719.5825 Ops/s 730.7763 Ops/s $\color{#d91a1a}-1.53\%$
test_ddpg_speed[reduce-overhead-backward] 2.3888ms 1.9400ms 515.4603 Ops/s 521.2545 Ops/s $\color{#d91a1a}-1.11\%$
test_sac_speed[False-None] 8.6344ms 8.3244ms 120.1287 Ops/s 123.8958 Ops/s $\color{#d91a1a}-3.04\%$
test_sac_speed[False-backward] 12.2261ms 11.2638ms 88.7798 Ops/s 90.9896 Ops/s $\color{#d91a1a}-2.43\%$
test_sac_speed[True-None] 2.2710ms 1.8565ms 538.6349 Ops/s 532.9118 Ops/s $\color{#35bf28}+1.07\%$
test_sac_speed[True-backward] 3.7937ms 3.6485ms 274.0858 Ops/s 273.9688 Ops/s $\color{#35bf28}+0.04\%$
test_sac_speed[reduce-overhead-None] 21.5470ms 12.1479ms 82.3187 Ops/s 83.1733 Ops/s $\color{#d91a1a}-1.03\%$
test_sac_speed[reduce-overhead-backward] 1.7011ms 1.6503ms 605.9394 Ops/s 597.0082 Ops/s $\color{#35bf28}+1.50\%$
test_redq_speed[False-None] 8.0049ms 7.5781ms 131.9592 Ops/s 130.6836 Ops/s $\color{#35bf28}+0.98\%$
test_redq_speed[False-backward] 12.0689ms 11.4446ms 87.3775 Ops/s 87.0566 Ops/s $\color{#35bf28}+0.37\%$
test_redq_speed[True-None] 2.5381ms 2.3414ms 427.0938 Ops/s 421.3612 Ops/s $\color{#35bf28}+1.36\%$
test_redq_speed[True-backward] 4.7865ms 4.2432ms 235.6705 Ops/s 231.0435 Ops/s $\color{#35bf28}+2.00\%$
test_redq_speed[reduce-overhead-None] 2.4525ms 2.3572ms 424.2349 Ops/s 421.0438 Ops/s $\color{#35bf28}+0.76\%$
test_redq_speed[reduce-overhead-backward] 4.5795ms 4.2560ms 234.9612 Ops/s 233.7168 Ops/s $\color{#35bf28}+0.53\%$
test_redq_deprec_speed[False-None] 9.3903ms 9.1358ms 109.4595 Ops/s 109.6508 Ops/s $\color{#d91a1a}-0.17\%$
test_redq_deprec_speed[False-backward] 12.9713ms 12.4567ms 80.2779 Ops/s 81.1737 Ops/s $\color{#d91a1a}-1.10\%$
test_redq_deprec_speed[True-None] 2.7808ms 2.6657ms 375.1390 Ops/s 371.8738 Ops/s $\color{#35bf28}+0.88\%$
test_redq_deprec_speed[True-backward] 4.8426ms 4.5260ms 220.9454 Ops/s 222.1272 Ops/s $\color{#d91a1a}-0.53\%$
test_redq_deprec_speed[reduce-overhead-None] 2.7667ms 2.6676ms 374.8707 Ops/s 369.1025 Ops/s $\color{#35bf28}+1.56\%$
test_redq_deprec_speed[reduce-overhead-backward] 4.9791ms 4.5101ms 221.7267 Ops/s 225.3961 Ops/s $\color{#d91a1a}-1.63\%$
test_td3_speed[False-None] 8.2582ms 8.0687ms 123.9362 Ops/s 124.9107 Ops/s $\color{#d91a1a}-0.78\%$
test_td3_speed[False-backward] 11.2750ms 10.6726ms 93.6975 Ops/s 97.1070 Ops/s $\color{#d91a1a}-3.51\%$
test_td3_speed[True-None] 1.6934ms 1.6676ms 599.6817 Ops/s 589.1207 Ops/s $\color{#35bf28}+1.79\%$
test_td3_speed[True-backward] 3.4516ms 3.3952ms 294.5371 Ops/s 305.3881 Ops/s $\color{#d91a1a}-3.55\%$
test_td3_speed[reduce-overhead-None] 49.7361ms 25.6451ms 38.9938 Ops/s 38.6358 Ops/s $\color{#35bf28}+0.93\%$
test_td3_speed[reduce-overhead-backward] 1.5556ms 1.4971ms 667.9597 Ops/s 729.8724 Ops/s $\textbf{\color{#d91a1a}-8.48\%}$
test_cql_speed[False-None] 17.3066ms 16.8468ms 59.3586 Ops/s 59.0820 Ops/s $\color{#35bf28}+0.47\%$
test_cql_speed[False-backward] 22.8552ms 22.3955ms 44.6519 Ops/s 45.5189 Ops/s $\color{#d91a1a}-1.90\%$
test_cql_speed[True-None] 3.3932ms 3.2910ms 303.8548 Ops/s 294.0153 Ops/s $\color{#35bf28}+3.35\%$
test_cql_speed[True-backward] 6.3148ms 5.7760ms 173.1308 Ops/s 171.8920 Ops/s $\color{#35bf28}+0.72\%$
test_cql_speed[reduce-overhead-None] 20.5242ms 12.9460ms 77.2439 Ops/s 75.2414 Ops/s $\color{#35bf28}+2.66\%$
test_cql_speed[reduce-overhead-backward] 2.1725ms 2.0308ms 492.4186 Ops/s 489.1112 Ops/s $\color{#35bf28}+0.68\%$
test_a2c_speed[False-None] 3.2900ms 3.2126ms 311.2780 Ops/s 304.8024 Ops/s $\color{#35bf28}+2.12\%$
test_a2c_speed[False-backward] 7.0028ms 6.4502ms 155.0340 Ops/s 154.6863 Ops/s $\color{#35bf28}+0.22\%$
test_a2c_speed[True-None] 1.4463ms 1.3689ms 730.5062 Ops/s 724.8984 Ops/s $\color{#35bf28}+0.77\%$
test_a2c_speed[True-backward] 3.1545ms 3.0858ms 324.0617 Ops/s 315.8961 Ops/s $\color{#35bf28}+2.58\%$
test_a2c_speed[reduce-overhead-None] 16.0154ms 9.0940ms 109.9630 Ops/s 111.7323 Ops/s $\color{#d91a1a}-1.58\%$
test_a2c_speed[reduce-overhead-backward] 1.8096ms 1.6281ms 614.2135 Ops/s 606.2644 Ops/s $\color{#35bf28}+1.31\%$
test_ppo_speed[False-None] 3.8079ms 3.7231ms 268.5947 Ops/s 268.0678 Ops/s $\color{#35bf28}+0.20\%$
test_ppo_speed[False-backward] 7.6042ms 7.1481ms 139.8979 Ops/s 141.9255 Ops/s $\color{#d91a1a}-1.43\%$
test_ppo_speed[True-None] 1.5214ms 1.4265ms 700.9924 Ops/s 686.2301 Ops/s $\color{#35bf28}+2.15\%$
test_ppo_speed[True-backward] 3.3109ms 3.2563ms 307.0992 Ops/s 315.7957 Ops/s $\color{#d91a1a}-2.75\%$
test_ppo_speed[reduce-overhead-None] 1.0660ms 0.9771ms 1.0235 KOps/s 1.0276 KOps/s $\color{#d91a1a}-0.40\%$
test_ppo_speed[reduce-overhead-backward] 1.7106ms 1.5868ms 630.2152 Ops/s 678.4137 Ops/s $\textbf{\color{#d91a1a}-7.10\%}$
test_reinforce_speed[False-None] 2.4290ms 2.2964ms 435.4600 Ops/s 434.9752 Ops/s $\color{#35bf28}+0.11\%$
test_reinforce_speed[False-backward] 3.6033ms 3.4333ms 291.2686 Ops/s 299.7215 Ops/s $\color{#d91a1a}-2.82\%$
test_reinforce_speed[True-None] 1.4191ms 1.3104ms 763.0992 Ops/s 745.4706 Ops/s $\color{#35bf28}+2.36\%$
test_reinforce_speed[True-backward] 3.2287ms 3.1202ms 320.4952 Ops/s 323.8900 Ops/s $\color{#d91a1a}-1.05\%$
test_reinforce_speed[reduce-overhead-None] 18.2215ms 10.1209ms 98.8050 Ops/s 99.9453 Ops/s $\color{#d91a1a}-1.14\%$
test_reinforce_speed[reduce-overhead-backward] 1.7484ms 1.6595ms 602.5857 Ops/s 591.9214 Ops/s $\color{#35bf28}+1.80\%$
test_iql_speed[False-None] 9.7488ms 9.2980ms 107.5497 Ops/s 106.2898 Ops/s $\color{#35bf28}+1.19\%$
test_iql_speed[False-backward] 13.6799ms 13.2433ms 75.5097 Ops/s 75.1869 Ops/s $\color{#35bf28}+0.43\%$
test_iql_speed[True-None] 2.4377ms 2.2419ms 446.0491 Ops/s 432.8899 Ops/s $\color{#35bf28}+3.04\%$
test_iql_speed[True-backward] 5.3963ms 4.9551ms 201.8122 Ops/s 191.2534 Ops/s $\textbf{\color{#35bf28}+5.52\%}$
test_iql_speed[reduce-overhead-None] 18.9237ms 11.2214ms 89.1151 Ops/s 89.0194 Ops/s $\color{#35bf28}+0.11\%$
test_iql_speed[reduce-overhead-backward] 2.1294ms 2.0751ms 481.8979 Ops/s 499.2254 Ops/s $\color{#d91a1a}-3.47\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.8178ms 6.3725ms 156.9244 Ops/s 154.5379 Ops/s $\color{#35bf28}+1.54\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5519ms 0.3228ms 3.0981 KOps/s 3.0419 KOps/s $\color{#35bf28}+1.85\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5088ms 0.2823ms 3.5429 KOps/s 3.2362 KOps/s $\textbf{\color{#35bf28}+9.48\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.3243ms 6.0955ms 164.0543 Ops/s 162.6039 Ops/s $\color{#35bf28}+0.89\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.0662ms 0.3492ms 2.8634 KOps/s 3.5595 KOps/s $\textbf{\color{#d91a1a}-19.56\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6193ms 0.2841ms 3.5194 KOps/s 4.1403 KOps/s $\textbf{\color{#d91a1a}-15.00\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5787ms 1.3290ms 752.4434 Ops/s 786.1329 Ops/s $\color{#d91a1a}-4.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4503ms 1.2234ms 817.4198 Ops/s 845.8550 Ops/s $\color{#d91a1a}-3.36\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.3473ms 6.2405ms 160.2439 Ops/s 158.7646 Ops/s $\color{#35bf28}+0.93\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.1152ms 0.5031ms 1.9876 KOps/s 2.0910 KOps/s $\color{#d91a1a}-4.95\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6747ms 0.4537ms 2.2042 KOps/s 2.1573 KOps/s $\color{#35bf28}+2.17\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.2029ms 6.0931ms 164.1212 Ops/s 160.9639 Ops/s $\color{#35bf28}+1.96\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.8169ms 0.2759ms 3.6239 KOps/s 3.2144 KOps/s $\textbf{\color{#35bf28}+12.74\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5444ms 0.2569ms 3.8928 KOps/s 3.4262 KOps/s $\textbf{\color{#35bf28}+13.62\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.3295ms 6.0949ms 164.0721 Ops/s 163.8878 Ops/s $\color{#35bf28}+0.11\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.9198ms 0.3276ms 3.0522 KOps/s 3.3393 KOps/s $\textbf{\color{#d91a1a}-8.60\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5521ms 0.3080ms 3.2470 KOps/s 3.4468 KOps/s $\textbf{\color{#d91a1a}-5.80\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.3624ms 6.2652ms 159.6122 Ops/s 158.6263 Ops/s $\color{#35bf28}+0.62\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0143ms 0.4617ms 2.1658 KOps/s 2.2179 KOps/s $\color{#d91a1a}-2.35\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6830ms 0.4571ms 2.1878 KOps/s 2.5327 KOps/s $\textbf{\color{#d91a1a}-13.62\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 7.0730ms 5.4863ms 182.2706 Ops/s 178.6309 Ops/s $\color{#35bf28}+2.04\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 6.5698ms 2.0576ms 485.9945 Ops/s 435.7539 Ops/s $\textbf{\color{#35bf28}+11.53\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.8518ms 1.2477ms 801.4529 Ops/s 843.6386 Ops/s $\textbf{\color{#d91a1a}-5.00\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 8.1542ms 5.6529ms 176.8995 Ops/s 179.4423 Ops/s $\color{#d91a1a}-1.42\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 7.7222ms 2.0321ms 492.0930 Ops/s 425.0501 Ops/s $\textbf{\color{#35bf28}+15.77\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.9707ms 1.2746ms 784.5485 Ops/s 852.3066 Ops/s $\textbf{\color{#d91a1a}-7.95\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.5144s 15.9633ms 62.6438 Ops/s 31.0774 Ops/s $\textbf{\color{#35bf28}+101.57\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 9.6864ms 2.2571ms 443.0386 Ops/s 501.2549 Ops/s $\textbf{\color{#d91a1a}-11.61\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.3547ms 1.3987ms 714.9417 Ops/s 799.5923 Ops/s $\textbf{\color{#d91a1a}-10.59\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 13.7876ms 13.4986ms 74.0819 Ops/s 71.5419 Ops/s $\color{#35bf28}+3.55\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.1786ms 17.5023ms 57.1354 Ops/s 58.7730 Ops/s $\color{#d91a1a}-2.79\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 18.8143ms 18.3311ms 54.5521 Ops/s 54.6418 Ops/s $\color{#d91a1a}-0.16\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.7266ms 17.8496ms 56.0238 Ops/s 58.4186 Ops/s $\color{#d91a1a}-4.10\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 19.3186ms 18.4748ms 54.1279 Ops/s 54.3281 Ops/s $\color{#d91a1a}-0.37\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.3452ms 19.0245ms 52.5638 Ops/s 54.3070 Ops/s $\color{#d91a1a}-3.21\%$

vmoens
vmoens reviewed Feb 12, 2025
View reviewed changes
Copy link
Contributor

@vmoens vmoens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice feature!
Do you think we could use Spec.enumerate() as a default? Presumably, envs that don't have a list of actions are also those where enumerate would fail

@kurtamohler
Copy link
Collaborator Author

kurtamohler commented Feb 13, 2025 •
edited
Loading

Ah I didn't know about enumerate! Yeah that would be a great way to implement it. It looks like the enumerate methods for Categorical/OneHot/etc. ignore the mask if it is set. So I think I should add an argument that will make it use the mask. Then I think ChessEnv and other envs that use action spec masks won't even need a specialized impl of all_actions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vmoens vmoens vmoens left review comments

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kurtamohler @vmoens @facebook-github-bot