mirror of
https://github.com/vale981/ray
synced 2025-03-06 10:31:39 -05:00
Add 0.8.1 release test results (#6943)
This commit is contained in:
Edward Oakes
GitHub
parent
4c97348cb6
commit
bfaee49880
7 changed files with 85 additions and 1 deletions
|
|
@ -144,7 +144,7 @@ This document describes the process for creating new releases.
|
|||
scripts. Make sure that it is finding the version of Ray that you just
|
||||
installed by checking ``ray.__version__`` and ``ray.__file__``.
|
||||
|
||||
Do this at least for MacOS and Linux, as well as for Python 2 and Python 3.
|
||||
Do this at least for MacOS and Linux.
|
||||
|
||||
9. **Upload to PyPI:** Now that you've tested the wheels on the PyPI test
|
||||
repository, they can be uploaded to the main PyPI repository. Be careful,
|
||||
|
|
|
|||
6
doc/dev/download_wheels.sh
Normal file
6
doc/dev/download_wheels.sh
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
wget https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp35-cp35m-manylinux1_x86_64.whl
|
||||
wget https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp36-cp36m-manylinux1_x86_64.whl
|
||||
wget https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp37-cp37m-manylinux1_x86_64.whl
|
||||
wget https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp35-cp35m-macosx_10_6_intel.whl
|
||||
wget https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp36-cp36m-macosx_10_6_intel.whl
|
||||
wget https://s3-us-west-2.amazonaws.com/ray-wheels/releases/$RAY_VERSION/$RAY_HASH/ray-$RAY_VERSION-cp37-cp37m-macosx_10_6_intel.whl
|
||||
21
doc/dev/release_logs/0.8.1/microbenchmark.txt
Normal file
21
doc/dev/release_logs/0.8.1/microbenchmark.txt
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
# NOTE: performance decreases are related to enabling reference counting and
|
||||
# pinning by default for objects passed using the ray API. 'single client get'
|
||||
# results dropped because we are no longer keeping a buffer reference in object
|
||||
# IDs for ray.put() objects. This shouldn't have a large impact on applications
|
||||
# and will be fixed in the future by caching buffers after the first ray.get()
|
||||
# on each object ID.
|
||||
|
||||
single client get calls per second 12550.57 +- 1835.19
|
||||
single client put calls per second 6791.78 +- 176.65
|
||||
single client put gigabytes per second 13.36 +- 5.7
|
||||
multi client put calls per second 13503.59 +- 179.1
|
||||
multi client put gigabytes per second 16.22 +- 1.36
|
||||
single client tasks sync per second 1295.56 +- 42.81
|
||||
single client tasks async per second 14825.7 +- 358.92
|
||||
multi client tasks async per second 43699.93 +- 627.98
|
||||
1:1 actor calls sync per second 2194.35 +- 57.18
|
||||
1:1 actor calls async per second 6873.68 +- 87.3
|
||||
1:1 actor calls concurrent per second 7285.91 +- 50.57
|
||||
1:n actor calls async per second 13290.25 +- 140.2
|
||||
n:n actor calls async per second 45354.88 +- 678.84
|
||||
n:n actor calls with arg async per second 13668.97 +- 105.62
|
||||
24
doc/dev/release_logs/0.8.1/rllib_regression.txt
Normal file
24
doc/dev/release_logs/0.8.1/rllib_regression.txt
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
+----------------------------------------+------------+-------+--------+------------------+-------------+----------+
|
||||
| Trial name | status | loc | iter | total time (s) | timesteps | reward |
|
||||
|----------------------------------------+------------+-------+--------+------------------+-------------+----------|
|
||||
| IMPALA_BreakoutNoFrameskip-v4_84d6d8e6 | TERMINATED | | 297 | 3606.54 | 6304000 | 362.33 |
|
||||
| IMPALA_BreakoutNoFrameskip-v4_84d75776 | TERMINATED | | 295 | 3605.31 | 6274500 | 361.79 |
|
||||
| IMPALA_BreakoutNoFrameskip-v4_84d7c2f6 | TERMINATED | | 294 | 3606.04 | 6267500 | 298.03 |
|
||||
| IMPALA_BreakoutNoFrameskip-v4_84d9bfa2 | TERMINATED | | 294 | 3609.24 | 6256500 | 358.66 |
|
||||
| PPO_BreakoutNoFrameskip-v4_84dafff2 | TERMINATED | | 550 | 3605.93 | 2750000 | 34.19 |
|
||||
| PPO_BreakoutNoFrameskip-v4_84dc19e6 | TERMINATED | | 546 | 3604.47 | 2730000 | 39.9 |
|
||||
| PPO_BreakoutNoFrameskip-v4_84dc8be2 | TERMINATED | | 545 | 3603.96 | 2725000 | 18.28 |
|
||||
| PPO_BreakoutNoFrameskip-v4_84dda946 | TERMINATED | | 549 | 3601.85 | 2745000 | 28.13 |
|
||||
| APEX_BreakoutNoFrameskip-v4_84e092fa | TERMINATED | | 113 | 3622.29 | 3535840 | 18.75 |
|
||||
| APEX_BreakoutNoFrameskip-v4_84e13336 | TERMINATED | | 113 | 3604.95 | 3556960 | 28.56 |
|
||||
| APEX_BreakoutNoFrameskip-v4_84e44026 | TERMINATED | | 113 | 3608.22 | 3453440 | 12.88 |
|
||||
| APEX_BreakoutNoFrameskip-v4_84e81cbe | TERMINATED | | 113 | 3600.83 | 3566880 | 32.35 |
|
||||
| A2C_BreakoutNoFrameskip-v4_84e9aa48 | TERMINATED | | 350 | 3609.93 | 3094500 | 102.92 |
|
||||
| A2C_BreakoutNoFrameskip-v4_84eb23fa | TERMINATED | | 350 | 3609.93 | 3000000 | 141.7 |
|
||||
| A2C_BreakoutNoFrameskip-v4_84eba21c | TERMINATED | | 349 | 3601.24 | 3009500 | 87.86 |
|
||||
| A2C_BreakoutNoFrameskip-v4_84eeb7c2 | TERMINATED | | 349 | 3600.57 | 3037500 | 83.63 |
|
||||
| DQN_BreakoutNoFrameskip-v4_84efd378 | TERMINATED | | 32 | 3628.85 | 320000 | 11.45 |
|
||||
| DQN_BreakoutNoFrameskip-v4_84f13bf0 | TERMINATED | | 31 | 3686.04 | 310000 | 10.33 |
|
||||
| DQN_BreakoutNoFrameskip-v4_84f68b8c | TERMINATED | | 32 | 3686.78 | 320000 | 17.51 |
|
||||
| DQN_BreakoutNoFrameskip-v4_84f7152a | TERMINATED | | 32 | 3601.57 | 320000 | 16.78 |
|
||||
+----------------------------------------+------------+-------+--------+------------------+-------------+----------+
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
== Status ==
|
||||
Memory usage on this node: 37.0/480.3 GiB
|
||||
Using FIFO scheduling algorithm.
|
||||
Resources requested: 0/640 CPUs, 0/8 GPUs, 0.0/2526.95 GiB heap, 0.0/128.42 GiB objects
|
||||
Result logdir: /home/ubuntu/ray_results/atari-impala
|
||||
Number of trials: 4 (4 TERMINATED)
|
||||
+---------------------------------------------+------------+-------+-----------------------------+--------+------------------+-------------+----------+
|
||||
| Trial name | status | loc | env | iter | total time (s) | timesteps | reward |
|
||||
|---------------------------------------------+------------+-------+-----------------------------+--------+------------------+-------------+----------|
|
||||
| IMPALA_BreakoutNoFrameskip-v4_511a1830 | TERMINATED | | BreakoutNoFrameskip-v4 | 39 | 699.024 | 3050000 | 44.21 |
|
||||
| IMPALA_BeamRiderNoFrameskip-v4_511b6726 | TERMINATED | | BeamRiderNoFrameskip-v4 | 37 | 664.972 | 3083500 | 449.72 |
|
||||
| IMPALA_QbertNoFrameskip-v4_511bd44a | TERMINATED | | QbertNoFrameskip-v4 | 40 | 707.975 | 3088000 | 495.464 |
|
||||
| IMPALA_SpaceInvadersNoFrameskip-v4_511c4858 | TERMINATED | | SpaceInvadersNoFrameskip-v4 | 38 | 680.926 | 3004500 | 380.35 |
|
||||
+---------------------------------------------+------------+-------+-----------------------------+--------+------------------+-------------+----------+
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
Finished in: 138.03558135032654s
|
||||
Average iteration time: 1.3803531074523925s
|
||||
Max iteration time: 2.4373393058776855s
|
||||
Min iteration time: 0.20983195304870605s
|
||||
15
doc/dev/release_logs/0.8.1/stress_tests/test_many_tasks.txt
Normal file
15
doc/dev/release_logs/0.8.1/stress_tests/test_many_tasks.txt
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
Stage 0 results:
|
||||
Total time: 20.65841770172119
|
||||
Stage 1 results:
|
||||
Total time: 109.76851415634155
|
||||
Average iteration time: 10.976841711997986
|
||||
Max iteration time: 11.290549039840698
|
||||
Min iteration time: 10.660207271575928
|
||||
Stage 2 results:
|
||||
Total time: 476.79087471961975
|
||||
Average iteration time: 95.35772457122803
|
||||
Max iteration time: 98.76972222328186
|
||||
Min iteration time: 90.59104466438293
|
||||
Stage 3 results:
|
||||
Actor creation time: 0.07228231430053711
|
||||
Total time: 2383.4370431900024
|
||||
Loading…
Add table
Reference in a new issue