* Exploration API (+EpsilonGreedy sub-class).
* Exploration API (+EpsilonGreedy sub-class).
* Cleanup/LINT.
* Add `deterministic` to generic Trainer config (NOTE: this is still ignored by most Agents).
* Add `error` option to deprecation_warning().
* WIP.
* Bug fix: Get exploration-info for tf framework.
Bug fix: Properly deprecate some DQN config keys.
* WIP.
* LINT.
* WIP.
* Split PerWorkerEpsilonGreedy out of EpsilonGreedy.
Docstrings.
* Fix bug in sampler.py in case Policy has self.exploration = None
* Update rllib/agents/dqn/dqn.py
Co-Authored-By: Eric Liang <ekhliang@gmail.com>
* WIP.
* Update rllib/agents/trainer.py
Co-Authored-By: Eric Liang <ekhliang@gmail.com>
* WIP.
* Change requests.
* LINT
* In tune/utils/util.py::deep_update() Only keep deep_updat'ing if both original and value are dicts. If value is not a dict, set
* Completely obsolete syn_replay_optimizer.py's parameters schedule_max_timesteps AND beta_annealing_fraction (replaced with prioritized_replay_beta_annealing_timesteps).
* Update rllib/evaluation/worker_set.py
Co-Authored-By: Eric Liang <ekhliang@gmail.com>
* Review fixes.
* Fix default value for DQN's exploration spec.
* LINT
* Fix recursion bug (wrong parent c'tor).
* Do not pass timestep to get_exploration_info.
* Update tf_policy.py
* Fix some remaining issues with test cases and remove more deprecated DQN/APEX exploration configs.
* Bug fix tf-action-dist
* DDPG incompatibility bug fix with new DQN exploration handling (which is imported by DDPG).
* Switch off exploration when getting action probs from off-policy-estimator's policy.
* LINT
* Fix test_checkpoint_restore.py.
* Deprecate all SAC exploration (unused) configs.
* Properly use `model.last_output()` everywhere. Instead of `model._last_output`.
* WIP.
* Take out set_epsilon from multi-agent-env test (not needed, decays anyway).
* WIP.
* Trigger re-test (flaky checkpoint-restore test).
* WIP.
* WIP.
* Add test case for deterministic action sampling in PPO.
* bug fix.
* Added deterministic test cases for different Agents.
* Fix problem with TupleActions in dynamic-tf-policy.
* Separate supported_spaces tests so they can be run separately for easier debugging.
* LINT.
* Fix autoregressive_action_dist.py test case.
* Re-test.
* Fix.
* Remove duplicate py_test rule from bazel.
* LINT.
* WIP.
* WIP.
* SAC fix.
* SAC fix.
* WIP.
* WIP.
* WIP.
* FIX 2 examples tests.
* WIP.
* WIP.
* WIP.
* WIP.
* WIP.
* Fix.
* LINT.
* Renamed test file.
* WIP.
* Add unittest.main.
* Make action_dist_class mandatory.
* fix
* FIX.
* WIP.
* WIP.
* Fix.
* Fix.
* Fix explorations test case (contextlib cannot find its own nullcontext??).
* Force torch to be installed for QMIX.
* LINT.
* Fix determine_tests_to_run.py.
* Fix determine_tests_to_run.py.
* WIP
* Add Random exploration component to tests (fixed issue with "static-graph randomness" via py_function).
* Add Random exploration component to tests (fixed issue with "static-graph randomness" via py_function).
* Rename some stuff.
* Rename some stuff.
* WIP.
* WIP.
* Fix SAC.
* Fix SAC.
* Fix strange tf-error in ray core tests.
* Fix strange ray-core tf-error in test_memory_scheduling test case.
* Fix test_io.py.
* LINT.
* Update SAC yaml files' config.
Co-authored-by: Eric Liang <ekhliang@gmail.com>
* Add regression tests
* Refactor, split RemoveSubmittedTaskReferences into submitted and finished paths
* Add nested return IDs to UpdateFinishedTaskRefs, rename WrapObjectIds
* Basic unit tests pass
* Fix unit test and add an out-of-order regression test
* Add stored_in_objects to ObjectReferenceCount, regression test now passes
* Add an Address to the ReferenceCounter so we can determine ownership
* Set the nested return IDs from the TaskManager
* Add another test
* Simplify
* Update src/ray/core_worker/reference_count_test.cc
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Apply suggestions from code review
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* comments
* Add python test
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
* Skeleton plus a unit test for simple borrower case
* First unit test passes - forward an ID and task returns with 1 submitted task pending on the inner ID
* Invariant for contained_in
* Unit test passes for testing task return without creating a borrower
* Wrap ref count functionality in test case
* Fix bad delete
* Unit test and fix for borrowers creating more borrowers
* Unit test and fix for simple borrowing, but owner sends call after borrower's ref count goes to 0
* Refactor:
- keep a sentinel ref count for task argument IDs
- keep contained_in_borrowed in addition to contained_in_owned
* Unit test for nested IDs passes
* Refactor so that an object ID can only be contained in 1 borrowed ID at a time
* Add check
* Fix
* Unit test (passes) to test nesting object IDs but no borrowers created
* Unit test for nested objects from different owners passes, refactor to unset contained_in when popping refs
* Unit tests for borrowers receiving an ObjectID from multiple sources,
skip adding ownership info if we already have it to handle duplicate
refs
* Unit test for returning object ID passes
* More unit tests for returning object IDs pass
* Add serialized ID tests
* fix serialization issue
* remove swap
* It builds!
* debugging and some fixes:
- register handler for WaitForRefRemoved
- don't create a python reference for arg IDs
- pass in client factory into ReferenceCounter
- fix bad decrement in PopBorrowerRefs
* Fix accounting for serialized IDs:
- don't decrement for IDs on dependency resolution, wait until task finished
- add object IDs that were inlined when building the arguments to the task spec, pin these on the task executor until task finishes
* mu_ -> mutex_
* lint
* fix build
* clear outer_object_id
* add direct call type check
* Fix test for direct call IDs and return IDs for actor calls
* Fix CoreWorkerClient.Addr()
* Remove unneeded lock
* Remove unnecessary ObjectID refs
* Fix worker holding serialized refs test
* Fix hex IDs
* fix
* fix tests
* fix tests
* refactor and cleanups
* lint
* Put inlined Ids in task args and some cleanup
* Add back gc.collect() line for test case
* Refactor and fixes:
- store inlined IDs in RayObject
- allow storing objects with inlined IDs in memory store
- pin objects that were promoted to plasma
* oops
* make sure worker ID is set in address, pass in rpc::Address to CoreWorkerClient
* todos
* cleanups and test builds
* Fix tests
* Add feature flag
* cleanups
* address comments and some cleanups
* cleanup
* fix recursive test
* Comments for tests
* Turn off ref counting by default
* Skip tests
* Fix some bugs for test_array.py, java build
* Don't include nested objects in the ref count when the feature flag is off
* C++ feature flag does not work...
* Remove
* Turn on python tests and add a warning when plasma objects are evicted before being pinned
* Fix build and remove irrelevant test
* Fix for java
* Revert "Fix build and remove irrelevant test"
This reverts commit 056cca9b263ed05b0f9ab2250907338edcbca2d5.
* Fix ray.internal.free
* Fixes and skip some flaky tests
* fix java build
* fix windows build
* Add IDs contained in owned objects
* Update src/ray/protobuf/core_worker.proto
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/core_worker/reference_count.cc
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/protobuf/core_worker.proto
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/protobuf/core_worker.proto
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/core_worker/reference_count.h
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/core_worker/reference_count.h
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Update src/ray/core_worker/reference_count.cc
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* Apply suggestions from code review
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
* update
* Try to fix ::test_direct_call_serialized_id_eviction
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
* Revert "Revert "Use Boost.Process instead of pid_t (#6510)" (#6909)"
This reverts commit bde575b8dd.
* Process wrapper, using Boost.Process on Windows
- Reverts bde575b8dd.
- Re-applies fb8e3615d5 after some refactoring.
* Remove Boost.Process dependency
* Don't open /proc file on Linux
* Change FATAL to ERROR and modify error message when process doesn't exist
* Avoid warning about swap being unlimited
Currently we get the following message on Jenkins:
"Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap."
Since we're not limiting swap anyway, we might as well avoid trying to.
https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details
* Fix escaping in re.search()
* Fix escaping in _noisy_layer()
* Raise a more descriptive error when dashboard data isn't found
* Don't error on dashboard files not being found when webui isn't required
* Change dashboard error to a warning instead
