Why are these changes needed?
Wasn't able to build Ray following these instructions : https://docs.ray.io/en/latest/ray-contribute/development.html#building-ray
It fails to run
pip install -e . --verbose # Add --user if you see a permission denied error.
I have a local installation of protobuf via Homebrew and bazel is using its headers against the protobuf that it is pulling into its sandbox. It is a known issue with bazel and one of the workarounds is to block the local dir so it doesn't accidentally pick up the header if someone happens to have it installed locally.
Manually tested by running
bazel build --verbose_failures --sandbox_debug -- //:ray_pkg
Without the fix I would get an error similar to https://gist.github.com/clarng/ff7b7bf7672802d1e3e07b4b509e4fc8
With the fix it builds
Show usage stats prompt when it's enabled.
Current UX are:
* The usage stats enabled or disabled message is shown every time in both terminal and dashboard.
* If users don't explicitly enable or disable usage stats, the first time they start a ray cluster interactively, they will be asked to confirm and will enable if no user action within 10s. If it's non-interactive, collection is enabled by default without confirmation.
* ray.init() doesn't collect usage stats
* Usage stats can be disabled via three approaches: 1. RAY_USAGE_STATS_ENABLED env var, 2. ray xxx --disable-usage-stats, 3. ray disable-usage-stats
Clean up the ci/ directory. This means getting rid of the travis/ path completely and moving the files into sensible subdirectories.
Details:
- Moves everything under ci/travis into subdirectories, e.g. ci/build, ci/lint, etc.
- Minor adjustments to some scripts (variable renames)
- Removes the outdated (unused) asan tests
`PATH` is easy to be changed in a terminal session. Different `$PATH` values lead to miss of bazel cache. e.g. `pip install python -e` and `bazel build //:all` don't share cache because Python modifies `PATH`.
`LC_ALL`, `LANG`, and python-related environment variables are only used by C++ worker tests, which invokes the `ray start` command when running tests with `bazel test`. Java worker is not affected because we don't use `bazel test` to run Java tests. So these env variables should stay `test_env`, not `action_env`.
This PR can greatly improve the cache hit rate of Bazel build and test.
Enables lineage reconstruction, which allows automatic recovery of task outputs, by default.
Also adds an info message to the driver whenever objects need to be reconstructed (not including recursive reconstruction).
## Why are these changes needed?
Ray wheel size limit is still at 100MB. Removing debug symbols would decrease Ray Linux wheel sizes.
## Related issue number
## Checks
* Revert "[CI] Remove config that disables Bazel test result cache (#18701)"
This reverts commit 098ff36faa.
* Remove all RLlib tests from BUILD that currently fail.
Co-authored-by: sven1977 <svenmika1977@gmail.com>
* exp backoff
* up
* format
* up
* up
* up
* up
* up
* format
* fix
* up
* format
* adjust ordering
* up
* Revert "[tune] Cache unstaged placement groups for potential re-use (#18706)"
This reverts commit 2e99fb215f.
* up
* update
* format
* up
* format
* fix
* Revert "Revert "[tune] Cache unstaged placement groups for potential re-use (#18706)""
This reverts commit 93425fdb986059e53699623a0fc8590c062e139b.
* up
* format
* fix lint
* up
* up
* up
* up
* check
* add test1
* format
* up
* add test
* up
* up
* up
* fix
* up
* up
* up
* add test
* format
* up
* up
* fix lint
* format
* fix
* format
* fix
* up
* Revert "Revert "[Build] include minimal debug info in C++ build; upgrade clang-format to 12 (#18840)" (#18886)"
This reverts commit f851a072f3.
* use gcc 8
* Add Bazel config for building with llvm. Upgrade C++ std to 17.
* Fix redis. Try fixing asan and tsan
* Fix asan and format
* Update comments.
Co-authored-by: Chen Shen <scv119@gmail.com>
* in progerss
* ASAN tests.
* d
* in progress
* in progress without the asan wheel
* Support the asan wheel.
* Support the asan wheels
* Not build a binary for asan
* Fix issues
* Remove a wrong build
* Separate out asan wheel build
* Try preparing more deps.
* ip
* Try different version
* done
* d
* Trial
* Another try
* Another try
* skip cpp build to see what happens
* add more des
* ip
* abc
* Try next
* completed
* try
* Try without static libasan
* dbg
* Try static link
* Fix issues
* abc
* Remove a flag and some methods in node manager including HandleDisconnectedActor, ResubmitTask, and HandleTaskReconstruction
* Make actor creator always required + remove raylet transport
* Remove actor reporter + remove FinishAssignedActorCreationTask
* Remove actor tasks.
* Remove finishactortask and switched it to finishactorcreation task
* Remove reconstruction policy.
* Remove lineage cache.
* Formatting.
* Remove actor frontier code.
* Removed build error.
* Revert "Remove reconstruction policy."
This reverts commit 9d25c9bced4da5fbcac5d484d51013345f16513b.
* Recover HandleReconstruction to mark expired objects as failed.
* Factor out --keep_going in Bazel --config=ci
* Remove Bazel --test_timeout=600 for Windows
* Use global --test_output for Bazel CI
Co-authored-by: Mehrdad <noreply@github.com>
