Clean up the ci/ directory. This means getting rid of the travis/ path completely and moving the files into sensible subdirectories.
Details:
- Moves everything under ci/travis into subdirectories, e.g. ci/build, ci/lint, etc.
- Minor adjustments to some scripts (variable renames)
- Removes the outdated (unused) asan tests
`PATH` is easy to be changed in a terminal session. Different `$PATH` values lead to miss of bazel cache. e.g. `pip install python -e` and `bazel build //:all` don't share cache because Python modifies `PATH`.
`LC_ALL`, `LANG`, and python-related environment variables are only used by C++ worker tests, which invokes the `ray start` command when running tests with `bazel test`. Java worker is not affected because we don't use `bazel test` to run Java tests. So these env variables should stay `test_env`, not `action_env`.
This PR can greatly improve the cache hit rate of Bazel build and test.
Enables lineage reconstruction, which allows automatic recovery of task outputs, by default.
Also adds an info message to the driver whenever objects need to be reconstructed (not including recursive reconstruction).
## Why are these changes needed?
Ray wheel size limit is still at 100MB. Removing debug symbols would decrease Ray Linux wheel sizes.
## Related issue number
## Checks
* Revert "[CI] Remove config that disables Bazel test result cache (#18701)"
This reverts commit 098ff36faa.
* Remove all RLlib tests from BUILD that currently fail.
Co-authored-by: sven1977 <svenmika1977@gmail.com>
* exp backoff
* up
* format
* up
* up
* up
* up
* up
* format
* fix
* up
* format
* adjust ordering
* up
* Revert "[tune] Cache unstaged placement groups for potential re-use (#18706)"
This reverts commit 2e99fb215f.
* up
* update
* format
* up
* format
* fix
* Revert "Revert "[tune] Cache unstaged placement groups for potential re-use (#18706)""
This reverts commit 93425fdb986059e53699623a0fc8590c062e139b.
* up
* format
* fix lint
* up
* up
* up
* up
* check
* add test1
* format
* up
* add test
* up
* up
* up
* fix
* up
* up
* up
* add test
* format
* up
* up
* fix lint
* format
* fix
* format
* fix
* up
* Revert "Revert "[Build] include minimal debug info in C++ build; upgrade clang-format to 12 (#18840)" (#18886)"
This reverts commit f851a072f3.
* use gcc 8
* Add Bazel config for building with llvm. Upgrade C++ std to 17.
* Fix redis. Try fixing asan and tsan
* Fix asan and format
* Update comments.
Co-authored-by: Chen Shen <scv119@gmail.com>
* in progerss
* ASAN tests.
* d
* in progress
* in progress without the asan wheel
* Support the asan wheel.
* Support the asan wheels
* Not build a binary for asan
* Fix issues
* Remove a wrong build
* Separate out asan wheel build
* Try preparing more deps.
* ip
* Try different version
* done
* d
* Trial
* Another try
* Another try
* skip cpp build to see what happens
* add more des
* ip
* abc
* Try next
* completed
* try
* Try without static libasan
* dbg
* Try static link
* Fix issues
* abc
* Remove a flag and some methods in node manager including HandleDisconnectedActor, ResubmitTask, and HandleTaskReconstruction
* Make actor creator always required + remove raylet transport
* Remove actor reporter + remove FinishAssignedActorCreationTask
* Remove actor tasks.
* Remove finishactortask and switched it to finishactorcreation task
* Remove reconstruction policy.
* Remove lineage cache.
* Formatting.
* Remove actor frontier code.
* Removed build error.
* Revert "Remove reconstruction policy."
This reverts commit 9d25c9bced4da5fbcac5d484d51013345f16513b.
* Recover HandleReconstruction to mark expired objects as failed.
* Factor out --keep_going in Bazel --config=ci
* Remove Bazel --test_timeout=600 for Windows
* Use global --test_output for Bazel CI
Co-authored-by: Mehrdad <noreply@github.com>
* Upload wheels to latest directory as well on GitHub Actions
* Fix bug in install-dependencies.sh
* Move out bazel build //:* from install_ray, since it isn't really necessary for that purpose
* Build wheels for different versions of Python on Windows
* Compile Windows in opt mode
Co-authored-by: Mehrdad <noreply@github.com>
