[Core] install jemalloc in Ray docker and use jemalloc in benchmark release tests (#26112)

There are mysterious memory usage growth in Ray clusters that disappear when running with jemalloc. Before we are able to figure out the root cause, it seems using jemalloc by default can be a good walkaround. Because of its efficiency, using jemalloc by default can be beneficial, but we need to run more benchmarks to verify.
This commit is contained in:
mwtian 2022-06-27 23:26:56 -07:00 committed by GitHub
parent 58bfad84d3
commit 513881584d
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 5 additions and 1 deletions

View file

@ -30,6 +30,7 @@ ENV HOME=/home/ray
RUN sudo apt-get update -y && sudo apt-get upgrade -y \ RUN sudo apt-get update -y && sudo apt-get upgrade -y \
&& sudo apt-get install -y \ && sudo apt-get install -y \
git \ git \
libjemalloc-dev \
wget \ wget \
cmake \ cmake \
g++ \ g++ \

View file

@ -1,5 +1,8 @@
base_image: {{ env["RAY_IMAGE_ML_NIGHTLY_GPU"] | default("anyscale/ray-ml:nightly-py37-gpu") }} base_image: {{ env["RAY_IMAGE_ML_NIGHTLY_GPU"] | default("anyscale/ray-ml:nightly-py37-gpu") }}
env_vars: {} env_vars: {"LD_PRELOAD": "/usr/lib/x86_64-linux-gnu/libjemalloc.so"}
debian_packages:
- libjemalloc-dev
python: python:
pip_packages: [] pip_packages: []