ray/dashboard/modules/runtime_env
Edward Oakes 10947c83b3
[runtime_env] Make pip installs incremental (#20341)
Uses a direct `pip install` instead of creating a conda env to make pip installs incremental to the cluster environment.

Separates the handling of `pip` and `conda` dependencies.

The new `pip` approach still works if only the base Ray is installed on the cluster and the user specifies libraries like "ray[serve]" in the `pip` field.  The mechanism is as follows:
- We don't actually want to reinstall ray via pip, since this could lead to version mismatch issues.  Instead, we want to use the Ray that's already installed in the cluster.
- So if "ray" was included by the user in the pip list, remove it
- If a library "ray[serve]" or "ray[tune, rllib]" was included in the pip list, remove it and replace it by its dependencies (e.g. "uvicorn", "requests", ..)

Co-authored-by: architkulkarni <arkulkar@gmail.com>
Co-authored-by: architkulkarni <architkulkarni@users.noreply.github.com>
2021-12-14 15:55:18 -08:00
..
__init__.py [runtime env] support create or delete runtime envs in agent (#15904) 2021-06-09 20:22:25 +08:00
runtime_env_agent.py [runtime_env] Make pip installs incremental (#20341) 2021-12-14 15:55:18 -08:00
runtime_env_consts.py [runtime env] Download runtime env(conda) in agent instead of setup_worker (#16525) 2021-06-25 19:39:05 +08:00