hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-06 10:31:39 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

[doc] Troubleshooting --dashboard-port (#11816)

Browse source
This commit is contained in:
dHannasch 2020-11-08 16:53:50 -07:00 • committed by GitHub
parent 0932320eb3
commit 64ca30c060
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 40 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

41
doc/source/configure.rst
View file

@ -136,9 +136,48 @@ In addition to ports specified above, the head node needs to open several more p
- ``--port``: Port of GCS. Default: 6379. - ``--port``: Port of GCS. Default: 6379.
- ``--redis-shard-ports``: Comma-separated list of ports for non-primary Redis shards. Default: Random values. - ``--redis-shard-ports``: Comma-separated list of ports for non-primary Redis shards. Default: Random values.
- ``--dashboard-port``: Port for accessing the dashboard. Default: 8265
- ``--gcs-server-port``: GCS Server port. GCS server is a stateless service that is in charge of communicating with the GCS. Default: Random value. - ``--gcs-server-port``: GCS Server port. GCS server is a stateless service that is in charge of communicating with the GCS. Default: Random value.
- If ``--include-dashboard`` is true (the default), then the head node must open ``--dashboard-port``. Default: 8265.
If ``--include-dashboard`` is true but the ``--dashboard-port`` is not open on
the head node, you will repeatedly get
.. code-block:: bash
WARNING worker.py:1114 -- The agent on node <hostname of node that tried to run a task> failed with the following error:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/grpc/aio/_call.py", line 285, in __await__
raise _create_rpc_error(self._cython_call._initial_metadata,
grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":4165,"referenced_errors":[{"description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":397,"grpc_status":14}]}"
(Also, you will not be able to access the dashboard.)
If you see that error, check whether the ``--dashboard-port`` is accessible
with ``nc`` or ``nmap`` (or your browser).
.. code-block:: bash
$ nmap -sV --reason -p 8265 $HEAD_ADDRESS
Nmap scan report for compute04.berkeley.edu (123.456.78.910)
Host is up, received reset ttl 60 (0.00065s latency).
rDNS record for 123.456.78.910: compute04.berkeley.edu
PORT STATE SERVICE REASON VERSION
8265/tcp open http syn-ack ttl 60 aiohttp 3.7.2 (Python 3.8)
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Note that the dashboard runs as a separate subprocess which can crash invisibly
in the background, so even if you checked port 8265 earlier, the port might be
closed *now* (for the prosaic reason that there is no longer a service running
on it). This also means that if that port is unreachable, if you ``ray stop``
and ``ray start``, it may become reachable again due to the dashboard
restarting.
If you don't want the dashboard, set ``--include-dashboard=false``.
Redis Port Authentication Redis Port Authentication
------------------------- -------------------------