The previous mechanism to communicate with a kernel was too low level from the
perspective of a client. The client interfaced directly with the subprocess
abstraction, `jupyter-ioloop`, and had to handle all "events" that occurred in
the `jupyter-ioloop`, e.g. when a channel was started or stopped. But in
reality such events should not be the concern of a client.
A client should only care about events that are directly related to kernel
messages and not events related to the implementation details of *how*
communication occurs.
This commit abstracts out the way in which a client communicates with its
kernel by introducing a new `jupyter-comm-layer` class. The
`jupyter-comm-layer` class takes care of managing the communication channel
between a kernel and its clients as well as sending events to all registered
clients. This way, clients operate solely at the level of events on the
communication layer. All a client does is register itself to receive events on
the communication layer and send events on the layer.
* jupyter-base.el (jupyter-session-endpoints): New function.
* jupyter-client.el (jupyter-kernel-client): Remove ioloop and channels slots.
Add kcomm slot.
(initialize-instance): Unconditionally stop channels.
(jupyter-initialize-connection): Change into a method call.
Call `jupyter-initialize-connection` on the `kcomm` slot.
(jupyter-with-client-buffer): Remove stale comment.
(jupyter-send): Call `jupyter-send` on the `kcomm` slot.
(jupyter-ioloop-handler): Remove all method definitions, replace `sent` and
`message` methods with their `jupyter-event-handler` equivalents.
(jupyter-hb-pause, jupyter-hb-unpause, jupyter-hb-beating):
(jupyter-channel-alive-p, jupyter-start-channel, jupyter-stop-channel):
(jupyter-start-channels, jupyter-stop-channels):
Replace with calls to their equivalents using the `kcomm` slot.
* jupyter-comm-layer.el: New file.
* jupyter-kernel-manager (jupyter-make-client): Set a client's `kcomm` slot to
`jupyter-channel-ioloop-comm`.
* jupyter-messages.el (jupyter-decode-message): Use `list` directly. There
seemed to be issues when using the new `jupyter-sync-channel-comm` due to
using quoted lists.
* test/jupyter-test.el: Add `jupyter-comm-layer` test. Update other tests.
* test/test-helper.el: Add `jupyter-comm-layer` mock objects. Update
`jupyter-echo-client`.
This also removes the use of `shell-command-to-string`, replacing it directly
with `process-file`. `shell-command-to-string` uses `shell-file-name` directly
which may not work properly when `default-directory` is a remote directory as
the remote directory may have a different shell than the local one. See #75.
For this case, no tunnels actually need to be created since docker can expose
ports using the -p flag to `docker run`. This is assumed here. A complete
example to start a kernel in a docker container using one of the official
Jupyter docker images is below:
docker pull jupyter/base-notebook
docker run --name test -it --rm -p 56406-56410:56406-56410 \
jupyter/base-notebook start.sh jupyter-kernel \
--ip=0.0.0.0 \
--KernelManager.control_port=56406 \
--KernelManager.hb_port=56407 \
--KernelManager.iopub_port=56408 \
--KernelManager.shell_port=56409 \
--KernelManager.stdin_port=56410
`accept-process-output` will return when any output has been read from a
process (after it has been handled by the process filter function) which is
usually the time when we want to check on the wait condition so there is no
need to have such a low time resolution.
This is to give packages like window-purpose a chance to work since passing in
the action alist directly seems to override the behavior of such packages.
See #27.
This is needed as opposed to eieio-instance-tracker so that we can create weak
references to objects. We don't want to have to manually delete an instance.
* jupyter-base.el (jupyter-instance-tracker): Do it.
(jupyter-all-objects): New function.
* jupyter-client.el (jupyter--clients): New variable.
(jupyter-kernel-client): Inherit from new class.
(jupyter-clients): Use jupyter-all-objects.
* jupyter-kernel-manager.el (jupyter-kernel-managers): New function.
* jupyter-repl.el (jupyter-output-buffer-marker, jupyter-output-buffer-request-id):
(jupyter--reset-output-buffer-p, jupyter-with-output-buffer): Do it.
(jupyter-repl-get-special-buffer): Do it and rename to jupyter-get-buffer-create.
* jupyter-client.el: See above.
* jupyter-base.el (jupyter-default-timeout)
(jupyter-long-timeout): Moved from jupyter-client.el
(jupyter-with-timeout): Do it.
* jupyter-client.el (jupyter-wait-until): Use it.
* jupyter-kernel-manager.el (jupyter-shutdown-kernel)
(jupyter-interrupt-kernel, jupyter-start-new-kernel):
(jupyter-start-kernel): Use it.
use timeout
The goal of this method is to act as a single entry point for insertion of
kernel results in any context. One would simply add another method to handle a
specific context.
* jupyter-base.el (jupyter-mime-types):
(jupyter-nongraphic-mime-types): New variables that give mime-types that can be
handled.
(jupyter-insert): New method for dispatching to code that inserts mimetype
representations in the current buffer.
* jupyter-mime.el: New file.
(jupyter-display-ids):
(jupyter-handle-control-codes):
(jupyter-fontify-buffers):
(jupyter-get-fontify-buffer):
(jupyter-fixup-font-lock-properties):
(jupyter-add-font-lock-properties):
(jupyter-fontify-according-to-mode):
(jupyter-insert-html):
(jupyter-markdown-mouse-map):
(juputer-markdown-follow-link-at-point):
(jupyter-insert-markdown):
(jupyter-insert-latex):
(jupyter-insert-ansi-coded-text): Moved from jupyter-repl.el, replaced
`jupyter-repl-` prefix with `jupyter-`.
(jupyter--shr-put-image): Ditto. Also add `shr-` prefix.
(jupyter--delete-javascript-tags): Ditto. Also mark as private functions.
(jupyter-insert-image): Ditto. Also mark as a public function.
(jupyter-insert): (DISPLAY-ID ...) Moved from jupyter-repl.el. Was
`jupyter-repl-insert-data-with-id`.
(jupyter-with-control-code-handling):
(jupyter-markdown-follow-link): Moved from jupyter-repl.el
(jupyter-insert): Implement methods to do the work previously done by
`jupyter-repl-insert-data`.
* jupyter-repl.el (jupyter-repl-graphic-mimetypes): Moved to jupyter-base.el,
inverted and renamed to `jupyter-nongraphic-mime-types`.
(jupyter-repl-graphic-data-p): Remove unused function.
(jupyter-repl-insert-data): Remove, replace calls with `jupyter-insert`.
(jupyter-repl-add-font-lock-properties):
(jupyter-repl-fixup-font-lock-properties):
(jupyter-repl-get-fontify-buffer):
(jupyter-repl-fontify-according-to-mode):
(jupyter-repl-delete-javascript-tags):
(jupyter-repl-put-image):
(jupyter-repl-insert-html):
(jupyter-repl-markdown-mouse-map):
(jupyter-repl-markdown-follow-link-at-point):
(jupyter-repl-insert-markdown):
(jupyter-repl-insert-latex):
(jupyter-repl--insert-image): Moved to jupyter-mime.el, which see.
(jupyter-repl-insert-data-with-id): Ditto. Changed to a `jupyter-insert` method
dispatched on a string argument.
(jupyter-repl-insert-ansi-coded-text): Ditto. Replace calls with
`jupyter-insert-ansi-coded-text`.
(jupyter-with-control-code-handling):
(jupyter-markdown-follow-link): Moved to jupyter-mime.el.
* jupyter-org-client.el (jupyter-handle-error): Replace
`jupyter-repl-insert-ansi-coded-text` with `jupyter-insert-ansi-coded-text`.
* jupyter-tests.el (jupyter-insert): Add tests for `jupyter-insert`
This function will prove useful in the future for applying text properties to
inserted text once the new `jupyter-insert` method is introduced.
* jupyter-base.el (jupyter-with-insertion-bounds): Add the new macro.
* jupyter-repl.el (jupyter-with-control-code-handling): Use it.
* Declare undeclared external functions
* Add ext: prefix to filename of external packages that may not be present on
every system
* Remove `company-grab-symbol-cons` function declaration since this function is
no longer used.
This replaces the behavior of the `:input-reply` message
type. A kernel sends an `:input-request` to the frontend
and the client sends an `:input-reply`.
Since IDs of requests are generated in the parent Emacs process, there is no
need to pass back the ID of a request from the child process that send
messages.
