So as to further generalize and separate out the abstract kernel manager class
from its various implementations.
* jupyter-kernel-manager.el (jupyter-meta-kernel): Fix documentation.
(jupyter-kernel-process, jupyter-command-kernel, jupyter-spec-kernel):
Move related functions and methods to `jupyter-kernel-process-manager.el`
(jupyter-kernel-manager-base): Remove class.
(jupyter-kernel-manager): Re-purpose class as the base class of all kernel
managers. The class corresponding to the old functionality is now named
`jupyter-kernel-process-manager`. Move all related functions and methods for
`jupyter-kernel-process-manager` to `jupyter-kernel-process-manager.el`.
* jupyter-kernel-process-manager.el: New file.
* jupyter-repl.el: Require `jupyter-kernel-process-manager` instead of
`jupyter-kernel-manager`.
* jupyter-server.el (jupyter-server-kernel-manager): Inherit from
`jupyter-kernel-manager` instead of `jupyter-kernel-manager-base`.
* test/jupyter-test.el (jupyter-kernel-lifetime)
(jupyter-command-kernel): Use `jupyter-kernel-process-manager` instead of
`jupyter-kernel-manager`.
* test/test-helper.el: Require `jupyter-kernel-process-manager` instead of
`jupyter-kernel-manager`.
* jupyter-env.el (jupyter-runtime-directory): Remove custom status.
New function definition.
* jupyter-kernel-manager.el (jupyter-write-connection-file):
Use new function.
* ob-jupyter.el (org-babel-jupyter--run-repl):
Remove setting of `jupyter-runtime-directory`.
* test/jupyter-test.el (jupyter-runtime-directory): New test.
This change localizes all `zmq` related functionality to `jupyter-ioloop` and
`jupyter-zmq-*` files.
* jupyter-channel-ioloop-comm.el: Add better commentary.
(jupyter-base): Require.
(jupyter-channel-ioloop-comm): Add `ioloop-class` slot
(initialize-instance [jupyter-channel-ioloop-comm]): Use it.
* jupyter-channel-ioloop.el (jupyter-base, jupyter-zmq-channel): Un-require.
(jupyter-ioloop-session, jupyter-ioloop-channels):
Rename to `jupyter-channel-ioloop-session` `jupyter-channel-ioloop-channels` and
update all callers.
(jupyter-channel-ioloop): Make into an abstract class.
(initialize-instance [jupyter-channel-ioloop]): Re-add
`jupyter-channel-ioloop-add-send-event`. Don't add to
`jupyter-ioloop-post-hook`.
(jupyter-channel-ioloop-recv-messages): Remove.
(jupyter-channel-ioloop--set-session, jupyter-ioloop-start)
(jupyter-channel-ioloop-add-send-event): Doc changes.
(jupyter-channel-ioloop-add-start-channel-event)
(jupyter-channel-ioloop-add-stop-channel-event):
Don't add/remove from the `jupyter-ioloop-poller`.
Now expected to be handled in the `jupyter-channel` subclass.
Update documentation. In addition, for the start-channel event,
do not attempt to add a channel if one doesn't already exist.
* jupyter-ioloop.el
(jupyter-ioloop-add-teardown):
Remove mention of `jupyter-channel-ioloop` behavior.
(jupyter-ioloop-add-arg-type): Update example variable.
(jupyter-ioloop-environment-p): New function.
* jupyter-kernel-manager.el (jupyter-channel): Require.
(jupyter-make-client): Require and use `jupyter-zmq-channel-ioloop`.
(jupyter-start-channels): Use `make-instance`.
(jupyter-interrupt-kernel): Remove `condition-case`. Not needed since
preventing socket blocking is now handled by `jupyter-recv`.
* jupyter-repl.el
(jupyter-connect-repl): Require and use `jupyter-zmq-channel-ioloop`.
* jupyter-zmq-channel-ioloop.el: New file.
* jupyter-zmq-channel.el (jupyter-ioloop-poller-remove)
(jupyter-ioloop-poller-add): New declares.
(jupyter-start-channel):
Add to `jupyter-ioloop-poller` when in `jupyter-ioloop-environment-p`.
(jupyter-stop-channel):
Only disconnect the socket from its endpoint instead of closing it, leave that
up to garbage collection.
Remove from `jupyter-ioloop-poller` when in `jupyter-ioloop-environment-p`.
(jupyter-recv): Handle non-blocking.
* test/jupyter-test.el
(jupyter-zmq-channel): Use non-blocking `zmq-send` since socket is no longer
closed when calling `jupyter-stop-channel`.
(jupyter-ioloop-test-eval-ioloop): Rename to `jupyter-test-ioloop-eval-event`,
update all callers, and move to `test/test-helper.el`.
(jupyter-channel-ioloop-send-event, jupyter-channel-ioloop-stop-channel-event)
(jupyter-channel-ioloop-start-channel-event): Fix tests for variable name
changes. Use `jupyter-test-channel-ioloop`. Update `jupyter-ioloop-poller`,
addition/removal from poller is now done in the `jupyter-channel` subclass by
checking `jupyter-ioloop-environment-p`.
* test/test-helper.el (jupyter-zmq-channel-ioloop): Require.
(initialize-instance [jupyter-echo-client]): Use it.
(jupyter-test-channel-ioloop): New macro.
(jupyter-test-ioloop-eval-event): New function.
* jupyter-channel-ioloop.el (jupyter-zmq-channel): Require.
(jupyter-channel-ioloop-add-start-channel-event): `sync` -> `zmq`
* jupyter-comm-layer (jupyter-comm--channel): Moved from `jupyter-channels.el`.
* jupyter-kernel-manager.el: `sync` -> `zmq`
* jupyter-zmq-channel-comm.el: New file.
* jupyter-channels.el: Mostly renamed to jupyter-zmq-channel.el. The
`jupyter-channel` class was moved to `jupyter-channel.el`. All that remains are
those classes dependent on `zmq`.
* test/jupyter-test.el: `sync` -> `zmq` where appropriate.
Extract `jupyter-channel` class from `jupyter-zmq-channel.el` into its own file
Having the `jupyter-comm-layer` abstraction means we do not need to do so.
* jupyter-base.el (zmq): Un-require.
(jupyter-socket-types): Move to `jupyter-channels.el`.
(jupyter-session): Don't mention zmq in doc string.
(jupyter-available-local-ports, jupyter-make-ssh-tunnel): New functions.
(jupyter-tunnel-connection): Use them.
* jupyter-channel-ioloop-comm.el: New file.
* jupyter-channels.el (jupyter-messages): Un-require.
(jupyter-comm-layer, zmq): New requires.
(jupyter-socket-types): Moved from `jupyter-base.el`.
(jupyter-send, jupyter-recv):
Implementations for `jupyter-session` moved from `jupyter-messages.el`.
(jupyter-sync-channel-comm): `jupyter-comm-layer` implementation for
`jupyter-sync-channel` objects moved from `jupyter-comm-layer.el`.
* jupyter-comm-layer.el (jupyter-channel-ioloop): Un-require.
(jupyter-sync-channel-comm): Move implementation to `jupyter-channels.el`.
(jupyter-ioloop-comm): Move implementation to new file `jupyter-ioloop-comm.el`.
(jupyter-channel-ioloop-comm):
Move implementation to new file `jupyter-channel-ioloop-comm.el`.
* jupyter-ioloop-comm.el: New file.
* jupyter-ioloop.el (zmq): Require.
* jupyter-kernel-manager.el
(jupyter-make-client): Ensure `jupyter-channel-ioloop-comm` is required.
* jupyter-messages.el (jupyter-send)
(jupyter-recv): Moved to `jupyter-channels.el`
* jupyter-repl.el
(jupyter-connect-repl): Ensure `jupyter-channel-ioloop-comm` is required.
* test/jupyter-test.el (jupyter-available-local-ports): New test.
* test/test-helper.el (jupyter-channel-ioloop-comm): New require.
Fixes#137
* jupyter-base.el (jupyter-kernelspec): Un-require.
(jupyter-command, jupyter-locate-python)
(jupyter-runtime-directory): Move to new file `jupyter-env.el`
(jupyter-include-other-output, jupyter-iopub-message-hook)
(jupyter-shell-message-hook)
(jupyter-stdin-message-hook): Move to `jupyter-client.el`
(jupyter-sha256, jupyter-hmac-sha256):
(jupytern-new-uuid): Move to `jupyter-messages.el`. Add declaration of
`jupyter-new-uuid` to account for its removal.
(jupyter-create-connection-info)
(jupyter-write-connection-file): Move to `jupyter-kernel-manager.el`
(jupyter-connect-endpoint, jupyter-connect-channel): Move to `jupyter-channels.el`
* jupyter-channels.el: Accept moved functions.
* jupyter-client.el: Accept moved variables.
* jupyter-kernel-manager.el: Accept moved functions.
(jupyter-env, jupyter-kernelspec): New requires.
* jupyter-kernelspec (jupyter-env): New require.
(jupyter-command): Remove declaration.
(jupyter-read-plist-from-string): New declaration.
* jupyter-messages.el: Accept moved functions.
(hmac-def, json): New requires.
* jupyter-org-extensions.el (jupyter-kernelspec): New require.
* jupyter-repl.el (jupyter-kernelspec): New require.
* jupyter-env.el: New file.
* ob-jupyter.el (jupyter-env, jupyter-kernelspec): New requires.
* test/jupyter-test.el (jupyter-env): New require.
Not really sure why this was done in the first place, but a kernel can be
forcibly killed and then asked to started up again, e.g. when restarting the
kernel.
If method arguments are not named it gives rise to errors like
`Args out of range: "", 0`
that originate in `help-function-arglist` when calling `describe-function` on
those methods.
This refactor implements a new class hierarchy to manage the lifetime of a
Jupyter kernel. The first node in this hierarchy is the
`jupyter-kernel-lifetime` class which defines a set of methods to manage the
lifetime of a kernel. An object that inherits from `jupyter-kernel-lifetime` is
stating that it has an association with a kernel and can be used to manage the
lifetime of the associated kernel.
The `jupyter-meta-kernel` class inherits from `jupyter-kernel-lifetime` and
mainly defines a `spec` slot used to hold the `kernelspec` from which a command
can be constructed to start a kernel and a `session` slot used to hold the
`jupyter-session` object that clients can use to establish communication with a
kernel once its live. Concrete classes that actually launch kernels are
intended to inherit from this class and use its slots.
`jupyter-kernel-process` manages the lifetime of a kernel started as a process
using the function `start-file-process`, `jupyter-command-kernel` calls the
`jupyter kernel` shell command to start a kernel, finally `jupyter-spec-kernel`
uses the `spec` slot to construct a shell command to start a kernel.
A `jupyter-kernel-manager` now consists of a `kernel` slot that holds a
`jupyter-meta-kernel` and a `control-channel` slot and inherits from
`jupyter-kernel-lifetime`. The `jupyter-kernel-lifetime` methods of the manager
just defer to those of `kernel` while also taking into account the
`control-channel`.
* jupyter-base.el (jupyter-write-connection-file): New function.
* jupyter-channel-ioloop.el
(jupyter-channel-ioloop-add-start-channel-event): Remove `sleep-for` call.
The startup message is not so important anymore.
* jupyter-client.el (jupyter-wait-until-startup: New function.
* jupyter-kernel-manager.el (jupyter-kernel-lifetime)
(jupyter-kernel, jupyter-kernel-process, jupyter-command-kernel)
(jupyter-spec-kernel): New classes.
(jupyter-kernel-manager): Inherit from jupyter-kernel-lifetime only and
implement its methods.
(jupyter-kernel-manager--cleanup, jupyter-kernel-managers)
(jupyter-delete-all-kernels, jupyter--kernel-sentinel)
(jupyter--start-kernel): Remove and remove related, their functionality has
been generalized in the new classes.
(jupyter-interrupt-kernel, jupyter-shutdown-kernel)
(jupyter-start-channels, jupyter-start-kernel, jupyter-kernel-alive-p)
(jupyter-kill-kernel): Refactor and implement to use the new class hierarchy.
* test/jupyter-test.el: Refactor tests to account for changes.
(jupyter-write-connect-file, jupyter-command-kernel): New tests.
* jupyter-kernelspec.el (jupyter-guess-kernelspec): New function.
The previous mechanism to communicate with a kernel was too low level from the
perspective of a client. The client interfaced directly with the subprocess
abstraction, `jupyter-ioloop`, and had to handle all "events" that occurred in
the `jupyter-ioloop`, e.g. when a channel was started or stopped. But in
reality such events should not be the concern of a client.
A client should only care about events that are directly related to kernel
messages and not events related to the implementation details of *how*
communication occurs.
This commit abstracts out the way in which a client communicates with its
kernel by introducing a new `jupyter-comm-layer` class. The
`jupyter-comm-layer` class takes care of managing the communication channel
between a kernel and its clients as well as sending events to all registered
clients. This way, clients operate solely at the level of events on the
communication layer. All a client does is register itself to receive events on
the communication layer and send events on the layer.
* jupyter-base.el (jupyter-session-endpoints): New function.
* jupyter-client.el (jupyter-kernel-client): Remove ioloop and channels slots.
Add kcomm slot.
(initialize-instance): Unconditionally stop channels.
(jupyter-initialize-connection): Change into a method call.
Call `jupyter-initialize-connection` on the `kcomm` slot.
(jupyter-with-client-buffer): Remove stale comment.
(jupyter-send): Call `jupyter-send` on the `kcomm` slot.
(jupyter-ioloop-handler): Remove all method definitions, replace `sent` and
`message` methods with their `jupyter-event-handler` equivalents.
(jupyter-hb-pause, jupyter-hb-unpause, jupyter-hb-beating):
(jupyter-channel-alive-p, jupyter-start-channel, jupyter-stop-channel):
(jupyter-start-channels, jupyter-stop-channels):
Replace with calls to their equivalents using the `kcomm` slot.
* jupyter-comm-layer.el: New file.
* jupyter-kernel-manager (jupyter-make-client): Set a client's `kcomm` slot to
`jupyter-channel-ioloop-comm`.
* jupyter-messages.el (jupyter-decode-message): Use `list` directly. There
seemed to be issues when using the new `jupyter-sync-channel-comm` due to
using quoted lists.
* test/jupyter-test.el: Add `jupyter-comm-layer` test. Update other tests.
* test/test-helper.el: Add `jupyter-comm-layer` mock objects. Update
`jupyter-echo-client`.
See https://github.com/dzop/emacs-jupyter/issues/70#issuecomment-476940527
* jupyter-kernel-manager.el
(jupyter--kernel-sentinel): Make ignored argument optional.
(jupyter--start-kernel): Don't set the process sentinel.
(jupyter-start-kernel): Report any errors after timeout.
Set the process sentinel as a last step.
When a kernel restarts, `jupyter-start-kernel` will overwrite the slots
corresponding to external resources of the old, shutdown, kernel, like the
connection file. Ensure resources are cleaned up before overwriting those
slots.
