- Add a `message-publisher` slot to `jupyter-request`, set when
sending a request.
- `*-request` functions now return a delayed request value instead of
a request. Update call sites.
- Add the functions `jupyter-messages`, `jupyter-idle`,
`jupyter-message-subscribed` and `jupyter-client-subscribed`.
- Move client subscription to a request's messages to
`jupyter-client-subscribed`.
- Request callbacks are now added via `jupyter-message-subscribed`.
- Reduce usage of callbacks in favor of searching `jupyter-messages`
where appropriate.
- Update tests
`jupyter-websocket-io` now returns, in addition to an IO function, a
kernel action subscriber. A client now sends messages to the
subscriber to interrupt or shutdown a kernel instead of doing so using
a `kernel` slot.
Also add the `jupyter-kernel-action` convenience function for
evaluating arbitrary code on a client's kernel.
* jupyter-kernel-process.el
* jupyter-server-kernel.el (jupyter-kernel): New method.
* jupyter-kernel.el (jupyter-kernel): Convert. Load files that handle
keywords and re-dispatch when ARGS does not have any connection
info. Document behavior.
* test/test-helper.el (jupyter-test-with-kernel-client)
* jupyter-server.el
(jupyter-connect-server-repl)
(jupyter-server-start-new-kernel): Use `jupyter-kernel`.
`(slot-boundp client 'manager)` always returns t since the `manager`
slot defaults to `nil` for every client. To check for a manager we
need to check for a non-nil `manager` slot.
* test/test-helper.el
(jupyter-test-with-client-cache): Check for a non-nil `manager` slot.
Remove `jupyter-channel-ioloop-comm.el`, `jupyter-comm-layer.el`, `jupyter-ioloop-comm.el`.
Remove the kcomm slot of a client, replace it with a kernel slot.
jupyter-server.el: Remove kernel communication related code.
Remove `jupyter-kernel-lifetime`.
* jupyter-kernel-manager.el (jupyter-kernel-lifetime): Do it. Update
all subclasses. All methods it defined no longer take a type. This
is in preparation of removing most of them altogether.
(jupyter-kernel-manager): Update doc.
* jupyter-kernel-process-manager.el
(jupyter-kernel-process): Update documentation.
(jupyter--kernel-died-process-sentinel): Remove type check.
* test/jupyter-test.el (jupyter-local-tcp-conn-info): Remove.
(jupyter-kernel-lifetime): Remove.
[conn] Have `jupyter-kernel-manager.el` re-direct to the new interface
* jupyter-kernel-manager.el
(jupyter-kernel-alive-p): Check that the kernel is
live, not an Emacs connection to it.
(jupyter-start-kernel)
[jupyter-kernel, jupyter-kernel-manager]: Use `jupyter-launch`.
(jupyter-shutdown-kernel) [jupyter-kernel-manager]: Use `jupyer-shutdown`.
(jupyter-kill-kernel) [jupyter-kernel]: Ditto. Remove a method
definition.
(jupyter-make-client): Use `jupyter-client`.
(jupyter-interrupt-kernel) [jupyter-kernel-manager]: Use `jupyer-interrupt`.
* jupyter-kernel-process-manager.el: Remove.
Don't add finalizer to cleanup process.
Cleanup is done in other ways, e.g. in
`jupyter--gc-kernel-processes`. Also it doesn't make sense to delete
a general kernel because the object that represents it in Emacs is no
longer accessible.
* jupyter-comm-layer.el
(jupyter-comm-initialize): Remove default method. This is in
preparation for moving over to classless communication.
* jupyter-channel-ioloop-comm (jupyter-connection): Require.
(jupyter--proxy-channel): New type.
(jupyter--make-channel-group, jupyter--channel-alive-p)
(jupyter--start-channel, jupyter--stop-channel)
(make-jupyter-async-connection): New functions.
(jupyter-channel-ioloop-comm): Remove `ioloop-class` slot, update all
callers. Remove `channels` slot, update all setters and
references. Add `conn` slot which holds a `jupyter-connection`.
(jupyter-comm-initialize): Initialize the `conn` slot to the
connection returned by `make-jupyter-async-connection`.
(jupyter-comm-start, jupyter-comm-stop)
(jupyter-comm-alive-p, jupyter-comm-id, jupyter-channel-alive-p)
(jupyter-stop-channel, jupyter-start-channel): Replace the body of these
functions with their equivalents in `conn`.
* jupyter-kernel-process-manager.el
(jupyter-make-client): Update `jupyter-channel-ioloop-comm` call.
* jupyter-repl.el:
(jupyter-connect-repl): Ditto.
* test/test-helper.el
(initialize-instance) [jupyter-echo-client]: Ditto. Replace setting of
`channel` slot with setting the `conn` clot.
The new files added in this commit will eventually replace the manager
and kernel classes and favor struct types to represent kernels instead
of classes. A kernel manager was a concept ripped from the
jupyter/jupyter_client reference implementation.
In Emacs the concept makes the client implementation more complicated
and is replaced by functions that manage the lifetime of a kernel:
`jupyter-launch`, `jupyter-shutdown`, and `jupyter-interrupt`.
* jupyter-kernel-manager.el
* jupyter-kernel-process-manager.el
* jupyter-server.el
* test/jupyter-server-test.el
* test/test-helper.el: Make `jupyter-kernel`, `jupyter-server-kernel`,
and `jupyter-kernel-process` private classes. The new files below
use the public names, but as structs now.
* jupyter-kernel.el
* jupyter-kernel-process.el
* jupyter-server-kernel.el: New files.
* jupyter-kernel-manager.el (jupyter-kernel): Set `spec` slot type to
`jupyter-kernelspec`.
(jupyter-kernel-name):
* jupyter-kernel-process-manager.el: Update all accessors.
* jupyter-kernelspec.el: Add `jupyter-kernelspec` struct type.
* jupyter-repl.el:
* jupyter-server.el:
* ob-jupyter.el:
* test/test-helper.el: Update all kernelspec using functions to use
the new type.
* jupyter-server.el (jupyter-ioloop, jupyter-server-ioloop): Remove requires.
(jupyter-server): Don't subclass `jupyter-ioloop-comm`.
(jupyter-server--connect-channels, jupyter-server--refresh-comm): Only do
something when the SERVER argument is a `jupyter-comm-layer`, i.e. a
`jupyter-server-ioloop-comm`. We don't check for that class directly since we
would end up requiring ZMQ.
(jupyter-comm-start) [jupyter-server-kernel-manager]: Use the right subclass of
`jupyter-server-abstract-kcomm` depending on if the `jupyter-server` object is
using ZMQ for communication.
(jupyter-server-make-instance): New function. Returns a `jupyter-server`
instance (a `jupyter-server-ioloop-comm`) that uses ZMQ if
`jupyter-server-use-zmq` is non-nil, and a plain old `jupyter-server` instance
otherwise.
(jupyter-current-server): `jupyter-server` -> `jupyter-server-make-instance`.
* ob-jupyter.el (org-babel-jupyter--server-repl): `jupyter-server` ->
`jupyter-server-make-instance`.
* test/test-helper.el (jupyter-test-with-notebook): `jupyter-server` ->
`jupyter-server-make-instance`.
* test/jupyter-test.el: Move all zmq related tests (and code) to
`test/jupyter-zmq-test.el`, add a `zmq` tag to the tests.
* test/test-helper.el (zmq, jupyter-zmq-channel-ioloop)
(jupyter-zmq-channel-ioloop-comm): Remove requires.
(jupyter-test-ioloop-eval-event, jupyter-test-channel-ioloop)
(initialize-instance) [jupyter-echo-client]: Remove uses of ZMQ by using
a `jupyter-mock-comm-layer` for the `kcomm` slot.
(jupyter-test-zmq-sockets): Move to `test/jupyter-zmq-test.el` along with all
related code.
* test/jupyter-zmq-test.el: Add tests moved from `test/jupyter-test.el` and all
related code.
Since we are using keyword to represent channels on the client level, use a
plist of channels on the comm layer level instead of channel slots. This allows
us to remove `jupyter-comm-channel` and simplify the relevant code somewhat.
So as to further generalize and separate out the abstract kernel manager class
from its various implementations.
* jupyter-kernel-manager.el (jupyter-meta-kernel): Fix documentation.
(jupyter-kernel-process, jupyter-command-kernel, jupyter-spec-kernel):
Move related functions and methods to `jupyter-kernel-process-manager.el`
(jupyter-kernel-manager-base): Remove class.
(jupyter-kernel-manager): Re-purpose class as the base class of all kernel
managers. The class corresponding to the old functionality is now named
`jupyter-kernel-process-manager`. Move all related functions and methods for
`jupyter-kernel-process-manager` to `jupyter-kernel-process-manager.el`.
* jupyter-kernel-process-manager.el: New file.
* jupyter-repl.el: Require `jupyter-kernel-process-manager` instead of
`jupyter-kernel-manager`.
* jupyter-server.el (jupyter-server-kernel-manager): Inherit from
`jupyter-kernel-manager` instead of `jupyter-kernel-manager-base`.
* test/jupyter-test.el (jupyter-kernel-lifetime)
(jupyter-command-kernel): Use `jupyter-kernel-process-manager` instead of
`jupyter-kernel-manager`.
* test/test-helper.el: Require `jupyter-kernel-process-manager` instead of
`jupyter-kernel-manager`.
* Start notebook on-demand during tests
* Do cleanup of sockets/processes when killing Emacs
* This should work around intermittent core dumps on Travis due to epoll
reconnect attempts.
* Simplify tests by creating and using setup/teardown macros
This change localizes all `zmq` related functionality to `jupyter-ioloop` and
`jupyter-zmq-*` files.
* jupyter-channel-ioloop-comm.el: Add better commentary.
(jupyter-base): Require.
(jupyter-channel-ioloop-comm): Add `ioloop-class` slot
(initialize-instance [jupyter-channel-ioloop-comm]): Use it.
* jupyter-channel-ioloop.el (jupyter-base, jupyter-zmq-channel): Un-require.
(jupyter-ioloop-session, jupyter-ioloop-channels):
Rename to `jupyter-channel-ioloop-session` `jupyter-channel-ioloop-channels` and
update all callers.
(jupyter-channel-ioloop): Make into an abstract class.
(initialize-instance [jupyter-channel-ioloop]): Re-add
`jupyter-channel-ioloop-add-send-event`. Don't add to
`jupyter-ioloop-post-hook`.
(jupyter-channel-ioloop-recv-messages): Remove.
(jupyter-channel-ioloop--set-session, jupyter-ioloop-start)
(jupyter-channel-ioloop-add-send-event): Doc changes.
(jupyter-channel-ioloop-add-start-channel-event)
(jupyter-channel-ioloop-add-stop-channel-event):
Don't add/remove from the `jupyter-ioloop-poller`.
Now expected to be handled in the `jupyter-channel` subclass.
Update documentation. In addition, for the start-channel event,
do not attempt to add a channel if one doesn't already exist.
* jupyter-ioloop.el
(jupyter-ioloop-add-teardown):
Remove mention of `jupyter-channel-ioloop` behavior.
(jupyter-ioloop-add-arg-type): Update example variable.
(jupyter-ioloop-environment-p): New function.
* jupyter-kernel-manager.el (jupyter-channel): Require.
(jupyter-make-client): Require and use `jupyter-zmq-channel-ioloop`.
(jupyter-start-channels): Use `make-instance`.
(jupyter-interrupt-kernel): Remove `condition-case`. Not needed since
preventing socket blocking is now handled by `jupyter-recv`.
* jupyter-repl.el
(jupyter-connect-repl): Require and use `jupyter-zmq-channel-ioloop`.
* jupyter-zmq-channel-ioloop.el: New file.
* jupyter-zmq-channel.el (jupyter-ioloop-poller-remove)
(jupyter-ioloop-poller-add): New declares.
(jupyter-start-channel):
Add to `jupyter-ioloop-poller` when in `jupyter-ioloop-environment-p`.
(jupyter-stop-channel):
Only disconnect the socket from its endpoint instead of closing it, leave that
up to garbage collection.
Remove from `jupyter-ioloop-poller` when in `jupyter-ioloop-environment-p`.
(jupyter-recv): Handle non-blocking.
* test/jupyter-test.el
(jupyter-zmq-channel): Use non-blocking `zmq-send` since socket is no longer
closed when calling `jupyter-stop-channel`.
(jupyter-ioloop-test-eval-ioloop): Rename to `jupyter-test-ioloop-eval-event`,
update all callers, and move to `test/test-helper.el`.
(jupyter-channel-ioloop-send-event, jupyter-channel-ioloop-stop-channel-event)
(jupyter-channel-ioloop-start-channel-event): Fix tests for variable name
changes. Use `jupyter-test-channel-ioloop`. Update `jupyter-ioloop-poller`,
addition/removal from poller is now done in the `jupyter-channel` subclass by
checking `jupyter-ioloop-environment-p`.
* test/test-helper.el (jupyter-zmq-channel-ioloop): Require.
(initialize-instance [jupyter-echo-client]): Use it.
(jupyter-test-channel-ioloop): New macro.
(jupyter-test-ioloop-eval-event): New function.
Having the `jupyter-comm-layer` abstraction means we do not need to do so.
* jupyter-base.el (zmq): Un-require.
(jupyter-socket-types): Move to `jupyter-channels.el`.
(jupyter-session): Don't mention zmq in doc string.
(jupyter-available-local-ports, jupyter-make-ssh-tunnel): New functions.
(jupyter-tunnel-connection): Use them.
* jupyter-channel-ioloop-comm.el: New file.
* jupyter-channels.el (jupyter-messages): Un-require.
(jupyter-comm-layer, zmq): New requires.
(jupyter-socket-types): Moved from `jupyter-base.el`.
(jupyter-send, jupyter-recv):
Implementations for `jupyter-session` moved from `jupyter-messages.el`.
(jupyter-sync-channel-comm): `jupyter-comm-layer` implementation for
`jupyter-sync-channel` objects moved from `jupyter-comm-layer.el`.
* jupyter-comm-layer.el (jupyter-channel-ioloop): Un-require.
(jupyter-sync-channel-comm): Move implementation to `jupyter-channels.el`.
(jupyter-ioloop-comm): Move implementation to new file `jupyter-ioloop-comm.el`.
(jupyter-channel-ioloop-comm):
Move implementation to new file `jupyter-channel-ioloop-comm.el`.
* jupyter-ioloop-comm.el: New file.
* jupyter-ioloop.el (zmq): Require.
* jupyter-kernel-manager.el
(jupyter-make-client): Ensure `jupyter-channel-ioloop-comm` is required.
* jupyter-messages.el (jupyter-send)
(jupyter-recv): Moved to `jupyter-channels.el`
* jupyter-repl.el
(jupyter-connect-repl): Ensure `jupyter-channel-ioloop-comm` is required.
* test/jupyter-test.el (jupyter-available-local-ports): New test.
* test/test-helper.el (jupyter-channel-ioloop-comm): New require.
