The old Emacs JSON API (for versions <= 27.2 at time of commit) was
structured in a way that replacing the symbol-function of json-encode
worked to get custom encoding for certain object types used by
emacs-jupyter. In the later API, the function json--print plays the
recursive role of json-encode. The code here works in either version,
and is simplified so that future updates to the API are easier to
accomodate.
Having the `jupyter-comm-layer` abstraction means we do not need to do so.
* jupyter-base.el (zmq): Un-require.
(jupyter-socket-types): Move to `jupyter-channels.el`.
(jupyter-session): Don't mention zmq in doc string.
(jupyter-available-local-ports, jupyter-make-ssh-tunnel): New functions.
(jupyter-tunnel-connection): Use them.
* jupyter-channel-ioloop-comm.el: New file.
* jupyter-channels.el (jupyter-messages): Un-require.
(jupyter-comm-layer, zmq): New requires.
(jupyter-socket-types): Moved from `jupyter-base.el`.
(jupyter-send, jupyter-recv):
Implementations for `jupyter-session` moved from `jupyter-messages.el`.
(jupyter-sync-channel-comm): `jupyter-comm-layer` implementation for
`jupyter-sync-channel` objects moved from `jupyter-comm-layer.el`.
* jupyter-comm-layer.el (jupyter-channel-ioloop): Un-require.
(jupyter-sync-channel-comm): Move implementation to `jupyter-channels.el`.
(jupyter-ioloop-comm): Move implementation to new file `jupyter-ioloop-comm.el`.
(jupyter-channel-ioloop-comm):
Move implementation to new file `jupyter-channel-ioloop-comm.el`.
* jupyter-ioloop-comm.el: New file.
* jupyter-ioloop.el (zmq): Require.
* jupyter-kernel-manager.el
(jupyter-make-client): Ensure `jupyter-channel-ioloop-comm` is required.
* jupyter-messages.el (jupyter-send)
(jupyter-recv): Moved to `jupyter-channels.el`
* jupyter-repl.el
(jupyter-connect-repl): Ensure `jupyter-channel-ioloop-comm` is required.
* test/jupyter-test.el (jupyter-available-local-ports): New test.
* test/test-helper.el (jupyter-channel-ioloop-comm): New require.
Fixes#137
* jupyter-base.el (jupyter-kernelspec): Un-require.
(jupyter-command, jupyter-locate-python)
(jupyter-runtime-directory): Move to new file `jupyter-env.el`
(jupyter-include-other-output, jupyter-iopub-message-hook)
(jupyter-shell-message-hook)
(jupyter-stdin-message-hook): Move to `jupyter-client.el`
(jupyter-sha256, jupyter-hmac-sha256):
(jupytern-new-uuid): Move to `jupyter-messages.el`. Add declaration of
`jupyter-new-uuid` to account for its removal.
(jupyter-create-connection-info)
(jupyter-write-connection-file): Move to `jupyter-kernel-manager.el`
(jupyter-connect-endpoint, jupyter-connect-channel): Move to `jupyter-channels.el`
* jupyter-channels.el: Accept moved functions.
* jupyter-client.el: Accept moved variables.
* jupyter-kernel-manager.el: Accept moved functions.
(jupyter-env, jupyter-kernelspec): New requires.
* jupyter-kernelspec (jupyter-env): New require.
(jupyter-command): Remove declaration.
(jupyter-read-plist-from-string): New declaration.
* jupyter-messages.el: Accept moved functions.
(hmac-def, json): New requires.
* jupyter-org-extensions.el (jupyter-kernelspec): New require.
* jupyter-repl.el (jupyter-kernelspec): New require.
* jupyter-env.el: New file.
* ob-jupyter.el (jupyter-env, jupyter-kernelspec): New requires.
* test/jupyter-test.el (jupyter-env): New require.
The previous mechanism to communicate with a kernel was too low level from the
perspective of a client. The client interfaced directly with the subprocess
abstraction, `jupyter-ioloop`, and had to handle all "events" that occurred in
the `jupyter-ioloop`, e.g. when a channel was started or stopped. But in
reality such events should not be the concern of a client.
A client should only care about events that are directly related to kernel
messages and not events related to the implementation details of *how*
communication occurs.
This commit abstracts out the way in which a client communicates with its
kernel by introducing a new `jupyter-comm-layer` class. The
`jupyter-comm-layer` class takes care of managing the communication channel
between a kernel and its clients as well as sending events to all registered
clients. This way, clients operate solely at the level of events on the
communication layer. All a client does is register itself to receive events on
the communication layer and send events on the layer.
* jupyter-base.el (jupyter-session-endpoints): New function.
* jupyter-client.el (jupyter-kernel-client): Remove ioloop and channels slots.
Add kcomm slot.
(initialize-instance): Unconditionally stop channels.
(jupyter-initialize-connection): Change into a method call.
Call `jupyter-initialize-connection` on the `kcomm` slot.
(jupyter-with-client-buffer): Remove stale comment.
(jupyter-send): Call `jupyter-send` on the `kcomm` slot.
(jupyter-ioloop-handler): Remove all method definitions, replace `sent` and
`message` methods with their `jupyter-event-handler` equivalents.
(jupyter-hb-pause, jupyter-hb-unpause, jupyter-hb-beating):
(jupyter-channel-alive-p, jupyter-start-channel, jupyter-stop-channel):
(jupyter-start-channels, jupyter-stop-channels):
Replace with calls to their equivalents using the `kcomm` slot.
* jupyter-comm-layer.el: New file.
* jupyter-kernel-manager (jupyter-make-client): Set a client's `kcomm` slot to
`jupyter-channel-ioloop-comm`.
* jupyter-messages.el (jupyter-decode-message): Use `list` directly. There
seemed to be issues when using the new `jupyter-sync-channel-comm` due to
using quoted lists.
* test/jupyter-test.el: Add `jupyter-comm-layer` test. Update other tests.
* test/test-helper.el: Add `jupyter-comm-layer` mock objects. Update
`jupyter-echo-client`.
`jupyter-channels.el` depends on the `jupyter-send` method defined in
`jupyter-messages.el` whereas `jupyter-messages.el` does not depend on any
functions in `jupyter-channels.el`.
* jupyter-channel.el: Require `jupyter-messages`
* jupyter-messages.el: Remove `jupyter-channels` require
* Handle the case of no time specification by defaulting to midnight when
decoding time
* Properly handle fractions in `jupyter--decode-time`
* Implement `jupyter--encode-time`
* Consider a time object to be a length 4 list of integers and encode the
object using `jupyter--encode-time` when encoding in `jupyter--encode`
* Add tests for time decoding/encoding
This introduces a way of keeping track of both the encoded and decoded parts of
a message, only decoding the parts of a message when needed, and only encoding
the parts of a message when needed.
If the objects passed to `jupyter--encode` or `jupyter--decode` are lists with
the first element being the symbol `message-part`, then the second element of
the list is interpreted as being the encoded message and the third element
being the decoded part of the message. If either the encoded or the decoded
part are nil, then `jupyter--encode` or `jupyter--decode` will fill those
message parts in and on subsequent calls, passing in the same list, no work
will need to be performed.
This avoids double encoding messages when relaying messages between the channel
subprocess and the browser displaying widgets. This speeds up the communication
process. The cost is storing two representations of the same message, i.e.
speed vs memory.
Fixes:
- Use `eq` instead of `equal` in message status predicates
- Fix time string decoding
- `parse-time-string` was returning all nil on Emacs 26
- Include validation for parent header in `jupyter--encode-message`
- Use `jupyter-message-header' to access the message header instead of `plist-get`
- Add missing namespace to sha256 function
- Any function defined should have a `jupyter-` prefix
- Remove `cl-lib` dependency in `jupyter-messages.el`
- Include `subr-x` in `jupyter-base.el`
- Use `tramp-file-name-user` instead of `tramp-file-name-real-user` since the
latter is missing in Emacs 26
