This avoids problems with a reply message and a status idle message being
received at identical times for a request. Two messages can be received at the
same time when the fractional seconds resolution of the time stamps received
from the kernel are not high enough. If the iopub channel had a higher priority
it would mean that the idle message would be received before the reply when
they have identical time stamps. Since request objects are removed from the
client when an idle message is received this would prevent any reply callbacks
from running.
To combat this problem, prioritize the shell messages over any other type of
message.
- Introduce `run-handlers-p` for `jupyter-request` objects. When this field is
non-nil, a clients `jupyter-handle-message` method will pass the message to
the channels `jupyter-handle-message` after running any callbacks. Otherwise
the channel handlers will not be run. This gives a way to suppress the
channel handlers for a request.
- Also remove the use of `jupyter-callback` since `jupyter-handle-message` for
a client just removes requests when an idle message is received. The callback
object was used to check if a callback actually ran.
This method dispatches to the appropriate message handlers based on the type of
channel passed to `jupyter-handle-message`. This method is also defined for a
`jupyter-kernel-client` in which case any callbacks for a message will be run
and then the message passed to the channel handlers.
- Only modify/access the `:jupyter-pending-requests' property of the ioloop
subprocess through `jupyter--ioloop-pop-request' and
`jupyter--ioloop-push-request'
- Push a message into a channels receive queue using
`jupyter-channel-push-message'
Where `jupyter-request-*` are the `jupyter-kernel-client` request methods. This
required the underlying messages request message functions to be renamed to
`jupyter-message-*-request` from `jupyter-*-request`.
Instead of directly sending every received message to the parent emacs process
at the moment we receive it. The messages are stored in a queue. Only when we
do not receive a message from the kernel for two polling periods or when the
queue is full will the messages be sent to the parent process. When the
messages are sent, they are first sorted by their timestamp and prioritized
based on channel.
`jupyter-request` encapsulates a request ID, request callbacks, and a flag
variable which tells you if the kernel has sent an idle message for this
request.
`jupyter-callback` encapsulates a callback function and a flag variable
determining if the callback has run at least once. The `callbacks` field of a
`jupyter-request` is an alist mapping reply types to `jupyter-callback`
objects.
Whenever a message is sent to the kernel a new `jupyter-request` object is
created and returned from one of the `jupyter-request-*` methods. This object
holds all the required information to track a message that the kernel is
handling. When a message is received from the kernel, the client checks its
`requests` hash table for the `jupyter-request` object associated with the
message and runs any callbacks for the message and updates the flag variables
of the request and callback if necessary.
The request is considered complete and removed from the `requests` hash table
when an idle message has been received for the request and all callbacks have
run at least once. Note that this is almost surely does not handle all cases
since there may be situations where you would like a callback to run multiple
times while an idle message has already been sent.
If `t` is passed as the `MSG-TYPE` argument of `jupyter-add-receive-callback`,
then it signifies that the associated callback will run for every message that
is received in response to a request.
With this new implementation, all communication between the kernel and the
client happens in a subprocess. When the client would like to send a message,
the parent emacs process generates the required plist and sends it to the
subprocess for encoding and sending to the kernel. When a message is received,
the subprocess decodes it and prints it to the pipe for the parent emacs
process to read.
This implementation also introduces the use of futures to avoid having to wait
for subprocess output when sending a message to the kernel. Every
`jupyter-request-*` function now returns a primitive future object which is
just a cons cell with the `car` equal to `:jupyter-future`. When the `cdr` of
the future is non-nil, then it is the message ID of the sent request. This acts
as a check to ensure that the message ID is available from the future object,
if the `cdr` is nil the ID is not available, but if the `cdr` is non-nil then
it is the message ID. The convenience function `jupyter-ensure-id` ensures that
the message ID is available and returns the ID.
The future acts as a stand in for the message ID of the encoded request which
will be retrieved from the subprocess once the message has been encoded and
sent to the kernel. This future object is meant to be passed to
`jupyter-add-receive-callback` and other related functions the same way as an
actual message id.
