| js | ||
| test | ||
| .ert-runner | ||
| .gitignore | ||
| Cask | ||
| COPYING | ||
| jupyter-base.el | ||
| jupyter-channel-ioloop.el | ||
| jupyter-channels.el | ||
| jupyter-client.el | ||
| jupyter-ioloop.el | ||
| jupyter-javascript.el | ||
| jupyter-julia.el | ||
| jupyter-kernel-manager.el | ||
| jupyter-kernelspec.el | ||
| jupyter-messages.el | ||
| jupyter-mime.el | ||
| jupyter-org-client.el | ||
| jupyter-python.el | ||
| jupyter-repl.el | ||
| jupyter-widget-client.el | ||
| jupyter.el | ||
| jupyter.png | ||
| ob-jupyter.el | ||
| README.org | ||
| widget.html | ||
- Installation
- Jupyter REPL
- Integration with
org-mode - API
- See also
An interface to communicate with Jupyter kernels in Emacs.
Installation
If you would like to try this package out, in your Emacs configuration add
(add-to-list 'load-path "<path>")
(require 'jupyter)
where <path> is the root directory for this project.
Dependencies
- markdown-mode (optional)
- https://jblevins.org/projects/markdown-mode/
- company-mode (optional)
- http://company-mode.github.io/
- emacs-websocket (optional)
- https://github.com/ahyatt/emacs-websocket
- simple-httpd (optional)
- https://github.com/skeeto/emacs-web-server
- emacs-zmq
- http://github.com/dzop/emacs-zmq
Widget support (EXPERIMENTAL)
There is also support for interacting with Jupyter widgets through an external browser. If a widget is to be displayed, an external browser is opened first to display the widget. In this case, Emacs acts as a relay for passing messages between the kernel and the external browser.
If you would like to try out this limited support, you will need to have node
installed on your system to build the necessary javascript. Then you will have
to run the following commands from the root project directory:
cd js
npm install -g yarn
npm install
yarn run build --progressJupyter REPL
To start a new kernel on the localhost and connect a REPL client to it, run
the command jupyter-run-repl. Alternatively you can connect to an existing
kernel by supplying the kernel's connection file to jupyter-connect-repl.
The REPL supports most of the rich output that a kernel may send to a client.
If the kernel requests a widget to be displayed, a browser is opened that
displays the widget. If the kernel sends image data, the image will be
displayed in the REPL buffer. If LaTeX is sent, it will be compiled (using
org-mode) and displayed. The currently available mimetypes and their
dependencies are:
DONE Rich kernel output
CLOSED: [2018-10-06 Sat 02:42]
- State "DONE" from "TODO" [2018-10-06 Sat 02:42]
A Jupyter kernel provides many representations of results that may be used by the frontend, in this case Emacs. Luckily, Emacs provides good support for most of the available representations.
The supported mimetypes along with their dependencies are shown below in order of priority if multiple representations are returned. Note, if a dependency is not available in your Emacs, a mimetype with a lower priority will be used to display output.
| Mimetype | Dependency |
|---|---|
application/vnd.jupyter.widget-view+json |
websocket, simple-httpd |
text/html |
Emacs built with libxml2 |
text/markdown |
markdown-mode |
text/latex |
org-mode |
image/svg+xml |
Emacs built with librsvg2 |
image/png |
none |
text/plain |
none |
Inspection
To send an inspect request to the kernel, press C-c C-f when the cursor is at
the location of the code you would like to inspect.
Completion
Completion is implemented through the completion-at-point interface. In
addition to completing symbols in the REPL buffer, completion also works in
buffers associated with a REPL. For org-mode users, there is even completion
in the org-mode buffer when editing the contents of a Jupyter source code
block.
REPL history
You can navigate through the REPL history using C-n and C-p or M-n and
M-p.
You can also search through the history using isearch. To search through
history, use the standard isearch keybindings: C-s to search forward
through history and C-s C-r to search backward.
Associating other buffers with a REPL
After starting a REPL, it is possible to associate the REPL with other buffers
if they pass certain criteria. Currently, the buffer must have the major-mode
that corresponds to the REPL's kernel language. To associate a buffer with a
REPL you can run the command jupyter-repl-associate-buffer.
jupyter-repl-associate-buffer will ask you for the REPL you would like to
associate with the current-buffer and enable the minor mode
jupyter-repl-interaction-mode. This minor mode populates the following
keybindings for interacting with the REPL:
| Key binding | Command |
|---|---|
C-c C-c |
jupyter-repl-eval-line-or-region |
C-c C-f |
jupyter-inspect-at-point |
C-c C-z |
jupyter-repl-pop-to-buffer |
C-c C-i |
jupyter-repl-interrupt-kernel |
C-c C-r |
jupyter-repl-restart-kernel |
Widget support
There is also support for Jupyter widgets integrated into the REPL. If any of the results returned by a kernel have a widget representation, a browser is opened and the widget is displayed in the browser. There is only one browser per client.
This feature is currently considered experimental and has only been tested for
simple uses of widgets. See jupyter-widget-client.
Integration with org-mode
For users of org-mode, integration with org-babel is provided through the
ob-jupyter library. To enable Jupyter support for source code blocks add
jupyter to org-babel-load-languages.
(org-babel-do-load-languages
'org-babel-load-languages
'((emacs-lisp . t)
(julia . t)
(python . t)
(jupyter . t))
Note, jupyter should be added as the last element when loading languages
since it depends on the values of variables such as org-src-lang-modes and
org-babel-tangle-lang-exts. After ob-jupyter has been loaded, new source
code blocks with names of the form jupyter-LANG will be available. LANG can
be any one of the kernel languages found on your system. See
jupyter-available-kernelspecs.
Every Jupyter source code block requires that the :session parameter be
specified since all interaction with a kernel is through a REPL. For example,
to interact with a python kernel you would create a new source block like so
#+BEGIN_SRC jupyter-python :session py
x = 'foo'
y = 'bar'
x + ' ' + y
#+END_SRC
By default, source blocks are executed synchronously. To execute a source block
asynchronously set the :async parameter to yes:
#+BEGIN_SRC jupyter-python :session py :async yes
x = 'foo'
y = 'bar'
x + ' ' + y
#+END_SRC
Since a particular language may have multiple kernels available, the default
kernel used for a language is the first kernelspec found by
jupyter-available-kernelspecs for the language. To change the kernel, set the
:kernel parameter:
#+BEGIN_SRC jupyter-python :session py :async yes :kernel python2
x = 'foo'
y = 'bar'
x + ' ' + y
#+END_SRC
Note, the same session name can be used for different values of :kernel since
the underlying REPL buffer for a source code block is a based on both
:session and :kernel.
In addition, any of the defaults for a language can be changed by setting
org-babel-default-header-args:jupyter-LANG to an appropriate value. For
example to change the default header arguments of the julia kernel, you can
set org-babel-default-header-args:jupyter-julia to something like
(setq org-babel-default-header-args:jupyter-julia '((:async . "yes")
(:session . "jl")
(:kernel . "julia-1.0")))Rich kernel output
In org-mode a code block returns scalar data (plain text, numbers, lists,
tables, …), an image file name, or code from another language. All of this
information must be specified in the code block's header arguments, but all of
this information is already provided in the messages passed between a Jupyter
kernel and its frontends.
When a kernel provides representations of results other than plain text, those richer representations are prioritized over plain text. For example if the kernel returns LaTeX code, the results are wrapped in a LaTeX source block. Similarly for HTML and markdown. If an image is returned, the image is automatically saved to file and a link to the file will be the result of the code block.
Below are the supported mimetypes ordered by priority
- text/org
- text/html
- text/markdown
- text/latex
- image/png, image/jpg, image/svg+xml
- text/plain
Image output without the :file header argument
For images sent by the kernel, if no :file parameter is provided to the code
block, a file name is automatically generated based on the image data and the
image is written to file in org-babel-jupyter-resource-directory. This is
great for quickly generating throw-away plots while your are working on your
code. Once you are happy with your results you can specify the :file
parameter to fix the file name.
Editing the contents of a code block
When editing a Jupyter code block's contents, i.e. by pressing C-c ' when at
a code block, jupyter-repl-interaction-mode is automatically enabled in the
edit buffer and the buffer will be associated with the REPL session of the code
block (see jupyter-repl-associate-buffer).
You may also bind the command org-babel-jupyter-scratch-buffer to an
appropriate key in org-mode to display a scratch buffer in the code block's
major-mode and connected to the code block's session.
Connecting to an existing kernel
To connect to an existing kernel, pass the kernel's connection file as the
value of the :session parameter. The name of the file must have a .json
suffix for this to work.
Remote kernels
If the connection file is a remote file name, i.e. has a prefix like /host:,
the kernel's ports are assumed to live on host. Before attempting to connect
to the kernel, ssh tunnels for the connection are created. So if you had a
remote kernel on a host named ec2 whose connection file is
/run/user/1000/jupyter/kernel-julia-0.6.json on that host, you would specify
the :session as
#+BEGIN_SRC jupyter-julia :session /ec2:/run/user/1000/jupyter/kernel-julia-0.6.json
...
#+END_SRCPassword handling for remote connections
Currently there is no password handling, so if your ssh connection requires a
password I suggest you instead use key-based authentication. Or if you are
connecting to a server using a pem file add something like
Host ec2
User <user>
HostName <host>
IdentityFile <identity>.pem
to your ~/.ssh/config file.
API
Naming conventions
Methods that send messages to a kernel are named jupyter-send-<msg-type> where
<msg-type> is an appropriate message type. The message types are identical to
those defined in the Jupyter spec with _ characters replaced by -
characters. So to send an execute-request you would call
jupyter-send-execute-request. Similarly, methods that are responsible for
handling messages received from a kernel are named jupyter-handle-<msg-type>.
Methods that require a message type as an argument such as
jupyter-add-callback should do so by passing a message type keyword such as
:execute-request.
TODO Overview
- State "TODO" from "NEXT" [2018-10-06 Sat 03:07]
- State "NEXT" from "TODO" [2018-10-02 Tue 00:53]
Class overview
-
jupyter-kernel-client - The base class for Jupyter frontends. Handles all message sending and receiving to/from a Jupyter kernel.
-
jupyter-kernel-manager - The base class for starting local kernel processes.
-
jupyter-widget-client - (EXPERIMENTAL) A subclass of
jupyter-kernel-clientthat adds support for displaying Jupyter widgets in an external browser. -
jupyter-repl-client - A subclass of
jupyter-kernel-clientthat implements a REPL. Note, ajupyter-repl-clientalso has ajupyter-widget-clientas a parent class. -
jupyter-org-client - A subclass of
jupyter-repl-clientthat adds support for evaluatingorg-modesource code blocks and inserting the results in theorg-modebuffer.
Communicating with the kernel
Initializing the connection and sending messages
For a jupyter-kernel-client to start communicating with a kernel, the
following steps are taken:
- Initialize the connection using
jupyter-initialize-connection - Start listening on the client's channels with
jupyter-start-channels - Send requests with the
jupyter-send-<msg-type>family of client methods
When starting a local kernel process, the first two steps are already taken care
of when using jupyter-start-new-kernel.
All message sending and receiving is asynchronous. Methods that send requests
to a kernel return a jupyter-request object that encapsulates the necessary
information to handle any future messages generated in response to the request.
Receiving messages
There are two methods to handle the reply messages sent by the kernel (1)
subclass the jupyter-kernel-client and override the
jupyter-handle-<msg-type> family of methods or (2) attach callbacks to the
jupyter-request objects returned by the jupyter-send-<msg-type> methods.
Both ways can occur in parallel.
When a message is received, jupyter-handle-message is called on the client to
kick off the message handling process. Any callbacks associated with the
jupyter-request of the message are evaluated and then the appropriate
jupyter-handle-<msg-type> method will be called. Finally, any completed
requests are dropped from the client's request table.
Note, the default handler methods of jupyter-kernel-client are no-ops with
the exception of jupyter-handle-input-request which requests input from the
user and sends it to the kernel.
jupyter-kernel-client
Represents a client connected to a Jupyter kernel.
Initializing a connection
jupyter-initialize-connection takes a client and a connection file as
arguments and configures the client to communicate with the kernel whose
connection information is contained in the connection file.
After initializing a connection, to begin communicating with a kernel call
jupyter-start-channels.
(let ((client (jupyter-kernel-client)))
(jupyter-initialize-connection client "kernel1234.json")
(jupyter-start-channels client))
jupyter-initialize-connection is mainly useful when initializing a remote
connection or connecting to an existing kernel. In order to start a new kernel
on the localhost use jupyter-start-new-kernel
(cl-destructuring-bind (manager client)
(jupyter-start-new-kernel "python")
BODY)
The above code starts a new python kernel and returns the
jupyter-kernel-manager object used to manage the lifetime of the local kernel
process and the jupyter-kernel-client connected to the manager's kernel.
jupyter-start-channels will already have been called on the returned client
when jupyter-start-new-kernel returns.
To create multiple client's connected to the kernel of a
jupyter-kernel-manager use jupyter-make-client.
Starting/stopping channels
To start a client's channels, use jupyter-start-channels. To stop a client's
channels, jupyter-stop-channels. To determine if at least one channel is
alive, jupyter-channels-running-p.
You may access each individual channel by accessing its corresponding slot in a
jupyter-kernel-client. To access the shell channel of a client
(oref client shell-channel)
this will give you the jupyter-channel object of the shell channel. By
accessing the channel slots of the client, individual channels may be started or
stopped.
Making requests to a kernel
To free up Emacs from having to process messages sent to and received from a kernel, an Emacs subprocess is created for every client. This subprocess is responsible for polling the client's channels for messages and taking care of message signing, encoding, and decoding. The parent Emacs process is only responsible for supplying the message property lists (the representation used for Jupyter messages in Emacs) when sending a message and will receive the decoded message property list when receiving a message. The exception to this is the heartbeat channel which is implemented using timers in the parent Emacs process.
Note, the message property lists should not be accessed directly. There are helper functions which should be used to access the message fields. See Message property lists.
The lifetime of a request
Sending a request to a kernel is done through one of the
jupyter-send-<msg-type> methods of a jupyter-kernel-client. The arguments
of the Jupyter message that each method represents are passed as keyword
arguments, the keywords all have names according to the Jupyter messaging spec
but with _ replaced by -. These methods construct the message property
lists based on their arguments and pass the constructed message to the
jupyter-send method of a client. The jupyter-send method then returns a new
jupyter-request representing the sent message.
(jupyter-send-execute-request client :code "1 + 2") ; Returns a `jupyter-request'
When a request is sent, the message ID of the request is added to the client's
request table which maps message IDs to their corresponding jupyter-request
objects.
When a message is received from the kernel the request that generated it is
found in the request table by using the jupyter-message-parent-id of the
message. The slots of the jupyter-request are updated, any callbacks
associated with the jupyter-request are run for the message, and the message
is dispatched to the appropriate channel handler method of the client (one of
the jupyter-handle-<msg-type> methods).
A request is considered complete and is dropped from the request table once a
status: idle message has been received for the request and it is not the most
recently made request.
DONE
jupyter-generate-request
CLOSED: [2018-10-01 Mon 16:09]
- State "DONE" from "TODO" [2018-10-01 Mon 16:09]
- State "TODO" from [2018-10-01 Mon 15:50]
When one of the send methods are called, a jupyter-request object is
instantiated by a call to jupyter-generate-request and the instantiated
request is returned by the send method so that the caller can attach their
callbacks as described above.
Most likely, subclasses would want to attach extra information to a request.
For example, an org-mode client that sends an :execute-request based on the
contents of a source code block might want to keep track of the code block's
buffer position so that it can insert the results at the right location when
they are ready.
This is the purpose of the jupyter-generate-request method. If a
jupyter-request object is not general enough for some purpose, a subclass of
jupyter-kernel-client can define a new request object, ensuring that the slots
of a jupyter-request are included, and return the new type of request when
jupyter-generate-request is called for a message.
For example, below is the definition of the jupyter-org-request type for
handling requests made in an org-mode buffer
(cl-defstruct (jupyter-org-request
(:include jupyter-request))
result-type
block-params
results
silent
id-cleared-p
marker
async)And the context specializers used are
(cl-defmethod jupyter-generate-request ((client jupyter-org-client) msg
&context (major-mode org-mode))
...) ; Return a `jupyter-org-request'
Notice that the major-mode context allows for jupyter-org-request objects
to be used by jupyter-generate-request when the request is generated in
org-mode buffers and to use the less specialized jupyter-request in other
contexts.
DONE
jupyter-drop-request
CLOSED: [2018-10-01 Mon 16:16]
- State "DONE" from "TODO" [2018-10-01 Mon 16:16]
- State "TODO" from [2018-10-01 Mon 15:50]
When a request is completed, i.e. when the kernel sends an idle message for a
request, you may want to do some final cleanup of the request. This is the
purpose of the jupyter-drop-request method, it gets called when an idle
message has been received for a kernel but only when the request is not the
most recently sent request.
Handling received messages
The handler methods of a jupyter-kernel-client are called whenever the
corresponding message is received from the kernel. They are intended to be
overwritten by subclasses and most of the default implementations do nothing
with the exception of the :input-reply, :comm-open, and :comm-close
messages. The :input-reply handler asks for input from the user through the
minibuffer and sends it to the kernel whereas the :comm-open / :comm-close
default message handlers store the state of open comms in the client's comms
slot.
The handler methods have the following signature
(cl-defmethod jupyter-handle-<msg-type> ((client jupyter-kernel-client) req arg1 arg2 ...)
BODY)
req will be the jupyter-request object that generated the message. arg1,
arg2, … will be the unwrapped message contents passed to the handler, their
number of arguments and their order are dependent on the message type.
Alternatively you may work with the full message property list by accessing the
jupyter-request-last-message slot of the juptyer-request object.
See message callbacks for another way of handling received messages.
A note on boolean arguments
For message types that have boolean message fields, the
symbol in the variable jupyter--false represents a false
value so when checking the contents of these arguments it
is best to explicitly check for t.
(if (eq arg1 t) ...)
This is because there are some ambiguities between
translating JSON values to their Emacs Lisp equivalents,
since nil in Emacs is used both as signifying false or
nothing whereas JSON has null for nothing.
Client local variables
Some variables which are used internally by jupyter-kernel-client have client
local values. For example the variable jupyter-include-other-output tells a
jupyter-kernel-client to pass IOPub messages originating from a different
client to their corresponding handlers and defaults to nil, i.e. do not
handle IOPub messages from other clients. To modify a client local variable you
would use jupyter-set
(jupyter-set client 'jupyter-include-other-output t)
and to retrieve the client local value, use jupyter-get
(jupyter-get client 'jupyter-include-other-output)
These functions just set/get the value of a buffer local variable in a private
buffer of the client. You may work with these buffer local variables directly
by using the jupyter-with-client-buffer macro, just be sure to use
setq-local if you are setting a new client local variable otherwise you may
change the global value of the variable. Alternatively you can define a
variable as automatically buffer local when set with defvar-local.
(jupyter-with-client-buffer client
(message "jupyter-include-other-output: %s" jupyter-include-other-output)
(setq-local jupyter-include-other-output (not jupyter-include-other-output)))Channel hooks
The channel hook variables jupyter-iopub-message-hook,
jupyter-shell-message-hook, and jupyter-stdin-message-hook are all client
local variables and functions can be added to or removed from them using
jupyter-add-hook and jupyter-remove-hook. See Channel hooks.
jupyter-kernel-manager
Manage the lifetime of a kernel on the localhost.
Kernelspecs
To get a list of kernelspecs on your system, as represented
in Emacs, use jupyter-available-kernelspecs which
processes the output of the shell command
jupyter kernelspec list
to construct the list of kernelspecs. This command also
supports remote hosts. So if the default-directory points
to a remote system, the returned kernelspecs are those on
the remote system.
To find all kernelspecs whose kernels match some regular
expression use jupyter-find-kernelspecs. In the case you
would like to get the kernelspec for a specific kernel, use
jupyter-get-kernelspec.
You may also use jupyter-completing-read-kernelspec in an
interactive spec to ask the user to select a kernel from
the list of available kernelspecs.
Managing the lifetime of a kernel
Starting a kernel
As was mentioned previously, to start a new kernel on the localhost and
create a connected client, use jupyter-start-new-kernel which takes a kernel
name and returns a jupyter-kernel-manager which manages the lifetime of the
kernel, and a connected jupyter-kernel-client.
(cl-destructuring-bind (manager client)
(jupyter-start-new-kernel "python")
BODY)
Instead of supplying an exact kernel name, you may also supply the prefix of
one. Then the first available kernel that has the same prefix will be started.
See jupyter-find-kernelspecs.
Stopping a kernel
To shutdown a kernel, use jupyter-shutdown-kernel. To check if a kernel is
alive, jupyter-kernel-alive-p.
Interrupting a kernel
To interrupt a kernel, use jupyter-interrupt-kernel.
Making clients connected to a kernel
Once you have a kernel manager you can make new jupyter-kernel-client (or a
subclass of one) instances using jupyter-make-client.
jupyter-widget-client
This class adds support for interacting with Jupyter widgets using an external
browser for the widget display. In order for this to work properly you will
need to have simple-httpd and the websocket packages installed, in
addition, you will have to build the required javascript files as described in
Widget support.
The default implementation of jupyter-widget-client overrides the following
methods of a jupyter-kernel-client
(jupyter-handle-comm-close)
(jupyter-handle-comm-open)
(jupyter-handle-comm-msg)Comm messages in Jupyter are a way to allow for custom messages between the kernel and a client. In the case of Jupyter widgets they are used to sync widget state between the kernel and client.
It would be amazing to add custom Jupyter widgets to Emacs using the built
widget library which would work for widgets such as text boxes, buttons, and
other simple widgets, but there doesn't seem to be a way to support more
complex widgets in Emacs that require embedded javascript.
The default implementation of jupyter-kernel-client only keeps track of open
comms through a client's comms slot. The jupyter-widget-client subclass
adds the functionality to display and interact with widgets through an external
browser. This works by relaying the comm messages between the browser and the
kernel through a websocket. For this to work, you will also need to have the
simple-httpd and websocket Emacs packages available.
This feature is currently experimental, but seems to work well. I was able to interact with an ipyleaflet map without any noticeable delay.
Callbacks and hooks
There are two main ways of evaluating code in response to a received message
from the kernel. You can either subclass jupyter-kernel-client and override
the handler methods or you can add message callbacks to the jupyter-request
objects returned by the send methods. In both cases, when a message of a
certain type is received for a request, the appropriate handler method or
callback runs. If both methods are used in parallel, the message callbacks will
run before the handler methods.
You can also add a hook to one of the jupyter-<channel>-message-hook client
local hooks. Where <channel> can be one of iopub, shell, or stdin.
jupyter-request callbacks
To add callbacks to a request, use jupyter-add-callback.
jupyter-add-callback accepts a jupyter-request object as its first argument
and alternating (message type, callback) pairs as the remaining arguments. The
callbacks are registered with the request object to run whenever a message of
the appropriate type is received. For example, to do something when a client
receives a :kernel-info-reply you would do the following:
(jupyter-add-callback (jupyter-send-kernel-info-request client)
:kernel-info-reply (lambda (msg)
(let ((info (jupyter-message-content msg)))
BODY)))To print out the results of an execute request:
(jupyter-add-callback (jupyter-send-execute-request client :code "1 + 2")
:execute-result (lambda (msg)
(message (jupyter-message-data msg :text/plain))))To add multiple callbacks to a request:
(jupyter-add-callback (jupyter-send-execute-request client :code "1 + 2")
:execute-result (lambda (msg)
(message (jupyter-message-data msg :text/plain)))
:status (lambda (msg)
(when (jupyter-message-status-idle-p msg)
(message "DONE!"))))There is also the possibility of running the same handler for different message types:
(jupyter-add-callback (jupyter-send-execute-request client :code "1 + 2")
'(:status :execute-result :execute-reply)
(lambda (msg)
(pcase (jupyter-message-type msg)
(:status ...)
(:execute-reply ...)
(:execute-result ...))))Channel hooks
Hook variables are available for each channel: jupyter-iopub-message-hook,
jupyter-stdin-message-hook, and jupyter-shell-message-hook. Unless you want
to run a channel hook for every client, use jupyter-add-hook to add a
function to one of the channel hooks. jupyter-add-hook only adds to the
client local value of the hook variables.
(jupyter-add-hook
client 'jupyter-iopub-message-hook
(lambda (msg)
(when (jupyter-message-status-idle-p msg)
(message "Kernel idle."))))
To remove a client local hook, use jupyter-remove-hook.
Channel hooks also provide a way of suppressing the handler methods. If any of the channel hooks return a non-nil value, the handler method for that message will be suppressed.
jupyter-inhibit-handlers
In addition to suppressing handler methods using channel hooks, to prevent a
client from running its handler methods for a particular request you can let
bind jupyter-inhibit-handlers to an appropriate value before the request is
made. For example, to prevent a client from running its stream handler for a
request you would do the following:
(let ((jupyter-inhibit-handlers '(:stream)))
(jupyter-send-execute-request client :code "print(\"foo\")\n1 + 2"))
jupyter-inhibit-handlers can be either a list of message types or t, the
latter meaning inhibit handlers for all message types. Alternatively you can
set the jupyter-request-inhibited-handlers slot of a jupyter-request
object. This slot can take the same values as jupyter-inhibit-handlers.
Waiting for messages
All message passing between the kernel and Emacs happens asynchronously. So if a code path in Emacs Lisp is dependent on some message already having been received, e.g. an idle message, there needs to be primitives that will block so there can be can guarantee that certain messages have been received.
The following functions all wait for different conditions to be met on the
received messages of a request and return the message that caused the function
to stop waiting or nil if no message was received within a timeout period.
The default timeout is jupyter-default-timeout seconds.
For example, to wait until an idle message has been received for a request:
(let ((timeout 4))
(jupyter-wait-until-idle
(jupyter-send-execute-request
client :code "import time\ntime.sleep(3)")
timeout))To wait until a message of a specific type is received for a request:
(jupyter-wait-until-received :execute-reply
(jupyter-send-execute-request client :code "[i*10 for i in range(100000)]"))
The most general form of the blocking functions is jupyter-wait-until which
takes a message type and a predicate function of a single argument. Whenever a
message is received that matches the message type, the message is passed to the
function to determine if jupyter-wait-until should return from waiting.
(defun stream-prints-50-p (msg)
(let ((text (jupyter-message-get msg :text)))
(cl-loop for line in (split-string text "\n")
thereis (equal line "50"))))
(let ((timeout 2))
(jupyter-wait-until
(jupyter-send-execute-request client :code "[print(i) for i in range(100)]")
:stream #'stream-prints-50-p
timeout))
The above code runs stream-prints-50-p for every stream message received
from a kernel (here assumed to be a python kernel) for an execute request that
prints the numbers 0 to 99 and waits until the kernel has printed the number 50
before returning from the jupyter-wait-until call. If the number 50 is not
printed before the two second timeout, jupyter-wait-until returns nil.
Otherwise it returns the stream message whose content contains the number 50.
Message property lists
There is really no need to construct or access message property lists directly.
The jupyter-send-<msg-type> client methods already handle creating them by
calling the jupyter-message-<msg-type> family of functions. Similarly, when a
message is received from a kernel the message properties are unwrapped and
passed as arguments to the jupyter-handle-<msg-type> client methods. If
required, the message property list is available in the
jupyter-request-last-message slot of the jupyter-request passed to the
jupyter-handle-<msg-type> client methods.
On the other hand, message callbacks pass the message property list directly to the callback. In this case, the following functions can be used to access the fields of the property list:
;; Get the `:content' propery of MSG
(jupyter-message-content msg)
;; Get the message type (one of the keys in `jupyter-message-types')
(jupyter-message-type msg)
;; Get the value of KEY in the MSG contents
(jupyter-message-get msg key)
;; Get the value of the MIMETYPE in MSG's :data property
;; MIMETYPE should be one of `:image/png', `:text/plain', ...
(jupyter-message-data msg mimetype)
Note that access of the message property lists should only occur through the
jupyter-message-* functions since the main parts of a message such as the
content and header are lazily decoded.
Convenience macros
jupyter-with-message-content gives a way to extract and
bind the keys of a jupyter-message-content easily
(jupyter-with-message-content msg (status ename)
...) ; status and ename keys of (jupyter-message-content msg) are bound
There is also jupyter-with-message-data which extracts
and binds the mimetypes of jupyter-message-data
(jupyter-with-message-data msg ((res text/plain))
...) ; res is bound to (jupyter-message-data msg :text/plain)Modify behavior depending on kernel language
Since Jupyter supports many different programming language kernels, each with varying degrees of support in Emacs there needs to be a general way of modifying the behavior of the client to take this into account.
This is achieved using the &context specializer of cl-defmethod. There are
currently two specializers in use, jupyter-lang and jupyter-repl-mode.
jupyter-lang is a context specializer that matches when the kernel language
of the jupyter-current-client is equal to the specializer's argument. For
example, below is the function that gets called in the REPL buffer when the
kernel language is julia for indenting the current line:
(cl-defmethod jupyter-indent-line (&context (jupyter-lang julia))
(call-interactively #'julia-latexsub-or-indent))There are many other entry points where methods may be overridden in such a way. Below is the full list of methods that can be overridden in this way
| Method | Purpose |
|---|---|
jupyter-code-context |
Return the code and position for inspect and complete requests |
jupyter-indent-line |
Indent the current cell in the REPL buffer |
jupyter-completion-prefix |
Return the completion prefix for the current completion context |
jupyter-completion-post-completion |
Evaluate code when a completion candidate has been selected |
jupyter-repl-after-init |
Evaluate code after a REPL buffer has been initialized |
jupyter-repl-after-change |
Called when input cell code changes |
jupyter-markdown-follow-link |
Follow a markdown link at point |
jupyter-org-transform-result |
Modify the result of a Jupyter code block before display in org-mode |
In addition to the jupyter-lang context, there is also the
jupyter-repl-mode context which is identical to the derived-mode context
but does its check against jupyter-repl-lang-mode if the
jupyter-current-client is a jupyter-repl-client. This is useful to modify
behavior depending on the major-mode that is used for a particular language.
For example for javascript kernels, it used to setup code highlighting when
js2-mode is used as the REPL languages major-mode since js2-mode does not
use font-lock.
org-mode
NEXT jupyter-org-client
- State "NEXT" from [2018-08-31 Fri 23:23]
A jupyter-org-client is a subclass of jupyter-kernel-client meant to
display the results of a Jupyter code block in an org-mode buffer.
Since the Jupyter spec provides rich output, a code block does not know before
obtaining the results from the kernel what type of results to expect. Typically
this is handled in the org-mode document by the user specifying the kind of
results it expects in header arguments. But the Jupyter messaging spec provides
enough information for the results of an execution so that the user does not
have to specify any header arguments.
A jupyter-org-client takes care of collecting the results of execution from
the kernel and inserting the results in an org-mode buffer. If the kernel
returns results that can be formatted as LaTeX, the results are wrapped in a
LaTeX code block. If the result is an image, a file link is inserted. Similarly
for all of the other supported mimetypes.
See also
- ob-ipython
- https://github.com/gregsexton/ob-ipython
- emacs-ipython-notebook
- https://github.com/millejoh/emacs-ipython-notebook
There are a few other packages similar to this one, the most notable one being
ob-ipython. I originally started out using this package, but was having
trouble getting it to work with some languages and I could never get completion
working consistently.
There is also ein which aims to be a fully integrated notebook environment in
Emacs with many features. I have not tried out this package yet, but looks like
a good solution for a complete notebook experience.
The current package differs from those above in that it aims to be a Jupyter
client API in Emacs to make it easy to build any type of frontend for
communicating with a kernel. In addition to serving as useful tools in their
own right, the included REPL client and org-mode client are examples of how
the API can be used.