Better support for completions in the Jupyter org-mode source blocks
A line context is often insufficient. For example, the Julia kernel will parse
up to the current point and return dictionary keys as completions or method
arguments. Similarly a Python kernel may use Jedi for completion which does
static analysis of the code block to resolve import paths for completion
results.
Change the behavior of `jupyter-org-sync-results` to return the org formatted
result string and modify the result parameters in `org-babel-execute:jupyter`
to add the "raw" result parameter so that the result string is directly
inserted into the buffer.
This is to work towards better support for in-lined code blocks.
This ensures that org-element objects are always returned by the functions like
`jupyter-org-file-link`. Previously functions that returned org objects would
return the object wrapped in a list.
Previously stream results would be placed directly inside the RESULTS drawer.
The issue with this is that sometimes the stream results can be represented as
org syntax.
Instead of inserting stream results directly into a RESULTS drawer, insert them
as either fixed-width or example-block org elements depending on
`org-babel-min-lines-for-block-output`. In addition, coalesce all stream
results so that multiple stream messages in succession produces only one
fixed-width or example-block element.
This may still not be the best place to put buffer modifications, but it
"spreads out the logic" a little more and simplifies
`jupyter-org--append-result`.
It seems `org-element-latex-fragment-parser` expects that a latex fragment can
really be parsed because there were false positives when it was called trying
to parse a latex environment such as `\begin{equation*}...\end{equation*}`. It
would consider `\begin{equation*}` as the fragment.
There was an ugly hack that destructively modified the source block parameters
supplied to the org-babel execute function to make `org-babel-insert-result` do
all of the insertion work. This relied too much on knowing the internals of
that function. I also could never figure out how to insert stream results in a
clean way.
Instead we manually insert the results by taking advantage of the `org-element`
API. Specifically the function `org-element-interpret-data` which takes an org
syntax tree and returns its printed representation. Now the
`jupyter-org-result` method returns either a string or a syntax tree. If the
latter is returned, it is filtered through `org-element-interpret-data` to
obtain the string representation for insertion.
In addition, all source blocks insert results in a RESULTS drawer. This allows
for inserting stream output as raw text in the drawer and allows for a way to
append results since the end of the drawer acts as an insertion point.
This is necessary to allow for the `jupyter-lang' method specializer to work in
contexts other than the REPL buffer, such as in an `org-mode` buffer when
handling `jupyter-org-request' objects.
* jupyter-client.el (jupyter-handle-message): (CLIENT ...) Do it.
* jupyter-org-client.el (jupyter-org-add-result):
Don't let bind `jupyter-current-client`.
Remove CLIENT argument. Update all callers.
This is in replacement of the `jupyter-org-prepare-result` and
`jupyter-org-transform-result`. A single method with both primary and :around
methods is sufficient.
* jupyter-org-client.el (jupyter-org-mime-types): New variable.
(jupyter-org-prepare-and-add-result): Remove.
(jupyter-handle-execute-result, jupyter-handle-display-data):
Do the work previously done by `jupyter-org-prepare-and-add-result`.
Abstract out the python handling to a `jupyter-org-result` method.
(jupyter-org--image-result): Add PARAMS argument.
Move setup done previously in `jupyter-org-prepare-result` to this function.
(jupyter-org-prepare-result): Remove.
Replace with calls to the `jupyter-org-result' method.
(jupyter-org-result): New method.
(text/html) Remove special handling of <img> tags, this was done due to old
versions of the Julia kernel.
(jupyter-org-transform-result): Remove.
Replace with a `jupyter-org-result` :around method.
Remove all callers.
* jupyter-python.el (jupyter-org-transform-result): Remove.
Replace with a `jupyter-org-result` :around method.
* jupyter-tests.el (jupyter-org-result): Add tests.
org-result
The goal of this method is to act as a single entry point for insertion of
kernel results in any context. One would simply add another method to handle a
specific context.
* jupyter-base.el (jupyter-mime-types):
(jupyter-nongraphic-mime-types): New variables that give mime-types that can be
handled.
(jupyter-insert): New method for dispatching to code that inserts mimetype
representations in the current buffer.
* jupyter-mime.el: New file.
(jupyter-display-ids):
(jupyter-handle-control-codes):
(jupyter-fontify-buffers):
(jupyter-get-fontify-buffer):
(jupyter-fixup-font-lock-properties):
(jupyter-add-font-lock-properties):
(jupyter-fontify-according-to-mode):
(jupyter-insert-html):
(jupyter-markdown-mouse-map):
(juputer-markdown-follow-link-at-point):
(jupyter-insert-markdown):
(jupyter-insert-latex):
(jupyter-insert-ansi-coded-text): Moved from jupyter-repl.el, replaced
`jupyter-repl-` prefix with `jupyter-`.
(jupyter--shr-put-image): Ditto. Also add `shr-` prefix.
(jupyter--delete-javascript-tags): Ditto. Also mark as private functions.
(jupyter-insert-image): Ditto. Also mark as a public function.
(jupyter-insert): (DISPLAY-ID ...) Moved from jupyter-repl.el. Was
`jupyter-repl-insert-data-with-id`.
(jupyter-with-control-code-handling):
(jupyter-markdown-follow-link): Moved from jupyter-repl.el
(jupyter-insert): Implement methods to do the work previously done by
`jupyter-repl-insert-data`.
* jupyter-repl.el (jupyter-repl-graphic-mimetypes): Moved to jupyter-base.el,
inverted and renamed to `jupyter-nongraphic-mime-types`.
(jupyter-repl-graphic-data-p): Remove unused function.
(jupyter-repl-insert-data): Remove, replace calls with `jupyter-insert`.
(jupyter-repl-add-font-lock-properties):
(jupyter-repl-fixup-font-lock-properties):
(jupyter-repl-get-fontify-buffer):
(jupyter-repl-fontify-according-to-mode):
(jupyter-repl-delete-javascript-tags):
(jupyter-repl-put-image):
(jupyter-repl-insert-html):
(jupyter-repl-markdown-mouse-map):
(jupyter-repl-markdown-follow-link-at-point):
(jupyter-repl-insert-markdown):
(jupyter-repl-insert-latex):
(jupyter-repl--insert-image): Moved to jupyter-mime.el, which see.
(jupyter-repl-insert-data-with-id): Ditto. Changed to a `jupyter-insert` method
dispatched on a string argument.
(jupyter-repl-insert-ansi-coded-text): Ditto. Replace calls with
`jupyter-insert-ansi-coded-text`.
(jupyter-with-control-code-handling):
(jupyter-markdown-follow-link): Moved to jupyter-mime.el.
* jupyter-org-client.el (jupyter-handle-error): Replace
`jupyter-repl-insert-ansi-coded-text` with `jupyter-insert-ansi-coded-text`.
* jupyter-tests.el (jupyter-insert): Add tests for `jupyter-insert`
* Declare undeclared external functions
* Add ext: prefix to filename of external packages that may not be present on
every system
* Remove `company-grab-symbol-cons` function declaration since this function is
no longer used.
* Define `jupyter-load-language-support` which takes a client and loads the
language support definitions of the client's kernel language.
* Call `jupyter-load-language-support` when initializing a REPL buffer in
`jupyter-repl-mode`. Note this also takes care of loading the kernel support
for a `jupyter-org-client' since a REPL buffer is setup before evaluating any
`org-mode` source code blocks.
* Move language specific methods to their own files named `jupyter-LANGUAGE.el`
