emacs-ipython-notebook/Enhancements.org

• Overview
• Design
  • Notebook Buffer
  • Notebooklist Buffer
  • Kernel communication
  • Contents API
  • Connecting to a running Kernel
• Enhancements/Fixes
  • Imenu/Speedbar Cooperation
  • Live links to other notebooks
  • Use polymode
  • Access password protected notebooks (issue #57)
  • Connect to non-python kernels
  • Synergies with pymacs?
  • Detect system path of opened notebook
  • Jump to notebook code in traceback (issue #42)
  • The Return of Worksheets

Overview

Design

Notebook Buffer

Notebook information is stored as a struct. Always associated with a buffer, ein:notebook-buffer is used to find buffer associated with a notebook.

Notebook does not hold cells, that is delegated to instances of the worksheet class. Instances are stored as a list in the `ein:$notebook-worksheets` slot.

Notebooklist Buffer

Kernel communication

Contents API

Connecting to a running Kernel

Entry point is [[file:lisp/ein-notebook.el::ein:notebook-start-kernel][ein:notebook-start-kernel]] which is called from ein:notebook-request-open-callback after successful call to the notebook server requesting the contents of a given notebook.

[[file:lisp/ein-kernel.el::ein:kernel-start][ein:kernel-start]] starts/gets a session with a running kernel using the REST API.

On a successful return ein creates a websocket channel (channels for IPython 2.x) via a call to websocket-open in the emacs-websocket package. The URL request is of the form:

ws://{server_address}:{port}/api/kernels/{kernel id from previous REST query}/channels?session_id={session id}

Enhancements/Fixes

Imenu/Speedbar Cooperation

Seems to be a couple ways of doing this:

1. Configuring `imenu-generic-expression` regex's.
2. Redefining imenu-create-index ala python.el.

(2) seems to be the more elegant solution.

EIN currently has minimal support for imenu through `ein:worksheet-imenu-create-index`, but all it does is look for headings. Somehow this fails to work with speedbar and also does not handle indexing Python code (i.e. variables, function, classes, etc.).

To get the speedbar working we will need to define a minor mode per the following instructions.

For /name/~-speedbar-menu-items~ can I just use imenu-generic-expression?

Maybe the way to do this is for each [[file:lisp/ein-cell.el::ein:codecell][codecell]] create a temp buffer with the text of that cell and call ein:imenu-create-index.

  (let ((text (ein:cell-get-text cell)))
    (with-temp-buffer
      (insert text)
      (ein:imenu-create-index)))

Still will need way to map temp buffer positions to actual positions in the notebook buffer (ein:cell-input-pos-min and ein:cell-input-pos-max)

Live links to other notebooks

1. Understand how org-mode does it.
2. Steal???
3. Profit!!!

Use polymode

Polymode uses indirect buffers, which may or may not be a good solution for ein notebooks.

Access password protected notebooks (issue #57)

This is what I have found out so far:

You can authenticate with the IPython/Jupyter notebook server using ein:notebooklist-login. After calling this a cookie is generated (very easy to see if you are using curl as the backend for emacs-request) and you can then use the REST API to list and get notebook data.

Once authenticated REST calls to get notebook json data and create sessions work fine. After EIN starts a session one can see the kernel is running from the web interface. The problem starts when ein tries to open a websocket connection to the kernel. The notebook server generates a 403 forbidden response. I think because emacs-websocket doesn't know anything about the security cookie generated during the curl request.

Not sure if that makes sense, but for the moment that is my theory on what's happening. Somehow we need to provide the security cookie with the websocket connect request.

<2015-06-09 Tue> SOLVED(?) - issue is that emacs-websocket needs to provide more info with the connection header:

1. Specify the port along with the url.
2. Pass along a security cookie.

Connect to non-python kernels

Synergies with pymacs?

Detect system path of opened notebook

Jump to notebook code in traceback (issue #42)

What needs to be done:

1. Carry notebook reference in the [[file:lisp/ein-traceback.el::ein:traceback][ein:traceback]] structure.
2. Look for <ipython-input-3-05c9758a9c21> in <module>(). The number 3 means input #3 in the notebook.
3. Find cell based on input number. Can iterate through list of cells () and look for matching input-prompt-number.
4. Call ein:cell-goto on that cell. May need to swap buffers first.

The Return of Worksheets

tkf/ein and IPython 2.x allowed for multiple worksheets within an individual notebook. This feature was removed in 3.0 since multiple worksheets do not make much sense in the context of a tabbed web browser interface. EIN's legacy code still supports worksheets, though at the moment that information is lost upon saving a notebook.

Having multiple worksheet support makes some sense for ein; below is thinking on how to reimplement this feature.

IPython nbformat 4 specifies a metadata key which can be used to store general information. Cell metadad has a tag key which is a "A list of string tags on the cell. Commas are not allowed in a tag."

Best place to set the tag key is when generating /hiro/emacs-ipython-notebook/src/commit/488bb67f092b4c5da5ba879fbb1be05c362d0748/content for saving a notebook.