{ "cells": [ { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "source": [ "# The Emacs IPython Notebook\n", "### JupyterCon 2018\n", "\n", "John Miller ([email](millejoh@mac.com) / [github](https://github.com/millejoh/) / [linkedin](www.linkedin.com/in/john-miller-3720a811\n", ")) ![The Mascot](_images/ein_logo.jpg)" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "source": [ "# The Plan" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "- Thanks\n", "- About me\n", "- Introducing EIN\n", "- About Emacs\n", "- A Short History\n", "- Feature dive\n", "- What's next\n", "- Demo?" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "source": [ "# Disclaimer" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "- This work is not sponsored by my employer (Honeywell UOP)\n", "- I am not representing my employer (today)\n", "- UOP does some cools things, find me later if you want to learn more!" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "source": [ "# Thanks!" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "fragment" } }, "source": [ "- Takafumi Arakami ([@tkf](https://github.com/tkf))\n", "\n", " - EIN does not exist without tkf" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "fragment" } }, "source": [ "- Fernando and the rest at Jupyter\n", "- O'Reilly - Andre Morrow in particular!\n", "- D.E. Shaw - supporting some work back in 2016-2017.\n", "- My kind, patient crew of Github ein [stargazers](https://github.com/millejoh/emacs-ipython-notebook/stargazers) (all 660 of you!)" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "# About John" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "- 1997: BS Chemical Engineering / Computer Science from CSU\n", "- Since then: [Honeywell UOP](https://www.uop.com)\n", "- Emacser since college (??)\n", "- Irregular Pythoner since 1998 (Zope to Pandas!)\n", "- EIN Maintainer since 2014/2015-ish (commit [20a7250](https://github.com/millejoh/emacs-ipython-notebook/commit/20a725012b6c8ffe9173fcfc8808dfb46fbb1824))" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "source": [ "# Introducing EIN" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "- A full-featured client for the Jupyter Notebook in the venerable Emacs editor\n", "- Tries to be a bit like [SLIME](https://common-lisp.net/project/slime/)\n", "- Work on emacs 24.5+ (but 25+ is best)\n" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "## Features\n", " - Written almost completely in elisp!\n", " - IDE features (syntax highlighting, jump to source, auto-completeion, popup help, help browser, etc.)\n", " - Integration with pdb via comint\n", " - Integration with [org-mode](https://orgmode.org/)\n", " - Works with non-python kernels!\n", " - Connect python buffers to running kernels." ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "source": [ "# About Emacs" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "The programmable programming editor.\n", "40 years of history.\n", "![real programmers](real_programmers.png)" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "source": [ "## Why do I use Emacs?" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "![Crazy emacs](_images/emacs.png)" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "- Lisp Machine's scrapier cousin from the wrong side of town\n", "- See Josh Stella's excellent blog posts ([#1](https://blog.fugue.co/2015-11-11-guide-to-emacs.html) and [#2](https://blog.fugue.co/2018-08-09-two-years-with-emacs-as-a-cto.html))\n", "- Focus without distraction\n", "- Not a black box with a shiny interface\n", "- Almost complete control over your editing environment\n", "- It's a tool building tool" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "# The Great Editor/IDE Wars" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "Let's not, actually. Brothers and sisters, read [More than Words](http://www.nhplace.com/kent/PS/Lambda.html).\n", "\n", "> In essence, I'll suggest that Lisp is a social phenomenon, akin to a\n", "> political party, and that what unifies Lisp are the people who are its\n", "> leaders, and the ways in which they respond (or fail to respond) to the needs\n", "> of that community. --- Kent Pitman" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "notes" } }, "source": [ "More than one political party is good! Diverse ideas are good! Extremism - not\n", "so good.\n", "\n", "The secret to peace - holding opposing thoughts in our heads?" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "source": [ "# A Brief History of EIN" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "fragment" } }, "source": [ "![Timeline](_images/Timeline.png)" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "- Created by [tkf](https://github.com/tkf) in 2012\n", "- First alternate client for IPython\n", "- tkf's last commit is on 17 Mar 2014 (1,795 commits later!)\n", "- I fork and push IPython 2.0 support on 14 April 2014\n", "- Supporting transition from IPython/Jupyter 1.0 -> 4.0 rough sailing!" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "# The Challenges" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "- Changes to the Contents API\n", "- Changes to communication protocol\n", "- Changes to the security model (I hate you _xsrf)\n", "- Changes to the notebook format\n", "- Debugging connection issues for other users\n", "- Legacy support (Emacs 24 vs 25, 26, 27...)\n" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "- Why?\n", " - John is agoraphobic outside of Emacs\n", " - John is allergic to javascript\n", " - EIN’s internal data structures tuned to earlier versions of notebook format\n", " - Behold the eldritch horrors of `ein:cell-to-nb4-json`\n", " - Every Emacs installation is different (advantage and disadvantage!)" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "# The Joys" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "- People (not just me) use it to do stuff!\n", "- Supportive community.\n", "- Solving the challenges.\n", "- I get to program in Lisp!" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "source": [ "# Feature Dive" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "## What is similar" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "- Visually alike, but more text-y\n", "- Cut, copy, past cells (even across notebooks!)\n", "- Inline images\n", "- Works with non-python kernels" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "## IDE-like Features" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "- Auto-completion\n", "- Jump to definition (local only!)\n", "- Popup (tooltip) help\n", "- Syntax highlighting (not perfect)\n", "- Help browser\n", "- Traceback view\n", "- Integration with ipdb" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "## Unique to Emacs?" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "- Run jupyter from inside Emacs!\n", "- Execute elisp from IPython\n", "- Integration with org-mode\n", "- Support for [hy](http://hylang.org)\n", "- Connect python buffer to a running notebook\n", "- Customizable using elisp (vs javascript)\n", "- %whos popup (`ein:pytools-whos`)\n", "- Run doctests for object at point" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "source": [ "## Needing some TLC" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "- Pandas to [SES](http://www.gnu.org/software/emacs/manual/html_node/ses/index.html)\n", "- Hierarchymagic (depends on [Hierarchymagic_ext](https://github.com/tkf/ipython-hierarchymagic))" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "subslide" } }, "source": [ "## What EIN does not do" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "- Widgets\n", "- Extensions\n", " - Emacs doesn't quite understand javascript.\n", " - Possible with some effort (see [timestamp](https://github.com/millejoh/emacs-ipython-notebook/blob/master/lisp/ein-timestamp.el))\n", " - [Skewer](https://github.com/skeeto/skewer-mode) package allows limited execution of javascript\n", "- Jupyterhub support exists, but is wonky" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "source": [ "# What's Next for EIN?" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "- Don't fall behind (easier said than done)\n", "- More robust!\n", "- Better integration with external documentation\n", "- Seamless support for remote Jupyterhub and remote connections\n", "- Support widgets (this is going to be tough!)\n", "- Better integration with the Emacs Python ecosystem\n", "- A full inspector (ala SLIME)\n", "- And much (too much!) more...\n" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "source": [ "# A Demonstration" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "%pwd" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "import sys\n", "# Watch me autocomplete and show an informative popup!\n", "# Try this before executing import sys with `ein:completion-backend`\n", "# set to `ein:use-ac-jedi-backend`. Eat your heart out Joel Grus and\n", "# JupyterLab!\n", "sys" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "print(\"Hello World!\")" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "## Behold the glory of Python and Emacs!" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "### Syntax highlighting and inline images." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "%matplotlib inline\n", "# Open via C-c ' and check out flycheck support\n", "import matplotlib.pyplot as plt\n", "import numpy as np" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "slide" } }, "outputs": [], "source": [ "plt." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "def plotnormal():\n", " return plt.plot(np.random.randn(1000), np.random.randn(1000), 'o', alpha=0.3)\n", "plotnormal()" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "### The help browser and jump to source" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "plt.plot?" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "### EIN combines python and ipython error handling with Emacs debugger support." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "%run non_existent_file" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "x = 1\n", "y = 4\n", "z = y / (1 - x)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "%run wiener_filtering.py" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "%debug" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "### What about org files" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "Take a looksy at [this](./org_demo.org)" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "### What was that about Hy?" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "skip" } }, "source": [ "Note - due to a bug in ein hy cell types are not saved between sessions, so you\n", "will need to manually set the cell type of the following two cells to Hy by\n", "calling `ein:worksheet-change-cell-type` then typing `h`" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [ { "data": { "text/plain": [ "2" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "(+ 1 1)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "(setv hy-var 42)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "hy_var" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "### What about Emacs Lisp?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "%run tools/emacslisp.py" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "res = EmacsLisp('(ein:dev-sys-info)')\n", "res" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "type(res)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "dir(res)" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "### We Support Magic!" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "%load tools/emacslisp.py" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "## Check out test.py" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "digits" ] }, { "cell_type": "markdown", "metadata": { "ein.tags": "worksheet-0", "slideshow": { "slide_type": "-" } }, "source": [ "# Don't look at me!" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "execute-time": [ "2018-10-15T02:00:45.633797Z", "2018-10-15T02:00:46.999049Z" ], "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "sns.set(context='talk')\n", "plt.rcParams[\"figure.dpi\"] = 144\n", "plt.rcParams[\"figure.figsize\"] = 16, 10" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "autoscroll": false, "collapsed": false, "ein.hycell": false, "ein.tags": "worksheet-0", "execute-time": [ "2018-10-15T03:20:48.024857Z", "2018-10-15T03:20:48.040723Z" ], "slideshow": { "slide_type": "-" } }, "outputs": [], "source": [ "digits" ] } ], "metadata": { "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Data Science (py36)", "name": "datascience" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.6" }, "name": "The Emacs IPython Notebook.ipynb", "rise": { "controls": false, "theme": "serif" } }, "nbformat": 4, "nbformat_minor": 2 }