I strongly agree with the premise of the article - HTML could be a fabulous substrate for computational notebooks!
But I didn't love the choices for how to implement it here. Dynamic, reactive HTML can be a lot more declarative than this, and Observable is cool, but strays from standard JS.
I started to build a reactive HTML system called Heximal that eventually will have notebook support, but it's declarative, based on HTML templates and custom elements with a expression / reactivity system (based on the TC39 Signals proposal) on top.
> and Observable is cool, but strays from standard JS.
The front end does but the underlying runtime is running just javascript, and the source code is basically javascript with some trivial macros which are fully captured in the MIT licensed acorn parser. That's why normal Javascript debugging expression work perfectly in Observablehq.
I love the Observable runtime. I wrote a decompiler for it so you can bidirectionally convert between the front end source and the compiled pure JS representation.
What is the need for the runtime? I've ported exported ObservableHQ code to plain JS, and the runtime bits were a lot of hard-to-read indirection. Lots of calling into function references with strings identifying the parameter names.
It seems like it might provide some kind of reactive signal abstraction, but modern signal libraries, and the TC39 Proposal, seem to do this in a lower-level and more ergonomic way.
It detects naming collisions, orchestrates recomputation with a topological sort and manages module reuse and normalizes generators and promise to a common abstraction. The parameter indirection is for separating the inner cell code from the inter-cell dependency graph. You need to do that for the partial recomputation to work without the user manually plumbing in the dependencies (like you have to do in React). I will read the TC39 I don't know it so can't compare.
Edit: yes seems like signals could be a major building block to do the recomputation part
Are you taking into account Observable Framework here? That came out in March and one of the major features was that it uses standard JavaScript, not the syntax hacks they invented for Observable Notebooks: https://observablehq.com/blog/observable-2-0#a-better-develo...
Really cool! I particularly liked your Python and SQLite demos. I find it interesting that you started with a through-the-web (TTW) editing loop, calling it a quine. While valid, the key weakness is persistance: TidlyWiki does basically the same thing, but saving your work is a headache because basically you need another headless server process to persist anything.
Personally, when wanting to do something similar[0], started with persistance, and of course this means files. Plain text files I can edit with my programmer's editor store and distribute using the file-system. This avoids the problem of having to recapitulate a programmers editor in the browser, which is non-trivial. In turn you have the problem of writing a fast server, but that is a rather fun problem to solve [1]. You side-stepped the problem by using basic content-editable sections, but it is a huge problem, second only to persistence.
One thing I see you face which I also faced is the fact you cannot export things from inside embedded scripts! This would be a nice feature to avoid mangling the global window object. (It would also, in my use case, make code generation easier.)
1 - Reflector, what I call the small node server that transforms, zips and caches your markdown, with a filewatcher invalidator: https://simpatico.io/reflector
With all due respect to the considerable thought and effort you put into this, the ergonomics of this approach are hideous. Why would I ever care about the styling elements when I'm just trying to do some exploratory data analysis. This is exactly why things like Jupyter notebooks excel. Regardless kudos to your curiosity and implementing alternate ideas.
Author: The ergonomics of this _are_ hideous, to my dismay, which was a lot of the motivation behind @celine/celine (https://maxbo.me/celine, aka me packaging the article up into a library).
It's still not quite there as a platform for exploratory data analysis - you don't have the instant reactivity of either a fully-fledged web code editor from Observable Notebooks or the hot-reloading file-watching Observable Framework. And the new Jupyter Kernel for Deno + VSCode is a pretty smooth experience too.
So while I agree that the ergonomics for exploratory analysis is uhhhh bad, I don't think the _publishing_ ergonomics is that bad. In fact, they're good. It's just a single file! I don't need to maintain some massive toolchain or pay some 3rd party service to just send someone a graph and some data munging - I just lob a HTML file at someone over Slack or host it somewhere. And the flexibility to style the analysis means that you can publish in environments where styling is important (blogs, or as a research paper).
From a brief look, I think he's trying to position it for "Exploratory DOM Analysis". The intro demo looks scheme/smalltalk like in that you are creating the structure and primitives as you go (as demonstrated by changing 'ivory' to 'red' and watching the syntax highlighting change in realtime).
I understand how we got here, but it's a shame that javascript frameworks and libraries aren't easier to just play with in the browser. It's just JS, you should be able to play with it quickly in a lightweight environment. This approach excels at that. This approach brings back the whimisical possibilities of HTML/JS. I'd love to see more stuff like that and less TS, rollup, webpack,...
edit:Actually after reading a bit, this is being proposed for data analysis. I think that's a poor fit for this approach
>when I'm just trying to do some exploratory data analysis
Isn't the point to have a unified platform to do exploratory data analysis which can then easily be published? It's not for throwaway Jupyter notebooks.
I think there's great value for an alternative to Jupyter notebooks that aren't just throwaway. The UX being really trash right now is something which can be improved IMO. The question is whether this setup is better than the JSON madness in Jupyter notebooks... I'm leaning yes personally.
Awesome work. Thank you for sharing. Yes, web technologies coming with web browsers, HTML, CSS, DOM, JAVASCRIPT, PHP, .., can be used directly inside web browsers. This is how I use them to build a tiny web text editor coming with a programming language : http://lambdaway.fr . Your opinion would be appreciated.
I enjoy the format of this post. Work from ground up to something interesting. Not tons of dependencies and framework here framework there blablabla hype.
To follow along it seems you can just copy paste the given snippets and understand step by step how it works. If I wrote something like this, I guess I would use literate programming in org-mode and export to HTML, to make it a blog post.
I still question fonts with built-in syntax highlighting.
It's JavaScript for the interactive/reactive parts, but the notebook is implemented in a self-contained HTML file: it handles everything from document structure to editing and display. It's also not dependant on 3rd party software (par browser), and is as durable, portable and easy-to-use as any HTML is.
In principle those external dependencies could be rolled together into one document and produce something truly freestanding. Once one decides to inline everything, opportunities arise to produce something simpler and more coherent.
I will soon make attempts to support this kind of HTML notebook in/for Raku.
Right now Raku's "notebook solutions" are Jupyter-based or Mathematica-based.
I don't get the argument for using web technologies to complicate this process. What does this added complexity solve? If anything, it slows the process and remove much needed features like auto-completions, snippets, and extensions. Also, it makes collaboration harder, since, most likely, others aren't familiar with your setup.
You only need to setup one environment once and use it forever with minor incremental improvements as tech evolves.
If you are a n/vim user, you can accomplish this by:
agreed, folks who want a reactive notebook should seriously consider emacs. there are many emacs users who use it for only a few things, and reactive notebooks is often one of them
Notebooks are hot these days! We also shipped our own version of a TypeScript notebook[1] but it takes quite different sides of the tradeoff: we want to run backend node code, so unlike this or observable we're not looking to run in the browser environment. Still, for many applications, this idea is a better take!
Seconding this. Thanks for sharing! Also my page crashed on iphone14 with chrome when attempting small changes to the pyodide cell (even just changing the axis labels)
The article was a great read! I wonder if you know about marimo.io; it's an open-source reactive notebook for Python which offers tons of great features. I have shifted my daily notebook usage (from what used to be colab and jupyter notebooks) to marimo notebooks over the past months.
So I've used Observable Notebooks for building interactive articles for a while now (my favorite https://observablehq.com/@mjbo/sydney-qms-panel-public-telep...). I wanted to see if I could replicate _most_ of the (reader's)
affordances of this platform in a context where I have more control.
The article is documenting me trying to do that replication. What's it used for? @celine/celine (https://maxbo.me/celine) is the packaged product that I now use for some private articles that I've been writing.
This is awesome - I have been toying with building something like this so that i can host (raku) code examples and do literate programming. FWIW there IS a good raku plugin for Jupyter Chatbooks (https://raku.land/zef:antononcube/Jupyter::Chatbook), but it just seems kinda wrong for raku to need python to do this.
Can this example be widened to use server side code (I am not too bothered about syntax highlighting as maybe can make a font for that) - but it would be great to have some kind of message layer (like zmq in Jupyter) to connect to a remote language kernel perhaps using web sockets???
Raku does not (yet) target wasm so that option is out.
> Can this example be widened to use server side code?
Author: Nothing stopping you from forcing a Jupyter notebook to self-host itself: https://hello-notebook-http-mode.fly.dev/ . Then you can just use HTTP to send messages back and forth between the browser and the notebook.
I was certain you were using something like https://pagedjs.org or https://printcss.net for the PDF export, but its just the SydJS or am I missing something?
I strongly agree with the premise of the article - HTML could be a fabulous substrate for computational notebooks!
But I didn't love the choices for how to implement it here. Dynamic, reactive HTML can be a lot more declarative than this, and Observable is cool, but strays from standard JS.
I started to build a reactive HTML system called Heximal that eventually will have notebook support, but it's declarative, based on HTML templates and custom elements with a expression / reactivity system (based on the TC39 Signals proposal) on top.
https://github.com/elematic/heximal
It's a bit like a mashup of HTMX, Tangle, Curvenote, and Polymer. Or like HTML if were natively reactive.
I think it will lend it self to graphical editing and notebook user cases quite well.
> and Observable is cool, but strays from standard JS.
The front end does but the underlying runtime is running just javascript, and the source code is basically javascript with some trivial macros which are fully captured in the MIT licensed acorn parser. That's why normal Javascript debugging expression work perfectly in Observablehq.
https://github.com/observablehq/parser
I love the Observable runtime. I wrote a decompiler for it so you can bidirectionally convert between the front end source and the compiled pure JS representation.
https://observablehq.com/@tomlarkworthy/observablejs-toolcha...
What is the need for the runtime? I've ported exported ObservableHQ code to plain JS, and the runtime bits were a lot of hard-to-read indirection. Lots of calling into function references with strings identifying the parameter names.
It seems like it might provide some kind of reactive signal abstraction, but modern signal libraries, and the TC39 Proposal, seem to do this in a lower-level and more ergonomic way.
It detects naming collisions, orchestrates recomputation with a topological sort and manages module reuse and normalizes generators and promise to a common abstraction. The parameter indirection is for separating the inner cell code from the inter-cell dependency graph. You need to do that for the partial recomputation to work without the user manually plumbing in the dependencies (like you have to do in React). I will read the TC39 I don't know it so can't compare.
Edit: yes seems like signals could be a major building block to do the recomputation part
"Observable is cool, but strays from standard JS"
Are you taking into account Observable Framework here? That came out in March and one of the major features was that it uses standard JavaScript, not the syntax hacks they invented for Observable Notebooks: https://observablehq.com/blog/observable-2-0#a-better-develo...
Really cool! I particularly liked your Python and SQLite demos. I find it interesting that you started with a through-the-web (TTW) editing loop, calling it a quine. While valid, the key weakness is persistance: TidlyWiki does basically the same thing, but saving your work is a headache because basically you need another headless server process to persist anything.
Personally, when wanting to do something similar[0], started with persistance, and of course this means files. Plain text files I can edit with my programmer's editor store and distribute using the file-system. This avoids the problem of having to recapitulate a programmers editor in the browser, which is non-trivial. In turn you have the problem of writing a fast server, but that is a rather fun problem to solve [1]. You side-stepped the problem by using basic content-editable sections, but it is a huge problem, second only to persistence.
One thing I see you face which I also faced is the fact you cannot export things from inside embedded scripts! This would be a nice feature to avoid mangling the global window object. (It would also, in my use case, make code generation easier.)
0 - Literate Markdown: https://simpatico.io/lit.md
1 - Reflector, what I call the small node server that transforms, zips and caches your markdown, with a filewatcher invalidator: https://simpatico.io/reflector
With all due respect to the considerable thought and effort you put into this, the ergonomics of this approach are hideous. Why would I ever care about the styling elements when I'm just trying to do some exploratory data analysis. This is exactly why things like Jupyter notebooks excel. Regardless kudos to your curiosity and implementing alternate ideas.
Author: The ergonomics of this _are_ hideous, to my dismay, which was a lot of the motivation behind @celine/celine (https://maxbo.me/celine, aka me packaging the article up into a library).
It's still not quite there as a platform for exploratory data analysis - you don't have the instant reactivity of either a fully-fledged web code editor from Observable Notebooks or the hot-reloading file-watching Observable Framework. And the new Jupyter Kernel for Deno + VSCode is a pretty smooth experience too.
So while I agree that the ergonomics for exploratory analysis is uhhhh bad, I don't think the _publishing_ ergonomics is that bad. In fact, they're good. It's just a single file! I don't need to maintain some massive toolchain or pay some 3rd party service to just send someone a graph and some data munging - I just lob a HTML file at someone over Slack or host it somewhere. And the flexibility to style the analysis means that you can publish in environments where styling is important (blogs, or as a research paper).
Pluto has a way to publish as HTML, even with some interactivity, via SliderServer. How is this different?
Well the edited artifact and the published artifact are one and the same. I think there's value in a "buildstepless" notebook format.
Thanks for the heads up about SliderServer btw, was experimenting with something similar with Jupyter: https://hello-notebook-http-mode.fly.dev/
yeah, but all the data analysis libs are in python.
From a brief look, I think he's trying to position it for "Exploratory DOM Analysis". The intro demo looks scheme/smalltalk like in that you are creating the structure and primitives as you go (as demonstrated by changing 'ivory' to 'red' and watching the syntax highlighting change in realtime).
I understand how we got here, but it's a shame that javascript frameworks and libraries aren't easier to just play with in the browser. It's just JS, you should be able to play with it quickly in a lightweight environment. This approach excels at that. This approach brings back the whimisical possibilities of HTML/JS. I'd love to see more stuff like that and less TS, rollup, webpack,...
edit:Actually after reading a bit, this is being proposed for data analysis. I think that's a poor fit for this approach
>when I'm just trying to do some exploratory data analysis
Isn't the point to have a unified platform to do exploratory data analysis which can then easily be published? It's not for throwaway Jupyter notebooks.
I think there's great value for an alternative to Jupyter notebooks that aren't just throwaway. The UX being really trash right now is something which can be improved IMO. The question is whether this setup is better than the JSON madness in Jupyter notebooks... I'm leaning yes personally.
Awesome work. Thank you for sharing. Yes, web technologies coming with web browsers, HTML, CSS, DOM, JAVASCRIPT, PHP, .., can be used directly inside web browsers. This is how I use them to build a tiny web text editor coming with a programming language : http://lambdaway.fr . Your opinion would be appreciated.
The original author of Lit recently left Google and is working on something very similar to this AFAIK.
https://github.com/elematic/heximal
I didn't realize he'd announced what he's been working on after. Thanks for the tip.
I would say that this is more just one of the things I'm experimenting with :)
I'm also working on some WYSIWYG component authoring and visual programming tools. It's all very exploratory still.
I enjoy the format of this post. Work from ground up to something interesting. Not tons of dependencies and framework here framework there blablabla hype.
To follow along it seems you can just copy paste the given snippets and understand step by step how it works. If I wrote something like this, I guess I would use literate programming in org-mode and export to HTML, to make it a blog post.
I still question fonts with built-in syntax highlighting.
Reactive HTML but it’s almost entirely JavaScript, or am I misunderstanding something?
It's JavaScript for the interactive/reactive parts, but the notebook is implemented in a self-contained HTML file: it handles everything from document structure to editing and display. It's also not dependant on 3rd party software (par browser), and is as durable, portable and easy-to-use as any HTML is.
It imports the ObservableHQ runtime in one of the first code blocks.
In principle those external dependencies could be rolled together into one document and produce something truly freestanding. Once one decides to inline everything, opportunities arise to produce something simpler and more coherent.
I actually did that in observable userspace https://observablehq.com/@tomlarkworthy/exporter
"Why does everyone use Colab or Observable to run notebooks in a browser?
Anyway, here's an alternative that uses Observable for the hard parts."
I will soon make attempts to support this kind of HTML notebook in/for Raku. Right now Raku's "notebook solutions" are Jupyter-based or Mathematica-based.
I tend to agree with the premise of this article -- that [HTML is underutilized as a medium for disseminating scientific writing].
Some recent work related to this done by Will Crichton:
https://willcrichton.net/nota/ https://willcrichton.net/notes/portable-epubs/
I don't get the argument for using web technologies to complicate this process. What does this added complexity solve? If anything, it slows the process and remove much needed features like auto-completions, snippets, and extensions. Also, it makes collaboration harder, since, most likely, others aren't familiar with your setup.
You only need to setup one environment once and use it forever with minor incremental improvements as tech evolves.
If you are a n/vim user, you can accomplish this by:
1- Data exploration: text -> jupyter notebook via https://github.com/untitled-ai/jupyter_ascending that uses jupytext. This way, you can efficiently edit and run code from your text editor.
2- Writing: you can use https://github.com/lervag/vimtex for LaTeX
On top of these, you can use tmux with tmuxp to open projects instantly.
You can have a seamless process with Emacs using org mode [1] [2] and/or Auctex.
[1] https://sqrtminusone.xyz/posts/2021-05-01-org-python/
[2] https://martibosch.github.io/jupyter-emacs-universe/
When I fire up my jupyter notebook in my browser, what technologies am I using?
When I fire up OPs website, what technologies am I using?
All good points.
Except, I do not like Python that much, I would rather use non-Python-centric tools. (That is why I talk about Raku and Mathematica.)
BTW, thanks for pointing to https://github.com/imbue-ai/jupyter_ascending !
agreed, folks who want a reactive notebook should seriously consider emacs. there are many emacs users who use it for only a few things, and reactive notebooks is often one of them
I'm saying this as someone who lives in Emacs: it's all fine until you want to show the notebook to someone else.
As a primarily (neo)vim user, I can relate.
Notebooks are hot these days! We also shipped our own version of a TypeScript notebook[1] but it takes quite different sides of the tradeoff: we want to run backend node code, so unlike this or observable we're not looking to run in the browser environment. Still, for many applications, this idea is a better take!
Kudos to the author.
https://github.com/srcbookdev/srcbook
First of all, incredible. This is right up my alley.
I tried editing the pyodide one (which rendered fine before editing) on my phone (iphone 13 + brave) and it crashed and reloaded the page.
Seconding this. Thanks for sharing! Also my page crashed on iphone14 with chrome when attempting small changes to the pyodide cell (even just changing the axis labels)
The article was a great read! I wonder if you know about marimo.io; it's an open-source reactive notebook for Python which offers tons of great features. I have shifted my daily notebook usage (from what used to be colab and jupyter notebooks) to marimo notebooks over the past months.
Author here! Happy to take questions!
I have no idea what this is. It's way over my head. Can you point me to a source that can explain what I'm looking at, and what it's used for?
So I've used Observable Notebooks for building interactive articles for a while now (my favorite https://observablehq.com/@mjbo/sydney-qms-panel-public-telep...). I wanted to see if I could replicate _most_ of the (reader's) affordances of this platform in a context where I have more control.
The article is documenting me trying to do that replication. What's it used for? @celine/celine (https://maxbo.me/celine) is the packaged product that I now use for some private articles that I've been writing.
This reminds me of https://tiddlywiki.com/
Ohh that Windows XP dinosaur cursor brought back so many memories!
As did the low res "Made with Microsoft GitHub Pages" banner at the bottom. What is this 90s aesthetic?
This is awesome - I have been toying with building something like this so that i can host (raku) code examples and do literate programming. FWIW there IS a good raku plugin for Jupyter Chatbooks (https://raku.land/zef:antononcube/Jupyter::Chatbook), but it just seems kinda wrong for raku to need python to do this.
Can this example be widened to use server side code (I am not too bothered about syntax highlighting as maybe can make a font for that) - but it would be great to have some kind of message layer (like zmq in Jupyter) to connect to a remote language kernel perhaps using web sockets???
Raku does not (yet) target wasm so that option is out.
Thanks for mentioning "Jupyter::Chatbook"!
There is also "RakuMode" for Mathematica: https://resources.wolframcloud.com/PacletRepository/resource...
> Can this example be widened to use server side code?
Author: Nothing stopping you from forcing a Jupyter notebook to self-host itself: https://hello-notebook-http-mode.fly.dev/ . Then you can just use HTTP to send messages back and forth between the browser and the notebook.
The cursor drives me nuts.
It brings me 20 years back. I liked the nostalgia :).
I think you mean thirty years, 2004 was almost web 2.0.
Call me crazy -- I like it's whimsical nature
Awesome implementation!
> 3. Export to .pdf for distribution
I was certain you were using something like https://pagedjs.org or https://printcss.net for the PDF export, but its just the SydJS or am I missing something?
Wow, this is lovely! I'm rather inspired to make something with it
This breaks so bad for me on my phone :-(
obervables move to modules makes a lot of this tenable. Im still using their hosted gui but this is inspiring me :)
You made my weed.