TODO: keep it simple. Just show how to connect it manually and do a data analysis chart like question word length?
Livebook started out as Elixir’s version of Jupyter Notebooks but has quickly gone beyond that. Now, Livebook is a very capable and powerful tool that can be put to many interesting uses.
One incredibly useful example is to create a runbook. This is basically a “compilation of routine procedures and operations that the system administrator or operator carries out.” When you realize that you can attach a local running Livebook to a remote live Elixir system, you can start to imagine some of the useful things it can do for you.
Metabase is a great tool for analyzing the data in your database. You point it at your database and it can help gain insight. You can create custom reports, or graphs, or data views that apply your business logic to the data. This is what defines an “active user”… how many active users do we have?
Metabase requires hosting your own server. The tool is nice because a non-developer (aka business person) can use it with some data guidance.
Using Livebook in this way is different. It executes your code! Your code already defines “an active user”. In the startup space, you don’t have time to create the admin pages for showing data charts and running reports. Because as soon as you run a report, you say… “oooh, so what about this?” Gaining insights leads you to ask more questions! This is good!
With a mature app, you’ve asked all the questions and decided on which ones are important and figured out how to answer them. You codify that into an admin report. How the report works isn’t changing frequently.
Using Livebook works really well for exploring the system. You can load a saved notebook re-run a report when needed, and expand or adjust things easily as well. Also, your analysis can go beyond just data. It can pull data from GenServers, read ETS tables, access BEAM statistics like RAM use, process counts, and more.
You can do this without having to learn a new query language like using Grafana or PromQL for Prometheus data. You just use Elixir!
TODO: Show a tictac graph of “active games” that shows data from a Registry ETS table.
A Common Startup Situation
In start-ups, or early-stage projects, the goal is to create an MVP, or Minimum Viable Product. The goal is to answer the question of “What’s the least we can build that solves the problem and validates the business idea?” Developers living with a project like that are working fast and furious to create the needed features for the clients of the system. That means all the Customer Support Dashboards and Admin Tools just don’t exist yet! And they probably won’t for some time. In this environment, any special needs are handled manually. Developers and Operations staff being assembling sets of custom scripts they use to “fix” things, “unlock” an account, or “run that weekly report”.
I’m making the case that Livebook can do more to solve this situation than you might have thought possible!
What Can I Do With It?
Using recent Livebook features like “inputs”, Vega-Lite for charts, and data table displays with Kino, Livebook has everything you need to pull this off!
You can connect a local running Livebook to your production Elixir application on Fly.io.
Once you have a remote connection, you can do things like:
- Using the “input” feature in Livebook, provide an email or username to find an account and make it an admin. For safety, break it out into a few steps with “checks” performed first.
- Locate an account and grant it “full membership”.
- Issue a refund.
- Generate a revenue report for the month. Including table data output and a graph.
- Report on customer sign-up rates.
These are just some ideas of the kinds of problems you can solve.
Why Livebook is a Good Solution
You can actually develop the solution in a notebook. Meaning you write out the code for doing the work. Defining and running the queries, operating on the data, performing the updates. Then, when you have a good working solution, you can turn that into actual project code, and refactor the notebook to just call that code.
The notebooks can live inside your project source code. So they are version controlled, kept up-to-date, and shared among the team. The notebooks are Markdown files so they are git-friendly too.
Creating a Notebook as a Runbook
First, it’s important to understand that a Notebook is designed to be run from top to bottom. You can’t run an Elixir cell at the bottom without it first running all the Elixir cells that come before it. This means you can’t create a single Notebook with many scripts for different purposes collected in one place. Each separate “purpose” needs it’s own Notebook.
To help with this, create an index.livemd file to use as an initial entry point and serves as a jumping off point into something more purpose built.
The index page can group, organize, and describe the collection of scripts.
A notebook is designed to execute from start to finish. The goal is to make it repeatable as well. This means it IS NOT A GOOD FIT for putting lots of different administrative scripts into a single notebook. Instead, break up notebooks to be single purpose. Then use the filename to locate and access the desire one.
How a Notebook Works for This
Why Livebook is a Good Solution
- notebooks are plain markdown and can be checked in to a git repository and be versioned and shared
- Livebook 0.2 added support for inputs, making it easier to work with dynamic situations
Current Livebook Shortcomings
Shortcomings for this specific purpose
TODO: I want a way to launch Livebook with commands to auto-attach using ENV or arguments and auto-load a specific notebook. Make it easy and immediate to launch TODO: I would like to either support the config I need using livebook as a hex package or executing it from a configured location. Then a script like “lb notebook/admin.md” could be executed to auto-load a local project notebook, detect the app name, IP address, launch livebook, attach to the node, etc.
Connecting Livebook to Your Fly.io App
Follow this guide for setting up a local Livebook that connects to your production app.
Where This Could Go in the Future
How to improve the process with Livebook.