more wip

Author: ianhi

paper/imgs/logistic_growth.png

@@ -1,16 +1,20 @@
 ---
-title: "mpl-interactions: A Python package for dynamic Matplotlib plots"
+title: "mpl-interactions: A Python package for Interactive Matplotlib Figures"
 tags:
   - Python
   - some-other-tags-maybe
 authors:
   - name: Ian Hunt-Isaak
     orcid: 0000-0002-7591-083X
-    affiliation: "1" # (Multiple affiliations must be quoted)
-  - name: Doeke Heksta
+    affiliation: 1 # (Multiple affiliations must be quoted)
+  - name: Doeke Hekstra
+    orcid: 0000-0003-2332-9223
+    affiliation: "1, 2" # (Multiple affiliations must be quoted)
 affiliations:
-  - name: John A. Paulson School of Engineering and Applied Sciences, Harvard University, USA
+  - name: John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA
     index: 1
+  - name: Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
+    index: 2
 date: 14 February 2023
 bibliography: paper.bib
 ---
@@ -19,23 +23,23 @@ bibliography: paper.bib
 
 <!-- A summary describing the high-level functionality and purpose of the software for a diverse, non-specialist audience. -->
 
-`mpl-interactions` enables users to transform static matplotlib figures to dynamic figures with control of the parameters of the plot elements through automatically generated widgets. It creates widgets and connects them to update plot elements automatically using a shorthand for widgets optimized for control of plots. In order to be as easy to use as possible it adds these features while otherwise staying as close to the `matplotlib.pyplot` interface as possible. It is built to be composable with itself while not interfering with static plotting elements. This allows for building any figure that `matplotlib` can produce while adding dynamism to specific parts as necessary to increase the utility of the plot. `mpl-interactions` has applications for data exploration, model building and pedagogy.
+Data exploration, model building and pedagogy all benefit from the ability to interactively update elements in Matplotlib figures. `mpl-interactions` enables this by making it easy for users to create matplotlib figures in which the displayed data can be dynamically controlled through widgets. These widgets can be automatically generated by passing arguments such as arrays or shorthands (such as a tuple of numbers to generate a slider) to modified pyplot functions. After creation of these widgets, `mpl-interactions` updates plot elements without further user intervention. For ease of use, it adds these features while otherwise staying close to the `matplotlib.pyplot` interface. `mpl-interactions` is built such that generated widgets are easy to reuse for multiple plot elements, while not interfering with static elements. This design allows for building any figure that `matplotlib` can produce, while adding interactivity to specific parts as desired.
+
+Complete Tutorials, Examples, and API documentation are available on https://mpl-interactions.readthedocs.io/en/stable/.
 
 # Statement of Need
 
 <!-- A Statement of need section that clearly illustrates the research purpose of the software and places it in the context of related work. -->
 
-A plot that dynamically updates as a user interacts with a widget such as a slider can be a very powerful tool in the scientific process as well as in pedagogy. For instance varying a parameter of a mathematical model plotted on top of data helps to understand the model - useful for both modelling in research and for pedagogy. Similarly it very useful during exploratory data analysis to interactively modify aspects of the plot such as points are displayed, or the threshold level of image.
-
-While it is possible to accomplish this using existing tools such as `matplotlib` and `ipywidgets`, the complexity of doing so dramatically slows down analysis and learning. `matplotlib` is a powerful Python library for visualizations. For static plots it is the defacto standard for academic work and many students and scientists have familirity produce data visualizations with it. In addition to the capability of creating static figures it has mechanisms for building interactive visualizations. Many of it's artists have APIs that allow updating them to reflect new data and it implements a basic set of widgets such as sliders and radio buttons. However, artist APIs are not consistent and sometimes under or undocumented, and positioning the widgets can be non-trivial. Learning, and then remembering the specifics of how to update an artist along with writing the boilerplate of callbacks significantly slows iteration when investigating data.
+The ability to interact dynamically with plots through widgets such as sliders can be a powerful tool in the scientific process and in pedagogy. For instance, varying a parameter of a mathematical model plotted on top of data helps to understand the relationship between the model and the data. Similarly, exploratory data analysis can be enhanced by interactively modifying aspects of the plot such as which points are displayed, or the threshold level of a displayed image.
 
-The `ipywidgets` makes widget creation easier, and handles layout for the user, but is still difficult to use quickly with matplotlib as doing so requires remembering how update matplotlib artists and writing approriate callbacks. The easiest approach for users is to use hte `ipywidgets`' `interact` function (footnote: this is inspiration for the package name) which automatically generates sliders and other widgets to control arguments to arbitrary python functions. However, the this has drawbacks as well, the recommended usage is to re-make the plot every time a parameter changes, and it only works in an context where ipywidgets can be displayed (e.g. jupyterlab). Finally, `ipywidgets` is a general framework, and thus constrained in how it's choices of how to intrepret shorthands for widget generation - the choices it makes are not always optimal for scientific plotting.
+Matplotlib provides mechanisms for updating elements (artists) in figures. However, the APIs for these artists are not consistent and some are under- or undocumented. Furthermore, the creation and positioning of the native Matplotlib widgets is nontrivial. While the `ipywidgets` library makes widget creation and positioning easier, it is difficult to integrate with matplotlib in a performant manner. The easiest way to do so is to use the `ipywidgets`' `interact()` function, which automatically generates sliders and other widgets to control arguments to arbitrary python functions. However, this can result in completely regenerating the figure which can be slow. Alternatively, the user needs to remember the specifics of how to update each individual artist. The final issue is that `ipywidgets` is a general framework, and thus constrained in its choices of how to interpret shorthands for widget generation -- as such, the choices it makes are not always optimal for scientific plotting.
 
-`mpl-interactions` builds on top of both of these packages in order make their power available while reducing the complexity of generating an interactive plot to a single line. It automates the creation of widgets, using a shorthand optimized for scientific plotting. It works anywhere matplotlib does, creating ipywidgets if they are available, but falling back to matplotlib widgets necessary. In addition when generating matplotlib widgets it automatically handles widget layout. After creating widgets it connects callbacks to update the matplotlib artists in the most performant way possible. Finally, `mpl-interactions` handles, updating the matplotlib axis limits as the plot elements update. Without this plot elements can easily go off screen.
+While `matplotlib` and `ipywidgets` provide the tools for controlling plots with widgets, the overhead of implementing such control can overwhelm its utility. `mpl-interactions` fills this gap by making it easy for users to generate widgets that dynamically control plots.
 
 # Overview
 
-`mpl-interactions` builds on top of the `matplotlib.pyplot`. While `matplotlib` requires users are to pass in arrays as arguments, `mpl-interactions` allows passing a function that returns numeric values. Parameters to these functions are specified by adding extra keyword arguments (`kwargs`) to the function call. Then `mpl-interactions` will generate the approriate widgets for the parameters and run the functions to generate the numerical data to plot. For example to plot a sinusoid and control it's amplitude and frequency using sliders a function returning the `y` values is defined and passed as the `y` parameter to the `plot` function. The ranges of the `A` and `f` parameters are defined using as extra kwargs using tuples as a widget shorthand.
+`mpl-interactions` provides several key features to make generating interactive figures simple. The first is what arguments are accepted. While `matplotlib` requires users to pass arrays as arguments, `mpl-interactions` allows passing a function that returns numeric values. Parameters to these functions are specified by adding extra keyword arguments (`kwargs`) to the plotting function call. Then `mpl-interactions` will generate the appropriate widgets for the parameters and run the functions to generate the numerical data to plot. For example, to plot a sinusoid and control its amplitude and frequency using sliders, a function returning the `y` values is defined and passed as the `y` parameter to the `plot` function. The ranges of the `A` and `f` parameters are defined as extra keyword arguments using tuples as a shorthand for what widget to generate.
 
 ```python
 import mpl_interactions.ipyplot as iplt
@@ -45,45 +49,95 @@ import numpy as np
 fig, ax = plt.subplots()
 
 def sinusoid(x, A, f):
-   return A*np.sin(x * f)
+    return A*np.sin(x * f)
+
 x = np.linspace(0, np.pi, 100)
 
 ctrls = iplt.plot(x, sinusoid, A=(1, 10), f = (.5, 2))
 plt.show()
 ```
 
-![Generated figure and sliders after running above example in jupyter lab.\label{fig:sinusoid}](imgs/sinusoid.png){ width=75% }
+![Multiple states of the figure resulting from moving the sliders after running above example in jupyter lab.\label{fig:sinusoid}](imgs/sinusoid-full.png)
 
-In addition the generated widgets can be re-used to control other elements in the figure, including scalar values. For example here a RangeSlider is created and used to control the thresholding of an image, as well as the position of two vertical lines on a histogram of pixel intensities.
+A second important feature of `mpl-interactions` is that interactive plot components are not isolated from each other. That is, the control widgets generated from one plotting call can be re-used to control other components. In addition to showing the re-use of control widgets this example demonstrates how matplotlib styling arguments (such as `vmin`) can be controlled through widgets.
 
 ```python
-import mpl_interactions.ipyplot as iplt
-import matplotlib.pyplot as plt
-import numpy as np
 N = 128
-im = np.random.randn(N * N).reshape(N, N)
+rng = np.random.default_rng(seed=1995)
+im = rng.normal(size=(N,N))
 
 fig, axs = plt.subplots(1, 2, figsize=(12, 5))
 
-# plot histogram of pixel intensities
-axs[1].hist(im.flatten(), bins="auto")
-axs[1].set_title("Histogram of Pixel Intensities")
 
 # create interactive controls
 ctrls = iplt.imshow(im, vmin_vmax=("r", im.min(), im.max()), ax=axs[0])
+
+# plot histogram of pixel intensities
+
+# by indexing the ctrls object it is possible to
+# re-use the the vmin and vmax created by imshow
+# to control the position of the axvlines
 iplt.axvline(ctrls["vmin"], ax=axs[1], c="k")
-_ = iplt.axvline(ctrls["vmax"], ax=axs[1], c="k")
+iplt.axvline(ctrls["vmax"], ax=axs[1], c="k");
+axs[1].hist(im.flatten(), bins="auto")
+axs[1].set_title("Histogram of Pixel Intensities")
+axs[1].set_xlabel('Pixel Intensity')
 ```
 
-Both of these examples are only a few lines different from their static counterpart and use a similar syntax for function calls as the pure matplotlib equivalent. This is the power of the library, allowing rapid iteration by allowing low effort addition of interactivity to static plots.
+![An imshow where the thresholds of the colorbar are controlled by the `vmin_vmax` range slider. While the slider is created by the `imshow` call, its interactive parameters can be reused to control the position of the veritcal lines on the histogram.](imgs/threshold.png)
 
-# Acknowledgements
+Finally, `mpl-interactions` allows the reuse of Python functions performing mathematical operations in multiple parts of user code, rather than writing requiring users to write a plotting specific version. For example below the `logistic_growth` function is used for both curve fitting and an interactive display in order to better understand the role of the model parameters.
 
-Ian was supported by the NDSEG + Doeke
+```python
+%matplotlib ipympl
+import matplotlib.pyplot as plt
+import numpy as np
+from scipy.optimize import curve_fit
+
+import mpl_interactions.ipyplot as iplt
+
+
+def logistic_growth(t, L, k, t0):
+    return L / (1 + np.exp(-k * (t - t0)))
+
+
+# create a synthetic dataset of logistic growth
+rng = np.random.default_rng(seed=1995)
+t_data = np.sort(rng.uniform(0, 10, size=50))
+y_data = logistic_growth(t_data, L=5, k=1, t0=1) + rng.normal(size=t_data.size, scale=0.1)
+
+
+# You can use the `logistic_growth` function to curve_fit
+popt, pcov = curve_fit(logistic_growth, t_data, y_data)
+
+
+# Now you can directly the same function to make
+# an interactive plot to better understand its parameters
+fig, axs = plt.subplots(1, 2, sharey=True)
+axs[0].plot(t_data, y_data, "o")
+axs[0].plot(t_data, logistic_growth(t_data, *popt))
+axs[0].set_title("Data + Fit")
+axs[1].set_title("Interactive Exploration")
+
+ctrls = iplt.plot(
+    np.linspace(0, 10),
+    logistic_growth,
+    L=(0.5, 10),
+    k=(0.1, 1),
+    t0=(0, 2.5),
+    ax=axs[1],
+    label="interactive",
+)
+axs[1].plot(t_data, logistic_growth(t_data, *popt), "--", label="fit")
+plt.legend()
+```
+
+![Generated figure and sliders after running above example in jupyter lab.\label{fig:logistic}](imgs/logistic_growth.png){ width=75% }
+
+# Acknowledgements
 
-Discussion: Kevin Dalton
-Contributions: John Russell, Remco De Boer, Samantha Hamilton,
+We thank Dr. K. Dalton for stimulating discussions, and [CONTRIBUTIONS]. This work was supported by a National Defense Science and Engineering Graduate Fellowship (IHI, add award number), the George W. Merck Fund of the New York Community Trust (award 338034, to DRH), and funds from Harvard University.
 
-- other contributors.
+In addition, many users have contributed features and bug fixes. Of particular note are Remco de Boer, John Russell, and Samantha Hamilton who made contributions to documentation and code, code, and documentation respectively. A full list of coding contributors can be found here: https://github.com/mpl-extensions/mpl-interactions/graphs/contributors
 
-Many users who have posted questions + feedback
+Finally, many users have contributed in ways other than coding. For example by raising issues either with the package or documentation. These users are recognized here: https://github.com/mpl-extensions/mpl-interactions#contributors-