|
| 1 | +# Overview |
| 2 | + |
| 3 | +*datalab*'s block system provides a modular approach to data processing and visualization. |
| 4 | +Each block type is a specialized component that handles specific kinds of data and operations, making it easy to extend the system's capabilities without modifying the core architecture. |
| 5 | +Typically, a given technique (e.g., XRD, NMR) will have its own block. |
| 6 | +Blocks can be implemented either in the main package, or as a plugin (see ["Plugins"](/plugins)). |
| 7 | + |
| 8 | +Data blocks are modular components that: |
| 9 | + |
| 10 | +1. Process specific file types and data formats for a technique or set of techniques, |
| 11 | +2. Generate visualizations and plots from this data to be shown in the UI, |
| 12 | +3. Store and manage their own state persistently in a database, |
| 13 | +4. Can be attached to individual items or collections in your data management system, |
| 14 | +5. Provide a mechanism for handling "events" through a decorator-based registration system, |
| 15 | +6. Expose a consistent API for creation, updating, and deletion. |
| 16 | +7. Handle logging, errors and warnings in a consistent way to show in the UI. |
| 17 | + |
| 18 | +## Block Lifecycle |
| 19 | + |
| 20 | +1. **Creation**: Blocks are instantiated with an item or collection ID |
| 21 | +2. **Initialization**: Initial state is set up, potentially including file data and defaults |
| 22 | +3. **Processing**: Data is processed, plots are generated, and state is updated |
| 23 | +4. **Serialization**: Block state is serialized for storage or transmission |
| 24 | +5. **Update**: Blocks can receive updates from the web interface |
| 25 | +6. **Deletion**: Blocks can be removed from items or collections |
| 26 | + |
| 27 | + |
| 28 | +## Web API |
| 29 | + |
| 30 | +The block system exposes several API endpoints: |
| 31 | + |
| 32 | +- `/add-data-block/`: Create and add a new block to an item |
| 33 | +- `/add-collection-data-block/`: Create and add a new block to a collection |
| 34 | +- `/update-block/`: Update an existing block's state |
| 35 | +- `/delete-block/`: Remove a block from an item |
| 36 | +- `/delete-collection-block/`: Remove a block from a collection |
| 37 | + |
| 38 | +## Creating a new block |
| 39 | + |
| 40 | +To create a new block type: |
| 41 | + |
| 42 | +1. Create a class that inherits from `DataBlock` |
| 43 | +2. Define the accepted file extensions and block metadata (descriptions will be used to populate the UI documentation automatically)- |
| 44 | +3. Implement data processing and visualization methods, with e.g., JSON-serialized Bokeh plots stored in the `self.data["bokeh_plot_data"]` attribute |
| 45 | +4. Any data to be stored in the database can be defined in the `self.data` attribute |
| 46 | +5. Register any event handlers using the `@event` decorator |
| 47 | +5. Add the block type to the `BLOCK_TYPES` registry |
| 48 | + |
| 49 | +By default, a generic UI component will be used in the *datalab* interface that |
| 50 | +will make use of titles, descriptions, accepted file extensions to render a |
| 51 | +simple user interface for the block. |
| 52 | +When the user loads the block in the UI, the block's `plot_functions` methods |
| 53 | +will be called in turn, which will either load from scratch, or load cached data |
| 54 | +for that block. |
| 55 | +If a JSON-serialized Bokeh plot is found in the block's data, this will be |
| 56 | +rendered in the UI. |
| 57 | + |
| 58 | +## Event system |
| 59 | + |
| 60 | +The event system allows external functions to be called by name, enabling clean interaction between the frontend and server-side block functionality. |
| 61 | +This is a new feature and this documentation will evolve alongside it. |
| 62 | + |
| 63 | +Currently, the event system allows: |
| 64 | + |
| 65 | +- Registration of event handlers in Python via the `@event` decorator |
| 66 | +- Access to available events at both class and instance levels |
| 67 | +- Runtime dispatch of events based on name |
| 68 | +- Support for event parameters passed as keyword arguments |
| 69 | +- Events can then be triggered by the front-end; for example, a Bokeh-based block can trigger an event in a Bokeh callback using the [`CustomEvent`](https://developer.mozilla.org/en-US/docs/Web/API/CustomEvent/CustomEvent) API, for example: |
| 70 | + ```javascript |
| 71 | + const event = new CustomEvent("block-event", { |
| 72 | + detail: { |
| 73 | + block_id: '<block_id>', |
| 74 | + event_name: '<event_name>', |
| 75 | + state_data: '<some data>', |
| 76 | + }, |
| 77 | + bubbles: true |
| 78 | + }); |
| 79 | + document.dispatchEvent(event); |
| 80 | + ``` |
| 81 | + The base data block (`DataBlockBase.vue`) will listen for such events registered as `'block-event'` and pass them to the appropriate server-side block. |
| 82 | + An example callback generator for an event consisting of a single parameter |
| 83 | + update can be found at [`generate_js_callback_single_float_parameter`][pydatalab.blocks.base.generate_js_callback_single_float_parameter]. |
| 84 | + |
| 85 | +## Future Directions |
| 86 | + |
| 87 | +Future updates to the block system will focus on: |
| 88 | + |
| 89 | +- Reducing boilerplate code required for new block types. |
| 90 | +- Enhanced automatic caching after block creation. |
| 91 | +- Improving the event system to enable richer UI interactions, e.g,. setting user parameters or controlling default plot styles. |
| 92 | +- Providing better support for custom user interfaces (i.e., allowing plugins to also specify custom Vue code). |
0 commit comments