🎉 Pipeline formalization, including scikit-learn block wrappers

CHANGELOG.md

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- (pipelines/*) add plaid_blocks.py and sklearn_block_wrappers.py: mechanisms to define ML pipeline on plaid datasets, that staisfy the sklearn conventions
 - (split.py) add mmd_subsample_fn to subsample datasets based on tabular input data
 - (CHANGELOG.md) initiale CHANGELOG
 - PLAID benchmarks and source code

docs/conf.py

@@ -157,6 +157,9 @@
 #     #     - Functions
 #     #     - Methods
 
+nb_execution_mode = 'auto'
+nb_execution_timeout = 300
+
 numfig = True
 
 # -----------------------------------------------------------------------------#
@@ -214,7 +217,6 @@
 
 # -----------------------------------------------------------------------------#
 
-
 def skip_logger_attribute(app, what, name, obj, skip, options):
     if what == "data" and "logger" in name:
         print(f"WILL SKIP: {what=}, {name=}")

docs/index.rst

@@ -64,38 +64,46 @@ It has been developped at SafranTech, the research center of `Safran group <http
 
 The code is hosted on `GitHub <https://github.com/PLAID-lib/plaid>`_
 
+
 .. toctree::
    :glob:
    :maxdepth: 1
    :caption: Getting Started
 
-   source/getting_started.md
+   source/quickstart.md
    source/description.md
 
 .. toctree::
    :glob:
    :maxdepth: 1
-   :caption: Advanced
+   :caption: Documentation
 
-   source/contributing.md
+   API Reference <autoapi/plaid/index>
+   Examples & Tutorials <source/notebooks.rst>
+   Data Conversion Guide <source/notebooks/convert_users_data_into_plaid>
+   Default Values Flowchart <source/default_values.rst>
 
 .. toctree::
    :glob:
    :maxdepth: 1
-   :caption: API Documentation
+   :caption: Going further
 
-   Autoapi <autoapi/plaid/index>
-   Basic examples <source/notebooks.rst>
-   Convert data into PLAID <source/notebooks/convert_users_data_into_plaid>
-   Default values flowchart <source/default_values.rst>
+   CGNS Standard <http://cgns.github.io/>
+   PLAID Benchmarks <source/plaid_benchmarks.rst>
 
 .. toctree::
    :glob:
    :maxdepth: 1
-   :caption: Going further
+   :caption: Citation
 
-   CGNS standard <http://cgns.github.io/>
-   PLAID Benchmarks <source/plaid_benchmarks.rst>
+   source/citation.md
+
+.. toctree::
+   :glob:
+   :maxdepth: 1
+   :caption: Contributing
+
+   source/contributing.md
 
 Indices and tables
 ==================

docs/refs.bib

@@ -19,7 +19,7 @@ @inproceedings{airfrans
 }
 
 @misc{casenave2025plaid,
-      title={Physics-Learning AI Datamodel (PLAID) datasets: a collection of physics simulations for machine learning},
+      title={{Physics-Learning AI Datamodel (PLAID) datasets: a collection of physics simulations for machine learning}},
       author={Casenave, F. and Roynard, X. and Staber, B. and Piat, W. and Bucci, M. A. and Akkari, N. and Kabalan, A. and Nguyen, X. M. V. and Saverio, L. and Carpintero Perez, R. and Kalaydjian, A. and Fouch\'{e}, S. and Gonon, T. and Najjar, G. and Menier, E. and Nastorg, M. and Catalani, G. and Rey, C.},
       year={2025},
       archivePrefix={arXiv},

docs/source/citation.md

@@ -0,0 +1,24 @@
+# How to Cite
+
+If you have used PLAID in your work and found it useful, please consider citing the following papers.
+
+## JOSS paper
+
+Currently under review:
+
+[![JOSS status](https://joss.theoj.org/papers/26b2e13a9fc8e012cc997ca28a7b565e/status.svg)](https://joss.theoj.org/papers/26b2e13a9fc8e012cc997ca28a7b565e)
+
+
+## PLAID datasets
+
+```bibtex
+@misc{casenave2025plaid,
+      title={{Physics-Learning AI Datamodel (PLAID) datasets: a collection of physics simulations for machine learning}},
+      author={Casenave, F. and Roynard, X. and Staber, B. and Piat, W. and Bucci, M. A. and Akkari, N. and Kabalan, A. and Nguyen, X. M. V. and Saverio, L. and Carpintero Perez, R. and Kalaydjian, A. and Fouch\'{e}, S. and Gonon, T. and Najjar, G. and Menier, E. and Nastorg, M. and Catalani, G. and Rey, C.},
+      year={2025},
+      archivePrefix={arXiv},
+      primaryClass={cs.LG},
+      url={https://arxiv.org/abs/2505.02974},
+}
+```
+

docs/source/contributing.md

@@ -1,12 +1,12 @@
-# Contributing to PLAID
+# How to Contribute
 
 Thank you for your interest in contributing to PLAID Project! We welcome contributions from the community to help make this project even better.
 
 ## Summary
 
 Everything you need to know if you want to contribute.
 
-- [Contributing to PLAID](#contributing-to-plaid)
+- [Contributing to PLAID](#how-to-contribute)
   - [Summary](#summary)
   - [1. Before contributing](#1-before-contributing)
   - [2. Reporting Issues](#2-reporting-issues)

docs/source/description.md

@@ -4,9 +4,8 @@ This page, still under construction, provides elements on PLAID functionalities.
 
 ---
 
-- [Description](#description)
-  - [1. Datamodel](#1-datamodel)
-  - [2. How to use it ?](#2-how-to-use-it-)
+- [1. Datamodel](#1-datamodel)
+- [2. How to use it ?](#2-how-to-use-it-)
 
 ---
 

docs/source/notebooks.rst

@@ -24,3 +24,4 @@ You can find here all basic detailed examples different part of PLAID-lib, nicel
    notebooks/init_with_tabular
    notebooks/interpolation
    notebooks/huggingface
+   notebooks/pipeline

docs/source/notebooks/config_pipeline.yml

@@ -0,0 +1,35 @@
+input_scalar_scaler:
+  in_features_identifiers:
+    - type: scalar
+      name: angle_in
+    - type: scalar
+      name: mach_out
+
+pca_nodes:
+  in_features_identifiers:
+    - type: nodes
+      base_name: Base_2_2
+  out_features_identifiers:
+    - type: scalar
+      name: reduced_nodes_*
+
+pca_mach:
+  in_features_identifiers:
+    - type: field
+      name: mach
+      base_name: Base_2_2
+  out_features_identifiers:
+    - type: scalar
+      name: reduced_mach_*
+
+regressor_mach:
+  in_features_identifiers:
+    - type: scalar
+      name: angle_in
+    - type: scalar
+      name: mach_out
+    - type: scalar
+      name: reduced_nodes_*
+  out_features_identifiers:
+    - type: scalar
+      name: reduced_mach_*

docs/source/notebooks/init_with_tabular.ipynb

@@ -20,7 +20,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -30,7 +30,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -40,7 +40,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -62,32 +62,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "tabular_data {\n",
-      "     scalar_0 : [-0.19303611 -0.18965825  0.12534278  0.16366327 -1.06532803  0.16960836\n",
-      " -0.50639747  0.35251503  1.61444411  0.20107186]\n",
-      "     scalar_1 : [-0.38257908 -0.82167722  1.23050277  1.17466345  1.22704241 -0.17093516\n",
-      "  0.30285162 -0.8562849  -1.27164055  0.34865076]\n",
-      "     scalar_2 : [ 2.26466948  0.77352161  1.82261031  0.08872893  0.39298522 -0.88340464\n",
-      " -0.29684834  0.48175612 -1.86906676 -0.87729029]\n",
-      "     scalar_3 : [ 0.21884728 -0.7854321  -1.41677387 -0.89415003 -0.59955508 -0.65567448\n",
-      " -0.98137585 -1.15201304 -1.28867388 -0.33766666]\n",
-      "     scalar_4 : [ 0.76753223  0.14741383  1.08377073  0.15641287 -0.69648491  0.0851449\n",
-      " -0.64294282  2.56287175  0.52314472 -1.41328651]\n",
-      "     scalar_5 : [ 1.13479088 -0.65772577  0.71878731 -0.33928161  0.45507802 -0.16504924\n",
-      " -1.05053809 -0.23645522 -2.18759612  1.12057703]\n",
-      "     scalar_6 : [-0.94867932 -0.61500724 -1.61546653  2.35936912 -0.20271597 -1.67890531\n",
-      "  0.45858461  1.73382506  0.71469664 -0.84691252]\n",
-      "}\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "# Generate random tabular data for multiple scalars\n",
     "nb_scalars = 7\n",
@@ -103,17 +80,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Initialized Dataset:  Dataset(10 samples, 7 scalars, 0 fields)\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "# Initialize a dataset with the tabular data\n",
     "dataset = initialize_dataset_with_tabular_data(tabular_data)\n",
@@ -129,17 +98,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "sample_1 = Sample(7 scalars, 0 timestamps, 0 fields, no tree)\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "# Retrieve and print the dataset and specific samples\n",
     "sample_1 = dataset[1]\n",
@@ -148,17 +109,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Scalar 'scalar_0' in Sample 1: -0.1896582515164902\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "# Access and display the value of a particular scalar within a sample\n",
     "scalar_value = sample_1.get_scalar(\"scalar_0\")\n",
@@ -167,25 +120,9 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Tabular Data Subset for Scalars 1, 3, and 5:\n",
-      "tabular_data_subset {\n",
-      "     scalar_1 : [-0.38257908 -0.82167722  1.23050277  1.17466345  1.22704241 -0.17093516\n",
-      "  0.30285162 -0.8562849  -1.27164055  0.34865076]\n",
-      "     scalar_3 : [ 0.21884728 -0.7854321  -1.41677387 -0.89415003 -0.59955508 -0.65567448\n",
-      " -0.98137585 -1.15201304 -1.28867388 -0.33766666]\n",
-      "     scalar_5 : [ 1.13479088 -0.65772577  0.71878731 -0.33928161  0.45507802 -0.16504924\n",
-      " -1.05053809 -0.23645522 -2.18759612  1.12057703]\n",
-      "}\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "# Retrieve tabular data from the dataset based on scalar names\n",
     "scalar_names = [\"scalar_1\", \"scalar_3\", \"scalar_5\"]\n",