GSOC: Probabilistic Machine Learning for Solar Forecasting: Applying Gaussian Mixture Models for output #448

murphytarra · 2025-08-25T20:22:29Z

Switchable Probabilistic Output: Quantile Regression ⇄ Gaussian Mixture Model (GMM)

Summary

This PR adds a configurable probabilistic head so users can choose between the existing quantile regression output and a new Gaussian Mixture Model (GMM) output.
When GMM is selected, the model predicts per-component weights, means, and standard deviations instead of fixed quantiles.

Main files touched: base_model.py, lightning_module.py

What’s changed

New output mode: gmm
- Predicts K mixture weights (softmax-normalized), means, and stds (softplus-constrained).
- Trains with negative log-likelihood of the Gaussian mixture.
Existing output mode: quantile
- Unchanged behaviour; trains with pinball loss.
Inference utilities for GMM:
- Compute mean/variance from mixture parameters. This is in each epoch update.
Tests added (finishing touches pending; to be pushed shortly).

API / Configuration

Minimal, explicit configuration:

# model config
num_gmm_components: 3            # only used if output_distribution == "gmm"

Use above instead of "num_quantiles" - if both given flagged error.

Output tensors

Quantile mode (unchanged):

Shape: [B, T, Q] (batch, horizon, num_quantiles)

GMM mode:

Weights: [B, T, K] (softmax over K)
Means: [B, T, K]
Stds: [B, T, K] (softplus > 0)

Internally we apply softmax to weights and softplus to stds to ensure valid parameters.

Backward compatibility

Default keeps quantile mode, so existing training/eval pipelines continue to work untouched.
If you opt into gmm, you will get different output heads and a different training loss

Training / Loss

Quantile: pinball loss over requested quantiles (unchanged).
GMM: exact negative log-likelihood of a K-component diagonal Gaussian mixture.

Evaluation & Inference Notes

For deterministic point forecasts in GMM mode, use mixture expected value:

$$\hat{y} = \sum_{k=1}^{K} \pi_k \mu_k$$

Migration guide

Staying with quantile: no action required.

Switching to gmm:

num_gmm_components: K in config file and get rid of num_quantiles.
Get rid of num_quantiles

Checklist:

[ x] My code follows OCF's coding style guidelines
[ x] I have performed a self-review of my own code
I have made corresponding changes to the documentation
[ x] I have added tests that prove my fix is effective or that my feature works
[ x] I have checked my code and corrected any misspellings

felix-e-h-p · 2025-08-28T10:17:21Z

pvnet/models/base_model.py

        history_minutes: int,
        forecast_minutes: int,
-        output_quantiles: list[float] | None = None,
+        output_quantiles: Optional[list[float]] | None = None,


I think this can be simplified to just:

output_quantiles: Optional[list[float]] = None,

coolio, will do :)

felix-e-h-p · 2025-08-28T10:19:35Z

pvnet/models/base_model.py

+        pis = F.softmax(logits, dim=-1)
+        return mus, sigmas, pis
+
+    def _quantiles_to_prediction(self, y_quantiles):


To: def _quantiles_to_prediction(self, y_quantiles: torch.Tensor) -> torch.Tensor:

Slotted in!

felix-e-h-p · 2025-08-28T10:26:56Z

pvnet/training/lightning_module.py

+            y_gmm:   (batch, forecast_len * num_components * 3)
+            y_true:  (batch, forecast_len)
+        """
+        mus, sigmas, pis = self._parse_gmm_params(y_gmm)


Forgive me if I am wrong - but I think this should be updated to:

self.model._parse_gmm_params(y_gmm)

Method is in base_model right - and Lightning should hold an instance of the model itself i.e. self.model

Yup! resolved :)

felix-e-h-p · 2025-08-28T10:27:59Z

pvnet/training/lightning_module.py

        if self.model.use_quantile_regression:
-            losses["quantile_loss"] = self._calculate_quantile_loss(y_hat, y)
-            y_hat = self.model._quantiles_to_prediction(y_hat)
+            losses["quantile_loss"] = self.model._calculate_quantile_loss(y_hat, y)


In line with line 91 potential update:

self._calculate_quantile_loss(y_hat, y)

sorry! missed this in some of the refactoring, changed it now :)

Think it's fine as is - perhaps maybe better with the reversion however I feel

felix-e-h-p · 2025-08-28T10:39:36Z

pvnet/models/late_fusion/late_fusion.py

        if self.use_quantile_regression:
-            # Shape: batch_size, seq_length * num_quantiles
-            out = out.reshape(out.shape[0], self.forecast_len, len(self.output_quantiles))
+            out = out.view(out.size(0), self.forecast_len, len(self.output_quantiles))


Solid refinement!

felix-e-h-p · 2025-08-29T09:23:28Z

tests/models/test_gmm_basemodel.py

+from pvnet.models.base_model import BaseModel
+
+
+class _MinimalGMMModel(BaseModel):


Brilliant - thanks for adding these in! Would you mind moving _MinimalGMMModel and potentially _build_y_gmm_from_params to conftest if all OK?

good shout :) moved :)

felix-e-h-p · 2025-08-29T09:31:15Z

pvnet/training/plots.py

    batch: TensorBatch,
    y_hat: torch.Tensor,
-    quantiles: list[float] | None,
+    model,


Great compatibility shift - thanks!

Not sure whether or not you feel it could be a strong addition, but what about an extra aspect that includes showing actual example samples from the mixture alongside say the mean?

the plotting currently also plots the std distribution error per step as well - is this what you mean?

Ah yeah, cheers sorry. Maybe like a few random samples from the predicted GMM distribution at each step and have those individually alongside the mean - completely up to you though

dfulu · 2025-09-01T15:19:32Z

Hi @murphytarra, thank you for all of your work on this. It looks really great already. I'll be doing an extra review on this PR as well as Felix.

Thought I'd drop in to say hello first, and from what I can see there are just a few places it could be tidied up a little bit

dfulu

Looks really good @murphytarra! I really like how you've integrated it

My comments are mostly around tidy ups. I presume you've used a linter which has the line length limit set to 80 rathe than 100. I'd prefer those line breaks to be undone where it has reduced readability

pvnet/models/late_fusion/late_fusion.py

pvnet/models/base_model.py

pvnet/training/lightning_module.py

pvnet/training/plots.py

pvnet/models/base_model.py

dfulu · 2025-09-02T08:32:18Z

Hi @murphytarra thanks for the changes!

I see the tests are currently failing but I think that's due to the github workflows now being out of date. If you merge the updated main branch into this branch it should fix the tests.

I'm going to go through and tag a few more lines which have been split where I think the split reduces readability. I'd appreciate it if you could revert them

pvnet/models/late_fusion/late_fusion.py

pvnet/models/base_model.py

dfulu · 2025-09-02T08:14:33Z

pvnet/models/base_model.py

        history_minutes: int,
        forecast_minutes: int,
-        output_quantiles: list[float] | None = None,
+        output_quantiles: Optional[list[float]] = None,


Could you change the type hint back here and match in the line below?

param: type | None = None is the current best practice for python > 3.10 rather than param: Optional[type] = None which was the practice in python<3.9

dfulu · 2025-09-02T08:15:09Z

pvnet/models/base_model.py

            output_quantiles: A list of float (0.0, 1.0) quantiles to predict values for. If set to
                None the output is a single value.
+            num_gmm_components: Number of Gaussian Mixture Model components to use for the model.
+            If None, output quantiles must be set. If  both None, the output is a single value.


Suggested change

If None, output quantiles must be set. If both None, the output is a single value.

If None, output quantiles must be set. If both None, the output is a single value.

pvnet/models/base_model.py

dfulu · 2025-09-02T08:37:33Z

tests/conftest.py

    gsp_ids = np.arange(0, 318)
    capacity = np.ones((len(times), len(gsp_ids)))
-    generation = np.random.uniform(0, 200, size=(len(times), len(gsp_ids))).astype(np.float32)
+    generation = np.random.uniform(0, 200, size=(len(times), len(gsp_ids))).astype(


Please unsplit

dfulu · 2025-09-02T08:37:47Z

tests/conftest.py

+) -> str:
+
    # Populate the config with the generated zarr paths
-    config = load_yaml_configuration(f"{_top_test_directory}/test_data/uk_data_config.yaml")


Please unsplit

dfulu · 2025-09-02T08:37:53Z

tests/conftest.py

+) -> str:
+
    # Populate the config with the generated zarr paths
-    config = load_yaml_configuration(f"{_top_test_directory}/test_data/site_data_config.yaml")


Please unsplit

dfulu · 2025-09-02T08:38:07Z

tests/conftest.py



 @pytest.fixture()
-def late_fusion_model_kwargs_site_history(raw_late_fusion_model_kwargs_site_history) -> dict:


Please unsplit

dfulu · 2025-09-02T08:38:14Z

tests/conftest.py



 @pytest.fixture()
-def late_fusion_model_site_history(late_fusion_model_kwargs_site_history) -> LateFusionModel:


Please unsplit

Merge branch 'tara_dev'

f50a665

murphytarra changed the title ~~GOSC: Probabilistic Machine Learning for Solar Forecasting: Applying Gaussian Mixture Models for output~~ GSOC: Probabilistic Machine Learning for Solar Forecasting: Applying Gaussian Mixture Models for output Aug 25, 2025

Tara Murphy added 4 commits August 26, 2025 12:51

corrected indentation

ea90672

corrected doc strings

c344676

added use gmm

2a1bc84

correct use gmm

c26dae1

felix-e-h-p reviewed Aug 28, 2025

View reviewed changes

add in tests

d2736c2

felix-e-h-p reviewed Aug 29, 2025

View reviewed changes

commits for review

d7dabca

felix-e-h-p requested a review from dfulu September 1, 2025 10:12

dfulu requested changes Sep 1, 2025

View reviewed changes

dfulu reviewed Sep 1, 2025

View reviewed changes

pvnet/models/base_model.py Show resolved Hide resolved

updates for dfulu

93c0b64

murphytarra requested a review from dfulu September 1, 2025 20:49

dfulu requested changes Sep 2, 2025

View reviewed changes

felix-e-h-p mentioned this pull request Sep 25, 2025

Quantile Regression VS GMM #484

Open

		from pvnet.models.base_model import BaseModel


		class _MinimalGMMModel(BaseModel):

	If None, output quantiles must be set. If both None, the output is a single value.
	If None, output quantiles must be set. If both None, the output is a single value.



		@pytest.fixture()
		def late_fusion_model_kwargs_site_history(raw_late_fusion_model_kwargs_site_history) -> dict:



		@pytest.fixture()
		def late_fusion_model_site_history(late_fusion_model_kwargs_site_history) -> LateFusionModel:

Uh oh!

GSOC: Probabilistic Machine Learning for Solar Forecasting: Applying Gaussian Mixture Models for output #448

Are you sure you want to change the base?

GSOC: Probabilistic Machine Learning for Solar Forecasting: Applying Gaussian Mixture Models for output #448

Uh oh!

Conversation

murphytarra commented Aug 25, 2025

Switchable Probabilistic Output: Quantile Regression ⇄ Gaussian Mixture Model (GMM)

Summary

What’s changed

API / Configuration

Output tensors

Backward compatibility

Training / Loss

Evaluation & Inference Notes

Migration guide

Checklist:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dfulu commented Sep 1, 2025

Uh oh!

dfulu left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dfulu commented Sep 2, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

dfulu left a comment •

edited

Loading