[ENH] add feature scaling support for EncoderDecoderDataModule

[PranavBhatP]

Reference Issues/PRs

fixes #1979

What does this implement/fix? Explain your changes.

Implements feature scaling for continuous targets and features in the D2 layer using scalers and target_normalizer params.

What should a reviewer concentrate their feedback on?

PR checklist

• The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.
• Added/modified tests
• Used pre-commit hooks when committing to ensure that code is compliant with hooks. Install hooks with pre-commit install.
  To run hooks independent of commit, execute pre-commit run --all-files

[codecov]

Codecov Report

❌ Patch coverage is 58.62069% with 24 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@aae13ba). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
pytorch_forecasting/data/data_module.py	58.62%	24 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1983   +/-   ##
=======================================
  Coverage        ?   86.79%           
=======================================
  Files           ?      160           
  Lines           ?     9520           
  Branches        ?        0           
=======================================
  Hits            ?     8263           
  Misses          ?     1257           
  Partials        ?        0           

Flag	Coverage Δ
cpu	86.79% <58.62%> (?)
pytest	86.79% <58.62%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[PranavBhatP]

Tests are failing on the notebook. Seems like its an issue caused by the training script in the notebook. Will try changing accelator = "cpu". Might resolve the issue.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[phoeenniixx]

I have doubt: Wouldn't using detach again detach the tensor from the computation graph? That would again lead to the same issue?

[PranavBhatP]

As far as my knowledge of pytorch goes, I think it's a good practice to use .detach() before converting the pytorch tensor to a numpy array. Anyways, the numpy array will not track the gradients, so this won't matter.

[phoeenniixx]

I think we could look at v1 implementation? Maybe that could help us with this issue, how the sklearn scalers are handled there?

[PranavBhatP]

Yes we will need a design doc to analyse v1 implementation. This is the best way forward.

[fkiraly]

agree - it seems like one more rabbit hole...

Is there perhaps something we can learn from pytorch-tabular?

[PranavBhatP]

I've replaced the loss function used in the notebook with nn.L1Loss(). I cannot think of a fix to the current issue with PTF metrics. Any thoughts yet, about what is causing it? @phoeenniixx @agobbifbk @fkiraly? Do we proceed with this PR by running the model in the notebook with nn metrics?

[fkiraly]

May I request an explanation why the notebook is running prior to the changes but now failing?

[PranavBhatP]

May I request an explanation why the notebook is running prior to the changes but now failing?

I did not get your question. Are you referring to why the tests were failing priority to the changes in the most recent commits? It is passing now...For context, the tests were failing on the notebook because training simply does not seem to work when we use in-house PTF metrics. More context on the initial failing test - https://github.com/sktime/pytorch-forecasting/actions/runs/18570677570/job/52943552548

However, the pipeline works perfectly fine when we use nn metrics. That is why we replace the metric parameter for the TFT model with nn.L1Loss() instead MAE().

[fkiraly]

I did not get you question.

I meant, did anything cause in this PR cause the failure in the notebook run?

If not, what caused the failure?

That notebooks pass is important, as it indicates usage patterns that direct users of pytorch-forecasting may rely on, and are more likely to rely on.

[phoeenniixx]

Do we proceed with this PR by running the model in the notebook with nn metrics?

I agree with @fkiraly and I am also not very comfortable with removing metrics support from here as I think only Point prediction losses would work with the models right now. For other nn metrics, we need to create the adapters.
Secondly, in any form, ptf should be able to support the inbuilt metrics.

[PranavBhatP]

Ok, I will revert the changes on the notebook. I will try to debug the issue with the pipeline then, will take some time.

[PranavBhatP]

@phoeenniixx @fkiraly @agobbifbk I think the pipeline for scaling works now. I will make changes to test_data_module.py so that all tests pass.

Pending questions

• A function for providing inverse normalization using the fitted scalers to be used by model's during output phase. Is this required? Or should it be implemented in model layer?
• I am currently running the scaling pipeline only using TorchNormalizer and EncoderNormalizer since these normalizers work adequately with pandas, numpy and torch tensors. I think its a hassle to include Standard and RobustScaler. Better to stick to in-house normalizers in v2?
• Also I am fitting EncoderNormalizer on the whole data for now. Since its correct usage would be to perform normalization on each encoding sequence/window, should this be done in _preprocess_data or directly in __getitem__ of _ProcessedEncoderDecoderDataset where windows are generated?

[agobbifbk]

Hello there,
some comments:

        for idx in train_indices:
            sample = self.time_series_dataset[idx]
            target = sample["y"]
            if isinstance(target, torch.Tensor):
                all_targets.append(target)
            else:
                all_targets.append(torch.tensor(target, dtype=torch.float32))

This works if all the data can fit in memory. Since we are planning to use a d1 layer that can work with several timeseries that may not fit in memory probably it is better to use partial_fit see here.
Since the partial fit is not available for each scaler it may worth to check this out.
Another think we are implementing in DSIPTS is the possibility to scale the data for group_id: if we have 2 time series with totally different scales (suppose selling data, one item is around 1M the other is 1K) probably we want to have two separate SET of scalers (for x and y), what do you think?

Not sure to understand the EncoderNormalizer, does it fit different scales for different lags?

Regarding the inverse transformation: I think the correct place for doing it is in the prediction phase that @phoeenniixx is implementing.

Do you manage also to have a look to the label encoding in the D1 layer (so we can also use categorical variable encoded as string in the original data)?

The rest seems fine, you added a piece to the puzzle, great :-)

[PranavBhatP]

This works if all the data can fit in memory. Since we are planning to use a d1 layer that can work with several timeseries that may not fit in memory probably it is better to use partial_fit see here.
Since the partial fit is not available for each scaler it may worth to check this out.
Another think we are implementing in DSIPTS is the possibility to scale the data for group_id: if we have 2 time series with totally different scales (suppose selling data, one item is around 1M the other is 1K) probably we want to have two separate SET of scalers (for x and y), what do you think?

Will take a look at it, and let you know my thoughts.

Do you manage also to have a look to the label encoding in the D1 layer (so we can also use categorical variable encoded as string in the original data)?

That is the next workitem after scaling.