Code

Feature/scheduler#49

Code

Closed

louisPoulain wants to merge 73 commits intomainMeteoSwiss/mlpp-lib:mainfrom feature/schedulerMeteoSwiss/mlpp-lib:feature/schedulerCopy head branch name to clipboard

+619-76

Commits

Commits on Jun 6, 2024

• Define a first draft for a Normalizer "meta-class"

  

  louisPoulain

  committed
  a766beb

• Extend normaliser class with new method.

  Show description for ddadee9
  

  louisPoulain

  committed
  ddadee9

Commits on Jun 7, 2024

• Continue working on normalizer class. Experimental, still need to be tested with the original standardizer class

  

  louisPoulain

  committed
  6bf868c

Commits on Jun 10, 2024

• Changed a lot the meta-normalizer architecture.

  

  louisPoulain

  committed
  a0d030c

• Early testing with the meta class and the initial standardizer class.

  

  louisPoulain

  committed
  3067f28

• Update gitignore to hide test files

  

  louisPoulain

  committed
  c261abc

• Add MinMaxScaler

  

  louisPoulain

  committed
  1a7764b

• Add and perform tests for MinMaxScaler

  

  louisPoulain

  committed
  d52f86a

Commits on Jun 11, 2024

• Change the testing to a more general framework.

  

  louisPoulain

  committed
  5b71fd7

• Correct a few issues

  

  louisPoulain

  committed
  e9fa689

• Add and test MinMaxScaler

  

  louisPoulain

  committed
  b4f5333

• Add RobustScaling (standard norma with median and iqr)

  

  louisPoulain

  committed
  181f8b8

• Make the test file work with an unlimited number of nromalizer subclasses

  

  louisPoulain

  committed
  ca556fe

• Add MaxAbsScaler, BoxCox and YeoJohnson transformations

  

  louisPoulain

  committed
  8c2761f

• Changed minor things

  

  louisPoulain

  committed
  5f562c6

• Minor changes, this test script is vowed to disappear

  

  louisPoulain

  committed
  7f3bd13

Commits on Jun 12, 2024

• Modification of the naming

  

  louisPoulain

  committed
  efb00d8

• Modofication to take into account the new class

  

  louisPoulain

  committed
  e4a91f1

• Modif ignore

  

  louisPoulain

  committed
  869472f

• Add a try/except to test if normalizer was fitted. To be changed in the future

  

  louisPoulain

  committed
  c6b0245

• Rename some parameters and change a bit the structure. Remove RobustScaler (issues with computing the median)

  

  louisPoulain

  committed
  6e4b25a

• Add function to bypass dask.nanmedian (not working currently)

  

  louisPoulain

  committed
  68c9ae2

• Few text changes

  

  louisPoulain

  committed
  65c88d2

Commits on Jun 13, 2024

• Add fillvalue in arguments, modify the way an object is created from to_dict

  

  louisPoulain

  committed
  e30ed07

• Reduce tolerance to 1e-7 for allclose tests

  

  louisPoulain

  committed
  bcbe64d

Commits on Jun 14, 2024

• Minor change

  

  louisPoulain

  committed
  86f8e64

• Add logger message to track the multinormalizer.

  

  louisPoulain

  committed
  f25ea28

• Debug a bit the lengths of datasets

  

  louisPoulain

  committed
  61399a2

Commits on Jun 17, 2024

• Add default normalizer

  

  louisPoulain

  committed
  41b7240

Commits on Jun 18, 2024

• Move save and load to the super-class

  

  louisPoulain

  committed
  e79aec4

• Remove old tests for multinormalizer. Reimplement them as pytest tests

  

  louisPoulain

  committed
  8ee56b7

• Add method to gather class attributes

  

  louisPoulain

  committed
  6416f74

• Correct typo in confest

  

  louisPoulain

  committed
  90f1c57

• Correct issue in conftest

  

  louisPoulain

  committed
  3968210

• Remove "return" statement

  

  louisPoulain

  committed
  0d3171b

• Remove RobustScaler (currently does not work with huge datasets)

  

  louisPoulain

  committed
  5642c0c

Commits on Jun 19, 2024

• Fix typo

  louisPoulain

  authored
  fec0033

• Update default fillvalues and multi.from_dict

  Show description for 4cd4a8e
  

  Louis Poulain--Auzéau

  committed
  4cd4a8e

• Modify fillvalue to -10.

  Show description for 2d4111d
  

  Louis Poulain--Auzéau

  committed
  2d4111d

• Minor modif

  

  Louis Poulain--Auzéau

  committed
  3a045cf

• Modify tests. Tolerance is set to 1e-6 for the inverse transform test

  

  Louis Poulain--Auzéau

  committed
  34cb53e

• Remove debugging code

  

  Louis Poulain--Auzéau

  committed
  9464260

• Modify assert with correct fillvalue

  

  Louis Poulain--Auzéau

  committed
  fc5d5c7

• Bug fix: in some cases, transform seemed to modify in-place the input dataset

  

  Louis Poulain--Auzéau

  committed
  bb0a47a

• Ensure fillvalue is -5 in this test

  

  Louis Poulain--Auzéau

  committed
  79d990b

• Modify floating point precision

  

  Louis Poulain--Auzéau

  committed
  6ad1c47

Commits on Jun 20, 2024

• Remove RobustScaler and YeoJohnson, refactor the classes into a normalization class handling all techniques to normalize and a normalizer (multinormalizer)

  

  louisPoulain

  committed
  804bc0b

• Change gitignore

  

  louisPoulain

  committed
  857dcd1

• Change code according to the new way normalizer are being defined

  

  louisPoulain

  committed
  edad113

• Remove unused function

  

  louisPoulain

  committed
  981e899

• Change code according to the new normalizer definition

  

  louisPoulain

  committed
  a825330

• Changes according to standardizers.py

  

  louisPoulain

  committed
  88b7bf5

• Changes according to standardizers.py

  

  louisPoulain

  committed
  4f15068

• Rename the test to rpevent it from being launched by pytest (test to be removed)

  

  louisPoulain

  committed
  8750db9

• Modify code to work with the new normalizers

  

  louisPoulain

  committed
  2ba7abc

• Clean a bit the code

  

  louisPoulain

  committed
  eb3e8be

• Update doc, remove print messages

  

  louisPoulain

  committed
  d5629b0

• Update test

  

  louisPoulain

  committed
  7866d46

• Remove import

  

  louisPoulain

  committed
  67ddcc6

• Modify the default normalization handling

  

  louisPoulain

  committed
  745f843

• Add fitting of data

  

  louisPoulain

  committed
  b3b2e37

Commits on Jun 21, 2024

• Minor simplification

  

  louisPoulain

  committed
  a3ae2fb

• Change names to reflect more accurately the roles of the classes

  

  louisPoulain

  committed
  1cfd1b7

Commits on Jun 25, 2024

• Revert to use "normalizer" as input name

  Show description for 5f76a3e
  

  louisPoulain

  committed
  5f76a3e

• Use again as pytesting. Removed the serialization test as serialization was moved to the data_transformer class

  

  louisPoulain

  committed
  0edf87d

• Rename the method according to #43 (comment)

  

  louisPoulain

  committed
  1a7129a

Commits on Jun 27, 2024

• Prevent DataLoader from not working in cases where the total number of samples is less than the batch number (especially useful for val and test)

  

  louisPoulain

  committed
  f6e11bc

Commits on Jun 28, 2024

• Correct log error message

  

  louisPoulain

  committed
  abaf96c

Commits on Jul 1, 2024

• Change seed setup

  

  louisPoulain

  committed
  a5c23a5

Commits on Jul 2, 2024

• Leave random as seed as previously

  

  louisPoulain

  committed
  8c7f807

Commits on Jul 3, 2024

• Prepare using schedulers in the yaml config

  

  louisPoulain

  committed
  fdd55c0

Commits on Jul 5, 2024

• Modify the creation of the scheduler

  

  louisPoulain

  committed
  cce2012

• Add a "metric" to track lr during training when using a scheduler

  

  louisPoulain

  committed
  0012614