Releases: Pale-Blue-Dot-97/Minerva
Minerva 0.23.0
minerva 0.23.0-beta
Updates
- Closes #152
- NEW: Added
mask_to_ohe
to convert mask tensors to one-hot-encoding (OHE). - NEW: Added
autoencoder_io
so that input and targets can be made the same for autoencoder experiments. - Added new tests (particularly for
trainer
) to close coverage gaps. - Added support for Python 3.11
- Bumped dependency versions to the latest version.
- Fixed bug in
STGLogger
for scene classifiers due to incorrect size of the logs. - Fixed the
UndefinedMetricWarning
in ROC plotting.
What's Changed
minerva v0.23.0
by @Pale-Blue-Dot-97 in #153
Full Changelog: v0.22.0-beta...v0.23.0-beta
Minerva v0.22.0-beta
* * __ ________ ____________ _ _____
* * / |/ / _/ | / / ____/ __ \ | / / | * *
* / /|_/ // // |/ / __/ / /_/ / | / / /| | *
* * / / / // // /| / /___/ _, _/| |/ / ___ | *
* /_/ /_/___/_/ |_/_____/_/ |_| |___/_/ |_| * *
THE REMOTE SENSING MACHINE LEARNING FRAMEWORK
==============> v0.22.0-beta <===============
This release focusses on adding support for third-party models to minerva
. There are also updates to the documentation, some minor bug fixes and new transforms.
What's Changed
- Reduce FCN Boilerplate by @Pale-Blue-Dot-97 in #150
- Added Support for Third Party Models by @Pale-Blue-Dot-97 in #151
Full Changelog: v0.21.0-beta...v0.22.0-beta
Minerva v0.21.0-beta
minerva v0.21.0-beta
* * __ ________ ____________ _ _____
* * / |/ / _/ | / / ____/ __ \ | / / | * *
* / /|_/ // // |/ / __/ / /_/ / | / / /| | *
* * / / / // // /| / /___/ _, _/| |/ / ___ | *
* /_/ /_/___/_/ |_/_____/_/ |_| |___/_/ |_| * *
THE REMOTE SENSING MACHINE LEARNING FRAMEWORK
==============> v0.21.0-beta <===============
This version mainly focuses on improving the user experience through a major shake-up in the structure of the experiment configs and major improvements to the documentation.
Updates
- Closes #146
- Closes #123
- Moved most of the more important hyperparameter options to the top-level of the config.
- Added most of these options to the CLI.
- Added extensive comments to the example configs to structure and explain the configs.
- Added a new
ASCII
banner art forminerva
. It used as a header for the configs and as a welcome sign inMinervaExp.py
. - Added some limited
argcomplete
functionality with the use ofchoices
option in CLI arguments that have restricted possible parameters. - Added
intersphinx
functionality to thesphinx
docs. This allows for the linking to thrid-party documentation. - Updated
example_config.rst
guide to new standard. - Updated and corrected all docstrings across
minerva
so thatsphinx
references are valid. - Improved the
README
and added a modified version to thesphinx
docs.
Full Changelog: v0.20.0-beta...v0.21.0-beta
Minerva v0.20.0
Update that mainly focuses on updating dependencies and hooks, reducing boilerplate and closing the distributed tests issue by removing that functionality from the pytest
coverage.
What's Changed
- Reduce model boilerplate by @Pale-Blue-Dot-97 in #144
- Resolved Distributed Tests by @Pale-Blue-Dot-97 in #145
Updates
- Closes #143
- Reduced the boilerplate nature of many of the model classes which are variants of one another.
- Added new
_ResNetX
class for ResNet variant classes to inherit from. - As #99 cannot be resolved, distributed functionality has had
pragma: no cover
comments added to remove it frompytest
coverage. - Added missing
max_r
parameter in the sampler definitions inexample_GeoCLR_config.yml
. - Added new
plot
andplot_random_sample
methods toPairedDataset
. These emulate theplot
methods included in all dataset classes intorchgeo
and are based on those used in theNAIP
dataset. - Updated existing
pre-commit
hooks to latest versions. - Added some new
pre-commit
hooks. - Resolved a security vulnerability from dependencies by using a new pre-release version of
tensorflow
. - Added
is_notebook
toutils.py
to check whether the code is being executed from a Juypter kernel. - Added
Visualise_Siamese_Pair.ipynb
example notebook that visualises both patches in a pair for Siamese learning. Placed in a newnotebooks
directory. - Minor bug fixes and improvements to some exception handling.
- Some improvements and updates to the
README
and documentation.
Full Changelog: v0.19.2-beta...v0.20.0-beta
`minerva` `v0.19.2`
minerva
v0.19.2-beta
This small update focuses on address some security vulnerabilities from dependencies and upgrading other dependencies as far as possible.
There are also some minor improvements to tests, package layout and type-hinting.
Updates
- Moved requirements files into
requirements
directory. - Added
perplexity
argument toutils.tsne_cluster
to account for changes inscikit-learn
that requireperplexity < n_samples
. - Minor
flake8
fixes. - Pinned
Werkzeug >= 2.2.3
to avoid a highlighted security vulnerability. - Bumped various dependencies up to the latest compatible versions.
- Removed some redundant dependencies.
- Added some missing dependencies to
setup.cfg
forminerva
. - Fixed a potential bug in
results_dir
fixture if thepath
did not exist when attempting to delete it and its contents. - Added some additional
pragma: no cover
comments to exclude some areas of the code, namely theSLURM
environment variables inrunner.py
as it seems unlikely these can ever be reliably tested.
Minerva Beta 0.19 Release
minerva
v0.19.0-beta
Welcome to the first beta
version of minerva
!
This latest update of minerva
focuses on adding KNN validation to the Trainer
for SSL models to use, along with other updates, improvements and bug fixes:
Updates
- Closes #140
- Adds
weighted_knn_validation
toTrainer
to validate SSL models using k-nearest neighbours classification. - Experiments with
model_type == "ssl"
or"siamese
will now automatically useweighted_knn_validation
rather than the standardepoch
for validation epochs. - Added
val_freq
to the config and CLI for setting the frequency at which validation epochs are run for every training epoch. Applies to SSL and Siamese models. - Restructured
siamese.py
to use a new abstractMinervaSiamese
class. This is to allow theforward
methods in these models to be built fromforward_pair
(forward pass of both heads of the network) andforward_single
(forward pass of a single head of the network). This was required for the KNN. sample_pairs
now only applies in training in general.- Validation and test datasets now ignore
sample_pairs
when constructed so they are suitable for the KNN. This seems a fair adjustment as validation or testing of a Siamese model is very likely to only require single-headed operation for downstream tasks. - Configs for SSL and Siamese models should be adjusted so that appropriate datasets and samplers are defined (i.e include labels and don't use paired samplers). See
example_GeoCLR_config.yml
for an example of this. - Added
KNNLogger
tologger.py
as a special logger used inweighted_knn_validation
. The usualSSLMetrics
metric logger can however still be used with the new functionality.
Improvements
- Results from tests will now be automatically cleaned up.
- Tests have been extended to cover new functionality and most of
Trainer
's possible operations. - Dependencies updated.
- Added some extra
pragma: no cover
on a few areas of code that are impractical to cover in tests. - Added
ToDefaultConfDir
context manager to switch to the default config directory in tests. - Added
set_seeds
pytest.fixture
so seeds are automatically set usingutils.set_seeds
across the whole test session.
Bug Fixes
- Fixed some broken test fixture paths.
- Fixed broken
_static
path forsphinx
docs. - Added missing
sphinx
docs formodels
modules andconfig_load.py
.
What's Changed
- Bump tensorflow from 2.7.0 to 2.7.1 by @dependabot in #29
- Distributed sampling by @Pale-Blue-Dot-97 in #32
- Merge
ds-beta
back intomain
by @Pale-Blue-Dot-97 in #33 - Bump tensorflow from 2.8.0 to 2.8.1 by @dependabot in #47
- Merge urgent bug fixes into
main
by @Pale-Blue-Dot-97 in #48 - Bump numpy from 1.21.2 to 1.22.0 by @dependabot in #49
- Merge
dist-dev
intomain
by @Pale-Blue-Dot-97 in #50 - 39 add test datasets by @Pale-Blue-Dot-97 in #51
- 40 add test samplers by @Pale-Blue-Dot-97 in #52
- 38 add test trainer by @Pale-Blue-Dot-97 in #53
- Merge New tests into Main by @Pale-Blue-Dot-97 in #54
- Simsiam dev by @Pale-Blue-Dot-97 in #55
- T sne dev by @Pale-Blue-Dot-97 in #56
- Moved distributed run framework to generic module by @Pale-Blue-Dot-97 in #61
- Torchvision weights dev by @Pale-Blue-Dot-97 in #64
- 65 runner testing by @Pale-Blue-Dot-97 in #66
models
restructuring by @Pale-Blue-Dot-97 in #75- Bump tensorflow from 2.9.1 to 2.9.3 by @dependabot in #76
- Mypy improv by @Pale-Blue-Dot-97 in #79
- Start of failing test for config paths by @metazool in #80
- Pathlib refactor by @Pale-Blue-Dot-97 in #83
- Convert
os.sep
andos.path
usage -->pathlib.Path
by @Pale-Blue-Dot-97 in #85 - Suggested tiny change to remove the specific type annotation from PathLike hints by @metazool in #86
- Flake8 fixes by @Pale-Blue-Dot-97 in #88
- Draft: option to log metrics via mlflow, given active run by @metazool in #87
- Config load tests by @Pale-Blue-Dot-97 in #93
- Tests convert os pathlib by @Pale-Blue-Dot-97 in #95
- Test models breakup by @Pale-Blue-Dot-97 in #98
- Draft: Test that covers Trainer when running distributed by @metazool in #100
- Test refactor by @Pale-Blue-Dot-97 in #101
- Fluffy sprint by @Pale-Blue-Dot-97 in #102
- Test refactor by @Pale-Blue-Dot-97 in #103
- Keeping fluffy-sprint for the development and PR purposes by @NavidCOMSC in #105
- Bump setuptools from 59.5.0 to 65.5.1 by @dependabot in #108
- Ddp logging by @NavidCOMSC in #107
- Revert "Ddp logging" by @Pale-Blue-Dot-97 in #109
- Minor Fluffy Sprint v2 dev by @Pale-Blue-Dot-97 in #111
- Added Support for Unionising Datasets by @Pale-Blue-Dot-97 in #113
- [Snyk] Fix for 2 vulnerabilities by @Pale-Blue-Dot-97 in #114
- [Snyk] Fix for 2 vulnerabilities by @Pale-Blue-Dot-97 in #115
- Circleci project setup by @Pale-Blue-Dot-97 in #116
- Unet dev by @Pale-Blue-Dot-97 in #118
- Coveralls dev by @Pale-Blue-Dot-97 in #117
- Add Docker Support by @Pale-Blue-Dot-97 in #120
- CircleCI Parallelisation Dev by @Pale-Blue-Dot-97 in #121
- Merge developments from
festive-lion
sprint by @Pale-Blue-Dot-97 in #122 - Added Qodana Integration by @Pale-Blue-Dot-97 in #126
- Added
onnx
support by @Pale-Blue-Dot-97 in #127 - Weights and Biases Support by @Pale-Blue-Dot-97 in #128
- [Snyk] Security upgrade numpy from 1.21.3 to 1.22.2 by @Pale-Blue-Dot-97 in #130
- [Snyk] Security upgrade numpy from 1.21.3 to 1.22.2 by @Pale-Blue-Dot-97 in #129
- [Snyk] Fix for 2 vulnerabilities by @Pale-Blue-Dot-97 in #131
- [Snyk] Fix for 7 vulnerabilities by @Pale-Blue-Dot-97 in #132
- [Snyk] Security upgrade numpy from 1.21.3 to 1.22.2 by @Pale-Blue-Dot-97 in #133
- [Snyk] Fix for 2 vulnerabilities by @Pale-Blue-Dot-97 in #134
- Introduced Caching to GitHub Actions Tests by @Pale-Blue-Dot-97 in #137
- Added KNN by @Pale-Blue-Dot-97 in #142
- [Snyk] Fix for 2 vulnerabilities by @Pale-Blue-Dot-97 in #141
New Contributors
- @dependabot made their first contribution in #29
- @Pale-Blue-Dot-97 made their first contribution in #32
- @metazool made their first contribution in #80
- @NavidCOMSC made their first contribution in #105
Full Changelog: https://github.com/Pale-Blue-Dot-97/Minerva/commits/0.19.0-beta