Where should the development home be for the ML Model Extension?

[rbavery]

The new MLM model extension is now released as version 1.0! It includes a validation package, JSON schema, and updated schema definitions to handle the complexity of how to run and deploy models in 2024. @fmigneault and I are excited to use and maintain this extension and we welcome folks to use the extension, provide contributions, and provide feedback in issues. The old ML Model extension is going to be deprecated in favor of this new MLM extension: stac-extensions/ml-model#16

On this note, where should the development home for the extension be? Currently it is in a fork within the CRIM org: https://github.com/crim-ca/mlm-extension

I think it'd be better for industry adoption, visibility, and community stewardship if the development home was instead within the stac org: https://github.com/stac-extensions

@fmigneault raised a concern that CRIM would like to maintain contribution visibility. One option is that we fork https://github.com/crim-ca/mlm-extension into https://github.com/stac-extensions and have the fork be the central hub of development. This would show clearly that the stac-extensions repo is derived form the CRIM repo.

I'm alright with this as long as the stac-extensions fork is the hub for development and hosts the identifier for the JSON schema. Curious what others think!

[gadomski]

maintain contribution visibility

Can you clarify what you mean by this? If we transfer the extension to the stac-extensions organization, the version control history comes along with it, so all contributions will still be attributed to the authors. Any other attribution could then be handled in the README or license information?

[rbavery]

I'm also of the opinion that attribution could be handled in the README with logos and text references to CRIM (and other contributors like Wherobots). There could be a full transfer to simplify the ownership. Some folks might get confused that a fork is where main development takes place.

If I understand correctly, CRIM would like to maintain a reference in the repo to the original repo with a fork. The link and message included in forked repos forked from https://github.com/crim-ca/dlm-extension might be important to them. @fmigneault do you have any new thoughts on this setup? Is a fork necessary?

[fmigneault]

I think that it would be acceptable if the contributions/logos are visible in the README (not just CRIM's but Wherobtos' ones also).
As long as it is not only the Git history. Most people do not bother to look deeper than the landing page.
I'll come back with more updates, I'll have to refer with others in the company.

[rbavery]

Thanks for checking in on this possibility Francis! I'm down to discuss in person at the next STAC Community meeting.

[m-mohr]

I was under the impression that actually mlm would be a v2 of ml-model, but that doesn't seem to be the case. (Why not?)

I don't think we should have forks in stac-extensions as it would water down where the main source would be. Why would we want to have a fork there? External repos can be added to the list of extensions on stac-extensions.github.io.
I'm relatively neutral on whether ml-model lives under stac-extensions or crim-ca. I don't think there is any major difference.

[rbavery]

I was under the impression that actually mlm would be a v2 of ml-model, but that doesn't seem to be the case. (Why not?)

I also was under this impression. thoughts @fmigneault ? While working on this PR, I thought we would transfer the DLM repo to stac-extensions/ml-model as version 2.

I'm relatively neutral on whether ml-model lives under stac-extensions or crim-ca. I don't think there is any major difference.

We (at Wherobots) would prefer the extension to live under a neutral org, similar to projects like geoparquet and the stac-spec itself. I think it's a helpful signal that the extension is owned and managed by the community and not any one company. I think it increases visibility and other ML related extensions (ml-aoi, ml-label) that are in use are under stac-extensions as well.

[fmigneault]

I'm open to discuss a merge of mlm as a ml-model v2, though there is not really any logic in the history between the two.
I'm not sure if that would cause more or less confusion in the end? This is partly why a different prefix was used.
Another issue is that I do not plan to spend time on offering "migration" scripts between v1 and v2.

[rbavery]

I'm supportive of not offering migration scripts and also don't plan on it. There's not enough active users of v1 for it to matter and the MLM offers many new necessary fields so automatic migration might not be possible.

I do think there's an advantage to discovery and encouraging community maintainership/contribution. Even though DLM didn't directly fork ml-model, I think it would make sense for users that the MLM is the newest progression of efforts to describe geospatial models and version 2 wouldn't look weird. As long as it is a non-fork repo and other past repos are archived or point to it, I think users will understand this is the extension to use today.

[m-mohr]

Bye the way, my question was just out of curiousity. I'm pretty neutral on whether it's a ml-model v2 or mlm.
If it's a v2, we don't necessarily need any migration scripts, but a documented migration path / a mapping would be good to have.

[emmanuelmathot]

IMO, this is not a ml-model v2 but a new extension addressing larger machine learning capabilities in STAC. We should keep ml-model as it is and deprecate it eventually when the mlm extension maturity will reach Candidate level.

[fmigneault]

I agree to keep them separate to help make thing clearer about the larger scope.

I think marking ml-model as deprecated earlier is better. Otherwise, users might be guided toward using it, which will lead to frustration later for migration, since the intention is to not maintain it anymore. Note that, if needed, a deprecated extension schema can still be used. It would just clearly state that one should not expect any fixes and development from it in the future.

[languith]

As we are the original maintainer of the stac-mlm extension, we would like to keep the repository within our organization at CRIM.
We understand the community’s preference for a neutral organization to enhance visibility and stewardship. However, CRIM, being a non-profit dedicated to applied research, aligns closely with these community values. Our continued management of the repository not only allows us to leverage our specialized expertise and dedicated resources for the evolution of the extension but also ensures rigorous maintenance and quality control.
We are committed to open collaboration and invite contributions from the community to ensure diverse input.
Furthermore, while we recognize the advantages of centralizing extensions under the stac-extensions organization, without a concerted effort to consolidate all extensions, maintaining this particular extension under CRIM allows us to demonstrate our active role in the geospatial domain and our commitment to the open-source.
We are open to discussions on how best to implement a governance model that ensures broad community involvement while keeping the development home at CRIM.

[jiayuasu]

Hi @languith

This is Jia from (https://wherobots.com/), and I want to express my appreciation for CRIM’s contributions to our collaborative projects.

I understand your concerns, but I believe transitioning the dlm-extension to a vendor-neutral GitHub repository under @stac-extensions is crucial for nurturing the growth of this open-source initiative. Experience has shown that few commercial organizations commit seriously to a repository that is not perceived as neutral, especially one aimed at establishing specifications rather than merely hosting software artifacts.

The stagnation observed in the development of https://github.com/crim-ca/dlm-extension over the past three years underscores the need for enhanced visibility and community collaboration, which can be achieved under the @stac-extensions umbrella.

As an active maintainer of Apache Sedona and participant in several open standards, including GeoParquet (https://github.com/opengeospatial/geoparquet), our commitment to open, collaborative development is profound. When Sedona transitioned to the Apache Software Foundation in 2020, all original git commit histories were preserved, dating back to its inception in 2015, as can be seen at https://github.com/apache/sedona.

Most importantly, according to stac-extensions/ml-model#13 , @rbavery and the community was expecting to move the dlm-extension to the stac-extensions based on @fmigneault-crim 's original communication with us in December last year.

While CRIM may continue development independently, uniting under @stac-extensions would avoid fragmentation and strengthen the project. I encourage CRIM to reconsider its position. Otherwise, we may need to reassess our commitment of time and resources to this project, given the potential for limited progress without broader collaborative support.

[rbavery]

Hi everyone, Francis and I (maintainers of the MLM) have agreed that the stac-extensions repo is the best home for the MLM extension.

We got approval from CRIM, who owns the current MLM github repo, to move the extension. Given that we don't want to have two competing specs and the MLM can accommodate docker-compose I think the ml-model extension should be marked deprecated.

The MLM extension has evolved quite a bit to meet a range of needs when describing models today. I think we should focus efforts on the MLM once a move is approved to the stac-extensions repo (working on this now in the Gitter channel). We are open to others providing input as contributors and maintainers as we continue to improve the schema and tooling around the MLM spec and would like the MLM to be collaborative and community driven.

[gadomski]

Awesome! Thanks for running this down.

Given that we don't want to have two competing specs and the MLM can accommodate docker-compose I think the ml-model extension should be marked deprecated.

Can you open an issue (or a PR) on that repo to this effect w/ the justification? 🙇🏼 Also, per https://github.com/orgs/stac-utils/discussions/4#discussioncomment-9240164, we should also update that extension w/ information re: migrating, etc.

[rbavery]

Np! There's currently a PR with some discussion stac-extensions/ml-model#16. I can PR an update to the README with info on a migration path and deprecation.

[gadomski]

Thanks! I should have looked before requesting the PR ... my bad 🤦🏼 .

[rbavery]

Np! Here is a PR with a migration path doc. Reviews appreciated from any STAC Extension maintainers: stac-extensions/ml-model#19

[fmigneault]

@rbavery @jiayuasu @gadomski @m-mohr @emmanuelmathot @languith
All validations have been obtained on CRIM's side. We can make the transition of the repository.

Here is the plan:

1. "someone" with sufficient access should create https://github.com/stac-extensions/mlm

   • please let me know who that person can be, and when this can be done
     (ideally soon, because timing with a scientific publication [see below] would help advertise this official location)
   • The "import repository" feature (https://github.com/new/import) should be used.
     This is to make it a "source" repository rather than yet another fork.
     (if that doesn't work somehow, I can also push the repository, but I will need access to do so instead)
   • maintainer permissions should be added for @fmigneault and @rbavery (minimally)
   • protection rules should be configured appropriately (eg: PR test conditions, min approvals, etc.)
     (if desired, I can do this step if sufficient access is granted to me)
   • pypi token for CI deployment of stac-model package (waiting for @rbavery access)

2. I will then map the contents of https://github.com/crim-ca/mlm-extension to redirect to https://github.com/stac-extensions/mlm as the new official repository

   • CRIM will keep the fork such that old definitions preserve valid links to schemas hosted under https://crim-ca.github.io/mlm-extension/
   • CRIM's README will include a deprecation notice redirecting to https://github.com/stac-extensions/mlm
     (Deprecate crim-ca/mlm-extension in favor of stac-extensions/mlm crim-ca/mlm-extension#40)

3. Transfers:

   • transfer any remaining issues from https://github.com/crim-ca/mlm-extension to https://github.com/stac-extensions/mlm
   • close outdated issues

4. Update and merge PR deprecated in favor of MLM extension stac-extensions/ml-model#16

   • mark ml-model as deprecated
   • add a link suggesting to use https://github.com/stac-extensions/mlm instead.

5. Open a PR updating https://github.com/stac-extensions/stac-extensions.github.io/ to:
   (Update MLM reference with official location stac-extensions/stac-extensions.github.io#52)

   • refer "Machine Learning Model (MLM)" to https://github.com/stac-extensions/mlm
   • remove external reference to https://github.com/crim-ca/mlm-extension

6. Update citation references and usages

   • Add a CITATION.cff with the associated upcoming ACM SIGSPATIAL 2024 paper presenting STAC MLM
     (add citation references stac-extensions/mlm#32)
   • Update the camera-ready links in the paper to use https://github.com/stac-extensions/mlm
   • Bump mlm to "candidate" maturity level (there are >3 usages already, as referenced in the paper)
     Update extension to 'candidate' maturity level stac-extensions/mlm#37
   • make sure mlm and ml-model are published with the appropriate maturity levels on https://stac-extensions.github.io/

7. Close/merge relevant issues/discussions/PRs:

   • deprecated in favor of MLM extension stac-extensions/ml-model#16
   • Revisiting the ML Model extension stac-extensions/ml-model#13
   • https://github.com/orgs/stac-utils/discussions/4

[gadomski]

@fmigneault I've created https://github.com/stac-extensions/mlm as a private repo and added you w/ maintainer permissions. I did not import the repo — I don't see any downside to you simply pushing the existing history to the new, blank repo.

I think that's all I need to do, but please let me know if there's more I can do to assist the transition.

[fmigneault]

@gadomski I was able to push the contents. However, I cannot adjust the CI/CD requirements, or generating releases.

[gadomski]

@fmigneault ok I've bumped you up to admin, you should be able to do anything I can at this point :-)

[fmigneault]

Thank you.