Document ORTE -> PRRTE MCA parameter changes

[qkoziol]

Document MCA parameter changes from move from ORTE to PRRTE.

Addresses Github issue #7668

[lrbison]

Reviewed for typos and wording. Also confirmed none of the deprecated options would apply to 5.0.0 version series. I cannot say if this is an exhaustive listing.

[jsquyres]

I rebased this PR on current main/HEAD, and added a 2nd suggested commit. This commit can be squashed or deleted.

[janjust]

I would squash the commit, but otherwise looking good to me.

[lrbison]

When replacing hwloc_base_use_hwthreads_as_cpus
What does this mean?

Value: [<Value>]:hwtcpus

I expected it to be perhaps just

rmaps_default_mapping_policy:hwtcpus

[rhc54]

When replacing hwloc_base_use_hwthreads_as_cpus What does this mean?

Value: [<Value>]:hwtcpus

I expected it to be perhaps just

rmaps_default_mapping_policy:hwtcpus

The cmd line option looks like --map-by :hwtcpus if you aren't specifying any mapping policy. However, note that this does NOT affect the default PRRTE behavior - it is a per-job directive. For mpirun, it doesn't matter as you are running a one-shot - unless your app does a comm_spawn, in which case it might (depends on whether or not you have set inherit).

If you want to affect the defaults, then you have to use the MCA parameter: PRTE_MCA_hwloc_base_use_hwthreads_as_cpus

[lrbison]

@rhc54 ah, thank you for explaining.

Jeff shared a rendered version of the table with me: https://ompi--11890.org.readthedocs.build/en/11890/mca.html#label-mca-backward-compat

And while most of these make sense to me to read, the rmaps_default_mapping_policy entries was hard for me to grok, as I expect it would be for users. I wonder if it would be worth an example explaining the concept of these parameters which get captured as extra colon-separated tags on top of other parameters.

However, note that this does NOT affect the default PRRTE behavior - it is a per-job directive

Hmm, am I hearing you correctly then: This table isn't correctly informing users how to take the previous MCA parameter hwloc_base_use_hwthreads_as_cpus and get the same effect using rmaps_default_mapping_policy because updating the default won't change the one-shot launch behavior. What is the correct parameter to direct the user towards?

[rhc54]

Hmm, am I hearing you correctly then: This table isn't correctly informing users how to take the previous MCA parameter hwloc_base_use_hwthreads_as_cpus and get the same effect using rmaps_default_mapping_policy because updating the default won't change the one-shot launch behavior.

Sorry - no, it is the other way around. Updating the default via MCA param will change the one-shot behavior. My point was that the cmd line options (e.g., --map-by foo) do not change the default behavior - they only impact the one-shot job request. So if you pass --map-by :hwt you will treat hwt's as cpus for that specific job. However, if that job does a comm_spawn, you may not treat hwt's as cpus for the child job (it would depend on whether or not inherit was set).

If you always want to treat hwt's as cpus, then you would set the MCA param so the default is set (probably in your default MCA param file, which is what I do).

[lrbison]

OK, thank you for clarifying

[jsquyres]

@rhc54 Thanks for all the clarifications.

@qkoziol @lrbison Can you guys fix up the content? Feel free to squash it all down to a single commit.

[qkoziol]

Working on adding some examples

[qkoziol]

Updated with examples

[jsquyres]

@qkoziol These are great examples!

One point, however -- all your examples show extra spaces around the = in the assignment of shell variables. That actually causes a syntax error; the extra spaces need to be removed:

# This is an error
$ export FOO = bar
zsh: bad assignment
$ FOO = bar
zsh: command not found: FOO

# It's also an error in bash
$ export FOO = bar
bash: export: `=': not a valid identifier
$ FOO = bar
bash: FOO: command not found

# Either of these are correct
$ export FOO=bar
$ FOO=bar

[janjust]

@qkoziol please open up v5.0 PR

[qkoziol]

@qkoziol please open up v5.0 PR

FYI: v5.0 PR: #11926