Howto handle hook changes

[fschulze]

I have a hook which needs to be changed. In essence it needs an additional argument.

The problem is, that plugins should be able to use the new argument, but still work on older versions where the argument didn't exist yet.

If I add the argument and use it in the hook implementation in the plugin, then the plugin doesn't work with older versions anymore: Argument(s) ... are declared in the hookimpl but can not be found in the hookspec

Using kwargs with defaults also doesn't work, the argument isn't passed then.

If I add a new hook, then both my implementations in the plugin are called, because the old hook still needs to be called for older plugins.

One idea would be the possibility to mark a hook specification deprecated with the name of the replacement and then when loading a plugin and it implements both hooks forget about the old hook. If it only supports the old hook, then add a warning about the deprecation.

[RonnyPfannschmidt]

this is a unsolved problem as of now - we didn't get the attempted solution to a thing that works

pytest also has this issue as the pytest_deselected hook is supposed to be invoked by externals but lacks a critical needed argument

[goodboy]

@fschulze why can't you add the new argument to the hookspec and then then call the hook using both the new and old argument (hookimpls don't have to declare all args in the spec or in the call - though the latter is not documented well / it's implied). Then when you are ready to deprecate the old argument use the new deprecation feature from #138 to slowly fade out the old argument?

Using kwargs with defaults also doesn't work, the argument isn't passed then.

I actually did implement this and it works (#15 leading to #43) but we never got to consensus on whether it was actually the right way to solve this issue.

[goodboy]

Shoot just realized #138 is only for deprecating full hooks and not specific args.

@RonnyPfannschmidt what do you think about something similar like:

@hookspec(
    warn_on_args={'arg1': DeprecationWarning("arg1 is deprecated and will be removed soon")
)
def oldhook(arg1, new_arg1):
    pass

[RonnyPfannschmidt]

@tgoodlet doesn't work even remotely since it ignores stuff like caller handling

[fschulze]

@tgoodlet that still wouldn't solve the issue, that the plugin should still work with older "hosts" where the argument wasn't there yet. And I don't want to deprecate the old argument, just introduce an additional one.

For me an entirely new hook would be fine, but then I need a way to prevent the old hook to be called in plugins that support the new hook.

[goodboy]

doesn't work even remotely since it ignores stuff like caller handling

@RonnyPfannschmidt explain? Also what are you referring to with this comment?

that still wouldn't solve the issue, that the plugin should still work with older "hosts" where the argument wasn't there yet.

@fschulze sorry, why wouldn't this work?

To make this work:

• add the new arg to the hook spec and call
• some new hookimpls receive the new arg
• some old hookimpls receive the old arg

I'm not sure I'm following what the problem is and I've opened #172 to demonstrate how you can accomplish this.

[RonnyPfannschmidt]

@tgoodlet hook calls are in plugins, now plugins are broken in fun ways as all of them would have a breaking change

so the proposed mechanism you showed is one that require breaking api changes

[goodboy]

hook calls are in plugins, now plugins are broken in fun ways as all of them would have a breaking change

I'm not following - where's the breakage? This new warn_on_args wouldn't conflict with anything (since it's an addition not a change to anything existing) and hook calls can continue to exist as they have inside plugins?

[RonnyPfannschmidt]

@tgoodlet plug-ins can no longer call hooks as the args changed

[goodboy]

@RonnyPfannschmidt right which is why you first deprecate and then later remove - just like any API progression? If you're going to deprecate and change an API at some point you have to stop supporting it. As per #172 you can definitely keep deprecated and new args working in tandem.

To me this is simple:

• warn_on_args causes deprecation warnings to emit at the hook call line as well as the hookimpl def
• old hook calls continue to work just fine but now with the warnings for any call or hookimpl that uses the deprecated args
• eventually hook call sites change when the arg is fully removed or else you'll get the classic pluggy error
• we'll need a slight bit of extra logic to warn differently on deprecated args (i.e. those listed in warn_on_args) which are in the spec but not in the impl

Implementing this will give good reason to finally bring in #57 as well since we'll want to keep track of which args per hookspec are deprecated.

[RonnyPfannschmidt]

@tgoodlet all callers must include all arguments - so i have no idea what you are talking about

only receivers can leave arguments out

[goodboy]

all callers must include all arguments - so i have no idea what you are talking about

Yes - no confusion there. So you change the spec to include warn_on_args and keep all your callers as is.
Then after enough time of these args being marked deprecated you change the call lines?

At some point the calls have to change, I don't see how you can ever get around that.

[RonnyPfannschmidt]

@tgoodlet removal isn't the problem here - adding them is

[goodboy]

Why? A call can not declare all args that are in the spec (you'll get a warning) as well as adding a new arg to both the call and spec works just fine with impls which don't declare it, again, as per #170?

Besides, warn_on_args can still have future warnings for introducing new args and everything should work the same way.

[RonnyPfannschmidt]

@tgoodlet the plugins will not have a extra arg, they will have a missing arg to the call and break

for example if pytest_deselected(items) changed to pytest_deselected(items, reason)
and my conftest only calls config.ihook.pytest_deselected(items=removed) - im going to be in trouble

[fschulze]

@tgoodlet

that still wouldn't solve the issue, that the plugin should still work with older "hosts" where the argument wasn't there yet.

@fschulze sorry, why wouldn't this work?

As I originally wrote, the plugin wouldn't work with older versions of the app defining the hookspec:

If I add the argument and use it in the hook implementation in the plugin, then the plugin doesn't work with older versions anymore: Argument(s) ... are declared in the hookimpl but can not be found in the hookspec

[goodboy]

for example if pytest_deselected(items) changed to pytest_deselected(items, reason)

The question is where is this changed. If it's in the spec then everything will work since warn_on_args is also on the spec.

and my conftest only calls config.ihook.pytest_deselected(items=removed)

@RonnyPfannschmidt I'm proposing this emits a warning and the call will still work - again #170.

I don't see why it's a problem that plugins do not work on old versions of pytest that don't support the reason arg - that's a pretty darn good reason for a new version of a plugin.

@fschulze If the spec doesn't define the new args then your call shouldn't either; the plugin should be aware of the host it is dealing with.

Trying to hack this inside the call machinery of pluggy is not correct imho. You should not be allowing a version of a plugin to work implicitly over multiple versions of a host without special explicit provisions (just like you would with any dependency that changes its api - pin a version of the plugin to versions of the host).

@RonnyPfannschmidt @fschulze this is the way I see it:

• you want to add a new argument
• change the hookspec in the host to introduce the new argument
• plugins not calling with the new argument see a future warning (under my proposal for a warn_on_args)
• plugins are eventually changed to include the new argument in hook calls likely via a version release or some change
• plugins now with the new argument can't call the old versions of the host since the old host's spec doesn't include this argument (results in an error)
• the version pin change: pin the new version of the plugin to only support newer versions of the host like you would with literally any other dependency API change ever..
• or do, the hack introspection approach: if you don't want to pin versions then simply modify the **kwargs to your hook call line based on introspecting the hookspec from the current version of the host - Encapsulate spec definitions with a class #57 would aid with this

Trying to make this work magically in pluggy I think is not correct - it should be the plugin's job (if anyone's) to figure out what API is supported in the host, not pluggy's. Version pinning to the host should be a thing; it's explicit demarcation of breaking changes and there's nothing wrong with having version(s) of a plugin tied to specific version of a host.

[fschulze]

@tgoodlet

Trying to make this work magically in pluggy I think is not correct - it should be the plugin's job (if anyone's) to figure out what API is supported in the host, not pluggy's. Version pinning to the host should be a thing; it's explicit demarcation of breaking changes and there's nothing wrong with having version(s) of a plugin tied to specific version of a host.

I would be fine with that, but at the moment I don't see how the plugin can do that. Do you have an example? Does it work already, or does pluggy need to add some introspection facilities?

If my plugin can check whether the host supports a specific hookspec during import, then I could just wrap my hook implementations in an if block and provide either the old implementation, or the new one. I'm not necessarily talking about arguments here, but whole hook functions. On the host I would just do something like myhost_oldhook, myhost_newhook. If myhost_newhook exists, then the plugin wouldn't provide myhost_oldhook. That way my plugin can support the old host without myhost_newhook and on the newer host the myhost_oldhook wouldn't be called for my plugin, but only myhost_newhook.

I could already import the hookspec module directly and introspect it, but normally the plugin doesn't know about that module, so I'd like to avoid that.

[RonnyPfannschmidt]

im out of this

[goodboy]

im out of this

😞

Does it work already, or does pluggy need to add some introspection facilities?

@fschulze not yet but like I said #57 would bring it. I just have to dig it all back up.
Iirc it would be something like pm.hook.myhook.spec would be references to a HookSpec type that can be used to determine the current spec's metadata.

[goodboy]

Just copying a comment from @fschulze from the #57 PR.

edit: sorry to have added this comment on this not entirely related PR, I thought I was replying on #170.

@tgoodlet

@fschulze take a look at this if you don't mind so we can figure out if it's going to meet your needs.
With this you can introspect spec args (and other things) using pm.hook..spec.

That again is from the host side, do you have anything in mind for the plugin side? Is that even possible?

I still think my proposal like this would be simpler:

Host side:

@hookspec(deprecated=True)
def myhost_oldhook(...):
...

@hookspec(replaces=myhost_oldhook)
def myhost_newhook(...):
...

Plugin side:

@hookimpl(optional=True)
def myhost_oldhook(...):
...

@hookimpl(optional=True)
def myhost_newhook(...):
...

Pluggy would then remove myhost_oldhook from the list. This way all the following work:

old host - old plugin
new host - old plugin (with deprecation warning)
old host - new plugin (because the hook impl is optional)
new host - new plugin (because pluggy removes the old hook impl from the list and new host only calls the new hook of the plugin)

[RonnyPfannschmidt]

pytest-dev/pytest#8463 as idea for a mechanism