Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove pool2d MLRoundingType - Simplify the operand layout support of conv2d and pooling 2d operations #770

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

fdwr
Copy link
Collaborator

@fdwr fdwr commented Oct 23, 2024

For #324

Rather than having two output shape parameters, an explicit output size and an output rounding mode, be more explicit in WebNN such that the caller passes the explicit output shape, resolving any higher-level policy beforehand (like with convTranspose2d):

dictionary MLPool2dOptions : MLOperatorOptions {
  sequence<[EnforceRange] unsigned long> windowDimensions;
  sequence<[EnforceRange] unsigned long> padding;
  sequence<[EnforceRange] unsigned long> strides;
  sequence<[EnforceRange] unsigned long> dilations;
  MLInputOperandLayout layout = "nchw";
- MLRoundingType roundingType = "floor";
  sequence<[EnforceRange] unsigned long> outputSizes;
};

Preview | Diff

index.bs Outdated
@@ -4688,24 +4682,8 @@ partial interface MLGraphBuilder {
1. Let |inputWidth| be |inputShape|[2].
1. Let |channels| be |inputShape|[3].
</dl>
1. If |outputSizes| is not given, then:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

outputSizes is optional, we should keep this checking. I feel the existing code has an issue, it should use outputSizes if it is given?

Copy link
Collaborator Author

@fdwr fdwr Oct 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the previous if looked like a typo (so that should be fixed in any case).

Well, with this change, outputSizes would be required (caller is explicit now). Do you think this simplification to be problematic/burdensome? 🤔

Copy link
Contributor

@huningxin huningxin Oct 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then outputSizes is not an option anymore, should it be moved to another parameter? Like

 MLOperand averagePool2d(MLOperand input, sequence<[EnforceRange] unsigned long> outputSizes, optional MLPool2dOptions options = {});

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://w3ctag.github.io/design-principles/#prefer-dictionaries suggests:

You should also consider accepting mandatory parameters through a dictionary, if it would make the API more readable, especially when they are of primitive types.

...but it also says:

The dictionary itself should be an optional argument, so that if the author is happy with all of the default options, they can avoid passing an extra argument.

I'd err on the side of putting required arguments into the dictionary - and making the dictionary itself required - because:

  • It's more readable, since primitive types are now described (as the Design Principles suggest)
    • averagePool2d(input, {outputSizes: [2, 3]}) > averagePool2d(input, [2, 3])
  • It's more readable, since it's otherwise not obvious to the caller why some configuration options belong in the options dictionary while others don't
    • averagePool2d(input, {outputSizes: [2, 3], layout: "nchw"}) > averagePool2d(input, [2, 3], {layout: "nchw"})
  • It's more extensible, since the options dict can be changed without changing the method signature (though making a non-required dictionary field required would still be a breaking change)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd err on the side of putting required arguments into the dictionary - and making the dictionary itself required

SGTM. Thanks for the explanation, @a-sully !

Copy link
Collaborator Author

@fdwr fdwr Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd err on the side of putting required arguments into the dictionary - and making the dictionary itself required

Fine with me. Will do.

and making the dictionary itself required

Hmm, the following doesn't work (bikeshed error as required can be used on fields but evidently not parameters):

  MLOperand averagePool2d(MLOperand input, required MLPool2dOptions options);

So I presume you meant to just remove the "= {}" instead?

  MLOperand averagePool2d(MLOperand input, MLPool2dOptions options);

index.bs Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
@fdwr fdwr requested a review from huningxin November 15, 2024 23:31
@@ -5395,7 +5385,7 @@ partial dictionary MLOpSupportLimits {

<details open algorithm>
<summary>
To <dfn for=MLGraphBuilder>calculate pool2d output sizes</dfn> given {{MLInputOperandLayout}} |layout|, [=/list=] of 4 unsigned integers |inputShape|, {{MLRoundingType}} |roundingType|, [=/list=] of 2 unsigned integers |windowDimensions|, [=/list=] of 4 unsigned integers |padding|, [=/list=] of 2 unsigned integers |strides|, [=/list=] of 2 unsigned integers |dilations|, and optional [=/list=] of 2 unsigned integers |outputSizes|, perform these steps. They return a [=/list=] of 4 unsigned integers.
To <dfn for=MLGraphBuilder>calculate pool2d output sizes</dfn> given {{MLInputOperandLayout}} |layout|, [=/list=] of 4 unsigned integers |inputShape|, [=/list=] of 2 unsigned integers <var ignore>windowDimensions</var>, [=/list=] of 4 unsigned integers <var ignore>padding</var>, [=/list=] of 2 unsigned integers <var ignore>strides</var>, [=/list=] of 2 unsigned integers <var ignore>dilations</var>, and optional [=/list=] of 2 unsigned integers |outputSizes|, perform these steps. They return a [=/list=] of 4 unsigned integers.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If windowDimensions, padding, strides and dilations parameters are not used in the algorithm steps, they probably can be removed.

outputSizes is not optional.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this algorithm is only invoked in one place (the "create pooling operations" steps) can we just move it inline?

1. Set |outputWidth| to ceiling(|outputWidth|).
1. Set |outputHeight| to ceiling(|outputHeight|).
</dl>
1. Let « |outputHeight|, |outputWidth| » be |outputSizes|.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add validation steps for user-supplied outputSizes? Chromium prototype does that: https://source.chromium.org/chromium/chromium/src/+/main:services/webnn/public/cpp/graph_validation_utils.cc;l=1179;drc=60130be50f68c89220d8ace62c87155025267a2c;bpv=1;bpt=1

If we add outputSizes validation steps, windowDimensions, padding, strides and dilations parameters would be required.

@@ -5451,7 +5427,7 @@ partial dictionary MLOpSupportLimits {
1. If |options|.{{MLPool2dOptions/dilations}}'s [=list/size=] is not 2, then [=exception/throw=] a {{TypeError}}.
1. If any value in |options|.{{MLPool2dOptions/dilations}} is not greater than 0, then [=exception/throw=] a {{TypeError}}.
1. Let |desc| be a copy of |input|.{{MLOperand/[[descriptor]]}}.
1. Let |outputShape| be the result of [=MLGraphBuilder/calculating pool2d output sizes=] given |options|.{{MLPool2dOptions/layout}}, |input|'s [=MLOperand/shape=], |options|.{{MLPool2dOptions/roundingType}}, |options|.{{MLPool2dOptions/windowDimensions}}, |options|.{{MLPool2dOptions/padding}}, |options|.{{MLPool2dOptions/strides}}, |options|.{{MLPool2dOptions/dilations}}, and |options|.{{MLPool2dOptions/outputSizes}} (if it [=map/exists=]).
1. Let |outputShape| be the result of [=MLGraphBuilder/calculating pool2d output sizes=] given |options|.{{MLPool2dOptions/layout}}, |input|'s [=MLOperand/shape=], |options|.{{MLPool2dOptions/windowDimensions}}, |options|.{{MLPool2dOptions/padding}}, |options|.{{MLPool2dOptions/strides}}, |options|.{{MLPool2dOptions/dilations}}, and |options|.{{MLPool2dOptions/outputSizes}} (if it [=map/exists=]).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The outputSizes validation above in 13.2 doesn't appear to be correct--shouldn't it be similar to this logic?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants