doc: add async_hooks migration note #45335

GeoffreyBooth · 2022-11-06T03:48:21Z

Implements nodejs/diagnostics#194 (comment), a docs addition informing users that the public async_hooks API will be going away at some unspecified future date; and some suggestions for other APIs to use instead. cc @Qard @mcollina @nodejs/diagnostics @nodejs/tsc

This doesn’t commit us to either a timeline or a process for async_hooks going internal, but I think merging in this PR can mark that we’ve decided on the goal we want to reach. Then we can discuss how we want to get there.

Trott · 2022-11-06T04:12:00Z

This feels like the domain deprecation. "We plan to get rid of this at some point (but actually...uh...we probably never will)." That's not a reason not to do it, but we should be aware that we have precedence for this and I'm not sure it's a precedent we don't want to break....

async_hooks was supposed to be the thing that let us get rid of domain. It's like that xkcd comic about standards.....

Trott

Text LGTM if @nodejs/diagnostics agrees with the content.

Qard · 2022-11-06T06:44:38Z

Well, we should be able to convert domains to AsyncLocalStorage. Though making async_hooks internal doesn't mean we can't continue to use the parts of it we need for other features like domains. Just means we can stop exposing it externally and can break compatibility to make it better. Whereas right now it's extremely difficult to make any changes to async_hooks without covering a whole lot of edge cases. 😬

Should we also change the status at this point? The "legacy" status is a more accurate representation of the current state, and "deprecated" is where we want to get it.

mcollina · 2022-11-06T09:46:23Z

Legacy is likely the correct status of this API.

mcollina · 2022-11-06T09:47:44Z

Note that we are exposing AsyncLocalStorage under async_hooks, we might want to move it elsewhere or just flag createHook as legacy.

doc/api/async_hooks.md

GeoffreyBooth · 2022-11-06T14:40:17Z

Should we also change the status at this point? The "legacy" status is a more accurate representation of the current state, and "deprecated" is where we want to get it.

Arguably the current status is the best, as technically it can be removed at any time. I think legacy means that it won't be removed but it also won't be improved, but we do want to remove this (from the user's perspective). We could go straight to deprecated now, to start the clock, and maybe remove in 20.0.0; but I was trying just to reach consensus about the goal here, to leave the plan for achieving it as a follow up.

nodejs-github-bot · 2022-11-06T17:24:55Z

CI: https://ci.nodejs.org/job/node-test-pull-request/47739/

Trott

You can test with a make doc-only but I'm pretty sure adding stuff to the stability banner line will break things.

doc/api/async_hooks.md

Flarna · 2022-11-06T18:01:03Z

doc/api/async_hooks.md

@@ -2,7 +2,10 @@

 <!--introduced_in=v8.1.0-->

-> Stability: 1 - Experimental
+> Stability: 1 - Experimental. We plan to remove this API


Really? I'm not aware of any plan for removal.

That’s what this PR is meant to declare. We’re agreeing that we want to eventually remove it, and the specifics of how will be determined.

Ok, in that cases I don't agree.
I see no benefit in removing something from public API surface which is need internally without providing a replacement. This only creates unneeded friction.
As I see no path to get a replacement as of now I see a similar topic here as in domains.

I see no benefit in removing something from public API surface which is need internally without providing a replacement.

async_hooks exposes internals. That’s why it shouldn’t remain a public API; but also why it’s fine to continue existing as an internal API.

The replacements are AsyncLocalStorage and Diagnostics Channel. If there are use cases currently served by async_hooks that are unachievable with those APIs, then those use cases are ones we should try to address before async_hooks is removed, either via AsyncLocalStorage and Diagnostics Channel or some other APIs. The point of this PR is to start that process: to encourage users depending on async_hooks to start migrating off of it, and report issues that they encounter so that collaborators can address them, by adding new capabilities to AsyncLocalStorage or Diagnostics Channel or other APIs that we consider stable or on a path toward stability.

Flarna · 2022-11-06T18:06:45Z

doc/api/async_hooks.md

-> Stability: 1 - Experimental
+> Stability: 1 - Experimental. We plan to remove this API
+> in the future; it will never become stable.
+> Please migrate to [`AsyncLocalStorage`][] and [Diagnostics Channel][] and


Could you add some more info here for users? In special I'm not aware of any use case where diagnostics channel is a replacement for async hooks. Diagnostics channel is a very generic API - similar to EventEmitter and it servers a different purpose. The only connection is that APMs tend to use both - but for different things.
Therefore I don't think we should mix this here as it generates likely more confusion.

AsyncLocalStore is actually the replacement for the most popular async hooks usecase. Concrete the use case to pass transactional context across async boundaries. Therefore I would put more focus on this.

The Diagnostics Channel is a replacement for the higher level use cases that people often use async_hooks for: tracking network requests, for example, rather than specific promises. AsyncLocalStorage is the replacement low-level API, Diagnostics Channel is a replacement high-level API for many of the use cases that both low-level APIs are used for.

Tracking active http requests is one example of a use case previously solved by async_hooks which can much more simply and performantly be solved by diagnostics_channel. My intent is to add more channels in various places to cover many of the other needs which async_hooks currently serves. If there are remaining use cases which neither AsyncLocalStorage nor diagnostics_channel with additional channels can solve, we can define new APIs which are more purpose-built and don't leak internals so much that performance optimization becomes next to impossible. One of the greatest challenges with async_hooks is that it's virtually impossible to refactor without breaking the ecosystem, which makes it extremely difficult to improve the performance. With higher-level APIs we can gain more control over the exact behaviour of the internals enabling us to rearrange things in ways that will perform much better.

Flarna · 2022-11-06T18:08:08Z

doc/api/async_hooks.md

+> Stability: 1 - Experimental. We plan to remove this API
+> in the future; it will never become stable.
+> Please migrate to [`AsyncLocalStorage`][] and [Diagnostics Channel][] and
+> other stable APIs.


This is quite vague. I think we should either remove this or point to concreate APIs (which I'm not aware of).

I am pointing to specific APIs. I don’t think we need any more detail here. This is a quick note informing people that this API will go away. It’s not our responsibility to tell them how to rewrite their code.

@Flarna Would this be better?

Suggested change

> other stable APIs.

> in the future; it will never become stable. If possible,

> please use other APIs instead such as [`AsyncLocalStorage`][] or

> [Diagnostics Channel][].

Qard · 2022-11-06T18:18:32Z

We can't just remove it at any time though. We want to remove it, but there will need to be a long deprecation cycle to do so and there's still work to be done to ensure the core use cases are adequately covered. Legacy status seems most suitable--it may be removed eventually but that will take a long time and it may have to remain but become actively discouraged, which legacy communicates well.

It is a feature that is currently depended on extensively, so the "could change at any time" definition of experimental is kind of a lie.

GeoffreyBooth · 2022-11-06T18:35:53Z

It is a feature that is currently depended on extensively, so the “could change at any time” definition of experimental is kind of a lie.

This PR isn’t changing the status. We can open a separate PR for that.

doc/api/async_hooks.md

build-breakage fixed

mcollina

Given how much async_hooks is used everywhere we cannot safely remove it. https://www.npmjs.com/package/cls-hooked is downloaded more than one million times per week as of today.

I agree it's not the API we want long term, but that's the API we have now. The ecosystem usage of this API ask us to treat it like any other stable API we have.

I don't see us removing it anytime soon, and we'd be sending the wrong message out. I don't see us improving this API in any way either. Legacy seems a perfect match for this situation. We should definitely point folks to AsyncLocalStorage (diagnostic_channel is yet another experimental thing, it's likely better to hold off).

Flarna · 2022-11-06T20:47:07Z

While I agree with the general move towards AsyncLocalStore I don't agree with the wording used in this PR and I doubt this will change based on commenting only.
Therefore I will create a PR during the next days as an alternative/addition to this one. I assume some sort of combination will finally succeed.

GeoffreyBooth · 2022-11-06T20:52:37Z

I will create a PR during the next days as an alternative/addition to this one. I assume some sort of combination will finally succeed.

Do you want to just propose alternative language here as a comment?

GeoffreyBooth · 2022-11-06T21:00:02Z

Given how much async_hooks is used everywhere we cannot safely remove it. npmjs.com/package/cls-hooked is downloaded more than one million times per week as of today.

I don’t see how this is relevant. cls-hooked can migrate to AsyncLocalStorage, just as it migrated from async_listener previously, and those million downloads per week will use the new API. Ditto with the instrumentation libraries that use async_hooks; those vendors could use a firm push to prioritize refactoring their code.

I agree it's not the API we want long term, but that's the API we have now. The ecosystem usage of this API ask us to treat it like any other stable API we have.

I don’t see us removing it anytime soon, and we’d be sending the wrong message out.

This API is not stable and will never be. It’s a borderline security risk in that it leaks internals without the user needing to pass --expose-internals; at best it’s a massive bug by design. We should be doing everything we can to encourage users to migrate off of it, not providing them the reassurance that it’ll never go away. The ecosystem usage will never decrease if we do nothing; it’s circular logic to argue that we can’t warn people off of an API because people are using the API. We may never actually carry out our threat of actually removing it, but we need to put that option firmly on the table to nudge people to get migrating.

Flarna · 2022-11-06T21:02:10Z

I plan to touch a bit more in the async_hooks docs to adress the exposes internals part and to be a bit more concreate regarding when to use AsyncLocalStorage.
I'm not planning to go the "we will remove this" announcement as this is the part where I don't agree.

mcollina

lgtm

anonrig

lgtm 🚀

doc/api/async_hooks.md

trevnorris · 2022-11-16T19:39:52Z

Fortunately it seems like this API will never be removed. Personally I use this API extensively whenever I need to debug very complex asynchronous issues which AsyncLocalStorage can't cover. I'll be happy to open issues that cover my use cases, and maybe someone else can come up with an API that would better suit the problem scope.

The Diagnostics Channel doesn't offer a substitute because it requires every location you want to observe to be tooled first. I use async hooks to figure out where I need to put those channels so it makes debugging easier in the future.

GeoffreyBooth · 2022-11-16T19:43:31Z

Fortunately it seems like this API will never be removed.

I wouldn’t count on that. The point of asking for use cases that aren’t covered by the mentioned APIs is so that they can be potentially deprecated in the future.

Trott

I think the discussion between @trevnorris and @Qard does need a resolution of some sort, but this is a doc-only change and we can easily update the text later, so I'm smashing Approve on this one for now.

@Qard

Approved per #45335 (comment): “+1 to @Qard’s suggested alternative”

nodejs-github-bot · 2022-11-21T00:29:53Z

Landed in ea92ca1...80dcc06

PR-URL: #45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

PR-URL: nodejs#45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

Add hints to migrate away from async hooks. Change docs at various places to be more clear that resources are internals and may change at any time. PR-URL: #45369 Refs: #45335 Reviewed-By: Geoffrey Booth <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Stephen Belanger <[email protected]> Reviewed-By: Chengzhong Wu <[email protected]>

PR-URL: #45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

Add hints to migrate away from async hooks. Change docs at various places to be more clear that resources are internals and may change at any time. PR-URL: #45369 Refs: #45335 Reviewed-By: Geoffrey Booth <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Stephen Belanger <[email protected]> Reviewed-By: Chengzhong Wu <[email protected]>

PR-URL: #45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

Add hints to migrate away from async hooks. Change docs at various places to be more clear that resources are internals and may change at any time. PR-URL: #45369 Refs: #45335 Reviewed-By: Geoffrey Booth <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Stephen Belanger <[email protected]> Reviewed-By: Chengzhong Wu <[email protected]>

PR-URL: #45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

Add hints to migrate away from async hooks. Change docs at various places to be more clear that resources are internals and may change at any time. PR-URL: #45369 Refs: #45335 Reviewed-By: Geoffrey Booth <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Stephen Belanger <[email protected]> Reviewed-By: Chengzhong Wu <[email protected]>

PR-URL: #45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

GeoffreyBooth added async_hooks Issues and PRs related to the async hooks subsystem. experimental Issues and PRs related to experimental features. diagnostics_channel Issues and PRs related to diagnostics channel labels Nov 6, 2022

nodejs-github-bot added the doc Issues and PRs related to the documentations. label Nov 6, 2022

Trott approved these changes Nov 6, 2022

View reviewed changes

aduh95 reviewed Nov 6, 2022

View reviewed changes

doc/api/async_hooks.md Outdated Show resolved Hide resolved

anonrig approved these changes Nov 6, 2022

View reviewed changes

GeoffreyBooth added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Nov 6, 2022

Trott previously requested changes Nov 6, 2022

View reviewed changes

doc/api/async_hooks.md Outdated Show resolved Hide resolved

Flarna reviewed Nov 6, 2022

View reviewed changes

Trott reviewed Nov 6, 2022

View reviewed changes

doc/api/async_hooks.md Outdated Show resolved Hide resolved

Trott force-pushed the async-hooks-note branch from 279632e to b4d0dbd Compare November 6, 2022 20:31

mcollina requested changes Nov 6, 2022

View reviewed changes

GeoffreyBooth added tsc-agenda Issues and PRs to discuss during the meetings of the TSC. and removed author ready PRs that have at least one approval, no pending requests for changes, and a CI started. labels Nov 6, 2022

Trott force-pushed the async-hooks-note branch from 5f961a2 to 3762422 Compare November 6, 2022 21:08

mcollina approved these changes Nov 16, 2022

View reviewed changes

anonrig approved these changes Nov 16, 2022

View reviewed changes

trevnorris suggested changes Nov 16, 2022

View reviewed changes

doc/api/async_hooks.md Show resolved Hide resolved

doc/api/async_hooks.md Show resolved Hide resolved

doc: add async_hooks migration note

65a6795

GeoffreyBooth force-pushed the async-hooks-note branch from 81b3c38 to 65a6795 Compare November 18, 2022 01:38

Trott changed the title ~~async_hooks: add migration note~~ doc: add async_hooks migration note Nov 18, 2022

Trott approved these changes Nov 18, 2022

View reviewed changes

GeoffreyBooth requested a review from trevnorris November 20, 2022 01:13

GeoffreyBooth added the commit-queue Add this label to land a pull request using GitHub Actions. label Nov 21, 2022

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Nov 21, 2022

nodejs-github-bot closed this Nov 21, 2022

nodejs-github-bot pushed a commit that referenced this pull request Nov 21, 2022

doc: add async_hooks migration note

80dcc06

PR-URL: #45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

GeoffreyBooth deleted the async-hooks-note branch November 21, 2022 00:33

ruyadorno pushed a commit that referenced this pull request Nov 21, 2022

doc: add async_hooks migration note

6f0bc09

PR-URL: #45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

ruyadorno mentioned this pull request Nov 24, 2022

v19.2.0 proposal #45615

Merged

danielleadams pushed a commit that referenced this pull request Dec 30, 2022

doc: add async_hooks migration note

916cb74

PR-URL: #45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

danielleadams mentioned this pull request Dec 30, 2022

v18.13.0 release proposal #46025

Merged

danielleadams pushed a commit that referenced this pull request Dec 30, 2022

doc: add async_hooks migration note

d05071b

PR-URL: #45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

danielleadams pushed a commit that referenced this pull request Jan 3, 2023

doc: add async_hooks migration note

dde8740

PR-URL: #45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

danielleadams pushed a commit that referenced this pull request Jan 4, 2023

doc: add async_hooks migration note

0bbcf4e

PR-URL: #45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

danielleadams pushed a commit that referenced this pull request Jan 5, 2023

doc: add async_hooks migration note

a8a8a87

PR-URL: #45335 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Rich Trott <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

doc: add async_hooks migration note #45335

doc: add async_hooks migration note #45335

GeoffreyBooth commented Nov 6, 2022

Trott commented Nov 6, 2022

Trott left a comment

Qard commented Nov 6, 2022

mcollina commented Nov 6, 2022

mcollina commented Nov 6, 2022

GeoffreyBooth commented Nov 6, 2022

nodejs-github-bot commented Nov 6, 2022

Trott left a comment

Flarna Nov 6, 2022

GeoffreyBooth Nov 6, 2022

Flarna Nov 6, 2022

GeoffreyBooth Nov 6, 2022

Flarna Nov 6, 2022

GeoffreyBooth Nov 6, 2022

Qard Nov 17, 2022

Flarna Nov 6, 2022

GeoffreyBooth Nov 6, 2022

Trott Nov 7, 2022

Qard commented Nov 6, 2022

GeoffreyBooth commented Nov 6, 2022

mcollina left a comment

Flarna commented Nov 6, 2022

GeoffreyBooth commented Nov 6, 2022

GeoffreyBooth commented Nov 6, 2022 •

edited

Loading

Flarna commented Nov 6, 2022

mcollina left a comment

anonrig left a comment

trevnorris commented Nov 16, 2022 •

edited

Loading

GeoffreyBooth commented Nov 16, 2022 •

edited

Loading

Trott left a comment

nodejs-github-bot commented Nov 21, 2022

-> other stable APIs.
+> in the future; it will never become stable. If possible,
+> please use other APIs instead such as [`AsyncLocalStorage`][] or
+> [Diagnostics Channel][].

doc: add async_hooks migration note #45335

doc: add async_hooks migration note #45335

Conversation

GeoffreyBooth commented Nov 6, 2022

Trott commented Nov 6, 2022

Trott left a comment

Choose a reason for hiding this comment

Qard commented Nov 6, 2022

mcollina commented Nov 6, 2022

mcollina commented Nov 6, 2022

GeoffreyBooth commented Nov 6, 2022

nodejs-github-bot commented Nov 6, 2022

Trott left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Qard commented Nov 6, 2022

GeoffreyBooth commented Nov 6, 2022

mcollina left a comment

Choose a reason for hiding this comment

Flarna commented Nov 6, 2022

GeoffreyBooth commented Nov 6, 2022

GeoffreyBooth commented Nov 6, 2022 • edited Loading

Flarna commented Nov 6, 2022

mcollina left a comment

Choose a reason for hiding this comment

anonrig left a comment

Choose a reason for hiding this comment

trevnorris commented Nov 16, 2022 • edited Loading

GeoffreyBooth commented Nov 16, 2022 • edited Loading

Trott left a comment

Choose a reason for hiding this comment

nodejs-github-bot commented Nov 21, 2022

GeoffreyBooth commented Nov 6, 2022 •

edited

Loading

trevnorris commented Nov 16, 2022 •

edited

Loading

GeoffreyBooth commented Nov 16, 2022 •

edited

Loading