liferay-npm-bundler does not support monorepo with Yarn workspaces with hoisted dependencies. Fails with ENOENT

[DannyMeister]

Issue type (mark with x)

• 🤔 Question
• 🐛 Bug report
• 🎁 Feature request
• 🤷‍♀️ Other

Version (mark with x)

• 2️⃣ v2.x
• 3️⃣ v3.x

Description

I'm not sure if this will be considered a bug or feature request. I cannot get the Liferay jar bundler to work for my React widget packages within a monorepo using Yarn workspaces.

Desired behavior:

When using Yarn workspaces, I would like the liferay-npm-bundler to support the hoisted node modules when trying to build my .jar files. If node_modules are hoisted to the root project, then the bundler commands should be able to still resolve their location correctly. Even better if it has full support for PnP.

Current behavior:

When upgrading from Yarn 1 to 3 (or 2) I tried the following sequence of configurations without success:

• Plug'n'Play (node_modules is not present, dependencies are zipped into root project's .yarn/cache). Result: lnbs-build fails with ENOENT error.
• yarn unplug on the liferay-npm-bundler[-*] packages so that they are unpacked to .yarn/unplugged. Result: lnbs-build fails with ENOENT error.
• Disable PnP by using the old node-modules linker (yarn config set nodeLinker node-modules). This uses a node_modules folder at the root project, but node_modules is not present in each workspace. Result: lnbs-build fails with ENOENT error.

The only way I have been able to successfully build my .jar is by disabling hoisting via nmHoistingLimits: "workspaces" in yarnrc.yml. This is a disappointing limitation which undermines much of the usefulness of Yarn workspaces.

In all of the failing configuraitons, the error is similar to the following, (which is specifically comes from the third variation). liferay-with-workspaces is the root project, while my-widget is the workspace containing the React project I wish to bundle for Liferay. Note that it looks for the non-existent node_modules directory under my-widget.

The system cannot find the path specified.
C:\git\liferay-with-workspaces\node_modules\liferay-npm-build-support\lib\util.js:60
        throw proc.error;
        ^

Error: spawnSync C:\git\liferay-with-workspaces\packages\my-widget\node_modules\.bin\liferay-npm-bundler ENOENT
    at notFoundError (C:\git\liferay-with-workspaces\node_modules\cross-spawn\lib\enoent.js:6:26)
    at Object.verifyENOENTSync (C:\git\liferay-with-workspaces\node_modules\cross-spawn\lib\enoent.js:48:16)
    at Function.spawnSync [as sync] (C:\git\liferay-with-workspaces\node_modules\cross-spawn\index.js:29:43)
    at Object.runNodeModulesBin (C:\git\liferay-with-workspaces\node_modules\liferay-npm-build-support\lib\util.js:56:40)
    at buildWith (C:\git\liferay-with-workspaces\node_modules\liferay-npm-build-support\lib\scripts\build\index.js:70:16)
    at Object.default_1 [as default] (C:\git\liferay-with-workspaces\node_modules\liferay-npm-build-support\lib\scripts\build\index.js:38:13)
    at Object.<anonymous> (C:\git\liferay-with-workspaces\node_modules\liferay-npm-build-support\bin\lnbs-build.js:8:40)
    at Module._compile (node:internal/modules/cjs/loader:1126:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1180:10)
    at Module.load (node:internal/modules/cjs/loader:1004:32) {
  code: 'ENOENT',
  errno: 'ENOENT',
  syscall: 'spawnSync C:\\git\\liferay-with-workspaces\\packages\\my-widget\\node_modules\\.bin\\liferay-npm-bundler',
  path: 'C:\\git\\liferay-with-workspaces\\packages\\my-widget\\node_modules\\.bin\\liferay-npm-bundler',
  spawnargs: []
}

Code within liferay-npm-build-support\lib\util.js's runNodeModulesBin function assumes that there will be a node_modules folder under the project being built:

    const proc = cross_spawn_1.default.sync(project_1.default.dir.join('node_modules', '.bin', script).asNative, args, {
        stdio: 'inherit',
    });

Yarn overrides the behavior of require and require.resolve (even to read out of zip files in the case of PnP) and so I think this can be fixed by using require.resolve for finding the needed script rather than the path combining logic that assumes things about the directory structure. I kind of hacked together something that showed it could work for my case within util.js on my local machine, but lack the time and experience for a PR that could take care of all the usages of runNodeModulesBin in all the various environments, package managers, and conditions under which it may be used.

Repro instructions (if applicable):

I have a minimal reproduction of the above error at https://github.com/DannyMeister/liferay-with-workspaces for the third variation of attempted configuration (hoisted non-PnP)

yarn install
yarn workspace my-widget build:liferay

Disclaimer: When trying to create a small repo to reproduce this, I might have left out things necessary to make it all fully work... for example, not sure if the gradle stuff we had in our large repo is necessary. Forgive me if that's the case, but you can ignore any further errors due to that if you are able to resolve the issues related to the bundler scripts trying to execute out of node_modules at the workspace level.

Other information (environment, versions etc):

Windows 10
Yarn 3.4.1
liferay-npm-build-support: 2.31.2
liferay-npm-bundler: 2.31.2
liferay-npm-bundler-preset-create-react-app: 2.31.2