yarn.lock should not include base registry(http://registry.npmjs.org)

[doxiaodong]

Do you want to request a feature or report a bug?

feature
What is the current behavior?
yarn.lock include registry
If the current behavior is a bug, please provide the steps to reproduce.

What is the expected behavior?

[email protected]:
  version "4.17.4"
  resolved "http://registry.npm.taobao.org/lodash/download/lodash-4.17.4.tgz#78203a4d1c328ae1d86dca6460e369b57f4055ae"

should be

...
resolved "/lodash/download/lodash-4.17.4.tgz#78203a4d1c328ae1d86dca6460e369b57f4055ae"
...

Reason
Most developers in china will use --registry=https://registry.npm.taobao.org , and then, yarn.lock will be

...
resolved "http://registry.npm.taobao.org/lodash/download/lodash-4.17.4.tgz#78203a4d1c328ae1d86dca6460e369b57f4055ae"
...

But for travis-ci, circleci, it seem to be slowly.
So, I think the regiistry should not be in yarn.lock

[bestander]

Yeah, that sounds like a useful feature.
@arcanis what do you think?

[arcanis]

Yup, completely agree. Ideally, I think the resolved field should be removed (because we don't care where the tarball come from, as long as it has the right hash), and replaced by a single hash field. The url can then be reconstructed at runtime when reading the lockfile.

[bestander]

@doxiaodong want to send an RFC?

We'll need to discuss backwards compatibility, this feature being an opt-in, defaults etc

[doxiaodong]

@bestander ， any template here? I don't know what and how to do that.

[bestander]

Sure, here are a couple of good examples https://github.com/yarnpkg/rfcs/pull/40/files https://github.com/yarnpkg/rfcs/pull/34/files

[doxiaodong]

I have sent a pr yarnpkg/rfcs#64

[csvan]

+1 for this. In our case (which I think is shared by many) we have one registry in our local dev environment, and another in our CI environment. Currently, tracking the lock file with git causes it to contain references to the dev registry which in turn leads to builds in CI failing. The only solution we have found so far is to not track the lock file at all, which incurs a performance hit during builds.

[yardik]

+1 for this. We use a local registry and would like it to ignore the one in the yarn.lock in favor of our local.

[el-davo]

+1 for this. In our situation we are using jenkins and sinopia in a kubernetes environment. In the kubernetes network sinopia has a different url to the external network url. It would be handy if we could switch between the 2 easily as it might be faster to use the internal kubernetes network url, while allowing developers to use the external registry url as they wont have access to the internal url

[vernak2539]

This would be nice to have. We have a mirror used outside our Build Environment that has one URL. Inside the Build Environment it has a different URL, and the original registry used to generate the lock file is not available.

We've noticed that passing the --registry flag doesn't overwrite from where the packages are fetched. Would be nice to have the lock file be registry agnostic

[bestander]

Suggest a simple backwards compatible RFC and send a PR, we'll get it sorted then.
Discussion in yarnpkg/rfcs#64 got derailed and suggests bigger changes than the team is eager to take.

[vernak2539]

@bestander I wrote one here. Not sure if it's enough, but it's a start

[aikar]

I would say remove concepts of urls/paths all together, and only list the name, version and hash.

It's easy to rebuild the url at download time instead, and opens potential for alternate url formats too.

[aikar]

@bestander when you say backwards compat RFC, do you mean newer yarn can still load older yarnrc, or that older yarns should still be able to load the newer yarnrc?

[bestander]

@aikar, conceptually I agree.
Although a major change in yarn.lock would need a detailed RFC with consideration of how breaking the change will be.

When I talk about backwards compatibility I mean that yarn.lock generated by a previous version of Yarn won't be compatible by the next version that strips the URLs and vice versa.
We'll need to think through how teams with people using multiple versions of Yarn can work.
It is doable but requires a responsible approach.

So far I am fine to settle on a minimal change to unblock people using npm mirrors.

[Maxxxz]

Met this question today ToT

[ashwinr]

This has sadly become a blocker for us. Npm works correctly using an integrity field in its lock file (in addition to a resolved field similar to yarn). Here's a solution that also preserves yarn backcompat: Introduce a new field called hash that's a sibling of resolved. Latest yarn would add this field wherever it's missing for each package entry in the lock file, when yarn install is run. When resolving a package, if hash is missing in any entry in the lock file, yarn falls back to the resolved field, which will keep backcompat with old lock files. Thoughts?

[bestander]

Here is the RFC we have agreed on, it should solve the title issue and is waiting for a champion to implement it: https://github.com/yarnpkg/rfcs/pull/84/files.

It is quite easy to implement it so please go ahead and send a PR.

As for talks about integrity fields we have one here: #816

[LinkMJB]

Wow, that was impeccable timing 🤣

https://status.yarnpkg.com/incident/14/

[arcanis]

Opened a new issue at #5892 to put this feature request inside the v2 high level goals 🙂

[BYK]

@arcanis shall we close this one in favor of #5892?

[lcw0622]

So this feature still under development?
It is really important for deploy in production environment when company has own internal registry.

[linonetwo]

Deploying website to now.sh always fail, and I can not even use YARN_REGISTRY="https://registry.yarnpkg.com" to override yarn.lock!

[coppyC]

you are right

[arcanis]

Closing this issue; we already have #5892.

This will be part of the v2, but probably not the v1.

[icyerasor]

What about at least logging that there are inconsistencies between the configured registry and the cached URLs in the lock?
Would probably save lots of dev hrs.

[kopax-polyconseil]

1. The checksum of the yarn lock change if you edit it, that's one problem
2. The git repo not being up to date after editing the yarn lock, a second problem.
3. A workaround would be to allow to specifiy the lock file then create a 2nd one with the first one using the other repo, that's not possible
4. A setting stronger than --repository that could take precedence over the lock file, also not possible.

Any plan to implement a solution within yarn after all those years ?

[arcanis]

Any plan to implement a solution within yarn after all those years ?

It got fixed about three years and two major versions ago.

[hmatfarafan]

Any plan to implement a solution within yarn after all those years ?

It got fixed about three years and two major versions ago.

My current problem is that it seems all official node docker images include version 1.x.x of yarn and it is a ban of our CI processes to use latest versions of yarn.

[chadlwilson]

it is a ban of our CI processes to use latest versions of yarn.

Well, no-one can help you with this. You either need to stop using Yarn, or workaround your policy to upgrade.

When used with corepack, making your build use Yarn 2+ with external node docker images is pretty trivial.

Yarn 2+ can essentially be treated as a build-time dependency like any other (not something pre-installed) and can be source controlled/locked alongside your code etc so doesn't really "affect" your CI environment except if you are doing something special to mount the Yarn cache for re-use between builds. In other words, Yarn 2+ does need to be, and is canonically never pre-installed on something like a container image. Rational explained on the linked docs.

[hmatfarafan]

@chadlwilson
Ok,
But if I want to be more clear, we use yarn.lock to be sure about keeping dependencies of the project intact. And then to speed up CI process, we prefer to set a hosted local registry and it cannot be done in yarn 1.x.
You say we can add Yarn 2+ as a build-time dependency. It means every CI run leads to run the process of Yarn 2+ installation again, and it is time consuming and makes the CI process slow down.

May keeping Yarn 1.x and downloading artifacts from original registry every time be simpler :-) .

[chadlwilson]

That’s now how it works or has to work (read the migration guide and/or my comment more closely around source controlling yarn itself) but this isn’t the right place for that discussion.