'npm install -d' causes permission issues when installing typegen

[morgante]

When trying to install the kpt function SDK (npm install -d) I get the following error:

npm info lifecycle [email protected]~postinstall: [email protected]

> [email protected] postinstall /generator/node_modules/create-kpt-functions
> go-npm install

Downloading from URL: https://storage.googleapis.com/kpt-functions/v0.16.4/typegen_darwin_amd64.tar.gz
internal/fs/utils.js:230
    throw err;
    ^

Error: EACCES: permission denied, rename 'bin/typegen' -> '/usr/local/bin/typegen'
    at Object.renameSync (fs.js:676:3)
    at /generator/node_modules/go-npm/bin/index.js:64:12
    at /generator/node_modules/go-npm/bin/index.js:53:9
    at ChildProcess.exithandler (child_process.js:294:7)
    at ChildProcess.emit (events.js:321:20)
    at maybeClose (internal/child_process.js:1028:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5) {
  errno: -13,
  syscall: 'rename',
  code: 'EACCES',
  path: 'bin/typegen',
  dest: '/usr/local/bin/typegen'

My package.json is virtually identical to the package.json in the examples: https://github.com/GoogleContainerTools/kpt-functions-sdk/blob/master/ts/demo-functions/package.json

[frankfarzan]

I repro'd this locally. For me the permission failure happens on when specifying -d flag to install. @morgante Can you verify removing -d flags eliminates the error?

It's interesting since the -d flag just enables verbose logging and should be innocuous.

In the happy bath the typegen binary in installed in this location:

./node_modules/create-kpt-functions/node_modules/.bin/typegen

and not in:

/usr/local/bin/typegen

[morgante]

Yes, removing -d eliminates the error for me too though that's very surprising.

Notably, the Docker build also seems to fail though.

[frankfarzan]

How's the docker command failing?

[morgante]

It fails on the npm ci step:

Sending build context to Docker daemon  959.5kB
Step 1/14 : FROM node:10-alpine as builder
 ---> a07f309c9753
Step 2/14 : RUN mkdir -p /home/node/app &&     chown -R node:node /home/node/app
 ---> Using cache
 ---> 6f5839478e84
Step 3/14 : USER node
 ---> Using cache
 ---> f571ae2d4482
Step 4/14 : WORKDIR /home/node/app
 ---> Using cache
 ---> 4c84293956e6
Step 5/14 : COPY --chown=node:node package*.json ./
 ---> Using cache
 ---> f744c24b1f7e
Step 6/14 : RUN npm ci
 ---> Running in 7b99026b13e9

> [email protected] postinstall /home/node/app/node_modules/create-kpt-functions
> go-npm install

Downloading from URL: https://storage.googleapis.com/kpt-functions/v0.16.4/typegen_linux_amd64.tar.gz

> [email protected] prepare /home/node/app
> npm run build


> [email protected] build /home/node/app
> tsc

Version 3.8.3
Syntax:   tsc [options] [file...]

Examples: tsc hello.ts
          tsc --outFile file.js file.ts
          tsc @args.txt
          tsc --build tsconfig.json

Options:
 -h, --help                                         Print this message.
 -w, --watch                                        Watch input files.
 --pretty                                           Stylize errors and messages using color and context (experimental).
 --all                                              Show all compiler options.
 -v, --version                                      Print the compiler's version.
 --init                                             Initializes a TypeScript project and creates a tsconfig.json file.
 -p FILE OR DIRECTORY, --project FILE OR DIRECTORY  Compile the project given the path to its configuration file, or to a folder with a 'tsconfig.json'.
 -b, --build                                        Build one or more projects and their dependencies, if out of date
 -t VERSION, --target VERSION                       Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019', 'ES2020', or 'ESNEXT'.
 -m KIND, --module KIND                             Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', 'es2020', or 'ESNext'.
 --lib                                              Specify library files to be included in the compilation.
                                                      'es5' 'es6' 'es2015' 'es7' 'es2016' 'es2017' 'es2018' 'es2019' 'es2020' 'esnext' 'dom' 'dom.iterable' 'webworker' 'webworker.importscripts' 'scripthost' 'es2015.core' 'es2015.collection' 'es2015.generator' 'es2015.iterable' 'es2015.promise' 'es2015.proxy' 'es2015.reflect' 'es2015.symbol' 'es2015.symbol.wellknown' 'es2016.array.include' 'es2017.object' 'es2017.sharedmemory' 'es2017.string' 'es2017.intl' 'es2017.typedarrays' 'es2018.asyncgenerator' 'es2018.asynciterable' 'es2018.intl' 'es2018.promise' 'es2018.regexp' 'es2019.array' 'es2019.object' 'es2019.string' 'es2019.symbol' 'es2020.bigint' 'es2020.promise' 'es2020.string' 'es2020.symbol.wellknown' 'esnext.array' 'esnext.symbol' 'esnext.asynciterable' 'esnext.intl' 'esnext.bigint'
 --allowJs                                          Allow javascript files to be compiled.
 --jsx KIND                                         Specify JSX code generation: 'preserve', 'react-native', or 'react'.
 -d, --declaration                                  Generates corresponding '.d.ts' file.
 --declarationMap                                   Generates a sourcemap for each corresponding '.d.ts' file.
 --sourceMap                                        Generates corresponding '.map' file.
 --outFile FILE                                     Concatenate and emit output to single file.
 --outDir DIRECTORY                                 Redirect output structure to the directory.
 --removeComments                                   Do not emit comments to output.
 --noEmit                                           Do not emit outputs.
 --strict                                           Enable all strict type-checking options.
 --noImplicitAny                                    Raise error on expressions and declarations with an implied 'any' type.
 --strictNullChecks                                 Enable strict null checks.
 --strictFunctionTypes                              Enable strict checking of function types.
 --strictBindCallApply                              Enable strict 'bind', 'call', and 'apply' methods on functions.
 --strictPropertyInitialization                     Enable strict checking of property initialization in classes.
 --noImplicitThis                                   Raise error on 'this' expressions with an implied 'any' type.
 --alwaysStrict                                     Parse in strict mode and emit "use strict" for each source file.
 --noUnusedLocals                                   Report errors on unused locals.
 --noUnusedParameters                               Report errors on unused parameters.
 --noImplicitReturns                                Report error when not all code paths in function return a value.
 --noFallthroughCasesInSwitch                       Report errors for fallthrough cases in switch statement.
 --types                                            Type declaration files to be included in compilation.
 --esModuleInterop                                  Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'.
 @<file>                                            Insert command line options and files from a file.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build: `tsc`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/node/.npm/_cacache/_logs/2020-05-28T23_23_44_412Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] prepare: `npm run build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] prepare script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/node/.npm/_logs/2020-05-28T23_23_44_430Z-debug.log
The command '/bin/sh -c npm ci' returned a non-zero code: 1
Error: Failed to build docker image
    at /Users/morgantep/code/google/pso/mp/mp-data-automation/generator/node_modules/create-kpt-functions/dist/cmd/docker_build.js:32:19
    at Object.processDockerfile (/Users/morgantep/code/google/pso/mp/mp-data-automation/generator/node_modules/create-kpt-functions/dist/cmd/docker_create.js:66:9)
    at Object.dockerBuild (/Users/morgantep/code/google/pso/mp/mp-data-automation/generator/node_modules/create-kpt-functions/dist/cmd/docker_build.js:23:21)
    at /Users/morgantep/code/google/pso/mp/mp-data-automation/generator/node_modules/create-kpt-functions/dist/cli.js:112:32
    at Generator.next (<anonymous>)
    at /Users/morgantep/code/google/pso/mp/mp-data-automation/generator/node_modules/create-kpt-functions/dist/cli.js:24:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/morgantep/code/google/pso/mp/mp-data-automation/generator/node_modules/create-kpt-functions/dist/cli.js:20:12)
    at main (/Users/morgantep/code/google/pso/mp/mp-data-automation/generator/node_modules/create-kpt-functions/dist/cli.js:37:12)
    at /Users/morgantep/code/google/pso/mp/mp-data-automation/generator/node_modules/create-kpt-functions/dist/cli.js:124:15
    at Generator.next (<anonymous>)
    at /Users/morgantep/code/google/pso/mp/mp-data-automation/generator/node_modules/create-kpt-functions/dist/cli.js:24:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/morgantep/code/google/pso/mp/mp-data-automation/generator/node_modules/create-kpt-functions/dist/cli.js:20:12)
    at /Users/morgantep/code/google/pso/mp/mp-data-automation/generator/node_modules/create-kpt-functions/dist/cli.js:122:8
    at Object.<anonymous> (/Users/morgantep/code/google/pso/mp/mp-data-automation/generator/node_modules/create-kpt-functions/dist/cli.js:130:4)
    at Module._compile (internal/modules/cjs/loader.js:1144:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1164:10)
    at Module.load (internal/modules/cjs/loader.js:993:32)
    at Function.Module._load (internal/modules/cjs/loader.js:892:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    at internal/main/run_main_module.js:17:47
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] kpt:docker-build: `kpt docker-build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] kpt:docker-build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/morgantep/.npm/_logs/2020-05-28T23_23_44_709Z-debug.log

[frankfarzan]

@morgante Can you file a separate issue for the docker build failure? Error messages shows typescript compiler failing, but doesn't provide details. If you can give me access to your function, I'll look into it.

[frankfarzan]

Let's make the following change to circumvent this problem all together:

Currently we use go-npm package [1] to download the typegen binary when the CLI package is installed [2]. Instead we can download typegen in type_create.ts[3]. We're already downloading the swagger file here anyways. We can then control exactly where the typegen binary is installed and avoid any NPM resolution/permission shenanigans.

[1] https://www.npmjs.com/package/go-npm
[2] https://github.com/GoogleContainerTools/kpt-functions-sdk/blob/f3df445fcff10448039584fc03cfff943a677e76/ts/create-kpt-functions/package.json#L70
[3] https://github.com/GoogleContainerTools/kpt-functions-sdk/blob/master/ts/create-kpt-functions/src/cmd/type_create.ts