Transitive dependencies

[robatwilliams]

Apply the marshalls not only to the package being installed, but also to the entire tree that will be installed.

Expected Behavior

Perform the checks on everything that will be installed, e.g. not only @vite/plugin-react, but also the 70+ transitive dependencies that the installation would cause.

Current Behavior

Currently only checks the package directly specified for installation.

Possible Solution

Walk the tree to perform the checks. Tree may often be large, so some thought on presenting the information to the user would be required.

Context

Large and popular projects are more likely to have mature security measures in place, but may still pull in many smaller projects which are more vulnerable. I would be more worried about what the marshalls say about those smaller ones.

[lirantal]

Indeed. The big problem here is that npq aims to be a pre-installation check and the actual tree for the resolved dependencies depends on both the package.json file and the package lock file as well as other .npmrc settings and so to avoid becoming a package manager and calculating the tree, it would've been ideal if we could hook into npm's lifecycle process but I'm not aware of any way to do that. Do you have any suggestions?

One way is to leverage arborist ourselves (it's the underlying engine that resolves package dependencies) and use it to calculate the after-install tree but I haven't experimented with this beyond entertaining the thought.

[robatwilliams]

npm install foo --dry-run produces some output, however it's not guaranteed to be parseable (unlike some other commands there are no --parseable or --json flags). yarn add does have a --json flag but doesn't (in 4.2.2) produce any output when used, and it doesn't have a dry run flag.

$ npm i react --dry-run --parseable
add fsevents 2.3.3
add @rollup/rollup-win32-ia32-msvc 4.18.0
add @rollup/rollup-win32-arm64-msvc 4.18.0
add @rollup/rollup-linux-x64-musl 4.18.0
add @rollup/rollup-linux-x64-gnu 4.18.0
add @rollup/rollup-linux-s390x-gnu 4.18.0
add @rollup/rollup-linux-riscv64-gnu 4.18.0
add @rollup/rollup-linux-powerpc64le-gnu 4.18.0
add @rollup/rollup-linux-arm64-musl 4.18.0
add @rollup/rollup-linux-arm64-gnu 4.18.0
add @rollup/rollup-linux-arm-musleabihf 4.18.0
add @rollup/rollup-linux-arm-gnueabihf 4.18.0
add @rollup/rollup-darwin-x64 4.18.0
add @rollup/rollup-darwin-arm64 4.18.0
add @rollup/rollup-android-arm64 4.18.0
add @rollup/rollup-android-arm-eabi 4.18.0
add @esbuild/win32-ia32 0.21.5
add @esbuild/win32-arm64 0.21.5
add @esbuild/sunos-x64 0.21.5
add @esbuild/openbsd-x64 0.21.5
add @esbuild/netbsd-x64 0.21.5
add @esbuild/linux-x64 0.21.5
add @esbuild/linux-s390x 0.21.5
add @esbuild/linux-riscv64 0.21.5
add @esbuild/linux-ppc64 0.21.5
add @esbuild/linux-mips64el 0.21.5
add @esbuild/linux-loong64 0.21.5
add @esbuild/linux-ia32 0.21.5
add @esbuild/linux-arm64 0.21.5
add @esbuild/linux-arm 0.21.5
add @esbuild/freebsd-x64 0.21.5
add @esbuild/freebsd-arm64 0.21.5
add @esbuild/darwin-x64 0.21.5
add @esbuild/darwin-arm64 0.21.5
add @esbuild/android-x64 0.21.5
add @esbuild/android-arm64 0.21.5
add @esbuild/android-arm 0.21.5
add @esbuild/aix-ppc64 0.21.5

added 38 packages in 787ms

18 packages are looking for funding
  run `npm fund` for details

Large and popular projects are more likely to have mature security measures in place, but may still pull in many smaller projects which are more vulnerable. I would be more worried about what the marshalls say about those smaller ones.

To add hard numbers from a real project: 43 direct, 1270 total.

[lirantal]

Ahh nice, I didn't realize npm has a --parseable flag, although that doesn't seem like something actually structured 😆 but yeah, good enough to work with by pattern matching.

so if I get it right that command output you shared includes all direct and indirect dependencies?

Here's an arborist example which seems to work good and doesn't require spinning up npm itself in an install mode:

const { Arborist } = require("@npmcli/arborist");
const path = require("path");

async function calculateNewDependencies(projectPath, newPackage) {
  const arb = new Arborist({
    path: projectPath,
  });

  try {
    const tree = await arb.loadActual();
    const idealTree = await arb.buildIdealTree({
      add: [newPackage],
    });

    console.log("New dependency tree:");
    idealTree.inventory.values().forEach((node) => {
      if (node.isProjectRoot) {
        console.log(`${node.name} (root)`);
      } else {
        console.log(`${node.name}@${node.version}`);
      }
    });
  } catch (err) {
    console.error(err);
  }
}

const projectPath = path.resolve("~/liran-experiments/a");
const newPackage = "lodash@4.17.21";

calculateNewDependencies(projectPath, newPackage);

So we can run it with a way to diff before/after states to figure out the new packages being added and then query them for the usual npq checks.

p.s. regarding package manager support, I'm fine with supporting npm only if there's no similar API on yarn/pnpm.

[lirantal]

Another thought here: it is worth re-thinking about the entire UX for npq if we are going to show transitive deps stats because the current checks and output is not going to scale well. Similarly, many transitive deps could show errors like no readme, no license, etc which could be noisy.

For context, here's a reference output from npq:

So besides the actual feature support, I think we'd need to re-think how to bubble up information from nested dependencies.

[robatwilliams]

Good, that covers the basic technical feasibility then.

Absolutely about scalability, I was drafting such a comment now. The UI presentation and also the number of API calls (not sure what they all are) could be multiplied by 100+. I expect some generated HTML view would be necessary, similar to how testing tools present a code coverage report.

[lirantal]

Right. The basics is JSON which is easy, from there we can output HTML and any other reports. The tricky part would be CLI output in interactive mode though an how to make it easily understood.

[robatwilliams]

The options I can think of for interactive CLI would be:

1. Use something more advanced than inquirer to allow browsing/expanding/collapsing data
2. Provide follow-up separate commands to query subsections of the information (approach similar to Git)
3. Simply print it all, possibly after a "Would you like to display detected possible issues in 18 of 70 transitive dependencies?"

I'd be happy with no. 3.

[uiii]

You can display a table with checks and crosses for each check type as a column

[lirantal]

There's a POC here: #391
However, not sure if there's value and how to capture it from nested deps vs the direct dependency that gets installed.

If someone wants to share some practical suggestions, I'm happy to re-visit.