[BUG] Transient ENOENT / file not found in npm ci

[biltongza]

I just got an ENOENT while running npm ci on my project. I reran npm ci and it worked fine. We had this issue on our Azure Devops pipeline for a while, but couldn't reproduce it on our machines so we assumed it was a problem with Azure's hosted agents. We logged it with Azure support but they closed it as a problem with npm, so we wrote a script to just retry npm ci 3 times and hope that it works once out of those 3 times. That has worked fine, but now that I have seen it on my own dev machine, I no longer believe that it is just a problem with Azure's agents.

Log attached:
2020-01-16T09_20_45_728Z-debug.log

I don't know how to reproduce this, but I am happy to provide more details on request, I just don't know what information is needed. I have a feeling this is related to #529, which also looks related to https://npm.community/t/10099

[michaelday008]

I experienced this on AWS and assumed it was their agent, but the issue is transient and sometimes I can run several jobs in a row with no issues.

The error I get is

Installing NPM packages...
--
58 | npm ERR! code ENOENT
59 | npm ERR! syscall open
60 | npm ERR! path /codebuild/output/srcxxxx/src/bitbucket.org/xxx/xxx/node_modules/fsevents/node_modules/needle/package.json
61 | npm ERR! errno -2
62 | npm ERR! enoent ENOENT: no such file or directory, open '/codebuild/output/srcxxx/src/bitbucket.org/xxx/xxx/node_modules/fsevents/node_modules/needle/package.json'
63 | npm ERR! enoent This is related to npm not being able to find a file.
64 | npm ERR! enoent
65 |  
66 | npm ERR! A complete log of this run can be found in:
67 | npm ERR!     /root/.npm/_logs/2020-07-20T20_16_21_621Z-debug.log

Strangely enough, the file is different sometimes.

fsevents/node_modules/npm-bundled/package.json

[rclayton-the-terrible]

I'm experiencing a similar problem. One of my teammates suggested that maybe it's an issue of running out of INodes on Linux?

Edit/Update: Not an inode issue - I created a post-install task to list inodes and only 11% used (unless somehow they were cleaned up by npm after the error).

[biltongza]

We get the same issue on Windows so not sure it's Linux specific.

[akash1810]

We use GitHub Actions for CI and frequently see this issue too using NPMv6 on Ubuntu.

We've recently added the Cache GitHub Action to our workflow which appears to have resolved this issue for us; we've performed over 300 builds with this change now.

The thinking is that it helps with this from the NPM docs:

npm is written to use resources efficiently on install, and part of this is that it tries to do as many things concurrently as is practical. Sometimes this results in race conditions and other synchronization issues.

Using a cache reduces the number of network requests being made and so there are fewer "race conditions and other synchronization issues".

[darcyclarke]

npm v6 is no longer in active development; We will continue to push security releases to v6 at our team's discretion as-per our Support Policy.

If your bug is preproducible on v7, please re-file this issue using our new issue template.

If your issue was a feature request, please consider opening a new RRFC or RFC. If your issue was a question or other idea that was not CLI-specific, consider opening a discussion on our feedback repo

Closing: This is an automated message.

[darcyclarke]

npm v6 is no longer in active development; We will continue to push security releases to v6 at our team's discretion as-per our Support Policy.

If your bug is preproducible on v7, please re-file this issue using our new issue template.

If your issue was a feature request, please consider opening a new RRFC or RFC. If your issue was a question or other idea that was not CLI-specific, consider opening a discussion on our feedback repo

Closing: This is an automated message.

[darcyclarke]

npm v6 is no longer in active development; We will continue to push security releases to v6 at our team's discretion as-per our Support Policy.

If your bug is preproducible on v7, please re-file this issue using our new issue template.

If your issue was a feature request, please consider opening a new RRFC or RFC. If your issue was a question or other idea that was not CLI-specific, consider opening a discussion on our feedback repo

Closing: This is an automated message.