Skip to content

Commit 161797c

Browse files
olebedevolebedev
committed
initial commit
0 parents  commit 161797c

29 files changed

+11280
-0
lines changed

‎.gitignore

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
.DS_Store
2+
node_modules
3+
4+
web*
5+
.npmrc
6+
.vscode

‎CODEOWNERS

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
* @olebedev

‎LICENSE

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2022 Canva
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎OWNERS

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[email protected]
2+
[email protected]

‎README.md

+49
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# js2nix
2+
3+
A tool that makes use of the [Nix] package manager to install [Node.js] dependencies declared in `package.json` and `yarn.lock` files. It is an experimental project to discover opportunities to use Nix for Node.js dependencies. Read more about the background problems [here](./docs/background.md).
4+
5+
### Goals of the project
6+
7+
- Provide incremental `node_modules` installations (don't build what have already been built) by turning every individual npm package into a self-containing Nix derivation
8+
- Handle dependency cycles
9+
- Allow overriding a package
10+
- Allow package life-cycle scripts and their overrides
11+
- Make the dependency graph explicit (expressed in Nix) and well-controlled with all the above
12+
- Remove the need to be check generated files into a code-base, provides IFD if required
13+
- Make the generation of the Nix expression pure, so no assumptions are made around missing SHAs, local packages locations, etc
14+
15+
### Details
16+
17+
It is implemented as a CLI tool written in JavaScript and as a Nix library that picks up that tool and executes it internally to generate a Nix expression out of the given tuple of `package.json` & `yarn.lock` files in a pure manner as a separate Nix derivation, that then can be imported into the Nix runtime and the generated Nix derivations will be built via the provided Nix library to install Node.js dependencies.
18+
19+
Then the artifact can be symlinked to some local location as a `node_modules` folder or can be placed, or picked up by Nix as a part of `NODE_PATH` to make it available for the Node.js resolution mechanism. Also, every derivation that represents an npm package is a first-class citizen in Nix and can be used independently, which is a convenient way to provide Node.js based CLIs in Nix. That is, if the npm package exposes a binary, it will be picked up by Nix and being made available in `PATH`, with no additional effort.
20+
21+
### A quick example
22+
23+
Let's create a file `package.nix` of a Nix expression with the following content:
24+
25+
```nix
26+
with import <nixpkgs> { };
27+
28+
let
29+
js2nix = callPackage (builtins.fetchGit {
30+
url = "ssh://[email protected]/canva-public/js2nix.git";
31+
ref = "main";
32+
}) { };
33+
nodeModules = js2nix.makeNodeModules ./package.json {
34+
tree = js2nix.load ./yarn.lock { };
35+
};
36+
in nodeModules
37+
```
38+
And then a `node_modules` folder can be created via:
39+
40+
```
41+
nix-build --max-jobs auto --out-link ./node_modules ./package.nix
42+
```
43+
44+
The `nodeModules` is a Nix derivation that contains a compatible with [Node.js module resolution algorithm](https://nodejs.org/api/modules.html#all-together) layout. Note that the layout of the resulting `node_modules` is similar to what `PNPN` package manager is providing, that is not a [flat](https://npm.github.io/how-npm-works-docs/npm3/how-npm3-works.html) layout but rather the canonical layout with symlinked (from the Nix store) npm packages into it.
45+
46+
To find out more about the project, its background, implementation details, how to use it please go to the [documentation space](./docs/README.md).
47+
48+
[nix]: https://nixos.org
49+
[node.js]: https://nodejs.org

‎__tests__/__snapshots__/host-deps-cycles.tests.js.snap

+137
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
// Jest Snapshot v1, https://goo.gl/fbAQLP
2+
3+
exports[`Host dependency cycle the "direct" case 1`] = `
4+
"A:
5+
- A+B
6+
7+
B:
8+
- A
9+
10+
A+B:
11+
- A+A
12+
13+
A+A:
14+
"
15+
`;
16+
17+
exports[`Host dependency cycle the "embedded" case 1`] = `
18+
"A:
19+
- B
20+
21+
B:
22+
- B+C
23+
24+
C:
25+
- C+D
26+
27+
D:
28+
- E
29+
30+
E:
31+
- C
32+
- F
33+
34+
F:
35+
- B
36+
37+
C+D:
38+
- C+E
39+
40+
C+E:
41+
- F
42+
- C+C
43+
44+
C+C:
45+
46+
B+C:
47+
- B+D
48+
49+
B+D:
50+
- B+E
51+
52+
B+E:
53+
- B+F
54+
- C+C
55+
56+
B+F:
57+
- B+B
58+
59+
B+B:
60+
"
61+
`;
62+
63+
exports[`Host dependency cycle the "none" case 1`] = `
64+
"A:
65+
- B
66+
67+
B:
68+
- C
69+
70+
C:
71+
- D
72+
73+
D:
74+
"
75+
`;
76+
77+
exports[`Host dependency cycle the "overlap" case 1`] = `
78+
"A:
79+
- B
80+
81+
B:
82+
- C
83+
84+
C:
85+
- C+D
86+
87+
D:
88+
- E
89+
90+
E:
91+
- C
92+
- E+F
93+
94+
F:
95+
- E
96+
97+
C+D:
98+
- C+E
99+
100+
C+E:
101+
- C+F
102+
- C+C
103+
104+
C+C:
105+
106+
C+F:
107+
- C+E
108+
109+
E+F:
110+
- E+E
111+
112+
E+E:
113+
"
114+
`;
115+
116+
exports[`Host dependency cycle the "transitive" case 1`] = `
117+
"A:
118+
- B
119+
120+
B:
121+
- B+C
122+
123+
C:
124+
- D
125+
126+
D:
127+
- B
128+
129+
B+C:
130+
- B+D
131+
132+
B+D:
133+
- B+B
134+
135+
B+B:
136+
"
137+
`;

‎__tests__/__snapshots__/render-tree.js.snap

+127
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
// Jest Snapshot v1, https://goo.gl/fbAQLP
2+
3+
exports[`Render nix expression a single module 1`] = `
4+
"# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
5+
6+
self: super: {
7+
\\"[email protected]\\" = self.buildNodeModule {
8+
id = { scope = \\"\\"; name = \\"A\\"; };
9+
version = \\"1.1.1\\";
10+
src = self.fetchurl {
11+
url = \\"https://registry.yarnpkg.com/A/A.tgz\\";
12+
sha1 = \\"sha1\\";
13+
};
14+
};
15+
}
16+
"
17+
`;
18+
19+
exports[`Render nix expression ambiguous url: no extention 1`] = `
20+
"# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
21+
22+
self: super: {
23+
\\"[email protected]\\" = self.buildNodeModule {
24+
id = { scope = \\"\\"; name = \\"A\\"; };
25+
version = \\"1.1.1\\";
26+
src = self.fetchurl {
27+
name = abort ''
28+
29+
30+
Failed to recognise the type of the archive of the \`[email protected]\` package served from
31+
\`https://registry.yarnpkg.com/A/A\`
32+
33+
Override \`\\"[email protected]\\".src.name\` attribute in order to provide a file name extension
34+
to let Nix recognise the type of the archive and unpack it appropriately. For example:
35+
36+
self: super: {
37+
\\"[email protected]\\" = super.\\"[email protected]\\".override (x: {
38+
src = x.src.override {
39+
# Let Nix to recognise the type of archive so it unpacks it appropriately
40+
name = \\"A.tgz\\";
41+
};
42+
});
43+
}
44+
45+
'';
46+
url = \\"https://registry.yarnpkg.com/A/A\\";
47+
sha1 = \\"sha1\\";
48+
};
49+
};
50+
}
51+
"
52+
`;
53+
54+
exports[`Render nix expression hosted modules 1`] = `
55+
"# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
56+
57+
self: super: {
58+
\\"[email protected]\\" = self.buildNodeModule {
59+
id = { scope = \\"\\"; name = \\"A\\"; };
60+
version = \\"1.1.1\\";
61+
src = self.fetchurl {
62+
url = \\"https://registry.yarnpkg.com/A/A.tgz\\";
63+
sha1 = \\"sha1\\";
64+
};
65+
modules = [
66+
self.\\"[email protected][email protected]\\"
67+
];
68+
};
69+
\\"[email protected][email protected]\\" =
70+
let id = { scope = \\"\\"; name = \\"A\\"; };
71+
in { inherit id; host = id; };
72+
\\"[email protected][email protected]\\" =
73+
self.\\"[email protected]\\".override {
74+
host = { scope = \\"\\"; name = \\"A\\"; };
75+
modules = [
76+
self.\\"[email protected][email protected]\\"
77+
];
78+
};
79+
\\"[email protected]\\" = self.buildNodeModule {
80+
id = { scope = \\"\\"; name = \\"B\\"; };
81+
version = \\"2.2.2\\";
82+
src = self.fetchurl {
83+
url = \\"https://registry.yarnpkg.com/B/B.tgz\\";
84+
sha1 = \\"sha1\\";
85+
};
86+
modules = [
87+
self.\\"[email protected]\\"
88+
];
89+
};
90+
}
91+
"
92+
`;
93+
94+
exports[`Render nix expression missing sha1 1`] = `
95+
"# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
96+
97+
self: super: {
98+
\\"[email protected]\\" = self.buildNodeModule {
99+
id = { scope = \\"\\"; name = \\"A\\"; };
100+
version = \\"1.1.1\\";
101+
src = self.fetchurl {
102+
url = \\"https://registry.yarnpkg.com/A/A.tgz\\";
103+
sha256 = abort ''
104+
105+
106+
Failed to infer \`sha256\` hash of the \`[email protected]\` package source from
107+
\`https://registry.yarnpkg.com/A/A.tgz\`.
108+
109+
Override \`\\"[email protected]\\".src.sha256\` attribute in order to provide this missing piece to Nix.
110+
For example:
111+
112+
self: super: {
113+
\\"[email protected]\\" = super.\\"[email protected]\\".override (x: {
114+
# The sha256 value is obtained via
115+
# nix-prefetch-url https://registry.yarnpkg.com/A/A.tgz
116+
src = x.src.override {
117+
sha256 = \\"<sha256>\\";
118+
};
119+
});
120+
}
121+
122+
'';
123+
};
124+
};
125+
}
126+
"
127+
`;

0 commit comments

Comments
 (0)