How are we going to import `@node-core/ui-components`? #236

avivkeller · 2025-03-24T14:24:16Z

npm does not support importing packages from subdirectories of git repositories.

This isn't something that the npm CLI team is interested in adding to the tool.

There are third-party solutions like GitPkg that allow this functionality in npm, but I'm not sure we should rely on a third-party tool to handle imports–lest something bad happen.

However, both Yarn (although, yarn requires the website to also use yarn) and pnpm do support this, so we could convert this repository to use one of them instead.

This would also require any project that runs api-docs-tooling use the same package manager (i.e. pnpm dlx instead of npx for execution).

Reference: nodejs/nodejs.org#7401 (comment)

The text was updated successfully, but these errors were encountered:

AugustinMauroy · 2025-03-24T14:39:52Z

Why not publish ui-components ?

avivkeller · 2025-03-24T14:52:38Z

That's an option, but I feel like we are going to be constantly updating that package, so it might be easier to keep it imported from the repository.

AugustinMauroy · 2025-03-24T14:54:32Z

That's an option, but I feel like we are going to be constantly updating that package, so it might be easier to keep it imported from the repository.

With a CD it's just a git tag to create no needed to create a changelog

avivkeller · 2025-03-24T15:30:47Z

Personally, I'm in favor of adopting support for pnpm

"@node-core/ui-components": "github:nodejs/nodejs.org#main&path:packages/ui-components"

However, I think we should get more opinions before making any decision.

flakey5 · 2025-03-24T18:46:58Z

This would also require any project that runs api-docs-tooling use the same package manager (i.e. pnpm dlx instead of npx for execution).

Would that interfere with us using node --run for nodejs/node#57343?

avivkeller · 2025-03-24T18:55:27Z

No, we'd just change the npm installation step to a pnpm installation step, but let's cross that bridge if we get to it.

node --run is package manager ambiguous.

AugustinMauroy · 2025-03-24T20:47:39Z

No, we'd just change the npm installation step to a pnpm installation step, but let's cross that bridge if we get to it.

node --run is package manager ambiguous.

it's mean we will need pnpm in node core ?

avivkeller · 2025-03-24T20:53:16Z

We wouldn't need to depend on it, we would just have to assume the user has it installed, which isn't ideal, but every one of our options has a drawback.

Switching package managers has the drawbacks of requiring Node.js doc-builders and linters to have it installed
Publishing to npm has the drawback of needing to setup a publishing automation, and not control versions by commit
Using a third party tool has the drawback of being a third party tool
We could also require the user to clone the website, and include like a local file, but that has the drawback of requiring the user to clone the website. (Example: https://stackoverflow.com/a/39195334)

ovflowd · 2025-03-24T21:03:47Z

I think we should publish the package, but not necessarily npm. It can be as an GitHub artifact. https://docs.github.com/en/packages/quickstart

avivkeller · 2025-03-24T21:22:43Z

It can be as an GitHub artifact.

That could work. Once the ui-components library lands, I'll make a CI/CD pipeline for uploading it to the GitHub packages registry.

I'm gonna leave this issue open until then, just in case anything changes.

ovflowd · 2025-03-24T22:51:27Z

It can be as an GitHub artifact.

That could work. Once the ui-components library lands, I'll make a CI/CD pipeline for uploading it to the GitHub packages registry.

I'm gonna leave this issue open until then, just in case anything changes.

Are you merging the PR today?

avivkeller · 2025-03-24T22:53:41Z

Yes, I don't think anyone else is going to object in the next hour, so I'm merge it now, but I'm going to add a workflow in a second PR, in case it needs a revert, we don't need to revert the entire UI package

avivkeller · 2025-03-25T17:24:29Z

We need to verify that Node.js can publish to the @node-core Github packages scope, otherwise, we might need to publish to npm. I know we control that scope on npm, but I'm not sure if we control it on Github Packages, I feel like we only control @nodejs (as that's the organization scope).

Edit: We do not control the @node-core scope (See https://github.com/orgs/community/discussions/52267#discussioncomment-5554918).

We have a few options:

Publish to npm, where we do control the @node-core scope
Rename the package to @nodejs/ui-components, and publish to Github Packages
Make an @node-core organization, and publish there (but I don't think that's the best idea)
All of the original options

bjohansebas · 2025-03-25T19:34:45Z

Of those options you gave, the best and simplest one is just renaming the package to nodejs/ui-components. It will only be used here, so I don't think the name change is that bad.

ovflowd · 2025-03-25T19:46:41Z

We need to verify that Node.js can publish to the @node-core Github packages scope, otherwise, we might need to publish to npm. I know we control that scope on npm, but I'm not sure if we control it on Github Packages, I feel like we only control @nodejs (as that's the organization scope).

Edit: We do not control the @node-core scope (See https://github.com/orgs/community/discussions/52267#discussioncomment-5554918).

We have a few options:

Publish to npm, where we do control the @node-core scope

Rename the package to @nodejs/ui-components, and publish to Github Actions

Make an @node-core organization, and publish there (but I don't think that's the best idea)

All of the original options

cc @nodejs/tsc and @nodejs/build looking for guidance here

ruyadorno · 2025-03-26T01:23:58Z

According to npm/npm#2974 (comment):

~~That's a 9 years old, outdated comment.~~

npm does not support importing packages from subdirectories of git repositories.

That's wrong, npm does support installing packages from subdirectories. I was there when we implemented it during the npm7 rewrite, although I acknowledge that it doesn't seem to have been properly documented back then. Here's how you use it:

  "dependencies": {
    "@node-core/ui-components": "nodejs/nodejs.org#main::path:packages/ui-components"
  }

I would suggest one of you to contribute back to the npm docs so that it's easier to you all and the rest of the world to know how to use it in the future. Here are some refs on how to use it from its tests.

ovflowd · 2025-03-26T10:20:24Z

According to npm/npm#2974 (comment):

That's a 9 years old, outdated comment.

npm does not support importing packages from subdirectories of git repositories.

That's wrong, npm does support installing packages from subdirectories. I was there when we implemented it during the npm7 rewrite, although I acknowledge that it doesn't seem to have been properly documented back then. Here's how you use it:
  "dependencies": {
    "@node-core/ui-components": "nodejs/nodejs.org#main::path:packages/ui-components"
  }
I would suggest one of you to contribute back to the npm docs so that it's easier to you all and the rest of the world to know how to use it in the future. Here are some refs on how to use it from its tests.

Thanks for chiming in, Ruy!

avivkeller · 2025-03-26T10:34:22Z

Thanks! Sorry about my outdated information, I'll check and PR the npm documentation to make this more clear.

avivkeller · 2025-05-18T20:09:26Z

@ruyadorno I just attempted to do that, and it appears to import the entire repository:

npm i "nodejs/nodejs.org#path:packages/ui-components"

Produces:

"node_modules/nodejs-website": {
      "resolved": "git+ssh://git@github.com/nodejs/nodejs.org.git#d57d7e42ec79ee060e9000c51082ab0f930e97ce",
      "license": "MIT",
      "dependencies": {
        "husky": "9.1.7",
        "lint-staged": "15.5.1",
        "turbo": "2.5.2"
      },
      "engines": {
        "node": "v22"
      }
    },

ruyadorno · 2025-05-18T20:49:06Z

@ruyadorno I just attempted to do that, and it appears to import the entire repository:
npm i "nodejs/nodejs.org#path:packages/ui-components"

Can you try again using the exact same specifier I provided in my example:

npm i "nodejs/nodejs.org#main::path:packages/ui-components"

avivkeller · 2025-05-18T21:12:58Z

It's the same result.

> npm i "nodejs/nodejs.org#main::path:packages/ui-components"

added 64 packages in 36s

32 packages are looking for funding
  run `npm fund` for details

"node_modules/nodejs-website": {
      "resolved": "git+ssh://git@github.com/nodejs/nodejs.org.git#663fad95788c9e0b398423c86b608f3410c998d8",
      "license": "MIT",
      "dependencies": {
        "husky": "9.1.7",
        "lint-staged": "15.5.1",
        "turbo": "2.5.2"
      },
      "engines": {
        "node": "v22"
      }
    },

avivkeller · 2025-05-19T10:57:05Z

https://openjs-foundation.slack.com/archives/CTEDKHCTB/p1747626798944489?thread_ts=1747599536.481849&channel=CTEDKHCTB&message_ts=1747626798.944489

:-/ Looks like we need to go back to deciding what to do.

MattIPv4 · 2025-05-19T14:27:49Z

👀 If we go down the GitHub Packages path, just want to flag that GitHub Packages requires auth always, even to read. For GitHub Actions consumption specifically, within the same org, you can pass the GITHUB_TOKEN into an .npmrc file and it'll have the right permissions, otherwise you need to generate a legacy personal access token on your account w/ the packages read scope which can then be used to auth w/ npm and stored in .npmrc.

As an example, https://github.com/alveusgg/data#installation + https://github.com/alveusgg/alveusgg/blob/0fb6b2efd001c6861c0c391dc37c311f3270f77e/.github/workflows/pull-request.yml#L23-L24

MattIPv4 · 2025-05-19T14:29:28Z

I do very much like the idea of just being able to do a git subpath import though, and it sounds like pnpm would support this. That'd mean anyone installing this package would also need to use pnpm though to resolve that sub-dependency correctly, unless... we change approach slightly with how api-docs-tooling is distributed and instead ship a bundled script with no sub-dependencies (or even just the ui-components bundled), similar to how Next.js ships itself via npm with a bunch of vendored/bundled packages.

ruyadorno · 2025-05-19T16:13:14Z

Sorry for the confusion folks, I finally had some time to take a look at the example today and it seems that although we did the work to parse git subdirectories from specifiers back in the day, it was never really integrated to the rest of the cli install 😢 💔

ovflowd · 2025-05-19T16:47:25Z

Sorry for the confusion folks, I finally had some time to take a look at the example today and it seems that although we did the work to parse git subdirectories from specifiers back in the day, it was never really integrated to the rest of the cli install 😢 💔

Shoo; Sadgers. Can this get somehow done on npm? Although relying on a new version wouldn't necessarily be a good idea, IDK.

avivkeller · 2025-05-19T20:50:00Z

Honestly, at this point, the simplest solution IMO is to use a package manager that already supports this functionality. While npm might support it in the future, that doesn’t help us during active development right now.

Regardless of whether we decide to bundle this or not (like @MattIPv4 suggested above, and on Slack), pnpm offers the capabilities we need to keep development moving on this repository in the short-term. If something comes up in the long-term (i.e. npm supporting this functionality), we can always switch back.

That said, I think we should hold off until the tooling lands in Node core—we don’t want any changes to this repo to interfere with that process.

ovflowd · 2025-05-20T09:55:35Z

I mean, we're reaching the point we already need node-core/ui-components. Another temporary solution for the time being would be Git Submodules. I'm also fine publishing to GitHub Package Registry, so it is org-specifc and not on NPM registry.

avivkeller · 2025-05-20T11:39:04Z

I'm also fine publishing to GitHub Package Registry, so it is org-specifc and not on NPM registry.

#236 (comment) may make it difficult for contributors to get started if they don't have the proper .npmrc.

Another temporary solution for the time being would be Git Submodules.

That'll work for development, but like you said, it's temporary, and (IIRC) npm install-ing this repository will not download the submodules.

ovflowd · 2025-05-21T12:23:39Z

#236 (comment) may make it difficult for contributors to get started if they don't have the proper .npmrc.

We can include the .npmrc within the repository, no?

ovflowd · 2025-05-21T12:24:20Z

That'll work for development, but like you said, it's temporary, and (IIRC) npm install-ing this repository will not download the submodules.

No, it would require users to initialize the submodule. It could be a preinstall script within the api-docs-tooling repo tbh. But just shooting random ideas here.

avivkeller · 2025-05-21T12:32:51Z

We can include the .npmrc within the repository, no?

No, the configuration file needs to have a GitHub PAT token.

avivkeller · 2025-05-21T12:34:36Z

No, it would require users to initialize the submodule. It could be a preinstall script within the api-docs-tooling repo tbh. But just shooting random ideas here.

I'm working on a postinstall/preinstall script similar to that which'll install the dependencies. Once I finish it (today?), I'll use it in the JSX generator for importing rehype-shiki, and we can see how it works.

ovflowd · 2025-05-21T12:44:31Z

We can include the .npmrc within the repository, no?

No, the configuration file needs to have a GitHub PAT token.

What? But an org-wide registry can be public, doesn't need a token.

avivkeller · 2025-05-21T12:47:47Z

What? But an org-wide registry can be public, doesn't need a token.

https://docs.github.com/en/packages/learn-github-packages/introduction-to-github-packages#authenticating-to-github-packages

You need an access token to publish, install, and delete private, internal, and public packages.

avivkeller · 2025-05-22T12:07:54Z

Currently, in #273, I'm using a package I made for us to import @node-core/rehype-shiki

avivkeller · 2025-05-22T23:11:10Z

We've explored a variety of ideas, but are now at the point where a decision is necessary, since upcoming generators depend on this functionality. Below is a summary of the final options based on previous discussions.

If you are a member of one of the following teams, please vote by reacting:

@nodejs/nodejs-website
@nodejs/web-infra

Only votes from these teams will be counted.

Voting Options

(feel free to vote for multiple options)

👍 Option 1: Stick with npm; publish @node-core packages to a public, unauthenticated registry¹
❤️ Option 2: Stick with npm; use a script/tool to install Git dependencies²
😄 Option 3: Switch to pnpm; bundle output into a single file for import from core

Please cast your vote using the appropriate emoji reaction. Thank you!

Note: The option "Switch to pnpm and require nodejs/node to depend on it" was excluded. It’s unlikely Node.js Core would adopt an external dependency for something that already has a similar internal solution.

GitHub Packages is not viable—see: https://github.com/nodejs/api-docs-tooling/issues/236#issuecomment-2891246006 ↩
I've published a tool for this. ↩

avivkeller mentioned this issue Mar 24, 2025

feat(ui): add ui-components package nodejs/nodejs.org#7401

Merged

avivkeller mentioned this issue Mar 24, 2025

feat(actions): publish @node-core/ui-components nodejs/nodejs.org#7566

Closed

avivkeller closed this as completed Mar 26, 2025

avivkeller mentioned this issue Mar 26, 2025

feat(install): add instructions for git packages npm/documentation#1525

Open

avivkeller reopened this May 18, 2025

avivkeller mentioned this issue May 18, 2025

feat(jsx): add jsx generator #273

Open

avivkeller mentioned this issue May 22, 2025

chore(ui): don't depend on pnpm syntax nodejs/nodejs.org#7771

Merged

How are we going to import @node-core/ui-components? #236

How are we going to import @node-core/ui-components? #236

Comments

avivkeller commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

AugustinMauroy commented Mar 24, 2025

Uh oh!

avivkeller commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AugustinMauroy commented Mar 24, 2025

Uh oh!

avivkeller commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

flakey5 commented Mar 24, 2025

Uh oh!

avivkeller commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AugustinMauroy commented Mar 24, 2025

Uh oh!

avivkeller commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ovflowd commented Mar 24, 2025

Uh oh!

avivkeller commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ovflowd commented Mar 24, 2025

Uh oh!

avivkeller commented Mar 24, 2025

Uh oh!

avivkeller commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bjohansebas commented Mar 25, 2025

Uh oh!

ovflowd commented Mar 25, 2025

Uh oh!

ruyadorno commented Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ovflowd commented Mar 26, 2025

Uh oh!

avivkeller commented Mar 26, 2025

Uh oh!

avivkeller commented May 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ruyadorno commented May 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

avivkeller commented May 18, 2025

Uh oh!

avivkeller commented May 19, 2025

Uh oh!

MattIPv4 commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MattIPv4 commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ruyadorno commented May 19, 2025

Uh oh!

ovflowd commented May 19, 2025

Uh oh!

avivkeller commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ovflowd commented May 20, 2025

Uh oh!

avivkeller commented May 20, 2025

Uh oh!

ovflowd commented May 21, 2025

Uh oh!

ovflowd commented May 21, 2025

Uh oh!

avivkeller commented May 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

How are we going to import `@node-core/ui-components`? #236

How are we going to import `@node-core/ui-components`? #236

avivkeller commented Mar 24, 2025 •

edited

Loading

avivkeller commented Mar 24, 2025 •

edited

Loading

avivkeller commented Mar 24, 2025 •

edited

Loading

avivkeller commented Mar 24, 2025 •

edited

Loading

avivkeller commented Mar 24, 2025 •

edited

Loading

avivkeller commented Mar 24, 2025 •

edited

Loading

avivkeller commented Mar 25, 2025 •

edited

Loading

ruyadorno commented Mar 26, 2025 •

edited

Loading

avivkeller commented May 18, 2025 •

edited

Loading

ruyadorno commented May 18, 2025 •

edited

Loading

MattIPv4 commented May 19, 2025 •

edited

Loading

MattIPv4 commented May 19, 2025 •

edited

Loading

avivkeller commented May 19, 2025 •

edited

Loading

avivkeller commented May 21, 2025 •

edited

Loading