Runtime requirements for external actions ignored #538

New Issue

bestlinuxgamers · 2024-04-16T22:11:37Z

bestlinuxgamers commented

2024-04-16 22:11:37 +00:00

I have got the following workflow that runs on ubuntu-22.04, but executes in the container ubuntu:23.10. The Docker image for ubuntu:23.10 doesn't ship node.

name: Test it
on: [push, workflow_dispatch]
permissions:
    contents: write
jobs:
    test:
        name: Test
        runs-on: ubuntu-22.04
        container: ubuntu:23.10
        steps:
            - name: Clone repository
              uses: actions/checkout@v4

When I run this workflow on github actions, it passes without a problem. When I run it on the act_runner with ubuntu-22.04:docker://gitea/runner-images:ubuntu-22.04, it throws the error:

Error: crun: executable file `node` not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found
Error: read unixpacket @->/proc/self/fd/14/attach: read: connection reset by peer

It's pretty clear: actions/checkout needs node to run, but ubuntu:23.10 doesn't provide node.
Apparently github has done something to ensure node support in every container that uses the checkout action. Is there anything I can do for it to work on act_runner too?

It seems like actions contain a file stating its runtime requiremnts (https://github.com/actions/checkout/blob/main/action.yml#L97)

Edit: This is related to upstream https://github.com/nektos/act/issues/732

I have got the following workflow that runs on `ubuntu-22.04`, but executes in the container `ubuntu:23.10`. The Docker image for `ubuntu:23.10` doesn't ship node. ```yaml name: Test it on: [push, workflow_dispatch] permissions: contents: write jobs: test: name: Test runs-on: ubuntu-22.04 container: ubuntu:23.10 steps: - name: Clone repository uses: actions/checkout@v4 ``` When I run this workflow on github actions, it passes without a problem. When I run it on the act_runner with `ubuntu-22.04:docker://gitea/runner-images:ubuntu-22.04`, it throws the error: ``` Error: crun: executable file `node` not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found Error: read unixpacket @->/proc/self/fd/14/attach: read: connection reset by peer ``` It's pretty clear: `actions/checkout` needs node to run, but `ubuntu:23.10` doesn't provide node. Apparently github has done something to ensure node support in every container that uses the checkout action. Is there anything I can do for it to work on act_runner too? It seems like actions contain a file stating its runtime requiremnts (https://github.com/actions/checkout/blob/main/action.yml#L97) Edit: This is related to upstream https://github.com/nektos/act/issues/732

wolfogre commented

2024-04-17 03:00:05 +00:00

GitHub has different execution mechanisms, which may require more work to investigate.

In fact, if you run run: node -v, you will find that Node is not installed in the image, but actions/checkout can still work.

If you do want to use ubuntu:23.10, my suggestion would be to run apt install node in the first step or build an image containing Node based on ubuntu:23.10.

BTW, your example file seems to have incorrect syntax. The correct way to write it should be:

name: Test it
on: [push, workflow_dispatch]
permissions:
    contents: write
jobs:
    test:
        name: Test
        runs-on: ubuntu-22.04
-        container: ubuntu:23.10
+        container:
+          image: ubuntu:23.10
+        steps:
            - name: Clone repository
              uses: actions/checkout@v4

GitHub has different execution mechanisms, which may require more work to investigate. In fact, if you run `run: node -v`, you will find that Node is not installed in the image, but `actions/checkout` can still work. If you do want to use `ubuntu:23.10`, my suggestion would be to run `apt install node` in the first step or build an image containing Node based on `ubuntu:23.10`. BTW, your example file seems to have incorrect syntax. The correct way to write it should be: ```diff name: Test it on: [push, workflow_dispatch] permissions: contents: write jobs: test: name: Test runs-on: ubuntu-22.04 - container: ubuntu:23.10 + container: + image: ubuntu:23.10 + steps: - name: Clone repository uses: actions/checkout@v4 ```

bestlinuxgamers commented

2024-04-18 12:25:00 +00:00

I think that not only for the sake of compatibility with github Actions, it would be great to further investigate how we could implement this. Because being able to run any action in any container is kind of a core requirement of the whole external action feature. Not every container provides a package manager to manually install node or any other runtime requirement.

My guess is that github ships the node package in a volume and executes it directly from that volume's path.

Also, thank you for your correction of my workflow example. I have fixed the error in my original post.

I think that not only for the sake of compatibility with github Actions, it would be great to further investigate how we could implement this. Because being able to run any action in any container is kind of a core requirement of the whole external action feature. Not every container provides a package manager to manually install node or any other runtime requirement. My guess is that github ships the node package in a volume and executes it directly from that volume's path. Also, thank you for your correction of my workflow example. I have fixed the error in my original post.

wolfogre added the

kind

proposal

label 2024-04-19 02:10:09 +00:00

silverwind commented

2024-04-19 08:47:32 +00:00

Actions code have a hard dependency on node 20 and we need to find a way so that actions code will always have node 20 available but user code must not see it (they need to use setup-node).

I think maybe it could be done by first extracting a node 20 tarball somewhere and inject it into PATH for when actions code run. Likely this is a issue in act.

For example a user can currently break actions/checkout by installing a older version of node. This is not happening in the github runner because it "shields" the version of node that actions code uses.

      - uses: actions/setup-node@v4
        with:
          node-version: '8'

Actions code have a hard dependency on node 20 and we need to find a way so that actions code will always have node 20 available but user code must not see it (they need to use `setup-node`). I think maybe it could be done by first extracting a node 20 tarball somewhere and inject it into `PATH` for when actions code run. Likely this is a issue in `act`. For example a user can currently break `actions/checkout` by installing a older version of node. This is not happening in the github runner because it "shields" the version of node that actions code uses. ```yaml - uses: actions/setup-node@v4 with: node-version: '8' ```

👍 1

lunny commented

2024-04-19 11:40:27 +00:00

Include node-20 to the official image? https://gitea.com/gitea/runner-images#default-images

bestlinuxgamers commented

2024-04-20 20:33:14 +00:00

Include node-20 to the official image? https://gitea.com/gitea/runner-images#default-images

I think this is already the case. But even if not, it would not fix the issue, since the usage of any sub-container image that doesn't contain node would not be affected by that.

> Include node-20 to the official image? https://gitea.com/gitea/runner-images#default-images I think this is already the case. But even if not, it would not fix the issue, since the usage of any sub-container image that doesn't contain node would not be affected by that.

wolfogre commented

2024-04-22 06:04:26 +00:00

I am confused by some comments, this issue is not related to the version of node or the official images.

The point is that when using some actions which have runtime requirements like node env, how to:

Inject a node env to the container to exec it if users are using a very simple image without node.
Or refuse to exec it and report "no required runtime" instead of "not found in $PATH".

I don't think it's easy to do it, and I'm not sure if it's worth it, so I tag this issue with proposal.

I am confused by some comments, this issue is not related to the version of node or the official images. The point is that when using some actions which have runtime requirements like node env, how to: - Inject a node env to the container to exec it if users are using a very simple image without node. - Or refuse to exec it and report "no required runtime" instead of "not found in $PATH". I don't think it's easy to do it, and I'm not sure if it's worth it, so I tag this issue with `proposal`.

bestlinuxgamers commented

2024-04-22 21:47:34 +00:00

I did some reverse-engineering on how github provides node.

It seems that github includes a read-only docker volume under /__e/ in every container (even if the runtime is not used by any action).
Each subfolder in this volume represents a runtime.
The following runtimes are currently available: node16, node16_alpine, node20, node20_alpine.

Now in the example of a Node-20 action, the following command is simply executed:
/__e/node20/bin/node /__w/_actions/<organization>/<action-name>/<tag>/<path-to-main>/main.js
Basically, node runs from the environment folder and gets the main class of the action as an argument.

I think that it is much easier than expected to implement something like this.
We would need:

The information about which binary file of a runtime folder is to be executed with which arguments. Maybe as a config entry like "node20/bin/node --example {path-to-action-main}"
The possibility to download/add runtimes (to the volume).
The possibility of limiting runtimes to the operating system (e.g. a linux node runtime wouldn't run on macos).

I did some reverse-engineering on how github provides node. It seems that github includes a read-only docker volume under `/__e/` in every container (even if the runtime is not used by any action). Each subfolder in this volume represents a runtime. The following runtimes are currently available: `node16`, `node16_alpine`, `node20`, `node20_alpine`. Now in the example of a Node-20 action, the following command is simply executed: `/__e/node20/bin/node /__w/_actions/<organization>/<action-name>/<tag>/<path-to-main>/main.js` Basically, node runs from the environment folder and gets the main class of the action as an argument. I think that it is much easier than expected to implement something like this. We would need: - The information about which binary file of a runtime folder is to be executed with which arguments. Maybe as a config entry like "`node20/bin/node --example {path-to-action-main}`" - The possibility to download/add runtimes (to the volume). - The possibility of limiting runtimes to the operating system (e.g. a linux node runtime wouldn't run on macos).

👍 1

bestlinuxgamers commented

2024-04-23 20:00:22 +00:00

I did some additional research and found out that this feature seems to be already implemented by upstream (https://github.com/nektos/act/pull/1988). When I run the workflow from my first post with upstram Act it passes without error.

Edit: Nevermind, only a special case for the checkout action with default arguments was implemented. That's why my action from the first post didn't fail. As soon as any arguments for checout are specified the workflow fails again.

~~I did some additional research and found out that this feature seems to be already implemented by upstream (https://github.com/nektos/act/pull/1988). When I run the workflow from my first post with upstram Act it passes without error.~~ Edit: Nevermind, only a special case for the `checkout` action with default arguments was implemented. That's why my action from the first post didn't fail. As soon as any arguments for checout are specified the workflow fails again.

ChristopherHX commented

2024-04-24 08:59:57 +00:00

I also think like wolfogre doing this increases a lot of complexity, due to binary incompatibilities. For example the non docker mode can currently run node actions on almost anything go supports as long someone preinstalled node in PATH

Doing this with a node binary management you might see a sightly decrease of platform support

Native Platforms

freebsd, can normally don't execute node for linux
macOS
windows
linux and it's different glibc and archs
... the list of my act based runner would be a lot longer, act_runner does decrease platform support already to some degree

Docker Platforms

glibc based (can only run statically linked or glibc node that is old enough)
alpine based (can only run statically linked or musle node)

Because being able to run any action in any container is kind of a core requirement of the whole external action feature

I use actions/runner (This is the program used by GitHub Hosted Runners) with gitea actions, but I'm unable to use many container configurations with node actions

I cannot use node20 actions on...

alpine based container images on anything not amd64
ubuntu:16.04
- glibc too old for node20
via 32bit images of the 64bit System
- host tools are mounted
- no 64bit glibc in image
riscv64 ppc64 images
- externals have wrong arch and dependencies
- no native port
- using qemu needs a self-hosted runner that set it up as binfmt
- you would need to install glibc amd64 dependencies into the container

With the current approuch you could just add node to the image regardless of any asumption needed by the runner

I also think like `wolfogre` doing this increases a lot of complexity, due to binary incompatibilities. For example the non docker mode can currently run node actions on almost anything go supports as long someone preinstalled node in PATH Doing this with a node binary management you might see a sightly decrease of platform support 1. Native Platforms - freebsd, can normally don't execute node for linux - macOS - windows - linux and it's different glibc and archs - ... the list of my act based runner would be a lot longer, act_runner does decrease platform support already to some degree 2. Docker Platforms - glibc based (can only run statically linked or glibc node that is old enough) - alpine based (can only run statically linked or musle node) > Because being able to run any action in any container is kind of a core requirement of the whole external action feature I use actions/runner (This is the program used by GitHub Hosted Runners) with gitea actions, but I'm unable to use many container configurations with node actions I cannot use node20 actions on... - alpine based container images on anything not amd64 - ubuntu:16.04 - glibc too old for node20 - via 32bit images of the 64bit System - host tools are mounted - no 64bit glibc in image - riscv64 ppc64 images - externals have wrong arch and dependencies - no native port - using qemu needs a self-hosted runner that set it up as binfmt - you would need to install glibc amd64 dependencies into the container With the current approuch you could just add node to the image regardless of any asumption needed by the runner

silverwind commented

2024-04-24 20:56:44 +00:00

Maybe we should make it a hard requirement for images that they have node in /__e/node<version>/bin/node available. <version> likely being something like 12,16,20 and then on startup, take the highest available?

That would at least solve the issue of the user being able to downgrade the node version with a setup-node action.

Maybe we should make it a hard requirement for images that they have node in `/__e/node<version>/bin/node` available. `<version>` likely being something like 12,16,20 and then on startup, take the highest available? That would at least solve the issue of the user being able to downgrade the node version with a `setup-node` action.

ChristopherHX commented

2024-04-24 22:01:42 +00:00

For the node downgrade issue:
I could think about grepping the full path of the executeable on job start like node -e "console.log(process.execPath)", then use the absolute path in all later node steps (act runs node actions via a custom code path)

For the node downgrade issue: I could think about grepping the full path of the executeable on job start like `node -e "console.log(process.execPath)"`, then use the absolute path in all later node steps (act runs node actions via a custom code path)

Sign in to join this conversation.

No Milestone

No Assignees

5 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: gitea/act_runner#538