Provide "docker run" options for the containers launched by the runner #79

New Issue

ghnp5 · 2023-03-25T16:15:43Z

ghnp5 commented

2023-03-25 16:15:43 +00:00

Hey,

Not sure if this is already possible,

But I'm finding it very needed to be able to provide extra options for the containers that the runner launches, so I can, for example, add volumes to it.

-v /path/to/dir/:/path/to/dir

If this is already possible, please let me know!

I understand this is something that wouldn't be possible on GitHub Actions, but since we're self-hosting Gitea on our servers, having some extra flexibility on this would be very great!

Thank you very much!!

Hey, Not sure if this is already possible, But I'm finding it very needed to be able to provide extra options for the containers that the runner launches, so I can, for example, add volumes to it. `-v /path/to/dir/:/path/to/dir` If this is already possible, please let me know! I understand this is something that wouldn't be possible on GitHub Actions, but since we're self-hosting Gitea on our servers, having some extra flexibility on this would be very great! Thank you very much!!

👍 2

wolfogre added the

kind

proposal

label 2023-03-26 07:54:11 +00:00

ChristopherHX commented

2023-04-04 12:08:15 +00:00

Due to missing container-options flag / config you currently have to add them into your workflow file

See second example with container: https://github.com/nektos/act/issues/1696#issuecomment-1483385747

on: push
jobs:
  _:
    runs-on: ubuntu-latest
    container:
      image: ubuntu:latest # Please use another image, which have node (nodejs) installed and in the PATH env var
      options: --network bridge # --container-options are no longer active
    steps:
    - run: sleep 10000

Not tested with act_runner, it works in act

Due to missing container-options flag / config you currently have to add them into your workflow file See second example with `container:` https://github.com/nektos/act/issues/1696#issuecomment-1483385747 ```yaml on: push jobs: _: runs-on: ubuntu-latest container: image: ubuntu:latest # Please use another image, which have node (nodejs) installed and in the PATH env var options: --network bridge # --container-options are no longer active steps: - run: sleep 10000 ``` _Not tested with act_runner, it works in act_

ghnp5 commented

2023-04-04 15:33:22 +00:00

Thanks. I didn't know this!

That should be enough for me, if it works!!

I'm giving it a try.

I see I cannot use env.XYZ like this:

    container:
      image: docker:latest
      options: -v ${{ env.ROOT_PATH }}:${{ env.ROOT_PATH }}

Which is OK - we can just hardcode the values.

However:

name: Test Job

on:
  push:
    branches:
      - test

env:
  ROOT_PATH: /root-path/

jobs:
  b_test1:
    name: Test 1
    runs-on: ubuntu-latest
    container:
      image: docker:latest
      options: -v /root-path/:/root-path/ -v /var/run/act/workflow/:/var/run/act/workflow/
    outputs:
      output1: ${{ steps.step1.outputs.test }}
    steps:
      - id: step1
        name: Test 1 Step 1
        run: |
          echo "User: $(whoami)"
          echo "Shell: $SHELL"
          echo $BASH_VERSION
          ls -l $(which sh)
          echo ""
          echo "GITHUB_OUTPUT = $GITHUB_OUTPUT"
          echo "GITHUB_STEP_SUMMARY = $GITHUB_STEP_SUMMARY"
          echo ""
          ls -l ${{ env.ROOT_PATH }}
          echo "test=hello" >> "$GITHUB_OUTPUT"
  b_test2:
    name: Test 2
    runs-on: ubuntu-latest
    needs: b_test1
    container:
      image: docker:latest
      options: -v /root-path/:/root-path/ -v /var/run/act/workflow/:/var/run/act/workflow/
    steps:
      - name: Test 2 Step 1
        env:
          OUTPUT1: ${{ needs.b_test1.outputs.output1 }}
        run: |
          echo "$OUTPUT1"
          echo ""
          ls -l ${{ env.ROOT_PATH }}

I was expecting echo "$OUTPUT1" (the 3rd line from the bottom) to print hello, but it doesn't.

Now - a few notes:

I run Gitea in Docker, with docker.sock mounted from the host
I understand -v /var/run/act/workflow/:/var/run/act/workflow/ mounts the host's /var/run/act/workflow/ folder, so this is not a valid test.
I saw the folder was created in the host, and contains all the Step files.

I deleted that folder.
I also tried without -v /var/run/act/workflow/:/var/run/act/workflow/.
I see the folder wasn't created in the host, but hello isn't printed either.
Not sure if the act_runner tries to add the volume itself, using the real folder or so.

The thing is that since both Gitea and act_runner run on Docker, I'm not exactly sure how could act_runner make this work for $GITHUB_OUTPUT.

Thanks. I didn't know this! That should be enough for me, if it works!! --- I'm giving it a try. I see I cannot use `env.XYZ` like this: ``` container: image: docker:latest options: -v ${{ env.ROOT_PATH }}:${{ env.ROOT_PATH }} ``` Which is OK - we can just hardcode the values. --- However: ``` name: Test Job on: push: branches: - test env: ROOT_PATH: /root-path/ jobs: b_test1: name: Test 1 runs-on: ubuntu-latest container: image: docker:latest options: -v /root-path/:/root-path/ -v /var/run/act/workflow/:/var/run/act/workflow/ outputs: output1: ${{ steps.step1.outputs.test }} steps: - id: step1 name: Test 1 Step 1 run: | echo "User: $(whoami)" echo "Shell: $SHELL" echo $BASH_VERSION ls -l $(which sh) echo "" echo "GITHUB_OUTPUT = $GITHUB_OUTPUT" echo "GITHUB_STEP_SUMMARY = $GITHUB_STEP_SUMMARY" echo "" ls -l ${{ env.ROOT_PATH }} echo "test=hello" >> "$GITHUB_OUTPUT" b_test2: name: Test 2 runs-on: ubuntu-latest needs: b_test1 container: image: docker:latest options: -v /root-path/:/root-path/ -v /var/run/act/workflow/:/var/run/act/workflow/ steps: - name: Test 2 Step 1 env: OUTPUT1: ${{ needs.b_test1.outputs.output1 }} run: | echo "$OUTPUT1" echo "" ls -l ${{ env.ROOT_PATH }} ``` I was expecting `echo "$OUTPUT1"` (the 3rd line from the bottom) to print `hello`, but it doesn't. Now - a few notes: 1) I run Gitea in Docker, with `docker.sock` mounted from the host 2) I understand `-v /var/run/act/workflow/:/var/run/act/workflow/` mounts the host's `/var/run/act/workflow/` folder, so this is not a valid test. I saw the folder was created in the host, and contains all the Step files. I deleted that folder. 3) I also tried without `-v /var/run/act/workflow/:/var/run/act/workflow/`. I see the folder wasn't created in the host, but `hello` isn't printed either. Not sure if the `act_runner` tries to add the volume itself, using the real folder or so. The thing is that since both Gitea and `act_runner` run on Docker, I'm not exactly sure how could `act_runner` make this work for `$GITHUB_OUTPUT`.

ChristopherHX commented

2023-04-04 15:59:08 +00:00

I see needs in your example, however that is not implemented (yet) in Gitea Actions and needs protocol changes..., the ones who implemented this in first place forget that (needs works in nektos/act)

Please read my list of defects:

https://github.com/go-gitea/gitea/issues/13539#issuecomment-1448888850
My issue about beeing unable to send job outputs gitea/actions-proto-def#4 to Gitea Actions

I'm still using my own GitHub Actions Simulator for gitea, which I posted in the initial gitea actions issue before this became reality

I see `needs` in your example, however that is not implemented (**yet**) in Gitea Actions and needs protocol changes..., the ones who implemented this in first place forget that (needs works in nektos/act) Please read my list of defects: - https://github.com/go-gitea/gitea/issues/13539#issuecomment-1448888850 - My issue about beeing unable to send job outputs https://gitea.com/gitea/actions-proto-def/issues/4 to Gitea Actions _I'm still using my own GitHub Actions Simulator for gitea, which I posted in the initial gitea actions issue before this became reality_

wolfogre commented

2023-04-06 01:49:37 +00:00

To be clear, needs works in Gitea Actions, what is not implemented is output.

To be clear, `needs` works in Gitea Actions, what is not implemented is `output`.

wolfogre referenced this issue

2023-04-06 02:18:19 +00:00

Support `outputs` #101

ghnp5 commented

2023-04-06 06:52:23 +00:00

Actually, not in the version I have right now. There have been a number of fixes since, also from issues I raised.

I was waiting for a new version of Gitea, but I've just realized that act_runner can be updated separately (of course, since I've downloaded a separate binary for it :-) !)

I'll look into upgrading it now with the latest version.

Actually, not in the version I have right now. There have been a number of fixes since, also from issues I raised. I was waiting for a new version of Gitea, but I've just realized that `act_runner` can be updated separately (of course, since I've downloaded a separate binary for it :-) !) I'll look into upgrading it now with the latest version.

ghnp5 commented

2023-04-06 06:53:14 +00:00

Although, some fixes go on Gitea, like - https://github.com/go-gitea/gitea/pull/23789

ChristopherHX commented

2023-04-06 07:37:28 +00:00

needs works in Gitea Actions, what is not implemented

I was refering to the needs context ${{ needs.<id>.result }}, because it has been empty.

> needs works in Gitea Actions, what is not implemented I was refering to the `needs` context `${{ needs.<id>.result }}`, because it has been empty.

👍 1

seepine commented

2023-04-13 02:27:35 +00:00

like this ?

jobs:
  b_test1:
    name: Test 1
    runs-on: ubuntu-latest
    container:
      image: docker:latest
      volumes: 
        - /root-path/:/root-path/
        - /var/run/act/workflow/:/var/run/act/workflow/

like this ? ``` jobs: b_test1: name: Test 1 runs-on: ubuntu-latest container: image: docker:latest volumes: - /root-path/:/root-path/ - /var/run/act/workflow/:/var/run/act/workflow/ ```

ghnp5 commented

2023-04-13 07:44:41 +00:00

Hey @seepine

I was thinking just something as simple as:

options: -v /root-path/:/root-path/ -v /var/run/act/workflow/:/var/run/act/workflow/

which seems to exist already, but there appears to be some caveats/limitations at the moment, based on the comments above.

I'd say we'll give it a bit of time while Gitea devs continue to implement what's missing, and make improvements, and then we can come back to this, to see what we can do to improve this scenario :-)

Hey @seepine I was thinking just something as simple as: `options: -v /root-path/:/root-path/ -v /var/run/act/workflow/:/var/run/act/workflow/` which seems to exist already, but there appears to be some caveats/limitations at the moment, based on the comments above. I'd say we'll give it a bit of time while Gitea devs continue to implement what's missing, and make improvements, and then we can come back to this, to see what we can do to improve this scenario :-)

👍 1

ednxzu commented

2023-07-29 18:26:59 +00:00

I see that both outputs and needs have been implemented, is needs.job.outputs.output supposed to be working on latest ? It doesn't seem to on my side.

nicorac commented

2023-07-31 07:19:33 +00:00

Sometimes you just want to inject the same options to all started containers (e.g. to set proxy settings), so it's better to "force" them through act_runner instead of repeating the same configuration over and over in each repository workflow.

I've built my own act_runner image with a simple patch to let container.options be injected in all started containers.

See #265

Sometimes you just want to inject the same options to all started containers (e.g. to set proxy settings), so it's better to "force" them through `act_runner` instead of repeating the same configuration over and over in each repository workflow. I've built my own `act_runner` image with a simple patch to let `container.options` be injected in all started containers. See [#265](https://gitea.com/gitea/act_runner/issues/265#issuecomment-744382)

nicorac referenced this issue from gitea/act

2023-10-17 06:53:35 +00:00

Patched options() to let container options propagate to job containers #80

nodiscc commented

2023-10-19 21:41:02 +00:00

Sometimes you just want to inject the same options to all started containers

In this case you may add them to options: in the act-runner configuration file. This is what I do [1] and it appears to work without problem.

Unless I misunderstand something, this issue looks like it is already fixed.

> Sometimes you just want to inject the same options to all started containers In this case you may add them to `options:` in the act-runner configuration file. This is what I do [[1]](https://github.com/nodiscc/xsrv/blob/master/roles/gitea_act_runner/templates/etc_act-runner_config.yaml.j2#L61) and it appears to work without problem. Unless I misunderstand something, this issue looks like it is already fixed.

nicorac commented

2023-10-20 06:34:11 +00:00

this issue looks like it is already fixed

No, it's not. options are applied only to default container.

If you use a custom container to run your jobs (and it happens most of the times, i.e. if you need Node.js, Python or other environments not available in default container), then those options are not applied.

You actually need to specify them within the job definition (which is exactly what I want to avoid) and pollute all of your repos.

Actually:

when using the default container, it gets the options defined in runner config
when using a custom container, defined in job config, it takes only the options defined in workflow, while (IMHO) it should take both

> this issue looks like it is already fixed No, it's not. `options` are applied only to **default container**. If you use a custom container to run your jobs (and it happens most of the times, i.e. if you need Node.js, Python or other environments not available in default container), then those `options` are not applied. You actually need to specify them within the job definition (which is exactly what I want to avoid) and pollute all of your repos. Actually: - when using the default container, it gets the `options` defined in runner config - when using a custom container, defined in job config, it takes **only** the options defined in workflow, while (IMHO) it should take **both**

👍 1

Sign in to join this conversation.