gitea/act_runner
62
108
Fork 81
Code Issues 139 Pull Requests 10 Actions Releases 21 Activity

@actions/cache with external cache server connecting wrong IP (0.0.0.0) #437

New Issue
Open
opened 2023-12-05 10:47:35 +00:00 by nex · 5 comments
nex commented 2023-12-05 10:47:35 +00:00
Copy Link

@zetta1325 Hello I'm having trouble using @actions/cache with external cache server, both the runner and the cache server are on a K8s, the step:

- name: Set cache dist
  uses: actions/cache@v3
  with:
    key: ${{ steps.regex-match.outputs.group1 }}-${{ steps.commonVars.outputs.commit_id_8 }}
    restore-keys: ${{ steps.regex-match.outputs.group1 }}
    path: apps/${{ steps.regex-match.outputs.group1 }}/package.json

runner config.yaml

cache:
#   # Enable cache server to use actions/cache.
  enabled: true
#   # The directory to store the cache data.
#   # If it's empty, the cache data will be stored in $HOME/.cache/actcache.
#   dir: ""
#   # The host of the cache server.
#   # It's not for the address to listen, but the address to connect from job containers.
#   # So 0.0.0.0 is a bad choice, leave it empty to detect automatically.
#   host: ""
#   # The port of the cache server.
#   # 0 means to use a random available port.
#   port: 0
#   # The external cache server URL. Valid only when enable is true.
#   # If it's specified, act_runner will use this URL as the ACTIONS_CACHE_URL rather than start a server by itself.
#   # The URL should generally end with "/".
  external_server: http://act-cache-server:7931/

I tried manually attach to the container created by the runner when running jobs and
nslookup act-cache-server correctly returned the cache server's K8s cluster internal IP.
curl http://act-cache-server:7931/ returned 404, so the network did work.

But some how runner is trying to connect to 0.0.0.0:7931 and having error :

⚠️:Failed to restore: downloadCache failed: connect ECONNREFUSED 0.0.0.0:7931

image

@zetta1325 Hello I'm having trouble using `@actions/cache` with external cache server, both the runner and the cache server are on a K8s, the step: ```yml - name: Set cache dist uses: actions/cache@v3 with: key: ${{ steps.regex-match.outputs.group1 }}-${{ steps.commonVars.outputs.commit_id_8 }} restore-keys: ${{ steps.regex-match.outputs.group1 }} path: apps/${{ steps.regex-match.outputs.group1 }}/package.json ``` runner `config.yaml` ```yml cache: # # Enable cache server to use actions/cache. enabled: true # # The directory to store the cache data. # # If it's empty, the cache data will be stored in $HOME/.cache/actcache. # dir: "" # # The host of the cache server. # # It's not for the address to listen, but the address to connect from job containers. # # So 0.0.0.0 is a bad choice, leave it empty to detect automatically. # host: "" # # The port of the cache server. # # 0 means to use a random available port. # port: 0 # # The external cache server URL. Valid only when enable is true. # # If it's specified, act_runner will use this URL as the ACTIONS_CACHE_URL rather than start a server by itself. # # The URL should generally end with "/". external_server: http://act-cache-server:7931/ ``` I tried manually attach to the container created by the runner when running jobs and `nslookup act-cache-server` correctly returned the cache server's K8s cluster internal IP. `curl http://act-cache-server:7931/` returned 404, so the network did work. But some how runner is trying to connect to 0.0.0.0:7931 and having error : ``` ⚠️:Failed to restore: downloadCache failed: connect ECONNREFUSED 0.0.0.0:7931 ``` ![image](/attachments/8fb89387-38be-46bc-96bc-17864189c6b1)
image.png
21 KiB
wolfogre commented 2023-12-06 04:00:06 +00:00
Owner
Copy Link

Could you please try to ping/dig/curl act-cache-server before the Set cache dist step and see what will happen?

Could you please try to ping/dig/curl `act-cache-server` before the `Set cache dist` step and see what will happen?
nex commented 2023-12-06 14:53:35 +00:00
Author
Copy Link

@wolfogre
Yes of course I've tried that.
ping does not work (as expected) because how K8s service works
dig returned the correct K8s cluster internal IP of the act-cache-server service (192.168.x.x)
curl http://act-cache-server:7931 returned 404, so the networking seems working

@wolfogre Yes of course I've tried that. `ping` does not work (as expected) because how K8s service works `dig` returned the correct K8s cluster internal IP of the `act-cache-server` service (192.168.x.x) `curl http://act-cache-server:7931` returned 404, so the networking seems working
Zettat123 commented 2023-12-26 04:01:02 +00:00
Member
Copy Link

Sorry for the late reply. I've tried in my k8s with gitea/act_runner:nightly-dind-rootless and the external cache server works well.
What type of image of act_runner are you using?

Sorry for the late reply. I've tried in my k8s with `gitea/act_runner:nightly-dind-rootless` and the external cache server works well. What type of image of `act_runner` are you using?
Zettat123 added the
kind
bug
reviewed
needs feedback
 labels 2023-12-26 04:01:17 +00:00
pikatenor commented 2024-02-25 13:36:08 +00:00
Copy Link

I ran into the same problem and found that the cache server is returning an incorrect URL as archiveLocation.

cache:
  enabled: true
  external_server: http://gitea-act-runner-cache.default.svc.cluster.local/

$ curl "http://gitea-act-runner-cache.default.svc.cluster.local/_apis/artifactcache/cache?keys=test&version=4bb8d4df808667b3ab8e06d15c4335a6d3b7f67c756feb10f9ecb621591d36e7"

{"archiveLocation":"http://0.0.0.0:80/_apis/artifactcache/artifacts/1","cacheKey":"test","result":"hit"}
logs excerpt
workflow
- name: test cache
  id: cache-test
  uses: actions/cache@v4
  with:
    path: cache-test
    key: test
log
⭐ Run Main test cache
[DEBUG] Writing entry to tarball workflow/outputcmd.txt len:0
[DEBUG] Writing entry to tarball workflow/statecmd.txt len:0
[DEBUG] Writing entry to tarball workflow/pathcmd.txt len:0
[DEBUG] Writing entry to tarball workflow/envs.txt len:0
[DEBUG] Writing entry to tarball workflow/SUMMARY.md len:0
[DEBUG] Extracting content to '/var/run/act'
[DEBUG] About to run action &{Cache GitHub Cache artifacts like dependencies and build outputs to improve workflow execution time map[enableCrossOsArchive:{An optional boolean when enabled, allows windows runners to save or restore caches that can be restored or saved respectively on other platforms false false} fail-on-cache-miss:{Fail the workflow if cache entry is not found false false} key:{An explicit key for restoring and saving the cache true } lookup-only:{Check if a cache entry exists for the given input(s) (key, restore-keys) without downloading the cache false false} path:{A list of files, directories, and wildcard patterns to cache and restore true } restore-keys:{An ordered list of keys to use for restoring stale cache if no cache hit occurred for key. Note `cache-hit` returns false in this case. false } save-always:{Run the post step to save the cache even if another step before fails false false} upload-chunk-size:{The chunk size used to split up large files during upload, in bytes false }] map[cache-hit:{A boolean value to indicate an exact match was found for the primary key }] {node20 map[] dist/restore/index.js  always() dist/save/index.js success() || github.event.inputs.save-always   [] []} {gray-dark archive}}
[DEBUG] type=remote-action actionDir=/home/rootless/.cache/act/actions-cache@v4 actionPath= workdir=/workspace/pikatenor/actions-test actionCacheDir=/home/rootless/.cache/act actionName=actions-cache@v4 containerActionDir=/var/run/act/actions/actions-cache@v4
[DEBUG] Removing /home/rootless/.cache/act/actions-cache@v4/.gitignore before docker cp
[DEBUG] /var/run/act/actions/actions-cache@v4
  🐳  docker cp src=/home/rootless/.cache/act/actions-cache@v4/ dst=/var/run/act/actions/actions-cache@v4/
[DEBUG] Writing tarball /tmp/act702315702 from /home/rootless/.cache/act/actions-cache@v4/
[DEBUG] Stripping prefix:/home/rootless/.cache/act/actions-cache@v4/ src:/home/rootless/.cache/act/actions-cache@v4/
[DEBUG] Extracting content from '/tmp/act702315702' to '/var/run/act/actions/actions-cache@v4/'
[DEBUG] executing remote job container: [node /var/run/act/actions/actions-cache@v4/dist/restore/index.js]
  🐳  docker exec cmd=[node /var/run/act/actions/actions-cache@v4/dist/restore/index.js] user= workdir=
[DEBUG] Exec command '[node /var/run/act/actions/actions-cache@v4/dist/restore/index.js]'
[DEBUG] Working directory '/workspace/pikatenor/actions-test'
  💬  ::debug::Resolved Keys:
  | ::debug::Resolved Keys:
  💬  ::debug::["test"]
  | ::debug::["test"]
  💬  ::debug::Checking zstd --quiet --version
  | ::debug::Checking zstd --quiet --version
  💬  ::debug::*** zstd command line interface 64-bits v1.4.4, by Yann Collet ***
  | ::debug::*** zstd command line interface 64-bits v1.4.4, by Yann Collet ***
  💬  ::debug::zstd version: null
  | ::debug::zstd version: null
  💬  ::debug::Resource Url: http://gitea-act-runner-cache.default.svc.cluster.local/_apis/artifactcache/cache?keys=test&version=4bb8d4df808667b3ab8e06d15c4335a6d3b7f67c756feb10f9ecb621591d36e7
  | ::debug::Resource Url: http://gitea-act-runner-cache.default.svc.cluster.local/_apis/artifactcache/cache?keys=test&version=4bb8d4df808667b3ab8e06d15c4335a6d3b7f67c756feb10f9ecb621591d36e7
  ⚙  ***
  | ::add-mask::***
  💬  ::debug::Cache Result:
  | ::debug::Cache Result:
  💬  ::debug::{"archiveLocation":"***","cacheKey":"test","result":"hit"}
  | ::debug::{"archiveLocation":"***","cacheKey":"test","result":"hit"}
  💬  ::debug::Archive Path: /tmp/ef4917b0-1c98-46bf-ab23-5d1a9bf11d1a/cache.tzst
  | ::debug::Archive Path: /tmp/ef4917b0-1c98-46bf-ab23-5d1a9bf11d1a/cache.tzst
  💬  ::debug::Use Azure SDK: false
  | ::debug::Use Azure SDK: false
  💬  ::debug::Download concurrency: 8
  | ::debug::Download concurrency: 8
  💬  ::debug::Request timeout (ms): 30000
  | ::debug::Request timeout (ms): 30000
  💬  ::debug::Cache segment download timeout mins env var: undefined
  | ::debug::Cache segment download timeout mins env var: undefined
  💬  ::debug::Segment download timeout (ms): 600000
  | ::debug::Segment download timeout (ms): 600000
  💬  ::debug::Lookup only: false
  | ::debug::Lookup only: false
  💬  ::debug::downloadCache - Attempt 1 of 2 failed with error: connect ECONNREFUSED 0.0.0.0:80
  | ::debug::downloadCache - Attempt 1 of 2 failed with error: connect ECONNREFUSED 0.0.0.0:80
  💬  ::debug::downloadCache - Attempt 2 of 2 failed with error: connect ECONNREFUSED 0.0.0.0:80
  | ::debug::downloadCache - Attempt 2 of 2 failed with error: connect ECONNREFUSED 0.0.0.0:80
  🚧  ::warning::Failed to restore: downloadCache failed: connect ECONNREFUSED 0.0.0.0:80
  | ::warning::Failed to restore: downloadCache failed: connect ECONNREFUSED 0.0.0.0:80
  | Cache not found for input keys: test
  ✅  Success - Main test cache

I wanted to deploy the cache server as a separate Deployment/Service from the runner, so I had set 0.0.0.0 as the host for the cache server.

cache:
  enabled: true
  host: "0.0.0.0"
  port: 80

(full Deployment/ConfigMap is here)

I think the problem is due to a limitation that the current act does not allow the ExternalURL to be set separately from the address that the cache server listens to.

pkg/artifactcache/handler.go Lines 113 to 118 in e6fec7324e
func (h *Handler) ExternalURL() string {
// TODO: make the external url configurable if necessary
return fmt.Sprintf("http://%s:%d",
h.outboundIP,
h.listener.Addr().(*net.TCPAddr).Port)
}

I would like to make the ExternalURL used for archiveLocation configurable for flexibility of deployment.

Is it better to send a PR to nextos/act or gitea-forked act? (this feature seems unnecessary when running act locally)

I ran into the same problem and found that the cache server is returning an incorrect URL as `archiveLocation`. ```yaml cache: enabled: true external_server: http://gitea-act-runner-cache.default.svc.cluster.local/ ``` ``` $ curl "http://gitea-act-runner-cache.default.svc.cluster.local/_apis/artifactcache/cache?keys=test&version=4bb8d4df808667b3ab8e06d15c4335a6d3b7f67c756feb10f9ecb621591d36e7" {"archiveLocation":"http://0.0.0.0:80/_apis/artifactcache/artifacts/1","cacheKey":"test","result":"hit"} ``` <details> <summary>logs excerpt</summary> ###### workflow ```yaml - name: test cache id: cache-test uses: actions/cache@v4 with: path: cache-test key: test ``` ###### log ``` ⭐ Run Main test cache [DEBUG] Writing entry to tarball workflow/outputcmd.txt len:0 [DEBUG] Writing entry to tarball workflow/statecmd.txt len:0 [DEBUG] Writing entry to tarball workflow/pathcmd.txt len:0 [DEBUG] Writing entry to tarball workflow/envs.txt len:0 [DEBUG] Writing entry to tarball workflow/SUMMARY.md len:0 [DEBUG] Extracting content to '/var/run/act' [DEBUG] About to run action &{Cache GitHub Cache artifacts like dependencies and build outputs to improve workflow execution time map[enableCrossOsArchive:{An optional boolean when enabled, allows windows runners to save or restore caches that can be restored or saved respectively on other platforms false false} fail-on-cache-miss:{Fail the workflow if cache entry is not found false false} key:{An explicit key for restoring and saving the cache true } lookup-only:{Check if a cache entry exists for the given input(s) (key, restore-keys) without downloading the cache false false} path:{A list of files, directories, and wildcard patterns to cache and restore true } restore-keys:{An ordered list of keys to use for restoring stale cache if no cache hit occurred for key. Note `cache-hit` returns false in this case. false } save-always:{Run the post step to save the cache even if another step before fails false false} upload-chunk-size:{The chunk size used to split up large files during upload, in bytes false }] map[cache-hit:{A boolean value to indicate an exact match was found for the primary key }] {node20 map[] dist/restore/index.js always() dist/save/index.js success() || github.event.inputs.save-always [] []} {gray-dark archive}} [DEBUG] type=remote-action actionDir=/home/rootless/.cache/act/actions-cache@v4 actionPath= workdir=/workspace/pikatenor/actions-test actionCacheDir=/home/rootless/.cache/act actionName=actions-cache@v4 containerActionDir=/var/run/act/actions/actions-cache@v4 [DEBUG] Removing /home/rootless/.cache/act/actions-cache@v4/.gitignore before docker cp [DEBUG] /var/run/act/actions/actions-cache@v4 🐳 docker cp src=/home/rootless/.cache/act/actions-cache@v4/ dst=/var/run/act/actions/actions-cache@v4/ [DEBUG] Writing tarball /tmp/act702315702 from /home/rootless/.cache/act/actions-cache@v4/ [DEBUG] Stripping prefix:/home/rootless/.cache/act/actions-cache@v4/ src:/home/rootless/.cache/act/actions-cache@v4/ [DEBUG] Extracting content from '/tmp/act702315702' to '/var/run/act/actions/actions-cache@v4/' [DEBUG] executing remote job container: [node /var/run/act/actions/actions-cache@v4/dist/restore/index.js] 🐳 docker exec cmd=[node /var/run/act/actions/actions-cache@v4/dist/restore/index.js] user= workdir= [DEBUG] Exec command '[node /var/run/act/actions/actions-cache@v4/dist/restore/index.js]' [DEBUG] Working directory '/workspace/pikatenor/actions-test' 💬 ::debug::Resolved Keys: | ::debug::Resolved Keys: 💬 ::debug::["test"] | ::debug::["test"] 💬 ::debug::Checking zstd --quiet --version | ::debug::Checking zstd --quiet --version 💬 ::debug::*** zstd command line interface 64-bits v1.4.4, by Yann Collet *** | ::debug::*** zstd command line interface 64-bits v1.4.4, by Yann Collet *** 💬 ::debug::zstd version: null | ::debug::zstd version: null 💬 ::debug::Resource Url: http://gitea-act-runner-cache.default.svc.cluster.local/_apis/artifactcache/cache?keys=test&version=4bb8d4df808667b3ab8e06d15c4335a6d3b7f67c756feb10f9ecb621591d36e7 | ::debug::Resource Url: http://gitea-act-runner-cache.default.svc.cluster.local/_apis/artifactcache/cache?keys=test&version=4bb8d4df808667b3ab8e06d15c4335a6d3b7f67c756feb10f9ecb621591d36e7 ⚙ *** | ::add-mask::*** 💬 ::debug::Cache Result: | ::debug::Cache Result: 💬 ::debug::{"archiveLocation":"***","cacheKey":"test","result":"hit"} | ::debug::{"archiveLocation":"***","cacheKey":"test","result":"hit"} 💬 ::debug::Archive Path: /tmp/ef4917b0-1c98-46bf-ab23-5d1a9bf11d1a/cache.tzst | ::debug::Archive Path: /tmp/ef4917b0-1c98-46bf-ab23-5d1a9bf11d1a/cache.tzst 💬 ::debug::Use Azure SDK: false | ::debug::Use Azure SDK: false 💬 ::debug::Download concurrency: 8 | ::debug::Download concurrency: 8 💬 ::debug::Request timeout (ms): 30000 | ::debug::Request timeout (ms): 30000 💬 ::debug::Cache segment download timeout mins env var: undefined | ::debug::Cache segment download timeout mins env var: undefined 💬 ::debug::Segment download timeout (ms): 600000 | ::debug::Segment download timeout (ms): 600000 💬 ::debug::Lookup only: false | ::debug::Lookup only: false 💬 ::debug::downloadCache - Attempt 1 of 2 failed with error: connect ECONNREFUSED 0.0.0.0:80 | ::debug::downloadCache - Attempt 1 of 2 failed with error: connect ECONNREFUSED 0.0.0.0:80 💬 ::debug::downloadCache - Attempt 2 of 2 failed with error: connect ECONNREFUSED 0.0.0.0:80 | ::debug::downloadCache - Attempt 2 of 2 failed with error: connect ECONNREFUSED 0.0.0.0:80 🚧 ::warning::Failed to restore: downloadCache failed: connect ECONNREFUSED 0.0.0.0:80 | ::warning::Failed to restore: downloadCache failed: connect ECONNREFUSED 0.0.0.0:80 | Cache not found for input keys: test ✅ Success - Main test cache ``` </details> I wanted to deploy the cache server as a separate Deployment/Service from the runner, so I had set `0.0.0.0` as the host for the cache server. ```yaml cache: enabled: true host: "0.0.0.0" port: 80 ``` (full Deployment/ConfigMap is [here](https://github.com/pikatenor/infra/blob/d9bfa214c304f1c583d1d0b3b678764774b3bdb2/dream0/default/gitea-runner-cache.yml)) I think the problem is due to a limitation that the current act does not allow the ExternalURL to be set separately from the address that the cache server listens to. https://gitea.com/gitea/act/src/commit/e6fec7324e5f34bce7f5071d2cd96a60a2af94f7/pkg/artifactcache/handler.go#L113-L118 I would like to make the `ExternalURL` used for `archiveLocation` configurable for flexibility of deployment. Is it better to send a PR to nextos/act or gitea-forked act? (this feature seems unnecessary when running act locally)
❤️ 1
Zettat123 commented 2024-03-06 06:50:39 +00:00
Member
Copy Link

@pikatenor Thank you for digging into this issue

Is it better to send a PR to nextos/act or gitea-forked act? (this feature seems unnecessary when running act locally)

I think it would be better to send a PR to gitea/act since accessing the cache server from outside is not necessary for act

@pikatenor Thank you for digging into this issue > Is it better to send a PR to nextos/act or gitea-forked act? (this feature seems unnecessary when running act locally) I think it would be better to send a PR to [gitea/act](https://gitea.com/gitea/act) since accessing the cache server from outside is not necessary for `act`
Zettat123 removed the
reviewed
needs feedback
 label 2024-03-06 06:51:06 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
renovate/github.com-nektos-act-0.x
renovate/alpine-3.x
renovate/actions-setup-go-5.x
renovate/golang.org-x-term-0.x
v0.2.10
v0.2.9
v0.2.8
v0.2.7
v0.2.6
v0.2.5
v0.2.4
v0.2.3
v0.2.2
v0.2.1
v0.2.0
v0.1.8
v0.1.7
v0.1.6
v0.1.5
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
v0.0.1
Labels
Clear labels   
kind
bug
Something is not working

  
kind
build

  
kind/compatible
  
kind
dependencies

  
kind
docs

  
kind
enhancement
New feature

  
kind
feature

  
kind
help wanted
Need some help

  
kind
proposal

  
kind
refactor

  
related
act
Related to act.

  
related
environment
Related to the environment.

  
related
exec

  
related
gitea
Related to Gitea.

  
related
workflow
Related to the workflow file or actions scripts.

  
reviewed
confirmed
confirmed issue

  
reviewed
duplicate
This issue or pull request already exists

  
reviewed
invalid
Something is wrong

  
reviewed
needs feedback

  
reviewed
wontfix
This won't be fixed

  
reviewed
workaround

No Label
kind
bug
kind
build
kind/compatible
kind
dependencies
kind
docs
kind
enhancement
kind
feature
kind
help wanted
kind
proposal
kind
refactor
related
act
related
environment
related
exec
related
gitea
related
workflow
reviewed
confirmed
reviewed
duplicate
reviewed
invalid
reviewed
needs feedback
reviewed
wontfix
reviewed
workaround
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: gitea/act_runner#437
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?