Unable to switch gitea.config.server.PROTOCOL back to http from https without deleting pod #344

New Issue

kindrobot · 2022-08-10T15:33:55Z

kindrobot commented

2022-08-10 15:33:55 +00:00

Summary

I use Gitea on k8s behind a Caddy reverse proxy which does SSL termination. I was trying to upgrade my Gitea helm deployment so that the public URL it showed was https (e.g. for cloning). I mistakenly changed gitea.config.server.PROTOCOL to https which caused gitea to fail to start because I didn't have a certificate set up. However, after switching PROTOCOL back to http and upgrading, gitea still tried to start HTTPS and crashed. I ended up having to delete the pod and upgrade to get it to "take" the http value for PROTOCOL.

Steps to reproduce

run helm3 install gitea-issue gitea-charts/gitea --values values.yaml with the following values.yml

global:
  storageClass: "nfs-csi"

replicaCount: 1

ingress:
  enabled: true
  hosts:
    - host: git-issue.kindrobot.ca
      paths:
        - path: /
          pathType: Prefix
  tls: []

gitea:
  admin:
    username: AzureDiamond
    password: hunter2
    email: "me@example.com"
  config:
    service:
      DISABLE_REGISTRATION: true
`

visit gitea instance; observe that it's running

run helm3 upgrade gitea-issue gitea-charts/gitea --values values.yaml with the following values

global:
  storageClass: "nfs-csi"

replicaCount: 1

ingress:
  enabled: true
  hosts:
    - host: git-issue.kindrobot.ca
      paths:
        - path: /
          pathType: Prefix
  tls: []

gitea:
  admin:
    username: AzureDiamond
    password: hunter2
    email: "me@example.com"
  config:
    server:
      PROTOCOL: https
    service:
      DISABLE_REGISTRATION: true
`

visit gitea instance; observe 503 nginx error (expected as SSL cert has not been configured)

run kubectl logs gitea-issue-0 observe

2022/08/10 15:20:39 cmd/web.go:208:listen() [I] Listen: https://0.0.0.0:3000
2022/08/10 15:20:39 cmd/web.go:212:listen() [I] AppURL(ROOT_URL): https://git-issue.kindrobot.ca/
2022/08/10 15:20:39 cmd/web_https.go:171:runHTTPS() [E] Failed to load https cert file  for tcp:0.0.0.0:3000: open : no such file or directory
2022/08/10 15:20:39 cmd/web.go:258:listen() [C] Failed to start server: open : no such file or directory
2022/08/10 15:20:39 cmd/web.go:260:listen() [I] HTTP Listener: 0.0.0.0:3000 Closed

run helm3 upgrade gitea-issue gitea-charts/gitea --values values.yaml with the following values

global:
  storageClass: "nfs-csi"

replicaCount: 1

ingress:
  enabled: true
  hosts:
    - host: git-issue.kindrobot.ca
      paths:
        - path: /
          pathType: Prefix
  tls: []

gitea:
  admin:
    username: AzureDiamond
    password: hunter2
    email: "me@example.com"
  config:
    server:
      PROTOCOL: http
    service:
      DISABLE_REGISTRATION: true
`

visit gitea instance; observe 503 nginx error (unexpected, because PROTOCOL was explicitly set back to http)

run kubectl logs gitea-issue-0 observe

2022/08/10 15:21:11 cmd/web.go:208:listen() [I] Listen: https://0.0.0.0:3000
2022/08/10 15:21:11 cmd/web.go:212:listen() [I] AppURL(ROOT_URL): https://git-issue.kindrobot.ca/
2022/08/10 15:21:11 cmd/web_https.go:171:runHTTPS() [E] Failed to load https cert file  for tcp:0.0.0.0:3000: open : no such file or directory
2022/08/10 15:21:11 cmd/web.go:258:listen() [C] Failed to start server: open : no such file or directory
2022/08/10 15:21:11 cmd/web.go:260:listen() [I] HTTP Listener: 0.0.0.0:3000 Closed

Work around

run kubectl delete gitea-issue-0
run helm3 upgrade gitea-issue gitea-charts/gitea --values values.yaml with PROTOCOL set to html
visit gitea instance; observe everything working fine

## Summary I use Gitea on k8s behind a Caddy reverse proxy which does SSL termination. I was trying to upgrade my Gitea helm deployment so that the public URL it showed was https (e.g. for cloning). I mistakenly changed `gitea.config.server.PROTOCOL` to `https` which caused gitea to fail to start because I didn't have a certificate set up. However, after switching `PROTOCOL` back to `http` and upgrading, gitea still tried to start HTTPS and crashed. I ended up having to delete the pod and upgrade to get it to "take" the `http` value for `PROTOCOL`. ## Steps to reproduce 1. run `helm3 install gitea-issue gitea-charts/gitea --values values.yaml` with the following values.yml ```yaml global: storageClass: "nfs-csi" replicaCount: 1 ingress: enabled: true hosts: - host: git-issue.kindrobot.ca paths: - path: / pathType: Prefix tls: [] gitea: admin: username: AzureDiamond password: hunter2 email: "me@example.com" config: service: DISABLE_REGISTRATION: true ` 1. visit gitea instance; observe that it's running 1. run `helm3 upgrade gitea-issue gitea-charts/gitea --values values.yaml` with the following values ```yaml global: storageClass: "nfs-csi" replicaCount: 1 ingress: enabled: true hosts: - host: git-issue.kindrobot.ca paths: - path: / pathType: Prefix tls: [] gitea: admin: username: AzureDiamond password: hunter2 email: "me@example.com" config: server: PROTOCOL: https service: DISABLE_REGISTRATION: true ` 1. visit gitea instance; observe 503 nginx error (expected as SSL cert has not been configured) 1. run `kubectl logs gitea-issue-0` observe ``` 2022/08/10 15:20:39 cmd/web.go:208:listen() [I] Listen: https://0.0.0.0:3000 2022/08/10 15:20:39 cmd/web.go:212:listen() [I] AppURL(ROOT_URL): https://git-issue.kindrobot.ca/ 2022/08/10 15:20:39 cmd/web_https.go:171:runHTTPS() [E] Failed to load https cert file for tcp:0.0.0.0:3000: open : no such file or directory 2022/08/10 15:20:39 cmd/web.go:258:listen() [C] Failed to start server: open : no such file or directory 2022/08/10 15:20:39 cmd/web.go:260:listen() [I] HTTP Listener: 0.0.0.0:3000 Closed ``` 1. run `helm3 upgrade gitea-issue gitea-charts/gitea --values values.yaml` with the following values ```yaml global: storageClass: "nfs-csi" replicaCount: 1 ingress: enabled: true hosts: - host: git-issue.kindrobot.ca paths: - path: / pathType: Prefix tls: [] gitea: admin: username: AzureDiamond password: hunter2 email: "me@example.com" config: server: PROTOCOL: http service: DISABLE_REGISTRATION: true ` 1. visit gitea instance; observe 503 nginx error (unexpected, because `PROTOCOL` was explicitly set back to `http`) 1. run `kubectl logs gitea-issue-0` observe ``` 2022/08/10 15:21:11 cmd/web.go:208:listen() [I] Listen: https://0.0.0.0:3000 2022/08/10 15:21:11 cmd/web.go:212:listen() [I] AppURL(ROOT_URL): https://git-issue.kindrobot.ca/ 2022/08/10 15:21:11 cmd/web_https.go:171:runHTTPS() [E] Failed to load https cert file for tcp:0.0.0.0:3000: open : no such file or directory 2022/08/10 15:21:11 cmd/web.go:258:listen() [C] Failed to start server: open : no such file or directory 2022/08/10 15:21:11 cmd/web.go:260:listen() [I] HTTP Listener: 0.0.0.0:3000 Closed ``` ## Work around 1. run `kubectl delete gitea-issue-0` 1. run `helm3 upgrade gitea-issue gitea-charts/gitea --values values.yaml` with `PROTOCOL` set to `html` 1. visit gitea instance; observe everything working fine

justusbunsi commented

2022-08-10 22:41:50 +00:00

It might be a bug with the checksums for configmap or secret content. Thanks for the detailed description.

justusbunsi commented

2022-08-21 15:46:52 +00:00

So, the checksums are generated as expected. It has something to do with statefulset being in a pending state from the previous attempt (using https). For some reason it does not replace the pod automatically in such case. Further investigate required. Maybe even forcing pod removal in such case using Helm hooks.

Right now it seems a "natural" behavior and the pod must be removed manually.

So, the checksums are generated as expected. It has something to do with statefulset being in a pending state from the previous attempt (using https). For some reason it does not replace the pod automatically in such case. Further investigate required. Maybe even forcing pod removal in such case using Helm hooks. Right now it seems a "natural" behavior and the pod must be removed manually.

pat-s commented

2023-02-26 12:59:50 +00:00

I guess this might be a case of "too much investigation/work needed" for too less gain?

@justusbunsi I'd favor closing here unless you have immediate plans to tackle this? Primarily to clean up a bit and not having such unclear/minor issues hanging around.

I guess this might be a case of "too much investigation/work needed" for too less gain? @justusbunsi I'd favor closing here unless you have immediate plans to tackle this? Primarily to clean up a bit and not having such unclear/minor issues hanging around.

justusbunsi commented

2023-02-26 15:48:43 +00:00

The behavior of not being able to restore default values in app.ini is a duplicate of #356 and as stated there, a regression of #239. This will be fixed sooner or later.

The requirement of replacing the pod itself is how the Chart currently works. For an actual on-the-fly configuration Gitea must be able to

reload app.ini changes without restarting the server, and
provide either an endpoint for posting a whole configuration file that is being processed or allow specifying an internal token that can be used to call existing API endpoints to configure the instance

This is something I am thinking about for quite some time now. And it requires some fundamental changes in Gitea and the Helm Chart. At some point I like to realize such kind of Configuration as Code for Gitea.

All in all I agree that we can close this issue as duplicate of #356.

The behavior of not being able to restore default values in app.ini is a duplicate of #356 and as stated there, a regression of #239. This will be fixed sooner or later. The requirement of replacing the pod itself is how the Chart currently works. For an actual on-the-fly configuration Gitea must be able to - reload app.ini changes without restarting the server, and - provide either an endpoint for posting a whole configuration file that is being processed or allow specifying an internal token that can be used to call existing API endpoints to configure the instance This is something I am thinking about for quite some time now. And it requires some fundamental changes in Gitea and the Helm Chart. At some point I like to realize such kind of Configuration as Code for Gitea. All in all I agree that we can close this issue as duplicate of #356.

pat-s commented

2023-05-19 20:09:18 +00:00

Duplicate of #356

pat-s closed this issue

2023-05-19 20:09:19 +00:00

Sign in to join this conversation.

No Milestone

No Assignees

3 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: gitea/helm-chart#344