ListMyRepos problems #507

Closed
opened 2021-03-29 09:48:29 +00:00 by pboguslawski · 7 comments

(1)
The line

https://gitea.com/gitea/go-sdk/src/branch/master/gitea/repo.go#L133

does not set defaults; changing "func (o ListOptions)" to "func (o *ListOptions)" in

https://gitea.com/gitea/go-sdk/src/branch/master/gitea/list_options.go

resolves this issue for us.

(2)
Gitea API should allow one to list all repos user has access to on one page using ListMyRepos with Page=1 and PageSize=0; now its possible using hack i.e. calling ListMyRepos with Page=1 and PageSize=very_large_int but it won't be possilbe after fixing (1) - setDefaults will force max PageSize value 50. This breaks Woodpecker integration (it wants to list all user repos in one api call):

https://github.com/laszlocph/woodpecker/blob/master/remote/gitea/gitea.go#L222

Regards,
Paweł

(1) The line https://gitea.com/gitea/go-sdk/src/branch/master/gitea/repo.go#L133 does not set defaults; changing "func (o ListOptions)" to "func (o *ListOptions)" in https://gitea.com/gitea/go-sdk/src/branch/master/gitea/list_options.go resolves this issue for us. (2) Gitea API should allow one to list all repos user has access to on one page using ListMyRepos with Page=1 and PageSize=0; now its possible using hack i.e. calling ListMyRepos with Page=1 and PageSize=_very_large_int_ but it won't be possilbe after fixing (1) - setDefaults will force max PageSize value 50. This breaks Woodpecker integration (it wants to list all user repos in one api call): https://github.com/laszlocph/woodpecker/blob/master/remote/gitea/gitea.go#L222 Regards, Paweł
Owner

FYI this will affect every ListX struct, since they all embed ListOptions.

FYI this will affect every `ListX` struct, since they all embed `ListOptions`.
Author
See also https://github.com/laszlocph/woodpecker/pull/182
6543 added this to the v0.14.1 milestone 2021-03-29 17:53:36 +00:00
6543 added the
kind/bug
label 2021-03-29 17:53:42 +00:00
Owner

@pboguslawski pulls are welcome ;)

@pboguslawski pulls are welcome ;)
Author

pulls are welcome ;)

We just tested change "func (o ListOptions)" to "func (o *ListOptions)" resolved issue with appying defaults but don't use this mod in production because we don't have enough exprerience in Gitea SDK/api to decide if its safe and won't break something else. That`s why issue not PR.

BTW: fix (2) is not necessarry because we've already proposed better solution for Woodpecker in https://github.com/laszlocph/woodpecker/pull/182; on the other hand option to have all repos returned in 1 api call would be nice (i.e. to avoid races) if it won't introduce gitea overloading.

Regards,
Paweł

> pulls are welcome ;) We just tested change "func (o ListOptions)" to "func (o *ListOptions)" resolved issue with appying defaults but don't use this mod in production because we don't have enough exprerience in Gitea SDK/api to decide if its safe and won't break something else. That`s why issue not PR. BTW: fix (2) is not necessarry because we've already proposed better solution for Woodpecker in https://github.com/laszlocph/woodpecker/pull/182; on the other hand option to have all repos returned in 1 api call would be nice (i.e. to avoid races) if it won't introduce gitea overloading. Regards, Paweł
Owner

gitea should not have reaces at all ... it it has it's a bug that should be fixed ...

most load for gitea list endpoint apis are loading(+calc) & serialize object attributes ... so to pagination you dont realy put more load on it ... the few more bits to handle the http(s) requests I think is managable

#508 should handle all cases :)

gitea should not have reaces at all ... it it has it's a bug that should be fixed ... most load for gitea list endpoint apis are loading(+calc) & serialize object attributes ... so to pagination you dont realy put more load on it ... the few more bits to handle the http(s) requests I think is managable #508 should handle all cases :)
jolheiser referenced this issue from a commit 2021-03-29 20:57:07 +00:00
Author

gitea should not have reaces at all ... it it has it's a bug that should be fixed

Races may occur problably on client side when client loops pages and repo list changes in the meantime in Gitea. To avoid this client may want to get all repos in one shot OR Gitea+client should use some kind of cursor and remember results for particular client query which is not practical nor safe (DOS possible to exhaust giteas resources).

most load for gitea list endpoint apis are loading(+calc) & serialize object attributes ... so to pagination you dont realy put more load on it ... the few more bits to handle the http(s) requests I think is managable

#508 should handle all cases :)

If generating all repo list in one shot is not risky in Gitea, consider allowing it (i.e. with PageSize=0) and document it in SDK; see also

https://github.com/laszlocph/woodpecker/pull/182#issuecomment-809962544

> gitea should not have reaces at all ... it it has it's a bug that should be fixed Races may occur problably on client side when client loops pages and repo list changes in the meantime in Gitea. To avoid this client may want to get all repos in one shot OR Gitea+client should use some kind of cursor and remember results for particular client query which is not practical nor safe (DOS possible to exhaust giteas resources). > most load for gitea list endpoint apis are loading(+calc) & serialize object attributes ... so to pagination you dont realy put more load on it ... the few more bits to handle the http(s) requests I think is managable > > #508 should handle all cases :) If generating all repo list in one shot is not risky in Gitea, consider allowing it (i.e. with PageSize=0) and document it in SDK; see also https://github.com/laszlocph/woodpecker/pull/182#issuecomment-809962544
Owner

@pboguslawski just follow the progress of #509 :)

@pboguslawski just follow the progress of #509 :)
Sign in to join this conversation.
No Milestone
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: gitea/go-sdk#507
No description provided.