Gitea federation task list #3

Open
opened 7 months ago by Ta180m · 2 comments
Ta180m commented 7 months ago (Migrated from github.com)
Owner

This is a detailed task list of things that need to be accomplished for Gitea federation.

For Gitea 1.18, I think we should aim to at least get the federated features of following, starring, issues and comments, and pull requests merged, as well as a UI for them.

User keypairs

This is done.

Discovery

This is done.

Actors

This is almost done.

  • Mapping Gitea's concepts ("People", "Teams") into AP "Actor" concepts
  • Mapping Gitea's actor concepts into the ontology
  • Assign Actor IRIs go-gitea@github.com/gitea#19133 #6
  • Manage inbox and outbox #6
  • Database storage for inbox and outbox #5
  • Generate user outbox from actions table
  • Send out activities to followers
  • Render followed user updates in your user feed
  • Periodically fetch and update cached remote users' data
  • Clean out avatar cache

ForgeFed

go-ap

We need to add ForgeFed types to go-ap. Mastodon discussion, mailing list discussion, current code

  • Implement Commit type
  • Implement Repository type
  • Implement Branch type
  • Implement Push type
  • Implement Ticket type
  • Tests
  • Export more JSON functions from go-ap/activitypub (solved by using a fork https://gitea.com/Ta180m/activitypub)

Sending/receiving activities

This is almost done.

  • Actor inbox and outbox IRIs go-gitea@github.com/gitea#19133
  • Addressing (currently we just send to the IRI in the To field which is a bad hack)
  • Transport (HTTPS with HTTP signatures) go-gitea@github.com/gitea#19133
  • Verify that actor or attributedTo matches HTTP signature.
  • Retry sending activities to servers that are down or have an outgoing activity queue?
  • sharedInbox? (probably not necessary)
  • C2S ActivityPub? (longshot)

Database representation of remote data

  • Remote federated users #2
  • Remote organizations
  • Remote repositories
  • External issues (and reference in comments) #12
  • External PRs (and reference in comments) #12
  • External projects (and reference in comments) #12

User eXperience and User Interface

See #8 for discussions

Serving/fetching ActivityStreams

  • Fetch remote ActivityStreams resources
  • Generate ActivityStreams representation of users and repos
  • Generate ActivityStreams representation of issues, PRs, and comments
  • Render remote users in the UI
  • Render remote organizations in the UI
  • Render remote repositories in the UI

Remote interactions

  • Federated following
  • Federated starring
  • Federated issues
  • Federated commenting
  • Federated forks #7
  • Federated PRs #7
  • Federated milestones
  • Federated wikis
  • Federated project boards (not yet in ForgeFed!)
  • Federated collaborators #11
  • Federated organization members #11
  • Cross-instance Git push
  • Federated private repos (use HTTP signatures for auth)
  • Federated discussions? (like GitHub discussions?)

Migrations

See #9

  • Username or repo renames
  • Moving a user or repo to a different instance
  • Official way to migrate entire GitHub account to a Gitea instance
  • Official way to migrate entire GitHub organization to a Gitea instance
  • Configurable cooldown period in which usernames cannot be claimed

Interoperability

  • Federated following with Mastodon, Pleroma, etc
  • Federated commenting on issues from Mastodon, etc mastodon@github.com/mastodon#18806
  • Handle Mastodon delegation
  • Federation with GitHub by redirecting github.com resources to a different domain that generates AS representations of GitHub data and can proxy issues and PRs via a bot account on GitHub, something like Forgeflux or Forgefriends?

Testing

  • Server-to-server federation integration testing (do we need to spin up two instances for these tests?)

Documentation

Moderation

See #10

  • Disable all federation features by default
  • Blocking a user
  • Blocking a server
  • Moderation UI
  • Limited federation mode where your instance only federates with whitelisted instances
  • Collection pagination
  • Rate limit federation requests
  • Rate limit user signups
  • Rate limit issue creation/commenting in UI
This is a detailed task list of things that need to be accomplished for [Gitea federation](https://gitea.com/Ta180m/gitea/wiki/Useful-links). For Gitea 1.18, I think we should aim to at least get the federated features of following, starring, issues and comments, and pull requests merged, as well as a UI for them. ## User keypairs This is done. - [x] Create public/private keypair [go-gitea@github.com/gitea#17071](https://github.com/go-gitea/gitea/pull/17071) - [x] HTTP signature middleware [go-gitea@github.com/gitea#19133](https://github.com/go-gitea/gitea/pull/19133) ## Discovery This is done. - [x] Nodeinfo endpoint [go-gitea@github.com/gitea#16758](https://github.com/go-gitea/gitea/issues/16758) [#19561](https://github.com/go-gitea/gitea/pull/19561) - [x] WebFinger [go-gitea@github.com/gitea#19462](https://github.com/go-gitea/gitea/pull/19462) - [x] ActivityPub-specific information in the WebFinger endpoint [go-gitea@github.com/gitea#19133](https://github.com/go-gitea/gitea/pull/19133) - [ ] Delegation? (WebFinger responses to a different domain, kinda like what Mastodon does with its WEB_DOMAIN variable) - [ ] Gitea federation for instances using a subdirectory? ## Actors This is almost done. - [x] Mapping Gitea's concepts ("People", "Teams") into AP "Actor" concepts - [x] Mapping Gitea's actor concepts into the ontology - [x] Assign Actor IRIs [go-gitea@github.com/gitea#19133](https://github.com/go-gitea/gitea/pull/19133) #6 - [x] Manage inbox and outbox #6 - [x] Database storage for inbox and outbox #5 - [x] Generate user outbox from actions table - [ ] Send out activities to followers - [ ] Render followed user updates in your user feed - [ ] Periodically fetch and update cached remote users' data - [ ] Clean out avatar cache ## ForgeFed - [x] Revive [ForgeFed](https://forgefed.org) - [x] Adapt Commit forgefed vocabulary for the needs of Gitea - [x] Adapt Repository forgefed vocabulary for the needs of Gitea - [x] Figure out the best way to represent pull requests in ForgeFed [ForgeFed@codeberg.org/ForgeFed#88](https://codeberg.org/ForgeFed/ForgeFed/issues/88) (I checked off this task since the Ticket+origin+target solution works really well for Gitea) - [ ] Issue state changes [ForgeFed@codeberg.org/ForgeFed#98](https://codeberg.org/ForgeFed/ForgeFed/issues/98) - [ ] Add milestones to ForgeFed - [ ] Add project board federation to ForgeFed - [ ] Add migrations to ForgeFed spec [ForgeFed@codeberg.org/ForgeFed#149](https://codeberg.org/ForgeFed/ForgeFed/issues/149) ## go-ap We need to add ForgeFed types to go-ap. [Mastodon discussion](https://social.exozy.me/@ta180m/108466642486831048), [mailing list discussion](https://lists.sr.ht/~mariusor/activitypub-go/%3C6aivY9YmilsvQ6CauRoaXbvXX2bdZEXOEv-xnaTD0JLvodqKPZ2FMdQHmXYKmEOdzrPZU9Ipro5sOynfe-hJbi5f31MiqOV8EI4rDOw88wg%3D%40proton.me%3E), [current code](https://gitea.com/Ta180m/gitea/src/branch/main/models/forgefed) - [x] Implement Commit type - [x] Implement Repository type - [x] Implement Branch type - [x] Implement Push type - [x] Implement Ticket type - [ ] Tests - [x] Export more JSON functions from go-ap/activitypub (solved by using a fork https://gitea.com/Ta180m/activitypub) ## Sending/receiving activities This is almost done. - [x] Actor inbox and outbox IRIs [go-gitea@github.com/gitea#19133](https://github.com/go-gitea/gitea/pull/19133) - [ ] Addressing (currently we just send to the IRI in the `To` field which is a bad hack) - [x] Transport (HTTPS with HTTP signatures) [go-gitea@github.com/gitea#19133](https://github.com/go-gitea/gitea/pull/19133) - [x] Verify that actor or `attributedTo` matches HTTP signature. - [ ] Retry sending activities to servers that are down or have an outgoing activity queue? - [ ] sharedInbox? (probably not necessary) - [ ] C2S ActivityPub? (longshot) ## Database representation of remote data - [x] Remote federated users #2 - [ ] Remote organizations - [x] Remote repositories - [ ] External issues (and reference in comments) #12 - [ ] External PRs (and reference in comments) #12 - [ ] External projects (and reference in comments) #12 ## User eXperience and User Interface See #8 for discussions - [ ] Use cases ([federated issues](https://forum.forgefriends.org/t/forgefriends-full-description/13#presentation-1) etc.) - [ ] Drafting UX in PenPot - [ ] Remote interactions pop-up UI (like Mastodon) - [x] `authorize_interaction?uri=` page (like Mastodon) - [ ] Federation handling for Gitea internal functions for creating issues, starring repos, etc - [ ] Global search bar? - [ ] Discoverability (See https://sepiasearch.org/, https://github.com/forgeflux-org/northstar) ## Serving/fetching ActivityStreams - [x] Fetch remote ActivityStreams resources - [x] Generate ActivityStreams representation of users and repos - [ ] Generate ActivityStreams representation of issues, PRs, and comments - [x] Render remote users in the UI - [ ] Render remote organizations in the UI - [ ] Render remote repositories in the UI ## Remote interactions - [x] Federated following - [x] Federated starring - [ ] Federated issues - [x] Federated commenting - [ ] Federated forks #7 - [ ] Federated PRs #7 - [ ] Federated milestones - [ ] Federated wikis - [ ] Federated project boards (not yet in ForgeFed!) - [ ] Federated collaborators #11 - [ ] Federated organization members #11 - [ ] Cross-instance Git push - [ ] Federated private repos (use HTTP signatures for auth) - [ ] Federated discussions? (like GitHub discussions?) ## Migrations See #9 - [ ] Username or repo renames - [ ] Moving a user or repo to a different instance - [ ] Official way to migrate entire GitHub account to a Gitea instance - [ ] Official way to migrate entire GitHub organization to a Gitea instance - [ ] Configurable cooldown period in which usernames cannot be claimed ## Interoperability - [x] Federated following with Mastodon, Pleroma, etc - [ ] Federated commenting on issues from Mastodon, etc [mastodon@github.com/mastodon#18806](https://github.com/mastodon/mastodon/issues/18806) - [ ] Handle Mastodon delegation - [ ] Federation with GitHub by redirecting github.com resources to a different domain that generates AS representations of GitHub data and can proxy issues and PRs via a bot account on GitHub, something like Forgeflux or Forgefriends? ## Testing - [ ] Server-to-server federation integration testing (do we need to spin up two instances for these tests?) ## Documentation - [x] [Federation.md](https://socialhub.activitypub.rocks/t/guide-for-new-activitypub-implementers/479#federationmd-31) ## Moderation See #10 - [x] Disable all federation features by default - [ ] Blocking a user - [ ] Blocking a server - [ ] Moderation UI - [ ] Limited federation mode where your instance only federates with whitelisted instances - [ ] Collection pagination - [ ] Rate limit federation requests - [ ] Rate limit user signups - [ ] Rate limit issue creation/commenting in UI
xy changed title from Gitea federation status to Are we federated yet? 6 months ago
xy changed title from Are we federated yet? to Gitea federation task list 6 months ago
Collaborator

Server-to-server federation integration testing (do we need to spin up two instances for these tests?)

Hosting two servers will need a redesign of the integrations tests(and a lot of Gitea's core design), so that should be a nuclear option. What about creating a "reference-design" server, which is just a stub of the federation capabilities(it doesn't need to do any processing)

> Server-to-server federation integration testing (do we need to spin up two instances for these tests?) Hosting two servers will need a redesign of the integrations tests(and a lot of Gitea's core design), so that should be a nuclear option. What about creating a "reference-design" server, which is just a stub of the federation capabilities(it doesn't need to do any processing)
xy commented 5 months ago
Owner

Hosting two servers will need a redesign of the integrations tests(and a lot of Gitea's core design), so that should be a nuclear option. What about creating a "reference-design" server, which is just a stub of the federation capabilities(it doesn't need to do any processing)

That's one possible idea, since it's actually really easy to implement a basic AP server (for instance, https://blog.joinmastodon.org/2018/06/how-to-implement-a-basic-activitypub-server/) since you only need to serve an actor object and a Webfinger endpoint.

Another idea could be to federate Gitea with itself.

> Hosting two servers will need a redesign of the integrations tests(and a lot of Gitea's core design), so that should be a nuclear option. What about creating a "reference-design" server, which is just a stub of the federation capabilities(it doesn't need to do any processing) That's one possible idea, since it's actually really easy to implement a basic AP server (for instance, https://blog.joinmastodon.org/2018/06/how-to-implement-a-basic-activitypub-server/) since you only need to serve an actor object and a Webfinger endpoint. Another idea could be to federate Gitea with itself.
This repo is archived. You cannot comment on issues.
No Label
No Milestone
No project
No Assignees
2 Participants
Due Date

No due date set.

Dependencies

No dependencies set.

Reference: xy/gitea#3
Loading…
There is no content yet.