Gitea federation task list #3

New Issue

This is a detailed task list of things that need to be accomplished for Gitea federation.

For Gitea 1.18, I think we should aim to at least get the federated features of following, starring, issues and comments, and pull requests merged, as well as a UI for them.

User keypairs

This is done.

Create public/private keypair go-gitea@github.com/gitea#17071
HTTP signature middleware go-gitea@github.com/gitea#19133

Discovery

This is done.

Nodeinfo endpoint go-gitea@github.com/gitea#16758 #19561
WebFinger go-gitea@github.com/gitea#19462
ActivityPub-specific information in the WebFinger endpoint go-gitea@github.com/gitea#19133
Delegation? (WebFinger responses to a different domain, kinda like what Mastodon does with its WEB_DOMAIN variable)
Gitea federation for instances using a subdirectory?

Actors

This is almost done.

Mapping Gitea's concepts ("People", "Teams") into AP "Actor" concepts
Mapping Gitea's actor concepts into the ontology
Assign Actor IRIs go-gitea@github.com/gitea#19133 #6
Manage inbox and outbox #6
Database storage for inbox and outbox #5
Generate user outbox from actions table
Send out activities to followers
Render followed user updates in your user feed
Periodically fetch and update cached remote users' data
Clean out avatar cache

ForgeFed

Revive ForgeFed
Adapt Commit forgefed vocabulary for the needs of Gitea
Adapt Repository forgefed vocabulary for the needs of Gitea
Figure out the best way to represent pull requests in ForgeFed ForgeFed@codeberg.org/ForgeFed#88 (I checked off this task since the Ticket+origin+target solution works really well for Gitea)
Issue state changes ForgeFed@codeberg.org/ForgeFed#98
Add milestones to ForgeFed
Add project board federation to ForgeFed
Add migrations to ForgeFed spec ForgeFed@codeberg.org/ForgeFed#149

go-ap

We need to add ForgeFed types to go-ap. Mastodon discussion, mailing list discussion, current code

Implement Commit type
Implement Repository type
Implement Branch type
Implement Push type
Implement Ticket type
Tests
Export more JSON functions from go-ap/activitypub (solved by using a fork https://gitea.com/Ta180m/activitypub)

Sending/receiving activities

This is almost done.

Actor inbox and outbox IRIs go-gitea@github.com/gitea#19133
Addressing (currently we just send to the IRI in the To field which is a bad hack)
Transport (HTTPS with HTTP signatures) go-gitea@github.com/gitea#19133
Verify that actor or attributedTo matches HTTP signature.
Retry sending activities to servers that are down or have an outgoing activity queue?
sharedInbox? (probably not necessary)
C2S ActivityPub? (longshot)

Database representation of remote data

Remote federated users #2
Remote organizations
Remote repositories
External issues (and reference in comments) #12
External PRs (and reference in comments) #12
External projects (and reference in comments) #12

User eXperience and User Interface

See #8 for discussions

Use cases (federated issues etc.)
Drafting UX in PenPot
Remote interactions pop-up UI (like Mastodon)
authorize_interaction?uri= page (like Mastodon)
Federation handling for Gitea internal functions for creating issues, starring repos, etc
Global search bar?
Discoverability (See https://sepiasearch.org/, https://github.com/forgeflux-org/northstar)

Serving/fetching ActivityStreams

Fetch remote ActivityStreams resources
Generate ActivityStreams representation of users and repos
Generate ActivityStreams representation of issues, PRs, and comments
Render remote users in the UI
Render remote organizations in the UI
Render remote repositories in the UI

Remote interactions

Federated following
Federated starring
Federated issues
Federated commenting
Federated forks #7
Federated PRs #7
Federated milestones
Federated wikis
Federated project boards (not yet in ForgeFed!)
Federated collaborators #11
Federated organization members #11
Cross-instance Git push
Federated private repos (use HTTP signatures for auth)
Federated discussions? (like GitHub discussions?)

Migrations

See #9

Username or repo renames
Moving a user or repo to a different instance
Official way to migrate entire GitHub account to a Gitea instance
Official way to migrate entire GitHub organization to a Gitea instance
Configurable cooldown period in which usernames cannot be claimed

Interoperability

Federated following with Mastodon, Pleroma, etc
Federated commenting on issues from Mastodon, etc mastodon@github.com/mastodon#18806
Handle Mastodon delegation
Federation with GitHub by redirecting github.com resources to a different domain that generates AS representations of GitHub data and can proxy issues and PRs via a bot account on GitHub, something like Forgeflux or Forgefriends?

Testing

Server-to-server federation integration testing (do we need to spin up two instances for these tests?)

Documentation

Federation.md

Moderation

See #10

Disable all federation features by default
Blocking a user
Blocking a server
Moderation UI
Limited federation mode where your instance only federates with whitelisted instances
Collection pagination
Rate limit federation requests
Rate limit user signups
Rate limit issue creation/commenting in UI

This is a detailed task list of things that need to be accomplished for [Gitea federation](https://gitea.com/Ta180m/gitea/wiki/Useful-links). For Gitea 1.18, I think we should aim to at least get the federated features of following, starring, issues and comments, and pull requests merged, as well as a UI for them. ## User keypairs This is done. - [x] Create public/private keypair [go-gitea@github.com/gitea#17071](https://github.com/go-gitea/gitea/pull/17071) - [x] HTTP signature middleware [go-gitea@github.com/gitea#19133](https://github.com/go-gitea/gitea/pull/19133) ## Discovery This is done. - [x] Nodeinfo endpoint [go-gitea@github.com/gitea#16758](https://github.com/go-gitea/gitea/issues/16758) [#19561](https://github.com/go-gitea/gitea/pull/19561) - [x] WebFinger [go-gitea@github.com/gitea#19462](https://github.com/go-gitea/gitea/pull/19462) - [x] ActivityPub-specific information in the WebFinger endpoint [go-gitea@github.com/gitea#19133](https://github.com/go-gitea/gitea/pull/19133) - [ ] Delegation? (WebFinger responses to a different domain, kinda like what Mastodon does with its WEB_DOMAIN variable) - [ ] Gitea federation for instances using a subdirectory? ## Actors This is almost done. - [x] Mapping Gitea's concepts ("People", "Teams") into AP "Actor" concepts - [x] Mapping Gitea's actor concepts into the ontology - [x] Assign Actor IRIs [go-gitea@github.com/gitea#19133](https://github.com/go-gitea/gitea/pull/19133) #6 - [x] Manage inbox and outbox #6 - [x] Database storage for inbox and outbox #5 - [x] Generate user outbox from actions table - [ ] Send out activities to followers - [ ] Render followed user updates in your user feed - [ ] Periodically fetch and update cached remote users' data - [ ] Clean out avatar cache ## ForgeFed - [x] Revive [ForgeFed](https://forgefed.org) - [x] Adapt Commit forgefed vocabulary for the needs of Gitea - [x] Adapt Repository forgefed vocabulary for the needs of Gitea - [x] Figure out the best way to represent pull requests in ForgeFed [ForgeFed@codeberg.org/ForgeFed#88](https://codeberg.org/ForgeFed/ForgeFed/issues/88) (I checked off this task since the Ticket+origin+target solution works really well for Gitea) - [ ] Issue state changes [ForgeFed@codeberg.org/ForgeFed#98](https://codeberg.org/ForgeFed/ForgeFed/issues/98) - [ ] Add milestones to ForgeFed - [ ] Add project board federation to ForgeFed - [ ] Add migrations to ForgeFed spec [ForgeFed@codeberg.org/ForgeFed#149](https://codeberg.org/ForgeFed/ForgeFed/issues/149) ## go-ap We need to add ForgeFed types to go-ap. [Mastodon discussion](https://social.exozy.me/@ta180m/108466642486831048), [mailing list discussion](https://lists.sr.ht/~mariusor/activitypub-go/%3C6aivY9YmilsvQ6CauRoaXbvXX2bdZEXOEv-xnaTD0JLvodqKPZ2FMdQHmXYKmEOdzrPZU9Ipro5sOynfe-hJbi5f31MiqOV8EI4rDOw88wg%3D%40proton.me%3E), [current code](https://gitea.com/Ta180m/gitea/src/branch/main/models/forgefed) - [x] Implement Commit type - [x] Implement Repository type - [x] Implement Branch type - [x] Implement Push type - [x] Implement Ticket type - [ ] Tests - [x] Export more JSON functions from go-ap/activitypub (solved by using a fork https://gitea.com/Ta180m/activitypub) ## Sending/receiving activities This is almost done. - [x] Actor inbox and outbox IRIs [go-gitea@github.com/gitea#19133](https://github.com/go-gitea/gitea/pull/19133) - [ ] Addressing (currently we just send to the IRI in the `To` field which is a bad hack) - [x] Transport (HTTPS with HTTP signatures) [go-gitea@github.com/gitea#19133](https://github.com/go-gitea/gitea/pull/19133) - [x] Verify that actor or `attributedTo` matches HTTP signature. - [ ] Retry sending activities to servers that are down or have an outgoing activity queue? - [ ] sharedInbox? (probably not necessary) - [ ] C2S ActivityPub? (longshot) ## Database representation of remote data - [x] Remote federated users #2 - [ ] Remote organizations - [x] Remote repositories - [ ] External issues (and reference in comments) #12 - [ ] External PRs (and reference in comments) #12 - [ ] External projects (and reference in comments) #12 ## User eXperience and User Interface See #8 for discussions - [ ] Use cases ([federated issues](https://forum.forgefriends.org/t/forgefriends-full-description/13#presentation-1) etc.) - [ ] Drafting UX in PenPot - [ ] Remote interactions pop-up UI (like Mastodon) - [x] `authorize_interaction?uri=` page (like Mastodon) - [ ] Federation handling for Gitea internal functions for creating issues, starring repos, etc - [ ] Global search bar? - [ ] Discoverability (See https://sepiasearch.org/, https://github.com/forgeflux-org/northstar) ## Serving/fetching ActivityStreams - [x] Fetch remote ActivityStreams resources - [x] Generate ActivityStreams representation of users and repos - [ ] Generate ActivityStreams representation of issues, PRs, and comments - [x] Render remote users in the UI - [ ] Render remote organizations in the UI - [ ] Render remote repositories in the UI ## Remote interactions - [x] Federated following - [x] Federated starring - [ ] Federated issues - [x] Federated commenting - [ ] Federated forks #7 - [ ] Federated PRs #7 - [ ] Federated milestones - [ ] Federated wikis - [ ] Federated project boards (not yet in ForgeFed!) - [ ] Federated collaborators #11 - [ ] Federated organization members #11 - [ ] Cross-instance Git push - [ ] Federated private repos (use HTTP signatures for auth) - [ ] Federated discussions? (like GitHub discussions?) ## Migrations See #9 - [ ] Username or repo renames - [ ] Moving a user or repo to a different instance - [ ] Official way to migrate entire GitHub account to a Gitea instance - [ ] Official way to migrate entire GitHub organization to a Gitea instance - [ ] Configurable cooldown period in which usernames cannot be claimed ## Interoperability - [x] Federated following with Mastodon, Pleroma, etc - [ ] Federated commenting on issues from Mastodon, etc [mastodon@github.com/mastodon#18806](https://github.com/mastodon/mastodon/issues/18806) - [ ] Handle Mastodon delegation - [ ] Federation with GitHub by redirecting github.com resources to a different domain that generates AS representations of GitHub data and can proxy issues and PRs via a bot account on GitHub, something like Forgeflux or Forgefriends? ## Testing - [ ] Server-to-server federation integration testing (do we need to spin up two instances for these tests?) ## Documentation - [x] [Federation.md](https://socialhub.activitypub.rocks/t/guide-for-new-activitypub-implementers/479#federationmd-31) ## Moderation See #10 - [x] Disable all federation features by default - [ ] Blocking a user - [ ] Blocking a server - [ ] Moderation UI - [ ] Limited federation mode where your instance only federates with whitelisted instances - [ ] Collection pagination - [ ] Rate limit federation requests - [ ] Rate limit user signups - [ ] Rate limit issue creation/commenting in UI

Server-to-server federation integration testing (do we need to spin up two instances for these tests?)

Hosting two servers will need a redesign of the integrations tests(and a lot of Gitea's core design), so that should be a nuclear option. What about creating a "reference-design" server, which is just a stub of the federation capabilities(it doesn't need to do any processing)

> Server-to-server federation integration testing (do we need to spin up two instances for these tests?) Hosting two servers will need a redesign of the integrations tests(and a lot of Gitea's core design), so that should be a nuclear option. What about creating a "reference-design" server, which is just a stub of the federation capabilities(it doesn't need to do any processing)

Hosting two servers will need a redesign of the integrations tests(and a lot of Gitea's core design), so that should be a nuclear option. What about creating a "reference-design" server, which is just a stub of the federation capabilities(it doesn't need to do any processing)

That's one possible idea, since it's actually really easy to implement a basic AP server (for instance, https://blog.joinmastodon.org/2018/06/how-to-implement-a-basic-activitypub-server/) since you only need to serve an actor object and a Webfinger endpoint.

Another idea could be to federate Gitea with itself.

> Hosting two servers will need a redesign of the integrations tests(and a lot of Gitea's core design), so that should be a nuclear option. What about creating a "reference-design" server, which is just a stub of the federation capabilities(it doesn't need to do any processing) That's one possible idea, since it's actually really easy to implement a basic AP server (for instance, https://blog.joinmastodon.org/2018/06/how-to-implement-a-basic-activitypub-server/) since you only need to serve an actor object and a Webfinger endpoint. Another idea could be to federate Gitea with itself.