Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapt linux repository generation to allow for the Mullvad browser #6358

Merged

Conversation

faern
Copy link
Member

@faern faern commented Jun 14, 2024

We want to add the Mullvad Browser into our Linux repositories. Currently the Linux repository generation is very tied to the app (since it's the only package in them). We need to change this so we can add more products into the repository, but also so we can eventually transfer the ownership of the repository generation to another team, once it's no longer app specific.

This PR moves in that direction. The general intended design is the following:

There is a new build-linux-repositories.sh script. We will run three instances of this (the services themselves are oneshot but will be triggered periodically by a systemd timer):

  • build-linux-repositories@production.service
  • build-linux-repositories@staging.service
  • build-linux-repositories@dev.service

Each run has a server environment (production, staging or dev). Each instance consume its own "inbox". This is where products dump notification files telling the repository building service it has new artifacts to pick up for a repository. See documentation at the top of build-linux-repositories.sh for more details.

Then each product we want to publish to the repositories are responsible for dumping their artifacts somewhere and write a "notification file" to the inbox of the above mentioned service.

The repository building service can trigger periodically (it does nothing if no .src files are present in its inbox)

What does each file do?

  • buildserver-build.sh - It's the script the app team run constantly to build the app when branches move and tags are pushed. It has very small changes
  • buildserver-config.sh configuration used by the app scripts. These are app specific and do not relate to the linux repository building service or the browser.
  • publish-app-to-repositories.sh - The script used by the app team to push app artifacts to the Linux repository building service.
  • build-linux-repositories-config.sh - Config specific to the repository building service. Some new variables, and some moved from buildserver-config.sh
  • prepare-{apt,rpm}-repository.sh - These take a directory of artifacts and produce repositories with metadata and signatures. These were around before, but now they have been adopted to handle more products and being run from build-linux-repositories.sh instead of buildserver-build.sh
  • build-linux-repositories.sh - The main script of the repository building service. Handle the parsing of the inbox, moving around of artifacts and triggering build and upload of repositories.
  • download-mullvad-browser.sh - Script provided by infra to download the browser, verify it, and send it to the repository building service.

This change is Reviewable

@faern faern requested review from raksooo and oskaralmlov June 14, 2024 13:04
Copy link

linear bot commented Jun 14, 2024

@faern faern force-pushed the adapt-linux-repository-generation-to-encompass-mullvad-des-866 branch 9 times, most recently from ff3ec8f to a49b9f6 Compare June 18, 2024 15:32
@faern faern marked this pull request as ready for review June 19, 2024 07:05
Copy link
Member

@raksooo raksooo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 8 of 14 files at r1.
Reviewable status: 8 of 14 files reviewed, all discussions resolved

Copy link

@oskaralmlov oskaralmlov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@faern faern force-pushed the adapt-linux-repository-generation-to-encompass-mullvad-des-866 branch 2 times, most recently from 42f1260 to b2299bd Compare June 19, 2024 12:31
Copy link

@oskaralmlov oskaralmlov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks Great To Me

@raksooo raksooo requested a review from oskaralmlov June 25, 2024 06:23
Copy link
Member

@raksooo raksooo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 14 files at r1, 1 of 5 files at r2, 2 of 7 files at r5, 2 of 2 files at r6, 3 of 3 files at r7, 4 of 6 files at r9, 1 of 3 files at r10, 2 of 2 files at r11.
Reviewable status: 13 of 14 files reviewed, 3 unresolved discussions (waiting on @faern and @oskaralmlov)


ci/linux-repository-builder/prepare-rpm-repository.sh line 49 at r11 (raw file):

fi

# Writes the mullvad.repo config file to the repository

This line is only 55 characters, you could fit this comment on fewer lines if you make them longer.

Copy link
Member

@raksooo raksooo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 14 files at r1.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @faern and @oskaralmlov)

@raksooo raksooo requested a review from oskaralmlov June 25, 2024 11:00
Copy link
Member

@raksooo raksooo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 5 of 5 files at r12.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @faern and @oskaralmlov)

Copy link
Member

@raksooo raksooo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 3 files at r13.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @faern and @oskaralmlov)

@faern faern force-pushed the adapt-linux-repository-generation-to-encompass-mullvad-des-866 branch from b1c482a to c40af9d Compare June 25, 2024 14:28
Copy link
Member Author

@faern faern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @oskaralmlov)


ci/build-linux-repositories@.service line 21 at r2 (raw file):

Previously, oskaralmlov (Oskar Almlöv) wrote…

You don't need to enable the service as the timer will take care of running it.

Done.

faern added 2 commits June 25, 2024 16:55
Creates a standalone service responsible for building deb/rpm
repositories and uploading them. This means the app build scripts no
longer do this. Instead the app is just sent to the "inbox" of the
repository service.

A separate script for downloading and verifying the Mullvad Browser is
added, which allows adding the browser into the same repositories
@faern faern force-pushed the adapt-linux-repository-generation-to-encompass-mullvad-des-866 branch from c40af9d to c13a42f Compare June 25, 2024 14:55
@faern faern merged commit dd0a415 into main Jun 25, 2024
13 of 14 checks passed
@faern faern deleted the adapt-linux-repository-generation-to-encompass-mullvad-des-866 branch June 25, 2024 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants