You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: ios/MullvadVPNUITests/README.md
+8-8
Original file line number
Diff line number
Diff line change
@@ -7,7 +7,7 @@ Tests can be triggered locally from Xcode in the Test navigator or by running te
7
7
There are five workflows running tests:
8
8
-[ios-end-to-end-tests.yml](https://github.com/mullvad/mullvadvpn-app/actions/workflows/ios-end-to-end-tests.yml) - super workflow which other workflows reuse. This is also the workflow you can manually trigger to run all tests or optionally specify which tests to run.
9
9
-[ios-end-to-end-tests-nightly.yml](https://github.com/mullvad/mullvadvpn-app/actions/workflows/ios-end-to-end-tests-nightly.yml) - scheduled nightly test run, running all tests.
10
-
-[ios-end-to-end-tests-merge-to-main.yml](https://github.com/mullvad/mullvadvpn-app/actions/workflows/ios-end-to-end-tests-merge-to-main.yml) - automatically tryggered by a PR merge to `main`.
10
+
-[ios-end-to-end-tests-merge-to-main.yml](https://github.com/mullvad/mullvadvpn-app/actions/workflows/ios-end-to-end-tests-merge-to-main.yml) - automatically triggered by a PR merge to `main`.
11
11
-[ios-end-to-end-tests-api.yml](https://github.com/mullvad/mullvadvpn-app/actions/workflows/ios-end-to-end-tests-api.yml) - manually triggered tests focusing on making sure the API is functioning as intended on stagemole.
12
12
-[ios-end-to-end-tests-settings-migration.yml](https://github.com/mullvad/mullvadvpn-app/actions/workflows/ios-end-to-end-tests-settings-migration.yml) - for now this is still manually triggered. Tests installing older version of the app, changing settings, upgrading the app and verifying that settings were correctly migrated.
13
13
@@ -17,7 +17,7 @@ When adding more files with test suites they must be added to the `MullvadVPNUIT
17
17
## Set up local environment
18
18
To run tests locally you need to make sure you have copied the configuration template `UITests.xcconfig.template` to `UITests.xcconfig` and set up the configuration attributes. The configuration attributes you're mostly likely to want to set custom values for are at the top:
19
19
```
20
-
// Pin code of the iOS device under test
20
+
// Pin code of the iOS device under test.
21
21
IOS_DEVICE_PIN_CODE =
22
22
23
23
// UUID to identify test runs. Should be unique per test device. Generate with for example uuidgen on macOS.
@@ -40,7 +40,7 @@ The test device must be on the office WiFi `app-team-ios-tests` in order to be a
40
40
41
41
### Set up of runner build environment
42
42
1. Install Xcode
43
-
2. Sign in with Apple id in Xcode
43
+
2. Sign in with Apple ID in Xcode
44
44
3. Download manual provisioning profiles in Xcode
45
45
4. Install Xcode command line tools `xcode-select --install`
46
46
5. Install yeetd
@@ -62,9 +62,9 @@ The test device must be on the office WiFi `app-team-ios-tests` in order to be a
62
62
### GitHub runner setup
63
63
1. Ask GitHub admin for new runner token and setup steps from GitHub. Set it up according to the steps, pass `--labels ios-test` to `config.sh` when running it. By default it will also have the labels `self-hosted` and `macOS` which are required as well.
64
64
2. Make sure GitHub actions secrets for the GitHub project are correctly set up:
65
-
-`IOS_DEVICE_PIN_CODE` - Device passcode if the device require it, otherwise leave blank. Devices used with CI should not require passcode.
66
-
-`IOS_HAS_TIME_ACCOUNT_NUMBER` - Production server account without time left
67
-
-`IOS_NO_TIME_ACCOUNT_NUMBER` - Production server account with time added to it
65
+
-`IOS_DEVICE_PIN_CODE` - Device passcode for the device you want to run tests on, otherwise leave blank. Devices used with CI should not require passcode.
66
+
-`IOS_HAS_TIME_ACCOUNT_NUMBER` - Production server account with time added to it.
67
+
-`IOS_NO_TIME_ACCOUNT_NUMBER` - Production server account with no time. Make sure that the account has not been deleted if left unused for too long.
68
68
-`TEST_DEVICE_IDENTIFIER_UUID` - unique identifier for the test device. Create new identifier with `uuidgen`.
69
69
-`PARTNER_API_TOKEN` - secret token for partner API. Optional and only intended to be used in CI when running tests against staging environment.
70
70
@@ -90,7 +90,7 @@ The iOS team NUC is hosting APIs consumed by tests:
90
90
The NUC is hosting a WiFi which test devices need to be on in order to be able to access the firewall and packet capture APIs. The SSID is `app-team-ios-tests`. The APIs running on the NUC are accessed by using IP address `8.8.8.8` and port `80` from test devices. This is a workaround for local network access not working from UI tests. `8.8.8.8` which is a public IP address is re-routed to the NUC. This way we don't need to allow local network access in order to access the local NUC.
91
91
## Troubleshooting
92
92
### Restarting services
93
-
The easiest way to restart test services running on the NUC is by SSH:ing into it at `192.168.105.1` as `root`(password is written on a sticker under it) and rebooting `sudo shutdown -r now`.
93
+
The easiest way to restart test services running on the NUC is by SSH:ing into it at `192.168.105.1` as `root`(password is written on a sticker under it) and rebooting `sudo shutdown -r now`.
94
94
95
95
## Gotchas
96
96
### GitHub actions concurrency
@@ -99,4 +99,4 @@ The way concurrency with GitHub actions work is that multiple workflows run conc
99
99
To make the test workflows not clash with each other the jobs output files to `~/workflow-outputs`. They create a directory which is unique for the test run, and after the test run finished the directory is removed. This is necessary because we cannot depend on the state of the working directory, since if we did test runs would be changing the working directory for each other.
100
100
101
101
### Packet capture API timeout
102
-
Tests always attempt to stop packet capture, but there is no guarantee that it can always be stopped. For example when running tests locally and stopping test execution mid packet capture the test cannot stop the packet capture. So the packet capture API has a timeout(5 minutes?) in place. If a packet capture session exceeds this duration it will be stopped. This means that tests cannot do packet capture exceeding this time limit(or we need to increase the limit).
102
+
Tests always attempt to stop packet capture, but there is no guarantee that it can always be stopped. For example when running tests locally and stopping test execution mid packet capture the test cannot stop the packet capture. So the packet capture API has a timeout(5 minutes) in place. If a packet capture session exceeds this duration it will be stopped. This means that tests cannot do packet capture exceeding this time limit.
0 commit comments