Skip to content

Commit 932a698

Browse files
delandelan
committed
Add build scripts for ubuntu2204 and windows10 images (#6)
1 parent ee1395b commit 932a698

File tree

4 files changed

+165
-49
lines changed

4 files changed

+165
-49
lines changed

‎README.md

Lines changed: 7 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -109,35 +109,16 @@ Windows 10 runner
109109

110110
Runners created from this image preinstall all dependencies (including those specified in the main repo, like GStreamer and Chocolatey deps), preload the main repo, and prebuild Servo in the release profile.
111111

112-
To build the base vm, first build a clean image:
112+
To build the base vm image:
113113

114114
- Download images into /var/lib/libvirt/images
115115
- Windows 10 (multi-edition ISO), English (United States): [Win10_22H2_English_x64v1.iso](https://www.microsoft.com/en-us/software-download/windows10ISO) (sha256 = a6f470ca6d331eb353b815c043e327a347f594f37ff525f17764738fe812852e)
116-
- VirtIO drivers: [virtio-win-0.1.240.iso](https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.240-1/virtio-win-0.1.240.iso) (sha256 = ebd48258668f7f78e026ed276c28a9d19d83e020ffa080ad69910dc86bbcbcc6)
117-
- Create zvol and libvirt guest with random UUID and MAC address
118-
- `zfs create -V 90G tank/base/servo-windows10.clean`
119-
- `virsh define windows10/guest.xml`
120-
- `virt-clone --preserve-data --check path_in_use=off -o servo-windows10.init -n servo-windows10.clean -f /dev/zvol/tank/base/servo-windows10.clean`
121-
- `virsh undefine servo-windows10.init`
122-
- Install Windows:
123-
- `genisoimage -J -o /var/lib/libvirt/images/servo-windows10.config.iso windows10/autounattend.xml`
124-
- `virsh start servo-windows10.clean`
125-
- Wait for the guest to shut down
126-
- Take a snapshot: `zfs snapshot tank/base/servo-windows10.clean@oobe`
127-
128-
Then build the base image:
129-
130-
- Clone the clean image: `zfs clone tank/base/servo-windows10{.clean@oobe,.new}`
131-
- Create a temporary libvirt guest: `virt-clone --preserve-data --check path_in_use=off -o servo-windows10.clean -n servo-windows10.new -f /dev/zvol/tank/base/servo-windows10.new`
132-
- Update new base image: `./mount-runner.sh servo-windows10.new $PWD/windows10/configure-base.sh`
133-
- Take a snapshot: `zfs snapshot tank/base/servo-windows10.new@configure-base`
134-
- Boot temporary vm guest: `virsh start servo-windows10.new`
135-
- Wait for the guest to shut down, which indicates Servo was built successfully
136-
- Take another snapshot: `zfs snapshot tank/base/servo-windows10.new@ready`
116+
- Run the build script: `windows10/build-image.sh`
117+
- FIXME: if Windows fails to autologon (stuck at lock screen): `virsh reboot servo-windows10.new`
137118
- Destroy the old base image (if it exists): `zfs destroy -r tank/base/servo-windows10`
138119
- Rename the new base image: `zfs rename tank/base/servo-windows10{.new,}`
120+
- Create the base libvirt guest (if it doesn’t exist): `virt-clone --preserve-data --check path_in_use=off -o servo-windows10.new -n servo-windows10 -f /dev/zvol/tank/base/servo-windows10`
139121
- Undefine the temporary libvirt guest: `virsh undefine servo-windows10.new`
140-
- Create the base libvirt guest (if it doesn’t exist): `virt-clone --preserve-data --check path_in_use=off -o servo-windows10.clean -n servo-windows10 -f /dev/zvol/tank/base/servo-windows10`
141122

142123
To clone and start a new runner:
143124

@@ -150,36 +131,13 @@ Ubuntu runner
150131

151132
Runners created from this image preinstall all dependencies (including those specified in the main repo, like mach bootstrap deps), preload the main repo, and prebuild Servo in the release profile.
152133

153-
To build the base vm, first build a clean image:
134+
To build the base vm image:
154135

155-
- Download images into /var/lib/libvirt/images
156-
- Ubuntu Server 22.04 cloud image: [jammy-server-cloudimg-amd64.img](https://cloud-images.ubuntu.com/jammy/20241217/jammy-server-cloudimg-amd64.img) (sha256 = 0d8345a343c2547e55ac815342e6cb4a593aa5556872651eb47e6856a2bb0cdd)
157-
- Create zvol and libvirt guest with random UUID and MAC address
158-
- `zfs create -V 90G tank/base/servo-ubuntu2204.clean`
159-
- `virsh define ubuntu2204/guest.xml`
160-
- `virt-clone --preserve-data --check path_in_use=off -o servo-ubuntu2204.init -n servo-ubuntu2204.clean -f /dev/zvol/tank/base/servo-ubuntu2204.clean`
161-
- `virsh undefine servo-ubuntu2204.init`
162-
- Install Ubuntu:
163-
- `qemu-img convert -f qcow2 -O raw /var/lib/libvirt/images/jammy-server-cloudimg-amd64.{img,raw}`
164-
- `dd status=progress bs=1M if=/var/lib/libvirt/images/jammy-server-cloudimg-amd64.raw of=/dev/zvol/tank/base/servo-ubuntu2204.clean`
165-
- `genisoimage -V CIDATA -R -o /var/lib/libvirt/images/servo-ubuntu2204.config.iso ubuntu2204/{user-data,meta-data}`
166-
- `virsh start servo-ubuntu2204.clean`
167-
- Wait for the guest to shut down
168-
- Take a snapshot: `zfs snapshot tank/base/servo-ubuntu2204.clean@fresh-install`
169-
170-
Then build the base image:
171-
172-
- Clone the clean image: `zfs clone tank/base/servo-ubuntu2204{.clean@fresh-install,.new}`
173-
- Create a temporary libvirt guest: `virt-clone --preserve-data --check path_in_use=off -o servo-ubuntu2204.clean -n servo-ubuntu2204.new -f /dev/zvol/tank/base/servo-ubuntu2204.new`
174-
- Update new base image: `./mount-runner.sh servo-ubuntu2204.new $PWD/ubuntu2204/configure-base.sh`
175-
- Take another snapshot: `zfs snapshot tank/base/servo-ubuntu2204.new@configure-base`
176-
- Boot temporary vm guest: `virsh start servo-ubuntu2204.new`
177-
- Wait for the guest to shut down, which indicates Servo was built successfully
178-
- Take another snapshot: `zfs snapshot tank/base/servo-ubuntu2204.new@ready`
136+
- Run the build script: `ubuntu2204/build-image.sh`
179137
- Destroy the old base image (if it exists): `zfs destroy -r tank/base/servo-ubuntu2204`
180138
- Rename the new base image: `zfs rename tank/base/servo-ubuntu2204{.new,}`
139+
- Create the base libvirt guest (if it doesn’t exist): `virt-clone --preserve-data --check path_in_use=off -o servo-ubuntu2204.new -n servo-ubuntu2204 -f /dev/zvol/tank/base/servo-ubuntu2204`
181140
- Undefine the temporary libvirt guest: `virsh undefine servo-ubuntu2204.new`
182-
- Create the base libvirt guest (if it doesn’t exist): `virt-clone --preserve-data --check path_in_use=off -o servo-ubuntu2204.clean -n servo-ubuntu2204 -f /dev/zvol/tank/base/servo-ubuntu2204`
183141

184142
To clone and start a new runner:
185143

‎common.sh

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,18 @@ set -a
88
set +a
99

1010
export SERVO_CI_MONITOR_DATA_PATH=${SERVO_CI_MONITOR_DATA_PATH-$script_dir/monitor/data}
11+
12+
# usage: trap print_undo_commands EXIT
13+
print_undo_commands() {
14+
exit_status=$?
15+
if [ $exit_status -ne 0 ]; then
16+
>&2 echo
17+
>&2 echo "Failed to build image!"
18+
fi
19+
if [ -n "$(cat $undo_commands)" ]; then
20+
>&2 echo
21+
>&2 echo "[*] To abort:"
22+
>&2 tac $undo_commands
23+
exit $exit_status
24+
fi
25+
}

‎ubuntu2204/build-image.sh

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
#!/usr/bin/env zsh
2+
# usage: ubuntu2204/build-image.sh
3+
image_dir=${0:a:h}
4+
script_dir=${0:a:h}/..
5+
. "$script_dir/common.sh"
6+
trap print_undo_commands EXIT
7+
cache_dir=$script_dir/cache
8+
. "$script_dir/download.sh"
9+
. "$script_dir/inject.sh"
10+
undo_commands=$(mktemp)
11+
image_name=servo-ubuntu2204
12+
13+
>&2 echo '[*] Caching downloads'
14+
download "$cache_dir" https://cloud-images.ubuntu.com/jammy/20241217/jammy-server-cloudimg-amd64.img 0d8345a343c2547e55ac815342e6cb4a593aa5556872651eb47e6856a2bb0cdd
15+
16+
>&2 echo '[*] Converting qcow2 image to raw image'
17+
qemu-img convert -f qcow2 -O raw "$cache_dir/jammy-server-cloudimg-amd64.img" "$cache_dir/jammy-server-cloudimg-amd64.raw"
18+
19+
>&2 echo '[*] Creating zvol and libvirt guest'
20+
zfs create -V 90G "$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new"
21+
>> $undo_commands echo "zfs destroy '$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new'"
22+
virsh define -- "$image_dir/guest.xml"
23+
>> $undo_commands echo "virsh undefine -- '$image_name.init'"
24+
virt-clone --preserve-data --check path_in_use=off -o "$image_name.init" -n "$image_name.new" -f "/dev/zvol/$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new"
25+
>> $undo_commands echo "virsh undefine -- '$image_name.new'"
26+
virsh undefine -- "$image_name.init"
27+
28+
>&2 echo '[*] Writing disk images'
29+
dd status=progress bs=1M if="$cache_dir/jammy-server-cloudimg-amd64.raw" of="/dev/zvol/$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new"
30+
genisoimage -V CIDATA -R -o "/var/lib/libvirt/images/$image_name.config.iso" "$image_dir/user-data" "$image_dir/meta-data"
31+
32+
>&2 echo '[*] Starting guest, to expand root filesystem'
33+
virsh start "$image_name.new"
34+
35+
>&2 echo '[*] Waiting for guest to shut down (max 40 seconds)' # normally ~19 seconds
36+
if ! time virsh event --timeout 40 -- "$image_name.new" lifecycle; then
37+
>&2 echo 'virsh event timed out!'
38+
exit 1
39+
fi
40+
41+
>&2 echo '[*] Waiting for partition block device to appear'
42+
partition_block_device=/dev/zvol/$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new-part1
43+
t=0; while ! test -e $partition_block_device; do
44+
if [ $t -ge $SERVO_CI_ZVOL_BLOCK_DEVICE_TIMEOUT ]; then
45+
>&2 printf '[!] Timed out waiting for block device: %s' $partition_block_device
46+
exit 1
47+
fi
48+
sleep 1
49+
t=$((t+1))
50+
done
51+
52+
>&2 echo '[*] Forcing update of partition block device geometry'
53+
# Dec 20 17:12:59 jupiter kernel: EXT4-fs (zd16p1): bad geometry: block count 23564539 exceeds size of device (548091 blocks)
54+
blockdev --rereadpt "/dev/zvol/$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new"
55+
56+
>&2 echo '[*] Configuring base image'
57+
./mount-runner.sh "$image_name.new" "$image_dir/configure-base.sh"
58+
59+
>&2 echo '[*] Starting guest, to apply changes'
60+
virsh start "$image_name.new"
61+
62+
>&2 echo '[*] Waiting for guest to shut down (max 2000 seconds)' # normally ~978 seconds
63+
if ! time virsh event --timeout 2000 -- "$image_name.new" lifecycle; then
64+
>&2 echo 'virsh event timed out!'
65+
exit 1
66+
fi
67+
68+
>&2 echo "[*] Taking zvol snapshot: $SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new@ready"
69+
zfs snapshot "$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new@ready"
70+
71+
# TODO: check that servo was actually built correctly
72+
73+
>&2 echo '[*] Done!'

‎windows10/build-image.sh

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
#!/usr/bin/env zsh
2+
# usage: windows10/build-image.sh
3+
image_dir=${0:a:h}
4+
script_dir=${0:a:h}/..
5+
. "$script_dir/common.sh"
6+
trap print_undo_commands EXIT
7+
cache_dir=$script_dir/cache
8+
. "$script_dir/download.sh"
9+
. "$script_dir/inject.sh"
10+
undo_commands=$(mktemp)
11+
image_name=servo-windows10
12+
13+
>&2 echo '[*] Caching downloads'
14+
download "$cache_dir" https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.240-1/virtio-win-0.1.240.iso ebd48258668f7f78e026ed276c28a9d19d83e020ffa080ad69910dc86bbcbcc6
15+
16+
>&2 echo '[*] Creating zvol and libvirt guest'
17+
zfs create -V 90G "$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new"
18+
>> $undo_commands echo "zfs destroy '$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new'"
19+
virsh define -- "$image_dir/guest.xml"
20+
>> $undo_commands echo "virsh undefine -- '$image_name.init'"
21+
virt-clone --preserve-data --check path_in_use=off -o "$image_name.init" -n "$image_name.new" -f "/dev/zvol/$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new"
22+
>> $undo_commands echo "virsh undefine -- '$image_name.new'"
23+
virsh undefine -- "$image_name.init"
24+
25+
>&2 echo '[*] Writing disk images'
26+
inject /var/lib/libvirt/images "$cache_dir/virtio-win-0.1.240.iso"
27+
genisoimage -J -o "/var/lib/libvirt/images/$image_name.config.iso" "$image_dir/autounattend.xml"
28+
29+
>&2 echo '[*] Starting guest, to install Windows'
30+
virsh start "$image_name.new"
31+
32+
>&2 echo '[*] Waiting for guest to shut down (max 640 seconds)' # normally ~313 seconds
33+
if ! time virsh event --timeout 640 -- "$image_name.new" lifecycle; then
34+
>&2 echo 'virsh event timed out!'
35+
exit 1
36+
fi
37+
38+
>&2 echo '[*] Waiting for partition block device to appear'
39+
partition_block_device=/dev/zvol/$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new-part1
40+
t=0; while ! test -e $partition_block_device; do
41+
if [ $t -ge $SERVO_CI_ZVOL_BLOCK_DEVICE_TIMEOUT ]; then
42+
>&2 printf '[!] Timed out waiting for block device: %s' $partition_block_device
43+
exit 1
44+
fi
45+
sleep 1
46+
t=$((t+1))
47+
done
48+
49+
>&2 echo '[*] Forcing update of partition block device geometry'
50+
# Dec 20 17:12:59 jupiter kernel: EXT4-fs (zd16p1): bad geometry: block count 23564539 exceeds size of device (548091 blocks)
51+
blockdev --rereadpt "/dev/zvol/$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new"
52+
53+
>&2 echo '[*] Configuring base image'
54+
./mount-runner.sh "$image_name.new" "$image_dir/configure-base.sh"
55+
56+
>&2 echo '[*] Starting guest, to apply changes'
57+
virsh start "$image_name.new"
58+
59+
>&2 echo '[*] Waiting for guest to shut down (max 2500 seconds)' # normally ~1218 seconds
60+
if ! time virsh event --timeout 2500 -- "$image_name.new" lifecycle; then
61+
>&2 echo 'virsh event timed out!'
62+
exit 1
63+
fi
64+
65+
>&2 echo "[*] Taking zvol snapshot: $SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new@ready"
66+
zfs snapshot "$SERVO_CI_ZFS_CLONE_PREFIX/$image_name.new@ready"
67+
68+
# TODO: check that servo was actually built correctly
69+
70+
>&2 echo '[*] Done!'

0 commit comments

Comments
 (0)