feat(EXC-1895): Improved GuestOS IO through larger dm_crypt sector size

[alexandru-uta]

This PR increases the dm_crypt sector size to 4KiB instead of the default 512 bytes. This brings a ~5X IO performance improvement in read performance, and ~3X in write performance.

Notes:

• the one important change is in setup_encryption.sh
• the other changes are such that the changes can be tested using ict testnet ...

Example fio output note the 16-17 GB/s IO throughput (compared to a much lower value on master):

root@hjh3j-22lsx-umjop-755u7-u5yww-dbhwk-ebnmo-rixxv-mthik-bpife-zqe:/var/lib/ic/data/ic_state# fio --filename=test1 --io_size=10GB --size=160GB --direct=1 --buffered=0 --invalidate 1 --norandommap --random_generator=lfsr --rw=read --bs=4m --ioengine=libaio --iodepth=64 --numjobs=16 --group_reporting --name=throughput-sw-job --eta-newline=1 | tee /var/lib/ic/data/perflog/fio_$(date "+%Y%m%d_%H%M%S").log
tee: /var/lib/ic/data/perflog/fio_20250212_114257.log: No such file or directory
throughput-sw-job: (g=0): rw=read, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=libaio, iodepth=64
...
fio-3.36
Starting 16 processes
throughput-sw-job: Laying out IO file (1 file / 163840MiB)

throughput-sw-job: (groupid=0, jobs=16): err= 0: pid=3671: Wed Feb 12 11:45:58 2025
  read: IOPS=4243, BW=16.6GiB/s (17.8GB/s)(160GiB/9653msec)
    slat (usec): min=224, max=49331, avg=659.03, stdev=1051.60
    clat (usec): min=1340, max=3275.6k, avg=226686.28, stdev=511217.95
     lat (usec): min=1624, max=3276.0k, avg=227345.30, stdev=511219.12
    clat percentiles (msec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    6], 20.00th=[    8],
     | 30.00th=[   10], 40.00th=[   14], 50.00th=[   18], 60.00th=[   26],
     | 70.00th=[   47], 80.00th=[  165], 90.00th=[  995], 95.00th=[ 1737],
     | 99.00th=[ 1871], 99.50th=[ 1921], 99.90th=[ 2333], 99.95th=[ 2769],
     | 99.99th=[ 3272]
   bw (  MiB/s): min= 1878, max=54283, per=100.00%, avg=17688.14, stdev=867.52, samples=276
   iops        : min=  459, max=13567, avg=4418.18, stdev=216.92, samples=276
  lat (msec)   : 2=0.06%, 4=3.23%, 10=27.10%, 20=23.44%, 50=16.95%
  lat (msec)   : 100=5.31%, 250=7.26%, 500=4.29%, 750=1.35%, 1000=1.05%
  lat (msec)   : 2000=9.77%, >=2000=0.20%
  cpu          : usr=0.41%, sys=16.81%, ctx=21339, majf=0, minf=1048747
  IO depths    : 1=0.1%, 2=0.1%, 4=0.2%, 8=0.3%, 16=0.6%, 32=1.2%, >=64=97.5%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=40960,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=16.6GiB/s (17.8GB/s), 16.6GiB/s-16.6GiB/s (17.8GB/s-17.8GB/s), io=160GiB (172GB), run=9653-9653msec

Disk stats (read/write):
    dm-4: ios=41883/599, sectors=335462400/4576, merge=0/0, ticks=8831634/32442, in_queue=8864076, util=98.83%, aggrios=1311249/998, aggsectors=335544320/7368, aggrmerge=0/0, aggrticks=272508461/45980, aggrin_queue=272554441, aggrutil=98.29%
    dm-1: ios=1311249/998, sectors=335544320/7368, merge=0/0, ticks=272508461/45980, in_queue=272554441, util=98.29%, aggrios=163989/866, aggsectors=335544320/7456, aggrmerge=1147260/144, aggrticks=544100/582, aggrin_queue=544841, aggrutil=56.82%
  vda: ios=163989/866, sectors=335544320/7456, merge=1147260/144, ticks=544100/582, in_queue=544841, util=56.82%