Add VDO

[JsGjKJzi]

The acap-rs workspace doesn't currently provide wrappers for the vdo library. When it is ready, I hope this merge request will provide that.

Immediate goals:

Provide safe Rust wrappers around the VDO functions, and abstract into rust structs and impls where appropriate.
Provide tests to verify functionality.
Since VDO uses peripheral hardware, I plan to build the tests and execute them on an Axis P1468-LE.

Checklist before requesting a review

• I have performed a self-review of my own code
• I have verified that the code builds perfectly fine on my local system
• I have added tests that prove my fix is effective or that my feature works
• I have commented my code, particularly in hard-to-understand areas
• I have verified that my code follows the style already available in the repository
• I have made corresponding changes to the documentation

[apljungquist]

I haven't looked at the bindings yet but this is neat alternative to cargo-acap-sdk for interacting with Cargo and a device, and it would be nice to have sooner rather than later; What do you think about dedicating a PR to it?

I implemented a slightly different take on it; I intend to remove it before merging that PR, but perhaps you like some of my ideas.

[JsGjKJzi]

If you want, I can submit a separate PR for it. Really it was just lingering, as I haven't experimented with an alternative for what I'm doing yet.

Something I don't like is using this requires sh or bash to be available which excludes windows. I'm a bit conflicted if this is abusing the runners functionality.

The command I run is usually something like cargo test --tests -p vdo -- --nocapture. Being able to exclude examples and doc tests as well as passing nocapture to the executable is important to me.

I think cargo-acap-sdk might be able to be modified to work like this. However, I haven't had time to sit down and look into it.

[apljungquist]

f you want, I can submit a separate PR for it. Really it was just lingering, as I haven't experimented with an alternative for what I'm doing yet.

I was running address sanitizer some more today and it's a massive help, so if you don't add it I think I will after all 😁

Something I don't like is using this requires sh or bash to be available which excludes windows. I'm a bit conflicted if this is abusing the runners functionality.

If it helps known contributors and doesn't get in the way of potential windows users, then I think it's OK. Especially if it only affects contributors and not downstream users of the binaries and libraries we publish. Besides, the windows users I have spoken to so far run WSL for everything except checking out code.

The command I run is usually something like cargo test --tests -p vdo -- --nocapture. Being able to exclude examples and doc tests as well as passing nocapture to the executable is important to me.

You may want to throw on --test-threads=1 as well; I found at least one of the C APIs doesn't work well when running tests in parallel (I don't remember which one).

I think cargo-acap-sdk might be able to be modified to work like this. However, I haven't had time to sit down and look into it.

This would be nice. I have thought the same, but it is not immediately clear to me how we would easily parse the arguments so I have resorted to using acap-ssh-utils directly. How to parse +nightly, needed for sanitizers, isn't obvious either.

[JsGjKJzi]

No problem. I'll submit a separate PR for that tonight Eastern US time.

I'd be interested in seeing when you've done with the address sanitizer. I was thinking about running something like valgrind to check for any memory leaks with all the pointers being passed across the Rust FFI. The cameras have a fairly locked down OS, so I haven't gone through the effort of getting valgrind set up. Maybe what you've done would help with the obvious things.

[JsGjKJzi]

@daniel-nielsen-se Does VDO or some other process invert the image? The following code works, but produces images that are upside down. Not sure if thats a bug in the library/code somehow or a characteristic of VDO or the cameras.

Although the PPM format is for interleaved data I still get an image by saving the VDO_FORMAT_PLANAR_RGB data, its just a 3x3 grayscale image.

use anyhow::Context;
use std::io::Write;
use std::{iter::Iterator, path::Path};
use vdo::{Stream, VdoFormat};

fn main() -> anyhow::Result<()> {
    let mut stream = Stream::builder()
        .channel(0)
        .format(VdoFormat::VDO_FORMAT_PLANAR_RGB)
        .resolution(1920, 1080)
        .build()
        .context("Unable to create stream")?;
    if let Ok(s) = stream.info() {
        s.dump();
    }
    let mut running_stream = stream.start().expect("failed to start stream");
    let buffer = running_stream
        .iter()
        .next()
        .expect("unable to fetch buffer from stream");
    let frame = buffer.frame().expect("failed to get frame from buffer");
    log::info!("frame size: {}", frame.size());
    log::info!("frame type: {:?}", frame.get_type());
    let buffer_slice = buffer.as_slice()?;
    let path = Path::new("test.ppm");
    let mut file = std::fs::OpenOptions::new()
        .read(true)
        .write(true)
        .create(true)
        .truncate(true)
        .open(path)
        .expect("unable to open file");
    file.write_all(b"P6\n1920 1080\n255\n")
        .expect("failed to write header");
    file.write_all(buffer_slice)
        .expect("unable to write image data");
    Ok(())
}

Also, VDO_FORMAT_RGB produces the following error on a P1468 camera. Could you help me understand why that is?

Error: Unable to create stream

Caused by:
    VDOError { code: 6, phrase: VDO_ERROR_CLOSED, message: Failed creating stream: Format 'rgb' is not supported }

[daniel-nielsen-se]

JsGjKJzi

Also, VDO_FORMAT_RGB produces the following error on a P1468 camera. Could you help me understand why that is?

Error: Unable to create stream

Caused by:
    VDOError { code: 6, phrase: VDO_ERROR_CLOSED, message: Failed creating stream: Format 'rgb' is not supported }

It is platform dependent. Basically VDO will only provide hardware accelerated formats. YUV(NV12 and Y800) however is supported by all current platforms, so if you want to write perfectly portable code, you need to first try the format that is fastest for your app, then if it doesn't work, fallback to YUV.

VDO_FORMAT_H264
VDO_FORMAT_H265 (Not Artpec-6)
VDO_FORMAT_JPEG
VDO_FORMAT_YUV
VDO_FORMAT_RGB (Ambarella CV, Artpec-9)
VDO_FORMAT_PLANAR_RGB (Ambarella CV, Artpec-8)
VDO_FORMAT_AV1 (Artpec-9)

The info comes from here
https://axiscommunications.github.io/acap-documentation/docs/api/src/api/vdostream/html/vdo-types_8h.html#a5ed136c302573571bf325c39d6d36246

[daniel-nielsen-se]

JsGjKJzi

@daniel-nielsen-se Does VDO or some other process invert the image? The following code works, but produces images that are upside down. Not sure if thats a bug in the library/code somehow or a characteristic of VDO or the cameras.

We are not affiliated with and take no responsibility for vooya, use at own risk, however it is my personal prefered application for viewing raw images.
https://www.offminor.de/downloads.html

Here you can see if you decode the image as if it was interleaved even though it is planar, it will look like your result.

then if you select planar in vooya it looks as expected.

so basically it's proof that your program is working, you only need to insert conversion code from
RRRGGGBBB to RGBRGBRGB.

When it comes to rotation, you can check the stream info if it contains a uint32_t rotation property.
'rotation'-----------------------: <uint32 0>
or it is
'rotation'-----------------------: <uint32 180>

The image can be upside down for many reasons.
a) If the rotation of your RGB is different from JPEG, then it is due to the internal hardware pipeline design of that product...
b) if the rotation of JPEG and RGB is the same. i.e. both are upside down, then it's some camera configuration.

[JsGjKJzi]

@daniel-nielsen-se

It is platform dependent. Basically VDO will only provide hardware accelerated formats. YUV(NV12 and Y800) however is supported by all current platforms, so if you want to write perfectly portable code, you need to first try the format that is fastest for your app, then if it doesn't work, fallback to YUV.

VDO_FORMAT_H264 VDO_FORMAT_H265 (Not Artpec-6) VDO_FORMAT_JPEG VDO_FORMAT_YUV VDO_FORMAT_RGB (Ambarella CV, Artpec-9) VDO_FORMAT_PLANAR_RGB (Ambarella CV, Artpec-8) VDO_FORMAT_AV1 (Artpec-9)

The info comes from here https://axiscommunications.github.io/acap-documentation/docs/api/src/api/vdostream/html/vdo-types_8h.html#a5ed136c302573571bf325c39d6d36246

Ah, I see now. Thank you for clearing that up.

When it comes to rotation, you can check the stream info if it contains a uint32_t rotation property.
'rotation'-----------------------: <uint32 0>
or it is
'rotation'-----------------------: <uint32 180>

The image can be upside down for many reasons.
a) If the rotation of your RGB is different from JPEG, then it is due to the internal hardware pipeline design of that product...
b) if the rotation of JPEG and RGB is the same. i.e. both are upside down, then it's some camera configuration.

The rotation was set to 0, but PLANAR_RGB and JPEG come out with different rotation, so it must be an internal nuance. That's fine, I was just worried it was something I was doing wrong. Thanks.

Also, thank you for the pointer to vooya. I hadn't heard about that and looks like it could be quite useful.