Foundations for BPF device integration: run-time part

[daphne-eu]

In GitLab by @pdamme on Aug 24, 2021, 14:46

We want to support IO kernels for BPF devices (computational storage). To support asynchronous IO in this context, we need to carry information on an open file between kernel calls. Thus, open/close operations are required. This is a kind of sideways access, later the compiler should support it automatically. In particular, the workflow looks as follows:

For POSIX:

• open(filename) -> File
• readCsv(File) -> Matrix
• close(File) -> void

For BPF

• open(dev) -> Target
• open(Target, filename) -> Descriptor
• readCsv(Descriptor) -> Matrix
• close(Target) -> void

The device in the open-call is essentially a string.
There will be different kinds of descriptors, based on the underlying hardware and techniques. In that sense, File could be considered a variant of a Descriptor, or at least they could have a common base class, such that an IO kernel (e.g. readCsv always takes the same kind of input).

Thanks @niclashedam for bringing up this topic and analyzing the requirements.

---

This issue is about the DaphneDSL/DaphneIR/compiler integration. We need

• data structures for Target
• data structures for Descriptors (potentially in a class hierarchy with File)
• kernels for open and close
• to move ReadCsv.h to src/runtime/local/kernels to make it accessible to kernel pre-compilation
• to register all those kernels in kernels.json

This issue is closely related to #107 on the compiler part.

[daphne-eu]

In GitLab by @pdamme on Aug 24, 2021, 15:48

Hi @niclashedam, there is now an initial (and slightly restricted, see the commit message 2337047) implementation of the compiler integration. You can now write something like this in DaphneDSL:

f = openFile("/some/file");
m = readCsv(f, 3, 2, ",");
close(f);

t = openDevice("/dev/some/device");
d = openFileOnTarget(t, "/some/file");
m = readCsv(d, 3, 2, ",");
close(t);

Surely it could look nicer. We can do this next week after the paper deadline. But this should already provide you a good starting point. Let me know if there are urgent problems.

[daphne-eu]

In GitLab by @pdamme on Aug 24, 2021, 15:51

Your next steps could be to implement the following kernels (plus the required data structures):

• openFile
• openDevice
• openFileOnTarget
• close with specializations for File and Target
• readCsv with specializations for File and Descriptor (partly exising)

All kernels must be in src/runtime/local/kernels and registered in kernels.json. The data structures should be in src/runtime/local/io.

[daphne-eu]

In GitLab by @niclashedam on Aug 26, 2021, 10:09

Sounds good. I may extend your functions a bit to allow for some extra flexibility. In some circumstances, it makes sense for a Descriptor to take multiple filenames (a local BPF file and a target path/URI).