2. Resource Graph

A resource graph contains a snapshot of the state of a cluster. This normalizes machine-specific information which common across jarvis repos. Resource graphs provide query interfaces which avoid having to repeat the same paths all over the place. This section describes the contents of a resource graph and the API available to Jarvis repos.

1.1. Resource Graph Contents

The resource graph contains information about the cluster hardware:

1. Block devices
2. Device partitions
3. Filesystem mount points
4. Capacities
5. Network protocols

1.2. Building a Resource Graph Automatically

Most of the information regarding resource graphs can be introspected. The main things that are not automatic:

1. Hostfile: which machines are we introspecting?
2. What is the per-user mount point of the storage devices?

jarvis resource-graph create \
  --hostfile=hostfile.txt

The resource graph will be stored under ${JARVIS_ROOT}/config/resource_graph.yaml

1.2. Storage Graph

The following command lists all block devices, their type, and their mount points:

lsblk -o NAME,SIZE,MODEL,TRAN,MOUNTPOINT

The following command lists all mounted filesystems and their capacities:

df -h

The following command lists all SPDK nvmes:

spdk_nvme list -c

1.3. Network Graph

To build the network graph, we collect the outputs from the following command:

fi_info

This will store the network information available per host.

1.4. CPU Graph

CPU information can be helpful for determining information such as the number of threads to use for a metadata service.

1.5. Memory Graph

Memory information can be useful for determining things like cache sizes.

1.6. Querying the Resource Graph

from jarvis_cd.resource_graph import ResourceGraph

rg = ResourceGraph()
rg.storage.query(type='nvme', count=1).mounts()

1.7. Future Ideas

• What if we have a smart NIC?
• What if we have FPGAs and ASICs?