Using the threadpool for improved stack memory usage

[sirknightj]

When using the sample application ./samples/kvsWebrtcClientMaster and having the viewer repeatedly join and leave, we have observed that the stack memory grows up and up until an asymptotic limit. Using the Instruments tool, we can check on the memory usage of the application. We can see amount of heap memory and also stack memory currently used.

Here's what a run of ./samples/kvsWebrtcClientMaster looks like using the latest master branch on my M1 MacOS. Each time the orange line moves up and down, is when the viewer joins and leaves. Note the size comparison (scale) of Heap Allocations compared to the VM: Stack in the chart below.

What needs to happen when the viewer joins? In the libwebsockets (LWS) message handler, we handle incoming messages by creating a new thread, then detaching it later. Here is an example:

amazon-kinesis-video-streams-webrtc-sdk-c/src/source/Signaling/LwsApiCalls.c

Lines 361 to 367 in 0ae9ec6

	retStatus = THREAD_CREATE(&pSignalingClient->reconnecterTracker.threadId, reconnectHandler, (PVOID) pSignalingClient);
	if (STATUS_FAILED(retStatus)) {
	ATOMIC_STORE_BOOL(&pSignalingClient->reconnecterTracker.terminated, TRUE);
	CHK(FALSE, retStatus);
	}
	CHK_STATUS(THREAD_DETACH(pSignalingClient->reconnecterTracker.threadId));

(Note that THREAD_CREATE and THREAD_DETACH macros are defined in PIC.)

For newer developers, how thread detach works is that once the thread's main method exits, the thread is automatically freed. If the thread's main method is slow to resolve (e.g. encounters a failure requiring retries), it may take a while before the main method finishes and thus the stack resources are slow to be reclaimed by the OS.

In order to limit the number of total threads that get created (in other words, decrease the asymptotic maximum), we can use a threadpool.

Here is what the memory usage graph looks like on my M1 MacOS after enabling the threadpool. I started the ./samples/kvsWebrtcClientMaster and used the JS SDK test page as viewer to repeatedly join and leave. You can see how many times that occurred from the Heap Allocations graph, which goes up and down each time the viewer joins and leaves.

In the current 1.9.0 release of the Amazon Kinesis Video Streams WebRTC SDK in C, the thread pool is not enabled by default. In the next release (Q4 2023), this feature will be ON by default. You can find more information on the configuration options in the README.

If you would like to use the threadpool in the current 1.9.0 release, you can use use-threadpool.patch to enable the thread pool on the current (1.9.0) master branch. Remember to run the cmake .. and make commands after applying the patch.