v811_spc009/external/libwebsockets/lib/misc/threadpool

Threadpool

Overview

An api that lets you create a pool of worker threads, and a queue of tasks that are bound to a wsi. Tasks in their own thread synchronize communication to the lws service thread of the wsi via LWS_CALLBACK_SERVER_WRITEABLE and friends.

Tasks can produce some output, then return that they want to "sync" with the service thread. That causes a LWS_CALLBACK_SERVER_WRITEABLE in the service thread context, where the output can be consumed, and the task told to continue, or completed tasks be reaped.

ALL of the details related to thread synchronization and an associated wsi in the lws service thread context are handled by the threadpool api, without needing any pthreads in user code.

Example

https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-server/minimal-ws-server-threadpool

Lifecycle considerations

Tasks vs wsi

Although all tasks start out as being associated to a wsi, in fact the lifetime of a task and that of the wsi are not necessarily linked.

You may start a long task, eg, that runs atomically in its thread for 30s, and at any time the client may close the connection, eg, close a browser window.

There are arrangements that a task can "check in" periodically with lws to see if it has been asked to stop, allowing the task lifetime to be related to the wsi lifetime somewhat, but some tasks are going to be atomic and longlived.

For that reason, at wsi close an ongoing task can detach from the wsi and continue until it ends or understands it has been asked to stop. To make that work, the task is created with a cleanup callback that performs any freeing independent of still having a wsi around to do it... the task takes over responsibility to free the user pointer on destruction when the task is created.

Reaping completed tasks

Once created, although tasks may run asynchronously, the task itself does not get destroyed on completion but added to a "done queue". Only when the lws service thread context queries the task state with lws_threadpool_task_status() may the task be reaped and memory freed.

This is analogous to unix processes and wait().

If a task became detached from its wsi, then joining the done queue is enough to get the task reaped, since there's nobody left any more to synchronize the reaping with.

User interface

The api is declared at https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-threadpool.h

Threadpool creation / destruction

The threadpool should be created at program or vhost init using lws_threadpool_create() and destroyed on exit or vhost destruction using first lws_threadpool_finish() and then lws_threadpool_destroy().

Threadpools should be named, varargs are provided on the create function to facilite eg, naming the threadpool by the vhost it's associated with.

Threadpool creation takes an args struct with the following members:

Member	function
threads	The maxiumum number of independent threads in the pool
max_queue_depth	The maximum number of tasks allowed to wait for a place in the pool

Task creation / destruction

Tasks are created and queued using lws_threadpool_enqueue(), this takes an args struct with the following members

Member	function
wsi	The wsi the task is initially associated with
user	An opaque user-private pointer used for communication with the lws service thread and private state / data
task	A pointer to the function that will run in the pool thread
cleanup	A pointer to a function that will clean up finished or stopped tasks (perhaps freeing user)

Tasks also should have a name, the creation function again provides varargs to simplify naming the task with string elements related to who started it and why.

The task function itself

The task function receives the task user pointer and the task state. The possible task states are

State	Meaning
LWS_TP_STATUS_QUEUED	Task is still waiting for a pool thread
LWS_TP_STATUS_RUNNING	Task is supposed to do its work
LWS_TP_STATUS_SYNCING	Task is blocked waiting for sync from lws service thread
LWS_TP_STATUS_STOPPING	Task has been asked to stop but didn't stop yet
LWS_TP_STATUS_FINISHED	Task has reported it has completed
LWS_TP_STATUS_STOPPED	Task has aborted

The task function will only be told LWS_TP_STATUS_RUNNING or LWS_TP_STATUS_STOPPING in its status argument... RUNNING means continue with the user task and STOPPING means clean up and return LWS_TP_RETURN_STOPPED.

If possible every 100ms or so the task should return LWS_TP_RETURN_CHECKING_IN to allow lws to inform it reasonably quickly that it has been asked to stop (eg, because the related wsi has closed), or if it can continue. If not possible, it's okay but eg exiting the application may experience delays until the running task finishes, and since the wsi may have gone, the work is wasted.

The task function may return one of

Return	Meaning
LWS_TP_RETURN_CHECKING_IN	Still wants to run, but confirming nobody asked him to stop. Will be called again immediately with LWS_TP_STATUS_RUNNING or LWS_TP_STATUS_STOPPING
LWS_TP_RETURN_SYNC	Task wants to trigger a WRITABLE callback and block until lws service thread restarts it with lws_threadpool_task_sync()
LWS_TP_RETURN_FINISHED	Task has finished, successfully as far as it goes
LWS_TP_RETURN_STOPPED	Task has finished, aborting in response to a request to stop

The SYNC or CHECKING_IN return may also have a flag LWS_TP_RETURN_FLAG_OUTLIVE applied to it, which indicates to threadpool that this task wishes to remain unstopped after the wsi closes. This is useful in the case where the task understands it will take a long time to complete, and wants to return a complete status and maybe close the connection, perhaps with a token identifying the task. The task can then be monitored separately by using the token.

Synchronizing

The task can choose to "SYNC" with the lws service thread, in other words cause a WRITABLE callback on the associated wsi in the lws service thread context and block itself until it hears back from there via lws_threadpool_task_sync() to resume the task.

This is typically used when, eg, the task has filled its buffer, or ringbuffer, and needs to pause operations until what's done has been sent and some buffer space is open again.

In the WRITABLE callback, in lws service thread context, the buffer can be sent with lws_write() and then lws_threadpool_task_sync() to allow the task to fill another buffer and continue that way.

If the WRITABLE callback determines that the task should stop, it can just call lws_threadpool_task_sync() with the second argument as 1, to force the task to stop immediately after it resumes.

The cleanup function

When a finished task is reaped, or a task that become detached from its initial wsi completes or is stopped, it calls the .cleanup function defined in the task creation args struct to free anything related to the user pointer.

With threadpool, responsibility for freeing allocations used by the task belongs strictly with the task, via the .cleanup function, once the task has been enqueued. That's different from a typical non-threadpool protocol where the wsi lifecycle controls deallocation. This reflects the fact that the task may outlive the wsi.

Protecting against WRITABLE and / or SYNC duplication

Care should be taken than data prepared by the task thread in the user priv memory should only be sent once. For example, after sending data from a user priv buffer of a given length stored in the priv, zero down the length.

Task execution and the SYNC writable callbacks are mutually exclusive, so there is no danger of collision between the task thread and the lws service thread if the reason for the callback is a SYNC operation from the task thread.

Thread overcommit

If the tasks running on the threads are ultimately network-bound for all or some of their processing (via the SYNC with the WRITEABLE callback), it's possible to overcommit the number of threads in the pool compared to the number of threads the processor has in hardware to get better occupancy in the CPU.