The task manager is built upon RabbitMQ and pocketbase and uses a subscriber-publisher model for real-time streaming. It has low-latency and high throughput for handling task feeds. The task manager is built using golang.

The task manager is used to queue tasks that takes time and can be processed asynchronously viz. sending email, processing image, running ML model on data, generating image through AI etc. It supports workers and can be scaled to handle millions of request by increasing the number of brokers.

The task data is persisted and backed-up using pocketbase. The data and task result is backed up automatically on s3 and the taskmanager exposes its functionality through rest API.