Asynchronous Programming

Rationale for Async Programming

To allow for non-blocking executions, instead of letting thread sitting idle while waiting for I/O operation to complete.
This yields control back to the system to perform other tasks, enabling for concurrency.

Concepts

Threads vs Processes

Features	Thread	Process
Memory	Shared memory space	Isolated memory
Overhead	Lightweight, simple to create/ switch	Heavyweight & slow start
Communication	Direct access	Requires socket/ IPC/ Pipes
Failures	Single crash will take down the thread	Independent, does not affect other processes

Parallelism vs Concurrency

Parallelism - Executing multiple operations at the same time.
Concurrency - Spinning up multiple tasks while allowing overlapping operations.

Key Challenges

1. Data Integrity [ Mutex/ Atomic Operations ]

Race Condition (i.e. Multiple threads updating shared column)
Atomicity for operations (i.e. must be all-or-nothing)

2. Deadlocks [ Lock Ordering/ Timeout ]

Circular waits on deadlocks
Starvation for lower-priority processes

3. Coordination[ Bounded Channels ]

Shared Memory (i.e. Deadlock management) vs Event-Driven Communication
Back-pressure of memory (i.e. Producer faster than consumer)
Callback vs async/ await (i.e. akin to synchronous for async programming)

4. Performance Overhead [ Lock Ordering/ Timeout ]

Context Switching (i.e. saving state of thread)
Synchronisation Overhead (i.e. mutex & semaphores incurs latency)

5. Non-deterministic Bugs [ Structured Concurrency ]

Zombie/ Orphan processes: Resource leaks from sub-tasks running in the background, leaking memory & consuming resources.
Stack traces: Error logs are difficult to debug because the sequence of events are jumbled rather than running consecutively

Architectural Solutions for Async Programming

Synchronisation Primitives

Mutex

Lock that guarantees that only a single thread can access the data at any time

Semaphores

Allows N number of threads to access the resource simultaneously

Read-Write Locks

Allows multiple threads to read data at the same time
When writer gains access to locks, all readers are temporarily blocked

Actor Model

[Note] Instead of sharing memory, every actor has its own private state

Communication - Mailbox

Uses Mailbox for communication with immutable messages
Data Consistency - Private State

Race conditions & deadlocks are virtually impossible as no two actors will share memory

Worker-Offloading Pattern

[Note] Performing CPU-intensive tasks might freeze the event loop for each user

Offload heavy tasks to workers
Depending on the type of tasks:
- Lightweight (i.e. email sending)
  - Use fire-and-forget
- For Resource-intensive (i.e. image processing, LLM calls)
  - Spin up Thread Pool or Distributed Task Queue

Communication Sequential Processes (CSP)

[Note] Data exists as states and is sent across channels

Communication - Channels

Data sent over channels instead of relying on databases
- Buffered channel: Requires handshake before sending message
- Unbuffered channel: Can send up to N items but sender is blocked once buffer is full.
Data Consistency - Ownership Transfers

Ownership Transfer: Only a single thread owns the data, no need for complex locks