Async Programming

Rationales for Async Programming

1. To allow for non-blocking executions, instead of letting thread sitting idle while waiting for I/O operation to complete.

2. This yields control back to the system to perform other tasks, enabling for concurrency.

Concepts

1. Threads vs Processes

Features	Thread	Process
Memory	Shared memory space	Isolated memory
Overhead	Lightweight, simple to create/ switch	Heavyweight & slow start
Communication	Direct access	Requires socket/ IPC/ Pipes
Failures	Single crash will take down the thread	Independent, does not affect other processes

2. Parallelism vs Concurrency

   • Parallelism - Executing multiple operations at the same time.

   • Concurrency - Spinning up multiple tasks while allowing overlapping operations.

Key Challenges

1. Data Integrity [ Mutex/ Atomic Operations ]

   • Race Condition (i.e. Multiple threads updating shared column)

   • Atomicity for operations (i.e. must be all-or-nothing)

2. Deadlocks [ Lock Ordering/ Timeout ]

   • Circular waits on deadlocks

   • Starvation for lower-priority processes

3. Coordination [ Bounded Channels ]

   • Shared Memory (i.e. Deadlock management) vs Event-Driven Communication

   • Back-pressure of memory (i.e. Producer faster than consumer)

   • Callback vs async/ await (i.e. akin to synchronous for async programming)

4. Performance Overhead [ Lock Ordering/ Timeout ]

   • Context Switching (i.e. saving state of thread)

   • Synchronisation Overhead (i.e. mutex & semaphores incurs latency)

5. Non-deterministic Bugs [ Structured Concurrency ]

   • Zombie/ Orphan processes (i.e. resource leaks)

   • Stack traces

Architectural Solutions for Async Programming

1. Synchronisation Primitives

   • Mutex (i.e. mutually exclusive)

   • Semaphores (i.e. )

   • Read-Write Locks (i.e. )

• Actor Model

Race conditions & deadlocks are virtually impossible as no two actors will share memory.

• Isolation: Each actor has a private state

• Mailboxes: Each actor communicates solely through immutable messages to each other’s mailboxes.

• Worker-Offloading Pattern ****[Implemented]

Performing CPU-intensive tasks might freeze the event loop for each user, this work can be delegated to workers.

• Fire-and-forget - For lightweight tasks

• Distributed Task Queue - Separate process

• Separate thread pool - Separate thread pool

• Communication Sequential Processes (CSP)

Data exists as states and is sent across channels

• Channels as Pipelines: Data sent over channels instead of relying on databases

  • Buffered channel: Requires handshake before sending message

  • Unbuffered channel: Can send up to N items but sender is blocked once buffer is full.

• Ownership Transfer: Only a single thread owns the data, no need for complex locks

• Back-pressure Mechanisms

• Structured Concurrency

Task scopes to ensure that all tasks are completed.

• No more orphans: No zombie thread in the event of failures or timeouts.