Semaphores & Condition Variables

Mutexes solve the problem of mutual exclusion—ensuring that only one thread accesses a shared resource at a time. But what about problems that require more expressive synchronization? What if you need to coordinate a fixed number of concurrent workers, signal from one thread to another that a condition has been met, or implement a producer-consumer pipeline where multiple threads process items at different rates?

This is where semaphores and condition variables come in. These are the higher-level synchronization primitives that let you express coordination patterns beyond simple mutual exclusion. Understanding them is essential for building any real concurrent system.

Overview

A semaphore is an integer counter that supports two atomic operations: wait (decrement) and signal (increment). When a thread calls wait and the counter is greater than zero, it decrements the counter and continues. When the counter is zero, the thread blocks until another thread calls signal. The key property that makes this work is that both operations are atomic—no two threads can simultaneously decrement past each other and miss a signal.

A condition variable is a synchronization primitive that lets a thread wait for an arbitrary condition (expressed as a boolean expression) to become true. Unlike a semaphore, which has only a counter, a condition variable is associated with a mutex and is used to wait for state changes that other threads control. You wait() on a condition variable while holding a lock, and another thread signal() or broadcast() when the state changes.

When to Use / When Not to Use

Use semaphores when: you need to limit concurrent access to a resource that can be safely used by N threads simultaneously (a counting semaphore with value N). Classic use cases include limiting the number of simultaneous connections to a database pool, restricting parallel file I/O operations, and implementing producer-consumer patterns where a bounded buffer has a fixed capacity.

Use condition variables when: you need to wait for a condition that depends on shared state. You must hold the associated mutex while calling wait(). The pattern is: acquire mutex, check condition, wait if false (atomically releasing the mutex and going to sleep), wake up, re-acquire mutex, check condition again. This is the standard monitor pattern used in nearly every multi-threaded coordination scenario.

Do not use a semaphore as a mutex. A binary semaphore (value 0 or 1) can be used for mutual exclusion, but it lacks the guarantee that the same thread that acquired the lock will be the one to release it. Mutexes enforce ownership semantics; semaphores do not. Using a semaphore as a mutex leads to subtle correctness bugs.

Do not use condition variables without a predicate. The spurious wakeup problem means that wait() can return even though no thread called signal(). Your condition check must be in a while loop, not an if statement.

Architecture or Flow Diagram

graph TD
    A[Producer puts item in buffer] --> B{Buffer full?}
    B -->|yes| C[Wait on semaphore empty]
    B -->|no| D[Put item in buffer]
    C --> E[Decrement empty count<br/>Sleep until signaled]
    D --> F[Increment fill count]
    F --> G[Signal full to wake Consumer]
    G --> H[Consumer woken]
    H --> I[Wait on full semaphore]
    I --> J[Decrement fill count]
    J --> K[Get item from buffer]
    K --> L[Signal empty to wake Producer]

The producer-consumer pattern with a bounded buffer: the empty semaphore tracks how many empty slots exist (initialized to N), and the full semaphore tracks how many filled slots exist (initialized to 0). When the buffer is full, producers wait on empty and sleep. When consumers remove items, they signal on empty to wake producers.

Core Concepts

Counting Semaphores

A semaphore wraps an integer counter with two atomic operations:

• wait() (or acquire()): If counter > 0, decrement and return immediately. If counter == 0, block until another thread increments.
• signal() (or release()): Increment the counter. If any threads are waiting, wake one of them.

With counter value 1, you have a binary semaphore—works like a mutex but without ownership semantics. With counter value N, you have a counting semaphore allowing up to N concurrent accesses.

Producer-Consumer with Bounded Buffer:

#include <semaphore.h>
#include <pthread.h>

#define BUFFER_SIZE 10
#define NUM_ITEMS 100

typedef struct {
    sem_t empty;    // Counts empty slots
    sem_t full;     // Counts filled slots
    pthread_mutex_t mutex;
    int buffer[BUFFER_SIZE];
    int in;
    int out;
} bounded_buffer_t;

void buffer_init(bounded_buffer_t *b) {
    sem_init(&b->empty, 0, BUFFER_SIZE);  // All slots empty initially
    sem_init(&b->full, 0, 0);             // No slots full initially
    pthread_mutex_init(&b->mutex, NULL);
    b->in = 0;
    b->out = 0;
}

void buffer_produce(bounded_buffer_t *b, int item) {
    sem_wait(&b->empty);                  // Wait for empty slot
    pthread_mutex_lock(&b->mutex);
    b->buffer[b->in] = item;
    b->in = (b->in + 1) % BUFFER_SIZE;
    pthread_mutex_unlock(&b->mutex);
    sem_post(&b->full);                   // Signal that a slot is now full
}

int buffer_consume(bounded_buffer_t *b) {
    sem_wait(&b->full);                   // Wait for full slot
    pthread_mutex_lock(&b->mutex);
    int item = b->buffer[b->out];
    b->out = (b->out + 1) % BUFFER_SIZE;
    pthread_mutex_unlock(&b->mutex);
    sem_post(&b->empty);                  // Signal that a slot is now empty
    return item;
}

The mutex protects the circular buffer data structure. The two semaphores track resource availability (empty slots vs. full slots) and handle blocking/waking without requiring the producer to poll.

Condition Variables

A condition variable is always paired with a mutex. The pattern:

1. Acquire the mutex
2. Check the condition (in a while loop!)
3. Call pthread_cond_wait() which atomically releases the mutex and goes to sleep
4. When woken, re-acquire the mutex and re-check the condition

#include <pthread.h>

typedef struct {
    pthread_mutex_t mutex;
    pthread_cond_t cond;
    int item_available;
    int shutdown_requested;
} shared_state_t;

void *consumer(void *arg) {
    shared_state_t *s = arg;
    pthread_mutex_lock(&s->mutex);
    while (!s->item_available && !s->shutdown_requested) {
        pthread_cond_wait(&s->cond, &s->mutex);  // Atomically: release mutex, sleep
    }
    if (s->shutdown_requested) {
        pthread_mutex_unlock(&s->mutex);
        return NULL;
    }
    int item = s->item_available;  // Consume the item
    s->item_available = 0;
    pthread_mutex_unlock(&s->mutex);
    return (void *)(intptr_t)item;
}

void producer_signal(shared_state_t *s, int item) {
    pthread_mutex_lock(&s->mutex);
    s->item_available = item;
    pthread_mutex_unlock(&s->mutex);
    pthread_cond_signal(&s->cond);  // Wake ONE waiting thread (broadcast for all)
}

Why the while loop? POSIX guarantees spurious wakeups—wait() can return without signal() being called. This happens because the kernel’s futex implementation cannot always distinguish “this thread was genuinely signaled” from “this thread was kicked out of a futex queue for some other reason.” The correct pattern is always while (!condition) pthread_cond_wait(...), never if (!condition).

Signal vs. Broadcast

• signal() (or pthread_cond_signal()): Wakes exactly one waiting thread. If multiple threads are waiting, the scheduler picks one—no guaranteed order.
• broadcast() (or pthread_cond_broadcast()): Wakes all waiting threads. Use when the condition change affects all waiters (e.g., “the shared resource has been updated, everyone should re-check”).

A common mistake: using signal() when the state change means “all waiters should proceed” (e.g., a shutdown notification). If multiple consumers should stop, use broadcast.

Production Failure Scenarios + Mitigations

The Semaphore Initialization Bug (Linux 2.6.32)

In some kernel versions, a race condition existed in the semaphore wakeup path that could cause a waiting thread to sleep indefinitely even after up() was called. The bug was in the interaction between the spinlock protecting the wait queue and the wakeup logic. It manifested as threads appearing to hang with 100% CPU usage (spinning in the sleep path). The fix involved reordering operations in the semaphore release path to ensure the wakeup was delivered before the semaphore counter was incremented.

Mitigation: Always use well-tested semaphore implementations from your language’s standard library. Do not implement your own semaphore primitives from atomics—the edge cases (especially wakeup ordering) are subtle and have caused real-world bugs in production kernels.

The Missing Broadcast in Connection Pool Shutdown

A connection pool implementation used a condition variable to signal when a connection became available. During shutdown, signal() was called to wake a single waiter. But three threads were waiting for connections, and one of them had the shutdown flag set. The signal went to the wrong thread, the shutdown thread remained blocked, and the pool’s shutdown method deadlocked waiting for all threads to terminate.

Mitigation: Use broadcast() for shutdown notifications so that all waiters see the shutdown request regardless of which one is chosen by the scheduler. Always validate the condition after waking—never assume the signal reached your specific thread for your specific reason.

The Lost Wakeup in Python threading.Event

Python’s threading.Event is implemented using a condition variable plus a boolean flag. A common user error: checking event.is_set() in a while loop, then calling event.wait() with a timeout. If the event is set between the check and the wait, the wait blocks indefinitely (the event was already “consumed” by the check). This is not a library bug—it’s a semantic subtlety in how events work vs. condition variables.

Mitigation: If you need persistent signaling (meaning the event stays set and all waiters see it), use threading.Event which maintains state. If you need transient signaling (notify waiters of a state transition), use a condition variable with a separate state variable you control.

Trade-off Table

Aspect	Counting Semaphore	Binary Semaphore	Condition Variable + Mutex
Use case	Limited resource pool (N workers)	Simple signaling / mutual exclusion	Arbitrary condition coordination
Blocking semantics	Blocks on decrement when count=0	Blocks when trying to “acquire” set semaphore	Blocks on wait until signal/broadcast
Ownership enforced	No — any thread can signal	No — any thread can signal	Yes — must hold associated mutex
Broadcast capability	Via post to all waiters (complex)	Via multiple signals	Native via broadcast()
State persistence	Counter persists until reset	State persists (set/clear)	State is external (your boolean)
Complexity	Low	Low	Medium

Implementation Snippets

Python: Semaphore-Based Thread Pool

import threading
import queue

class ThreadPool:
    def __init__(self, num_workers):
        self._workers = num_workers
        self._task_queue = queue.Queue()
        self._semaphore = threading.Semaphore(num_workers)
        self._threads = []

    def submit(self, task, *args):
        self._task_queue.put((task, args))

    def start(self):
        for _ in range(self._workers):
            t = threading.Thread(target=self._worker)
            t.start()
            self._threads.append(t)

    def _worker(self):
        while True:
            task, args = self._task_queue.get()
            try:
                task(*args)
            finally:
                self._task_queue.task_done()

    def shutdown(self):
        self._task_queue.join()  # Wait for all tasks to complete

Python: Condition Variable for Producer-Consumer

import threading

class BlockingQueue:
    def __init__(self, maxsize=0):
        self._mutex = threading.Lock()
        self._not_empty = threading.Condition(self._mutex)
        self._not_full = threading.Condition(self._mutex)
        self._queue = []
        self._maxsize = maxsize

    def put(self, item, timeout=None):
        with self._not_full:
            while len(self._queue) >= self._maxsize:
                if not self._not_full.wait(timeout=timeout):
                    return False  # Timed out
            self._queue.append(item)
            self._not_empty.notify()  # Wake one consumer
        return True

    def get(self, timeout=None):
        with self._not_empty:
            while not self._queue:
                if not self._not_empty.wait(timeout=timeout):
                    return None  # Timed out
            item = self._queue.pop(0)
            self._not_full.notify()  # Wake one producer
        return item

C: Condition Variable with Predicate

#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>

typedef struct {
    pthread_mutex_t mutex;
    pthread_cond_t cond;
    int counter;
    int done;
} shared_data_t;

void wait_for_count(shared_data_t *s, int target) {
    pthread_mutex_lock(&s->mutex);
    while (s->counter < target) {  // ALWAYS while, never if!
        pthread_cond_wait(&s->cond, &s->mutex);
    }
    pthread_mutex_unlock(&s->mutex);
}

void increment_and_signal(shared_data_t *s) {
    pthread_mutex_lock(&s->mutex);
    s->counter++;
    if (s->counter >= 10) {
        s->done = 1;
        pthread_cond_broadcast(&s->cond);  // Wake all waiters
    }
    pthread_mutex_unlock(&s->mutex);
}

Observability Checklist

• Semaphore wait time: How long threads spend blocked on sem_wait(). Alert if p99 > 10ms for production systems
• Queue depth histogram: For producer-consumer patterns, track buffer occupancy to detect producer starvation or consumer overload
• Signal-to-wake latency: Time between signal() call and waiting thread resuming. High latency indicates scheduler pressure
• Spurious wakeup rate: Count how often wait() returns without a corresponding signal(). High rates suggest incorrect condition variable usage
• Thread count: Track how many threads are in a wait state vs. running state
• Lock order inversion: Use tools like helgrind or ThreadSanitizer to detect lock order violations that can lead to deadlock with condition variables

Security/Compliance Notes

Resource exhaustion via semaphore: If an untrusted process can call sem_wait() on a semaphore that is only incremented by a trusted process, a malicious actor could decrement the counter to zero and cause denial of service for all threads depending on it. Use proper permissions on named semaphores and never expose semaphore file descriptors to untrusted code.

Condition variable race on shutdown: When shutting down a multi-threaded system, ensure all condition variable waits are terminated before the associated mutex is destroyed. Destroying a mutex while threads are waiting on its condition variable is undefined behavior and can lead to crashes.

For compliance: any synchronization primitive protecting access to sensitive data (PII, financial records) must be documented. Use static analysis to find missing signal/broadcast calls on condition variables—threads waiting on an unset condition will sleep forever.

Common Pitfalls / Anti-patterns

1. Using if instead of while for condition variable wait. Spurious wakeups mean wait() can return even without signal(). Always re-check the condition in a while loop: while (!condition) cv.wait().

2. Calling signal() without holding the mutex. On some implementations (including POSIX), calling pthread_cond_signal() without holding the associated mutex produces undefined behavior. Always hold the mutex when signaling.

3. Broadcasting when signaling would suffice. broadcast() forces all waiting threads to wake and re-acquire the mutex, causing contention. If only one waiter should act on the condition, use signal().

4. Forgetting to call sem_post() after sem_wait(). If a semaphore is never incremented after being decremented, all subsequent waiters block forever. Use RAII wrappers or structured scope guards for semaphore operations.

5. Using a semaphore where a condition variable is needed. A semaphore with value 1 used as a mutex has no ownership semantics—if Thread A acquires and Thread B signals, Thread B can release a lock it never owned. Use condition variables + mutex when you need stateful waiting.

Quick Recap Checklist

• Semaphores provide atomic increment/decrement on an integer counter; threads block when decrementing a zero counter
• Use counting semaphores to limit concurrent access to N resources (e.g., connection pools, worker pools)
• Condition variables always use a while loop for the condition check due to spurious wakeups
• Always hold the associated mutex when calling signal() or broadcast()
• Use broadcast() for shutdown notifications so all waiters see the signal
• Do not use a semaphore as a mutex—it lacks ownership semantics
• Semaphores can be used to implement barriers (initialize to N, wait N times)

Interview Questions

Further Reading

• Concurrency Fundamentals — The problem space and why synchronization is needed
• Mutex Implementation — How mutexes are implemented in userspace and kernel
• Semaphores — Counting semaphores for resource management
• Readers-Writer Locks — Optimizing for read-heavy workloads
• Lock-Free Structures — Advanced techniques for highly concurrent systems

Conclusion

Semaphores and condition variables extend the basic mutual exclusion model into expressive coordination primitives. Semaphores excel at managing access to a fixed pool of resources—a counting semaphore with value N enforces that no more than N threads can acquire the resource simultaneously. Condition variables solve the harder problem of waiting for arbitrary state changes that other threads control, but only when used correctly: always with a while loop for spurious wakeup protection and always while holding the associated mutex.

The producer-consumer pattern with bounded buffers demonstrates the full power of these primitives working together. Beyond that, semaphores implement barriers, readers-writers problems, and thread pools. Condition variables are the foundation of monitors, which are the dominant synchronization model in higher-level languages. When designing concurrent systems, reach for condition variables when your problem involves “wait until X becomes true” and semaphores when it involves “ensure no more than N workers access this resource simultaneously.”

For continued learning, explore the relationship between condition variables and the broader monitor pattern, study semaphore implementations in Linux (which use futexes internally), and examine how asyncio in Python implements its condition-event primitives without native CV support.