Inter-Process Communication
Process isolation is the feature that makes an operating system safe, and the obstacle that makes cooperation hard. IPC is the set of controlled channels the kernel provides so isolated processes can exchange data on purpose rather than by accident.
Why Isolation Forces the Question
Two processes cannot read each other's memory. That is not a limitation to work around — it is the guarantee that a bug in one program cannot corrupt another, enforced by the MMU on every single memory access.
So inter process communication operating system support — IPC — is the deliberate exception: a set of channels where the kernel mediates an exchange it has verified both sides consented to. Every mechanism below is a different answer to the same question — how do you move bytes across an isolation boundary without discarding the isolation.
The mechanisms differ along four axes worth holding in mind. Does it preserve message boundaries or deliver a raw byte stream? Does data get copied through the kernel or shared in place? Does it work between unrelated processes, or only between a parent and its children? And does it survive the machine boundary?
Threads, by contrast, need none of this. They already share an address space, so they communicate by writing a variable — which is exactly why they need locks and processes do not.
- The MMU blocks cross-process access on every instruction
- IPC is a mediated, consented exception to that rule
- Axes: boundaries, copying, relatedness, network reach
- Threads skip IPC and pay in synchronisation instead
Pipes and FIFOs
A pipe is a unidirectional byte stream with a kernel buffer between the two ends, typically 64 KB on Linux. pipe() returns two descriptors: read from one, write to the other. It is the mechanism behind ls | wc -l, where the shell creates the pipe, forks, and wires one process's stdout to the other's stdin.
The flow control is automatic and is the elegant part. Writing to a full pipe blocks the writer; reading an empty pipe blocks the reader. A fast producer cannot overwhelm a slow consumer, and neither needs to know the other's speed — the buffer plus blocking is a complete producer-consumer solution with no code from you.
Ordinary pipes only work between related processes, because the descriptors have to be inherited across fork. A named pipe or FIFO fixes this by giving the pipe a filesystem path: mkfifo /tmp/chan lets any two unrelated processes open it by name.
The limitation is that a pipe is a byte stream with no message boundaries. Write 100 bytes then 50, and the reader may receive 150 in one read, or 3 then 147. Any record structure has to be imposed by you — length prefixes or delimiters — which is precisely the framing work message queues do for you.
- Unidirectional byte stream over a ~64 KB kernel buffer
- Blocking on full and empty gives free flow control
- Ordinary pipes need a shared ancestor; FIFOs need only a path
- No message boundaries — you must frame the data yourself
Terms, operations, and practical uses
Stream channels
- PipeUnidirectional byte stream over a ~64 KB kernel buffer.
- FIFOA named pipe, so unrelated processes can open it by path.
- Flow controlBlocking on full and empty solves producer-consumer for free.
- No boundariesA stream has no records — you must frame the data.
Shared memory
- MappingOne physical region mapped into several address spaces.
- Zero copyThe only mechanism that never copies through the kernel.
- Your problemThe kernel supplies no synchronisation at all.
Message channels
- Message queueDiscrete messages with boundaries and priorities.
- Unix socketLocal, and faster than TCP loopback — no network stack.
- SignalNotification, not data; handlers must be async-signal-safe.
Three channels, three different guarantees
# Three IPC styles, modelled to expose the difference that matters:
# what each does to message boundaries, and whether it copies.
pipe_buf = []
def pipe_write(data): # byte stream: boundaries are lost on the way in
pipe_buf.extend(data)
pipe_write("hello")
pipe_write("world")
reads = 0
while pipe_buf: # 10 bytes come back in chunks of 4, not as 2 writes
del pipe_buf[:4]
reads += 1
shm = {"value": 0} # shared memory: mapped, so nothing is copied
shm["value"] = 42
copies = 0
queue = ["hello", "world"] # message queue: each send stays one message
print(f"pipe: {reads} reads for 2 writes | shm: {copies} copies "
f"| queue: {len(queue)} messages")#include <deque>
#include <iostream>
#include <string>
#include <vector>
using namespace std;
int main() {
// Three IPC styles, modelled to expose the difference that matters:
// what each does to message boundaries, and whether it copies.
deque<char> pipeBuf;
auto pipeWrite = [&](const string& data) { // byte stream: boundaries lost
for (char c : data) pipeBuf.push_back(c);
};
pipeWrite("hello");
pipeWrite("world");
int reads = 0;
while (!pipeBuf.empty()) { // 10 bytes come back in 4s, not as 2 writes
for (int i = 0; i < 4 && !pipeBuf.empty(); i++) pipeBuf.pop_front();
reads++;
}
int shmValue = 0; // shared memory: mapped, so nothing is copied
shmValue = 42;
int copies = 0;
vector<string> queue = {"hello", "world"}; // each send stays one message
cout << "pipe: " << reads << " reads for 2 writes | shm: " << copies
<< " copies | queue: " << queue.size() << " messages\n";
}import java.util.ArrayDeque;
import java.util.Deque;
import java.util.List;
class Main {
public static void main(String[] args) {
// Three IPC styles, modelled to expose the difference that matters:
// what each does to message boundaries, and whether it copies.
Deque<Character> pipeBuf = new ArrayDeque<>();
for (String data : new String[]{"hello", "world"}) // boundaries lost
for (char c : data.toCharArray()) pipeBuf.add(c);
int reads = 0;
while (!pipeBuf.isEmpty()) { // 10 bytes come back in 4s, not as 2 writes
for (int i = 0; i < 4 && !pipeBuf.isEmpty(); i++) pipeBuf.poll();
reads++;
}
int shmValue = 42; // shared memory: mapped, nothing copied
int copies = 0;
List<String> queue = List.of("hello", "world"); // one message each
System.out.println("pipe: " + reads + " reads for 2 writes | shm: "
+ copies + " copies | queue: " + queue.size() + " messages");
}
}two 5-byte writes, one shared value, two messagespipe: 3 reads for 2 writes | shm: 0 copies | queue: 2 messagesRun the example step by step
Shared Memory
Shared memory maps one region of physical memory into two or more address spaces. After the setup call — shm_open plus mmap, or shmget/shmat in the older System V API — each process reads and writes it with ordinary loads and stores.
It is the fastest IPC there is, and the reason is structural: every other mechanism copies data from the sender's buffer into the kernel and out again into the receiver's. Shared memory copies nothing. Sending a megabyte costs a pointer dereference. This is what databases, video pipelines, and high-frequency trading systems use.
The price is that the kernel now provides no synchronisation whatsoever. Two processes writing the same region race exactly like two unsynchronised threads, and you must supply the coordination yourself — usually a semaphore or a mutex placed inside the shared region itself and marked process-shared.
That inversion is the thing to remember. Pipes and message queues give you synchronisation for free and charge you a copy; shared memory gives you the speed and charges you the correctness problem.
- One physical region mapped into several address spaces
- The only mechanism with no copy through the kernel
- Kernel supplies zero synchronisation — that is now your job
- Locks must live inside the shared region and be process-shared
Message Queues, Sockets, and Signals
A message queue stores discrete messages rather than a byte stream, so a 100-byte send is received as exactly one 100-byte message. POSIX queues (mq_open) add priorities, so an urgent message can jump ahead. The framing pipes lack is the whole selling point, paid for with a kernel copy in each direction.
Sockets are the only mechanism that does not care whether the peer is on this machine. A Unix domain socket connects processes on one host and is markedly faster than a TCP loopback connection because it skips the entire network stack; the same API over TCP reaches another machine. That portability is why sockets dominate service-to-service communication, and why containers and daemons expose Unix sockets rather than pipes.
Signals are not really data transfer — they carry a single number and interrupt the target's execution. SIGTERM, SIGINT, SIGCHLD are notification, not communication. Signal handlers run asynchronously between any two instructions, so only async-signal-safe functions may be called inside them; calling printf from a handler is a genuine and common bug.
Choosing is usually mechanical. Related processes, streaming data, no framing needs — a pipe. Unrelated processes, discrete records — a message queue or Unix socket. Large volume, latency critical, willing to own the locking — shared memory. Might cross a machine — sockets, always.
| Mechanism | Boundaries | Copies | Needs relation? | Crosses machines |
|---|---|---|---|---|
| Pipe | No — byte stream | 2 (in and out) | Yes, shared ancestor | No |
| FIFO / named pipe | No — byte stream | 2 | No, opened by path | No |
| Shared memory | N/A — raw memory | 0 | No | No |
| Message queue | Yes, preserved | 2 | No | No |
| Unix socket | Either mode | 2 | No | No |
| TCP socket | Yes, stream or datagram | 2 + network stack | No | Yes |
| Signal | Number only, no data | — | No | No |
- Message queues preserve boundaries and support priorities
- Unix sockets beat TCP loopback by skipping the network stack
- Signals are notification; handlers must be async-signal-safe
- Pick by framing needs, relatedness, volume, and network reach