♪ NOW PLAYING 0 Old Flame Like A Dream | *** thanks for stopping by my corner of the web *** best viewed at 800x600 *** sign my guestbook *** Polar Codes: Capacity by ConstructionChannel Coding Every capacity-approaching code this series has met so far runs on randomness. Turbo codes get theirs from an interleaver (a pseudo-random permutation that manufactures ensemble-like behavior inside a single code); LDPC codes get theirs from a randomly sampled sparse graph, analyzed through the ensemble it was drawn from. That was also Shannon’s own move in 1948: average over all codes, show the average is good, conclude a good code exists, and say nothing about how to find it. The series opener framed the sixty-year hunt that followed as the search for an explicit answer. In 2009, Erdal Arıkan delivered it: a deterministic construction, with no ensemble and no random draw anywhere, that provably achieves the symmetric capacity of any binary-input memoryless channel, with encoding and decoding both running in . The mechanism is a single recursive idea called channel polarization, and this page builds it from one XOR gate up to the full theorem, then confronts the honest catch that kept polar codes out of practice for another few years. Symmetric Capacity and the Bhattacharyya ParameterLet be a binary-input discrete memoryless channel (B-DMC), . Arıkan’s analysis runs on two scalar parameters of , and it matters that there are two, because they do different jobs. The first is the symmetric capacity, the mutual information between input and output when the input is equiprobable:
The second is the Bhattacharyya parameter, a reliability measure:
upper-bounds the maximum-likelihood error probability of a single use of to send one bit, so small means a reliable channel. The two parameters track each other: exactly when (a good channel), and exactly when (a useless one). Arıkan’s Proposition 1 makes the coupling precise: and . Why carry both? Because measures rate and turns out to be the quantity that polarization conserves, which is what drives the main theorem; while measures reliability, satisfies clean one-step recursions, and directly bounds error probability, which is what drives the performance theorem and the code construction. The proofs need each of them in its own role. One special case is worth memorizing before anything else. For the binary erasure channel BEC(), which delivers the input bit with probability and an erasure symbol otherwise, both parameters are available in closed form: and . The BEC is the one channel where every quantity in this page can be computed exactly, which is why every worked example uses it. Channel Combining and SplittingHere is the entire invention at size two. Take two independent copies of and, before transmitting, couple their inputs with a single XOR:
The combined two-bit channel is . Now split it back into two single-bit channels, defined so that they are decoded in order, first: Read the definitions carefully, because the asymmetry between them is the whole trick. , the channel seen by , observes both outputs but must treat as unknown interference summed over; it is worse than a plain use of . , the channel seen by , observes both outputs and receives as a known side input; it is better than . Notice what that side input assumes: gets to act as if has already been decoded correctly. This “genie” assumption is written directly into the channel definition here, and successive cancellation decoding will later have to actually earn it, one bit at a time. How much worse and how much better? Two exact statements answer this, and they are the load-bearing wall of the whole theory:
Total capacity is preserved; it is merely redistributed. Nothing is created and nothing is destroyed: capacity is sorted, moved out of one channel and into the other. If you retain a single sentence from this page, make it that one, because everything that follows (the recursion, the polarization theorem, even the punchline of the martingale proof) is that sentence applied over and over. Both statements have proofs short enough to keep in full. Conservation is nothing but the chain rule for mutual information. The map is a bijection (invert it: , ), so if are i.i.d. uniform then so are . Then
the last step because the two channel uses are independent. (The middle step uses the fact that equals the conditional term , since is independent of .) Extremization is the observation that adding observations cannot decrease mutual information:
and conservation then forces . The Bhattacharyya parameters transform just as cleanly:
with equality in the second relation if and only if is a BEC, a detail that will matter enormously when we try to construct codes for other channels. The first relation is an exact computation from the definition. The output alphabet of is the triple , so
since the inner bracket equals for either value of . All of this abstraction becomes concrete in two lines on the erasure channel. For BEC(), where is just the erasure probability: the first channel must recover , which requires both observations, so it is erased whenever either copy erases, with probability . The second channel already knows , so it can recover from either observation ( directly, or ), and is erased only when both copies erase, with probability . That two-line argument, “either erases” versus “both erase,” is the entire polarization mechanism made tangible, and it matches the two formulas exactly. Recursive Construction for Length NOne XOR moved capacity around a little. Arıkan’s next move is to do it again, recursively, on copies of : pair up channels and transform, then pair up the resulting synthetic channels and transform those, levels deep. Formally the length- combined channel is built from two length- ones,
and the split channels obey the one-step recursions you will use constantly: The clean way to picture this is a binary tree of depth . The root is itself; every node branches into a "" child (the worse transform) and a "" child (the better one). Each synthetic channel sits at a leaf, and the binary expansion of spells out the path that reaches it: index is a path through the polarization tree. A channel that took mostly "" steps has been squared toward again and again and is nearly noiseless; one that took mostly "" steps has been degraded toward and is nearly useless. The claim of the polarization theorem, proved below, is that as grows almost every leaf ends up at one extreme or the other, and the fraction landing at the good extreme is exactly . If that claim holds, the code writes itself. Choose the most reliable synthetic channels as the information set and send data on those; freeze the remaining inputs to values known in advance by both sides. The frozen bits are not overhead grudgingly accepted: they are the code. Freezing is precisely how the encoder avoids the bad synthetic channels, and the set defines the code completely. A Worked Example: Code Construction on the BECNothing makes the mechanism believable like running it. Take BEC(), so and , and recall that for the BEC both recursions are exact equalities: and . Iterating from :
Check the conservation law at every level: the row sums are , , , , always , because for the BEC the parameter is an erasure probability and erasure probability is conserved exactly. This is the arithmetic sanity check to run whenever you compute these by hand; if a row does not sum to , you made a slip. Now sort the values: . They are visibly drifting toward the two extremes and away from the middle, though at the drift is still weak, a weakness we will return to with numbers attached. Push the same exact recursion to large and the drift becomes a cliff: ![]() For a rate- code at we need the indices with the smallest :
Look at that set for a moment, because it kills a tempting misconception: is not simply “the last indices.” Index 4 () beats index 5 (), because index 4’s path through the tree ends in two "" steps while index 5’s ends in two "" steps. The reliability ordering is genuinely non-monotonic in the index, and it is channel-dependent besides. How good is this code? Theorem 3 (stated properly below) bounds the block error probability of successive cancellation decoding by the sum of the information-set Bhattacharyya parameters:
Useless at , and the honesty is the point: polarization at small block lengths has barely begun, and this bound quantifies exactly the finite-length weakness Arıkan himself reports. Finally, the encoder in matrix form. The recursive XOR structure works out to with , built from Kronecker powers of the kernel: where is the bit-reversal permutation (for it reorders rows to ). The frozen positions are set to fixed values, conventionally zeros. Two convention warnings are worth flagging right here. First, only reorders; many modern treatments and the 5G standard drop it and work with the natural order , giving the same code up to a permutation, so always check which convention a paper uses before comparing index sets. Second, the choice of frozen values is irrelevant only for symmetric channels (Arıkan proves this in his §VI); for asymmetric channels it genuinely matters, so “frozen bits are zero” is a convention, not a theorem. Encoding and Successive-Cancellation Decodinghas the recursive butterfly structure of an FFT, and encoding inherits the FFT’s cost: stages of XORs each, XOR operations total, computable fully in place with no matrix multiply and no memory beyond the codeword itself. Encoding a polar code is essentially free, a striking contrast with LDPC, where the sparse parity-check matrix gives no directly usable encoder and real systems need dedicated encoding structure. Decoding is where the definitions of §2 come due. Successive cancellation (SC) decodes strictly in order. At step : Note the in the conditioning: every decision assumes all previous decisions were correct. That is what “successive cancellation” means, it is exactly the decoder paying for the genie assumption baked into the definition of , and it is also the algorithm’s central flaw, because a single early error corrupts everything downstream. Hold that thought for the finale. Computationally, everything reduces to two update functions applied on a binary tree of depth . Given two log-likelihood ratios and arriving from the channel side:
combines two LLRs whose underlying bits are XOR-coupled, and
combines them once the coupling bit (the partial sum) is known. Stop and recognize . It is identical to the check-node update of LDPC belief propagation: the same tanh rule, the same min-sum approximation, for the same reason (a check node also computes the LLR of an XOR of bits). And is the variable-node update with a sign flip. Polar SC decoding and LDPC message passing are built from the same two primitives; they differ only in the graph the primitives run on and the schedule that fires them. This series has been telling a story of three apparently unrelated code families, and at this moment the story collapses into one: everyone is doing sum and tanh-product updates on a graph, and the families differ in where the graph comes from. Both functions drop straight out of the splitting definitions, and the derivation is short enough to keep. Write the channel LLRs and . For , expand by summing over : under the transmitted pair is and under it is , so
Divide numerator and denominator by to get the LLR of the first synthetic channel:
To see this equals the tanh form, set and use . Over the common denominator, has numerator , while has numerator , so , matching exactly. For , the second synthetic channel has available as side information: and both cases at once read . The asymmetry of the transform is now visible as algebra. attenuates: , so the "" channel’s evidence is weaker than either input. adds: it accumulates both LLRs, so the "" channel’s evidence is stronger. That is polarization, restated in two lines about absolute values. One practical note falls out of the min-sum approximation, which keeps only the dominant exponentials for large : unlike LDPC min-sum, polar decoders usually apply it without a correction scaling factor, because SC makes a hard decision immediately and never iterates, so there is no feedback loop for min-sum’s over-confidence to amplify. The full decoder is a depth- tree recursion: descend left applying , wait for the left subtree’s decoded bits, descend right applying with those bits as partial sums, then combine and pass upward. Condensed: Three details of this recursion are each a classic source of bugs, so they earn explicit notice. First, the line computing Complexity is time, and with a small observation, memory: a naive implementation stores an LLR array at every node for space, but at any instant only one array per level is live, and level holds values, so suffices. The first of those three details deserves its own paragraph, because it is not a bug hazard but an architectural sentence. The step for any right subtree cannot start until its left sibling has fully decoded, and this dependency chain runs the entire length of the code: SC decoding is inherently serial, bit after bit after bit. Where LDPC belief propagation updates every node in the graph simultaneously and soaks up parallel hardware, SC decoding fundamentally resists it. This serial dependency will haunt the hardware story when the series reaches it. The Polarization TheoremEverything so far assumed the channels actually do polarize. Arıkan’s Theorem 1 states it: for any B-DMC and any , as , In words: the fraction of near-perfect synthetic channels converges to exactly the capacity, the fraction of near-useless ones to exactly one minus the capacity, and nothing survives in the middle. The proof is the elegant core of the paper, and it fits in five steps. Define a random walk down the polarization tree: at each level, step to the "" child or the "" child with probability each, and let be the symmetric capacity of the synthetic channel you are standing on after steps. Choosing a uniformly random leaf this way makes statements about the walk statements about the fraction of leaves, which is what the theorem needs.
The last line is the punchline of the entire subject, and it is worth saying slowly. The fraction of perfect channels equals the capacity because capacity was conserved at every single step. Conservation makes the capacity process a martingale; the martingale property survives to the limit; and so whatever mass ends up at must, on average, equal what you started with. The sorting metaphor was not decoration: the proof is literally an accounting identity. And notice what the proof does not use: no distance spectrum, no ensemble average, no union bound over codewords. It is barely a page. A companion argument runs the same walk on , which is a supermartingale ( and ) also converging to , and that version is what actually feeds the error bound. Finite-Length PerformanceThe performance statement is Arıkan’s Theorem 3: for the polar code defined by information set ,
and for any fixed rate , choosing as the best indices gives , independently of the rate, with encoding and decoding. This is the headline of the paper and the answer to the series question: an explicit, deterministic code family that provably achieves the symmetric capacity of any B-DMC with quasi-linear complexity. Shannon’s 1948 promise, made constructive, sixty-one years later. But look at that convergence rate. is slow: pushing to this way would need on the order of . Arıkan does not hide it; his §V-D numerical example concludes, in his own words, that “the example also shows that the rate of polarization is too slow to make near-capacity polar coding under SC decoding feasible in practice.” His Figure 7 is the quantitative form of that sentence: ![]() Even at , holding the error bound below costs you rate: about against a capacity of . Arıkan and Telatar later tightened the asymptotics dramatically, to for any , but the finite-length gap is real and it sets up this page’s ending: capacity-achieving does not mean best at practical lengths. At the block lengths real systems use, a plain SC-decoded polar code loses to a well-designed LDPC code. Asymptotic optimality and finite-length competitiveness are different properties, and in 2009 polar codes had only the first. Code Construction and Comparison with Reed-Muller CodesOne more practical matter hides inside “choose the indices with smallest ”: computing those values. Arıkan’s §IX is refreshingly honest that this is hard in general, because the exact recursion holds only for the BEC; for any other channel the synthetic channels’ output alphabets grow exponentially and no efficient exact algorithm is known. The methods that emerged form a ladder:
Two consequences follow. First, a polar code is channel-specific: the information set depends on the design SNR, and a set designed for one operating point loses performance at another. One code per design SNR is an awkward property for a standard, and the 5G row of the table is precisely the engineering compromise that resolves it: a single tabulated nested ordering that is good enough across operating points. Second, polarization itself helps construction: as grows, most values cluster so close to or that even a coarse estimate classifies them correctly. Arıkan closes his paper with a comparison that carries a design lesson bigger than polar codes. Reed-Muller codes have exactly the same matrix structure ; they differ only in choosing by row Hamming weight instead of by . His §X proves that the RM rule is asymptotically unreliable under SC decoding. The reason is that the RM rule is channel-blind: it maximizes minimum distance, which is the right criterion for maximum-likelihood decoding, but SC is not ML. SC is a sequential procedure whose error probability is governed by , a channel-dependent reliability sum that the row-weight rule ignores entirely. (Amusingly, at on BEC() the two rules happen to select the same set ; the agreement is a small- coincidence, and the RM choice degrades as grows.) The general lesson: the right design criterion depends on the decoder you will actually use. RM optimizes for ML, polar optimizes for SC, and this is the same lesson the turbo story taught when the effective free distance and the error coefficient, not , turned out to govern iteratively decoded performance. Codes are not good or bad in the abstract; they are good or bad for a decoder. Check Yourself
Answers
Where This Leaves UsArıkan’s 2009 paper closed the question this series opened: a fully explicit, deterministic code with encoding and decoding and a one-page martingale proof that it achieves capacity, no random ensemble anywhere. Along the way it quietly unified the field, since the SC decoder’s and are exactly the check-node and variable-node primitives of LDPC message passing, running on a different graph under a different schedule. But the same paper is candid about its two weaknesses: polarization is slow, so at practical block lengths the plain SC-decoded polar code loses to LDPC; and SC decoding is greedy and irrevocable, so one early wrong decision corrupts everything after it, with no mechanism for repair. A provably capacity-achieving code that ships worse performance than its theoretically ungrounded rivals: that is where 2009 left polar codes, and it took until Tal and Vardy to change it. Their fix, keeping a list of candidate decoding paths and letting a CRC pick the winner, not only rescued the decoder but exposed and then repaired a deeper weakness of the code itself, and it carried polar codes all the way into the 5G standard. That story, list decoding, the CRC trick, and the hardware endgame, is the finale. ← LDPC Reborn: Density EvolutionIndexList Decoding and the Road to 5G → |