*** thanks for stopping by my corner of the web *** best viewed at 800x600 *** sign my guestbook ***

Polar Codes: Capacity by Construction

Channel Coding

Every capacity-approaching code this series has met so far runs on randomness. Turbo codes get theirs from an interleaver (a pseudo-random permutation that manufactures ensemble-like behavior inside a single code); LDPC codes get theirs from a randomly sampled sparse graph, analyzed through the ensemble it was drawn from. That was also Shannon’s own move in 1948: average over all codes, show the average is good, conclude a good code exists, and say nothing about how to find it. The series opener framed the sixty-year hunt that followed as the search for an explicit answer. In 2009, Erdal Arıkan delivered it: a deterministic construction, with no ensemble and no random draw anywhere, that provably achieves the symmetric capacity of any binary-input memoryless channel, with encoding and decoding both running in O(NlogN)O(N \log N). The mechanism is a single recursive idea called channel polarization, and this page builds it from one XOR gate up to the full theorem, then confronts the honest catch that kept polar codes out of practice for another few years.

Symmetric Capacity and the Bhattacharyya Parameter

Let W:XYW: \mathcal{X} \to \mathcal{Y} be a binary-input discrete memoryless channel (B-DMC), X={0,1}\mathcal{X} = \{0,1\}. Arıkan’s analysis runs on two scalar parameters of WW, and it matters that there are two, because they do different jobs.

The first is the symmetric capacity, the mutual information between input and output when the input is equiprobable:

I(W)=yYxX12W(yx)log2W(yx)12W(y0)+12W(y1)  [0,1].I(W) = \sum_{y\in\mathcal{Y}}\sum_{x\in\mathcal{X}} \tfrac{1}{2} W(y|x)\,\log_2 \frac{W(y|x)}{\tfrac12 W(y|0) + \tfrac12 W(y|1)} \;\in [0,1].

The second is the Bhattacharyya parameter, a reliability measure:

Z(W)=yYW(y0)W(y1)  [0,1].Z(W) = \sum_{y\in\mathcal{Y}} \sqrt{W(y|0)\,W(y|1)} \;\in [0,1].

Z(W)Z(W) upper-bounds the maximum-likelihood error probability of a single use of WW to send one bit, so small ZZ means a reliable channel. The two parameters track each other: I(W)1I(W) \approx 1 exactly when Z(W)0Z(W) \approx 0 (a good channel), and I(W)0I(W) \approx 0 exactly when Z(W)1Z(W) \approx 1 (a useless one). Arıkan’s Proposition 1 makes the coupling precise: I(W)log221+Z(W)I(W) \ge \log_2 \frac{2}{1+Z(W)} and I(W)1Z(W)2I(W) \le \sqrt{1 - Z(W)^2}.

Why carry both? Because II measures rate and turns out to be the quantity that polarization conserves, which is what drives the main theorem; while ZZ measures reliability, satisfies clean one-step recursions, and directly bounds error probability, which is what drives the performance theorem and the code construction. The proofs need each of them in its own role.

One special case is worth memorizing before anything else. For the binary erasure channel BEC(ϵ\epsilon), which delivers the input bit with probability 1ϵ1-\epsilon and an erasure symbol otherwise, both parameters are available in closed form: I(W)=1ϵI(W) = 1 - \epsilon and Z(W)=ϵZ(W) = \epsilon. The BEC is the one channel where every quantity in this page can be computed exactly, which is why every worked example uses it.

Channel Combining and Splitting

Here is the entire invention at size two. Take two independent copies of WW and, before transmitting, couple their inputs with a single XOR:

x1=u1u2,x2=u2.x_1 = u_1 \oplus u_2, \qquad x_2 = u_2.

The basic size-two channel combining transform W W u1 u2 y1 y2 x1 x2
The size-two transform. A single XOR couples the two inputs before transmission, so the first channel carries the modulo-two sum of both bits while the second carries the second bit alone. The two channel uses stay independent; all that changes is which combination of the input bits each one is asked to convey.

The combined two-bit channel is W2(y1,y2u1,u2)=W(y1u1u2)W(y2u2)W_2(y_1, y_2 \mid u_1, u_2) = W(y_1 \mid u_1 \oplus u_2)\, W(y_2 \mid u_2). Now split it back into two single-bit channels, defined so that they are decoded in order, u1u_1 first:

W2(1)(y1,y2u1)=u212W2(y1,y2u1,u2),W2(2)(y1,y2,u1u2)=12W2(y1,y2u1,u2).W_2^{(1)}(y_1,y_2 \mid u_1) = \sum_{u_2} \tfrac{1}{2}\, W_2(y_1,y_2 \mid u_1,u_2), \qquad W_2^{(2)}(y_1,y_2,u_1 \mid u_2) = \tfrac{1}{2}\, W_2(y_1,y_2 \mid u_1,u_2).

Read the definitions carefully, because the asymmetry between them is the whole trick. W2(1)W_2^{(1)}, the channel seen by u1u_1, observes both outputs y1,y2y_1, y_2 but must treat u2u_2 as unknown interference summed over; it is worse than a plain use of WW. W2(2)W_2^{(2)}, the channel seen by u2u_2, observes both outputs and receives u1u_1 as a known side input; it is better than WW. Notice what that side input assumes: W2(2)W_2^{(2)} gets to act as if u1u_1 has already been decoded correctly. This “genie” assumption is written directly into the channel definition here, and successive cancellation decoding will later have to actually earn it, one bit at a time.

How much worse and how much better? Two exact statements answer this, and they are the load-bearing wall of the whole theory:

I(W2(1))+I(W2(2))=2I(W)(conservation)\boxed{\,I(W_2^{(1)}) + I(W_2^{(2)}) = 2\,I(W)\,} \qquad \text{(conservation)} I(W2(1))    I(W)    I(W2(2))(extremization)\boxed{\,I(W_2^{(1)}) \;\le\; I(W) \;\le\; I(W_2^{(2)})\,} \qquad \text{(extremization)}

Total capacity is preserved; it is merely redistributed. Nothing is created and nothing is destroyed: capacity is sorted, moved out of one channel and into the other. If you retain a single sentence from this page, make it that one, because everything that follows (the recursion, the polarization theorem, even the punchline of the martingale proof) is that sentence applied over and over.

Both statements have proofs short enough to keep in full. Conservation is nothing but the chain rule for mutual information. The map (u1,u2)(x1,x2)(u_1,u_2) \mapsto (x_1,x_2) is a bijection (invert it: u2=x2u_2 = x_2, u1=x1x2u_1 = x_1 \oplus x_2), so if U1,U2U_1, U_2 are i.i.d. uniform then so are X1,X2X_1, X_2. Then

I(W2(1))+I(W2(2))=I(U1;Y1Y2)+I(U2;Y1Y2U1)=I(U1U2;Y1Y2)=I(X1X2;Y1Y2)=2I(W),I(W_2^{(1)}) + I(W_2^{(2)}) = I(U_1;Y_1Y_2) + I(U_2;Y_1Y_2 \mid U_1) = I(U_1U_2;Y_1Y_2) = I(X_1X_2;Y_1Y_2) = 2I(W),

the last step because the two channel uses are independent. (The middle step uses the fact that I(W2(2))=I(U2;Y1Y2U1)I(W_2^{(2)}) = I(U_2; Y_1Y_2U_1) equals the conditional term I(U2;Y1Y2U1)I(U_2;Y_1Y_2 \mid U_1), since U2U_2 is independent of U1U_1.) \blacksquare

Extremization is the observation that adding observations cannot decrease mutual information:

I(W2(2))=I(U2;Y1Y2U1)    I(U2;Y2)=I(W),I(W_2^{(2)}) = I(U_2; Y_1Y_2U_1) \;\ge\; I(U_2;Y_2) = I(W),

and conservation then forces I(W2(1))=2I(W)I(W2(2))I(W)I(W_2^{(1)}) = 2I(W) - I(W_2^{(2)}) \le I(W). \blacksquare

The Bhattacharyya parameters transform just as cleanly:

Z(W2(2))=Z(W)2Z(W2(1))2Z(W)Z(W)2\boxed{\,Z(W_2^{(2)}) = Z(W)^2\,} \qquad \boxed{\,Z(W_2^{(1)}) \le 2Z(W) - Z(W)^2\,}

with equality in the second relation if and only if WW is a BEC, a detail that will matter enormously when we try to construct codes for other channels. The first relation is an exact computation from the definition. The output alphabet of W2(2)W_2^{(2)} is the triple (y1,y2,u1)(y_1, y_2, u_1), so

Z(W2(2))=y1,y2,u112W(y1u1)W(y20)12W(y1u1 ⁣ ⁣1)W(y21)Z(W_2^{(2)}) = \sum_{y_1,y_2,u_1} \sqrt{\tfrac12 W(y_1|u_1)W(y_2|0)\cdot \tfrac12 W(y_1|u_1\!\oplus\!1)W(y_2|1)} =12u1[y1W(y1u1)W(y1u1 ⁣ ⁣1)]y2W(y20)W(y21)=122Z(W)Z(W)=Z(W)2,= \tfrac12 \sum_{u_1}\Big[\sum_{y_1}\sqrt{W(y_1|u_1)W(y_1|u_1\!\oplus\!1)}\Big]\cdot \sum_{y_2}\sqrt{W(y_2|0)W(y_2|1)} = \tfrac12 \cdot 2Z(W)\cdot Z(W) = Z(W)^2,

since the inner bracket equals Z(W)Z(W) for either value of u1u_1. \blacksquare

All of this abstraction becomes concrete in two lines on the erasure channel. For BEC(ϵ\epsilon), where Z=ϵZ = \epsilon is just the erasure probability: the first channel must recover u1=x1x2u_1 = x_1 \oplus x_2, which requires both observations, so it is erased whenever either copy erases, with probability 1(1ϵ)2=2ϵϵ21 - (1-\epsilon)^2 = 2\epsilon - \epsilon^2. The second channel already knows u1u_1, so it can recover u2u_2 from either observation (u2=y2u_2 = y_2 directly, or u2=y1u1u_2 = y_1 \oplus u_1), and is erased only when both copies erase, with probability ϵ2\epsilon^2. That two-line argument, “either erases” versus “both erase,” is the entire polarization mechanism made tangible, and it matches the two ZZ formulas exactly.

Recursive Construction for Length N

One XOR moved capacity around a little. Arıkan’s next move is to do it again, recursively, on N=2nN = 2^n copies of WW: pair up channels and transform, then pair up the resulting synthetic channels and transform those, nn levels deep. Formally the length-NN combined channel is built from two length-N/2N/2 ones,

WN(y1Nu1N)=WN/2 ⁣(y1N/2|u1,oddNu1,evenN)WN/2 ⁣(yN/2+1N|u1,evenN),W_N(y_1^N \mid u_1^N) = W_{N/2}\!\left(y_1^{N/2} \,\middle|\, u_{1,\text{odd}}^N \oplus u_{1,\text{even}}^N\right)\cdot W_{N/2}\!\left(y_{N/2+1}^N \,\middle|\, u_{1,\text{even}}^N\right),

and the split channels obey the one-step recursions you will use constantly:

  I(WN(2i1))+I(WN(2i))=2I(WN/2(i))Z(WN(2i))=Z(WN/2(i))2Z(WN(2i1))2Z(WN/2(i))Z(WN/2(i))2(= for BEC)  \boxed{\;\begin{aligned} I(W_N^{(2i-1)}) + I(W_N^{(2i)}) &= 2\,I(W_{N/2}^{(i)})\\[2pt] Z(W_N^{(2i)}) &= Z(W_{N/2}^{(i)})^2\\[2pt] Z(W_N^{(2i-1)}) &\le 2Z(W_{N/2}^{(i)}) - Z(W_{N/2}^{(i)})^2 \quad (= \text{ for BEC}) \end{aligned}\;}

The clean way to picture this is a binary tree of depth nn. The root is WW itself; every node branches into a "-" child (the worse transform) and a "++" child (the better one). Each synthetic channel WN(i)W_N^{(i)} sits at a leaf, and the binary expansion of i1i-1 spells out the path that reaches it: index ii is a path through the polarization tree. A channel that took mostly "++" steps has been squared toward Z0Z \approx 0 again and again and is nearly noiseless; one that took mostly "-" steps has been degraded toward Z1Z \approx 1 and is nearly useless. The claim of the polarization theorem, proved below, is that as nn grows almost every leaf ends up at one extreme or the other, and the fraction landing at the good extreme is exactly I(W)I(W).

If that claim holds, the code writes itself. Choose the KK most reliable synthetic channels as the information set A\mathcal{A} and send data on those; freeze the remaining NKN - K inputs to values known in advance by both sides. The frozen bits are not overhead grudgingly accepted: they are the code. Freezing is precisely how the encoder avoids the bad synthetic channels, and the set A\mathcal{A} defines the code completely.

A Worked Example: Code Construction on the BEC

Nothing makes the mechanism believable like running it. Take W=W = BEC(1/21/2), so Z(W)=0.5Z(W) = 0.5 and I(W)=0.5I(W) = 0.5, and recall that for the BEC both ZZ recursions are exact equalities: Z=2ZZ2Z^- = 2Z - Z^2 and Z+=Z2Z^+ = Z^2. Iterating from Z=0.5Z = 0.5:

NNZ(WN(i))Z(W_N^{(i)}) for i=1Ni = 1 \dots N
10.50.5
20.75,  0.250.75,\; 0.25
40.9375,  0.5625,  0.4375,  0.06250.9375,\; 0.5625,\; 0.4375,\; 0.0625
80.99609,  0.87891,  0.80859,  0.31641,  0.68359,  0.19141,  0.12109,  0.003910.99609,\; 0.87891,\; 0.80859,\; \mathbf{0.31641},\; 0.68359,\; \mathbf{0.19141},\; \mathbf{0.12109},\; \mathbf{0.00391}

Check the conservation law at every level: the row sums are 0.50.5, 1.01.0, 2.02.0, 4.04.0, always NϵN\epsilon, because for the BEC the parameter ZZ is an erasure probability and erasure probability is conserved exactly. This is the arithmetic sanity check to run whenever you compute these by hand; if a row does not sum to NϵN\epsilon, you made a slip.

Now sort the N=8N=8 values: 0.004, 0.121, 0.191, 0.316, 0.684, 0.809, 0.879, 0.9960.004,\ 0.121,\ 0.191,\ 0.316,\ 0.684,\ 0.809,\ 0.879,\ 0.996. They are visibly drifting toward the two extremes and away from the middle, though at N=8N=8 the drift is still weak, a weakness we will return to with numbers attached. Push the same exact recursion to large NN and the drift becomes a cliff:

Sorted Bhattacharyya parameters of the synthetic channels of BEC(1/2), computed by the exact Z recursion for N from 2^3 up to 2^16. As N grows the sorted curve steepens into a step function with its transition at 1 - I(W) = 0.5: half the channels head to Z = 0 and half to Z = 1, so the fraction of near-perfect channels is exactly the capacity.
Sorted Bhattacharyya parameters of the synthetic channels of BEC(1/2), computed by the exact Z recursion for N from 2^3 up to 2^16. As N grows the sorted curve steepens into a step function with its transition at 1 - I(W) = 0.5: half the channels head to Z = 0 and half to Z = 1, so the fraction of near-perfect channels is exactly the capacity.

For a rate-1/21/2 code at N=8N = 8 we need the K=4K = 4 indices with the smallest ZZ:

A={4,6,7,8},Ac={1,2,3,5}  (frozen).\mathcal{A} = \{4, 6, 7, 8\}, \qquad \mathcal{A}^c = \{1, 2, 3, 5\} \;\text{(frozen)}.

Look at that set for a moment, because it kills a tempting misconception: A\mathcal{A} is not simply “the last KK indices.” Index 4 (Z=0.316Z = 0.316) beats index 5 (Z=0.684Z = 0.684), because index 4’s path through the tree ends in two "++" steps while index 5’s ends in two "-" steps. The reliability ordering is genuinely non-monotonic in the index, and it is channel-dependent besides.

How good is this code? Theorem 3 (stated properly below) bounds the block error probability of successive cancellation decoding by the sum of the information-set Bhattacharyya parameters:

Pe    iAZ(WN(i))=0.316+0.191+0.121+0.004=0.633.P_e \;\le\; \sum_{i\in\mathcal{A}} Z(W_N^{(i)}) = 0.316 + 0.191 + 0.121 + 0.004 = 0.633.

Useless at N=8N = 8, and the honesty is the point: polarization at small block lengths has barely begun, and this bound quantifies exactly the finite-length weakness Arıkan himself reports.

Finally, the encoder in matrix form. The recursive XOR structure works out to x1N=u1NGN\mathbf{x}_1^N = \mathbf{u}_1^N G_N with GN=BNFnG_N = B_N F^{\otimes n}, built from Kronecker powers of the 2×22 \times 2 kernel:

F=(1011),F2=(1000110010101111),G4=B4F2=(1000101011001111),F = \begin{pmatrix}1&0\\1&1\end{pmatrix}, \qquad F^{\otimes 2} = \begin{pmatrix}1&0&0&0\\1&1&0&0\\1&0&1&0\\1&1&1&1\end{pmatrix}, \qquad G_4 = B_4 F^{\otimes 2} = \begin{pmatrix}1&0&0&0\\1&0&1&0\\1&1&0&0\\1&1&1&1\end{pmatrix},

where BNB_N is the bit-reversal permutation (for N=4N = 4 it reorders rows to 1,3,2,41, 3, 2, 4). The frozen positions uAc\mathbf{u}_{\mathcal{A}^c} are set to fixed values, conventionally zeros. Two convention warnings are worth flagging right here. First, BNB_N only reorders; many modern treatments and the 5G standard drop it and work with the natural order GN=FnG_N = F^{\otimes n}, giving the same code up to a permutation, so always check which convention a paper uses before comparing index sets. Second, the choice of frozen values is irrelevant only for symmetric channels (Arıkan proves this in his §VI); for asymmetric channels it genuinely matters, so “frozen bits are zero” is a convention, not a theorem.

Encoding and Successive-Cancellation Decoding

FnF^{\otimes n} has the recursive butterfly structure of an FFT, and encoding inherits the FFT’s cost: n=log2Nn = \log_2 N stages of N/2N/2 XORs each, N2log2N\frac{N}{2}\log_2 N XOR operations total, computable fully in place with no matrix multiply and no memory beyond the codeword itself. Encoding a polar code is essentially free, a striking contrast with LDPC, where the sparse parity-check matrix gives no directly usable encoder and real systems need dedicated encoding structure.

Decoding is where the definitions of §2 come due. Successive cancellation (SC) decodes u1,u2,,uNu_1, u_2, \dots, u_N strictly in order. At step ii:

u^i={frozen valueiA,argmaxu{0,1}WN(i) ⁣(y1N,u^1i1u)iA.\hat{u}_i = \begin{cases} \text{frozen value} & i \notin \mathcal{A},\\[4pt] \arg\max_{u\in\{0,1\}} W_N^{(i)}\!\left(y_1^N, \hat{u}_1^{i-1} \mid u\right) & i \in \mathcal{A}. \end{cases}

Note the u^1i1\hat{u}_1^{i-1} in the conditioning: every decision assumes all previous decisions were correct. That is what “successive cancellation” means, it is exactly the decoder paying for the genie assumption baked into the definition of W2(2)W_2^{(2)}, and it is also the algorithm’s central flaw, because a single early error corrupts everything downstream. Hold that thought for the finale.

Computationally, everything reduces to two update functions applied on a binary tree of depth log2N\log_2 N. Given two log-likelihood ratios aa and bb arriving from the channel side:

f(a,b)=2artanh ⁣(tanha2tanhb2)sign(a)sign(b)min(a,b)f(a,b) = 2\,\mathrm{artanh}\!\left(\tanh\tfrac{a}{2}\tanh\tfrac{b}{2}\right) \approx \mathrm{sign}(a)\,\mathrm{sign}(b)\,\min(|a|,|b|)

combines two LLRs whose underlying bits are XOR-coupled, and

g(a,b,u^s)=b+(12u^s)ag(a,b,\hat{u}_s) = b + (1 - 2\hat{u}_s)\,a

combines them once the coupling bit u^s\hat{u}_s (the partial sum) is known. Stop and recognize ff. It is identical to the check-node update of LDPC belief propagation: the same tanh rule, the same min-sum approximation, for the same reason (a check node also computes the LLR of an XOR of bits). And gg is the variable-node update with a sign flip. Polar SC decoding and LDPC message passing are built from the same two primitives; they differ only in the graph the primitives run on and the schedule that fires them. This series has been telling a story of three apparently unrelated code families, and at this moment the story collapses into one: everyone is doing sum and tanh-product updates on a graph, and the families differ in where the graph comes from.

Both functions drop straight out of the N=2N = 2 splitting definitions, and the derivation is short enough to keep. Write the channel LLRs L1=lnW(y10)W(y11)L_1 = \ln\frac{W(y_1|0)}{W(y_1|1)} and L2=lnW(y20)W(y21)L_2 = \ln\frac{W(y_2|0)}{W(y_2|1)}. For ff, expand W2(1)W_2^{(1)} by summing over u2u_2: under u1=0u_1 = 0 the transmitted pair is (u2,u2)(u_2, u_2) and under u1=1u_1 = 1 it is (1u2,u2)(1 \oplus u_2, u_2), so

W2(1)(y1,y20)=12[W(y10)W(y20)+W(y11)W(y21)],W_2^{(1)}(y_1,y_2 \mid 0) = \tfrac12\big[W(y_1|0)W(y_2|0) + W(y_1|1)W(y_2|1)\big], W2(1)(y1,y21)=12[W(y11)W(y20)+W(y10)W(y21)].W_2^{(1)}(y_1,y_2 \mid 1) = \tfrac12\big[W(y_1|1)W(y_2|0) + W(y_1|0)W(y_2|1)\big].

Divide numerator and denominator by W(y11)W(y21)W(y_1|1)W(y_2|1) to get the LLR of the first synthetic channel:

L(1)=lneL1eL2+1eL1+eL2.L^{(1)} = \ln\frac{e^{L_1}e^{L_2} + 1}{e^{L_1} + e^{L_2}}.

To see this equals the tanh form, set t=tanhL12tanhL22=(eL11)(eL21)(eL1+1)(eL2+1)t = \tanh\frac{L_1}{2}\tanh\frac{L_2}{2} = \frac{(e^{L_1}-1)(e^{L_2}-1)}{(e^{L_1}+1)(e^{L_2}+1)} and use 2artanh(t)=ln1+t1t2\,\mathrm{artanh}(t) = \ln\frac{1+t}{1-t}. Over the common denominator, 1+t1 + t has numerator (eL1+1)(eL2+1)+(eL11)(eL21)=2eL1+L2+2(e^{L_1}+1)(e^{L_2}+1) + (e^{L_1}-1)(e^{L_2}-1) = 2e^{L_1+L_2} + 2, while 1t1 - t has numerator 2eL1+2eL22e^{L_1} + 2e^{L_2}, so 1+t1t=eL1+L2+1eL1+eL2\frac{1+t}{1-t} = \frac{e^{L_1+L_2}+1}{e^{L_1}+e^{L_2}}, matching L(1)L^{(1)} exactly. \blacksquare

For gg, the second synthetic channel has u1u_1 available as side information:

L(2)=lnW(y1u10)W(y20)W(y1u11)W(y21)={L1+L2u1=0,L1+L2u1=1,L^{(2)} = \ln\frac{W(y_1 \mid u_1 \oplus 0)\,W(y_2 \mid 0)}{W(y_1 \mid u_1 \oplus 1)\,W(y_2 \mid 1)} = \begin{cases} L_1 + L_2 & u_1 = 0,\\ -L_1 + L_2 & u_1 = 1,\end{cases}

and both cases at once read L(2)=L2+(12u1)L1L^{(2)} = L_2 + (1 - 2u_1)L_1. \blacksquare

The asymmetry of the transform is now visible as algebra. ff attenuates: f(a,b)min(a,b)|f(a,b)| \le \min(|a|,|b|), so the "-" channel’s evidence is weaker than either input. gg adds: it accumulates both LLRs, so the "++" channel’s evidence is stronger. That is polarization, restated in two lines about absolute values. One practical note falls out of the min-sum approximation, which keeps only the dominant exponentials for large L1,L2|L_1|, |L_2|: unlike LDPC min-sum, polar decoders usually apply it without a correction scaling factor, because SC makes a hard decision immediately and never iterates, so there is no feedback loop for min-sum’s over-confidence to amplify.

The full decoder is a depth-log2N\log_2 N tree recursion: descend left applying ff, wait for the left subtree’s decoded bits, descend right applying gg with those bits as partial sums, then combine and pass upward. Condensed:

Python
# α: LLR vector of length m = 2^(n-d) at a depth-d node
# β: returned hard-decision vector (partial sums) of the same length

def decode(node, α):
    if node.is_leaf:                       # m == 1, one bit
        if node.frozen: return [0]
        return [0] if α[0] >= 0 else [1]   # threshold the LLR

    m = len(α) // 2
    αl = [ f(α[i], α[i+m])         for i in range(m) ]
    βl = decode(node.left, αl)
    αr = [ g(α[i], α[i+m], βl[i])  for i in range(m) ]   # needs βl!
    βr = decode(node.right, αr)
    return [βl[i] ^ βr[i] for i in range(m)] + βr        # combine upward

û = decode(root, [channel_LLR(y[i]) for i in range(N)])

Three details of this recursion are each a classic source of bugs, so they earn explicit notice. First, the line computing αr cannot run until βl has returned: the right subtree’s input depends on the left subtree’s output. Second, the returned β vectors are partial sums, the re-encoded values at that tree level, not information bits; the information bits are read off at the leaves as they are decided. Third, the combine step [βl ^ βr] + βr is precisely the encoder’s butterfly run in reverse: SC decoding re-encodes as it goes, which is also why the partial sums exist at all.

Complexity is O(NlogN)O(N \log N) time, and with a small observation, O(N)O(N) memory: a naive implementation stores an LLR array at every node for O(NlogN)O(N \log N) space, but at any instant only one array per level is live, and level dd holds 2nd2^{n-d} values, so d=0n2nd=2N1\sum_{d=0}^{n} 2^{n-d} = 2N - 1 suffices.

The first of those three details deserves its own paragraph, because it is not a bug hazard but an architectural sentence. The gg step for any right subtree cannot start until its left sibling has fully decoded, and this dependency chain runs the entire length of the code: SC decoding is inherently serial, bit after bit after bit. Where LDPC belief propagation updates every node in the graph simultaneously and soaks up parallel hardware, SC decoding fundamentally resists it. This serial dependency will haunt the hardware story when the series reaches it.

The Polarization Theorem

Everything so far assumed the channels actually do polarize. Arıkan’s Theorem 1 states it: for any B-DMC WW and any δ(0,1)\delta \in (0,1), as N=2nN = 2^n \to \infty,

{i:I(WN(i))(1δ,1]}N    I(W),{i:I(WN(i))[0,δ)}N    1I(W).\frac{\left|\{i : I(W_N^{(i)}) \in (1-\delta,\, 1]\}\right|}{N} \;\to\; I(W), \qquad \frac{\left|\{i : I(W_N^{(i)}) \in [0,\, \delta)\}\right|}{N} \;\to\; 1 - I(W).

In words: the fraction of near-perfect synthetic channels converges to exactly the capacity, the fraction of near-useless ones to exactly one minus the capacity, and nothing survives in the middle.

The proof is the elegant core of the paper, and it fits in five steps. Define a random walk down the polarization tree: at each level, step to the "-" child or the "++" child with probability 1/21/2 each, and let InI_n be the symmetric capacity of the synthetic channel you are standing on after nn steps. Choosing a uniformly random leaf this way makes statements about the walk statements about the fraction of leaves, which is what the theorem needs.

  1. {In}\{I_n\} is a martingale: by conservation, E[In+1In]=12(I+I+)=In\mathbb{E}[I_{n+1} \mid I_n] = \tfrac12(I^- + I^+) = I_n, and it is bounded in [0,1][0,1].
  2. Bounded martingales converge almost surely, so InII_n \to I_\infty.
  3. Convergence forces the two branches to stop separating: I+I0|I^+ - I^-| \to 0 along the walk. But the extremization inequality says the branches genuinely separate unless the channel is already extremal, with I{0,1}I \in \{0,1\}.
  4. Therefore I{0,1}I_\infty \in \{0,1\} almost surely: the limit is a Bernoulli random variable.
  5. The martingale property pins down its mean: E[I]=I0=I(W)\mathbb{E}[I_\infty] = I_0 = I(W), so Pr[I=1]=I(W)\Pr[I_\infty = 1] = I(W). \blacksquare

The last line is the punchline of the entire subject, and it is worth saying slowly. The fraction of perfect channels equals the capacity because capacity was conserved at every single step. Conservation makes the capacity process a martingale; the martingale property survives to the limit; and so whatever mass ends up at I=1I = 1 must, on average, equal what you started with. The sorting metaphor was not decoration: the proof is literally an accounting identity. And notice what the proof does not use: no distance spectrum, no ensemble average, no union bound over codewords. It is barely a page. A companion argument runs the same walk on ZnZ_n, which is a supermartingale (Z+=Z2ZZ^+ = Z^2 \le Z and Z2ZZ2Z^- \le 2Z - Z^2) also converging to {0,1}\{0,1\}, and that version is what actually feeds the error bound.

Finite-Length Performance

The performance statement is Arıkan’s Theorem 3: for the polar code defined by information set A\mathcal{A},

Pe(N,K,A)    iAZ(WN(i)),\boxed{\,P_e(N, K, \mathcal{A}) \;\le\; \sum_{i\in\mathcal{A}} Z(W_N^{(i)})\,},

and for any fixed rate R<I(W)R < I(W), choosing A\mathcal{A} as the NRNR best indices gives Pe=O(N1/4)P_e = O(N^{-1/4}), independently of the rate, with O(NlogN)O(N \log N) encoding and decoding. This is the headline of the paper and the answer to the series question: an explicit, deterministic code family that provably achieves the symmetric capacity of any B-DMC with quasi-linear complexity. Shannon’s 1948 promise, made constructive, sixty-one years later.

But look at that convergence rate. N1/4N^{-1/4} is slow: pushing PeP_e to 10610^{-6} this way would need NN on the order of 102410^{24}. Arıkan does not hide it; his §V-D numerical example concludes, in his own words, that “the example also shows that the rate of polarization is too slow to make near-capacity polar coding under SC decoding feasible in practice.” His Figure 7 is the quantitative form of that sentence:

Arıkan's Figure 7 for BEC(1/2): achievable rate versus the block error bound, computed exactly from the BEC Z recursions for N up to 2^20. The solid curves plot the rate obtainable while holding the bound sum of Z over the information set below a target P_e; the dashed curves use max of Z as a lower-bound companion. Even at N around 10^6, keeping P_e below 1e-10 limits the rate to about 0.45 against a capacity of 0.5.
Arıkan's Figure 7 for BEC(1/2): achievable rate versus the block error bound, computed exactly from the BEC Z recursions for N up to 2^20. The solid curves plot the rate obtainable while holding the bound sum of Z over the information set below a target P_e; the dashed curves use max of Z as a lower-bound companion. Even at N around 10^6, keeping P_e below 1e-10 limits the rate to about 0.45 against a capacity of 0.5.

Even at N=220106N = 2^{20} \approx 10^6, holding the error bound below 101010^{-10} costs you rate: about 0.450.45 against a capacity of 0.50.5. Arıkan and Telatar later tightened the asymptotics dramatically, to Pe2NβP_e \approx 2^{-N^\beta} for any β<1/2\beta < 1/2, but the finite-length gap is real and it sets up this page’s ending: capacity-achieving does not mean best at practical lengths. At the block lengths real systems use, a plain SC-decoded polar code loses to a well-designed LDPC code. Asymptotic optimality and finite-length competitiveness are different properties, and in 2009 polar codes had only the first.

Code Construction and Comparison with Reed-Muller Codes

One more practical matter hides inside “choose the KK indices with smallest ZZ”: computing those ZZ values. Arıkan’s §IX is refreshingly honest that this is hard in general, because the exact ZZ^- recursion holds only for the BEC; for any other channel the synthetic channels’ output alphabets grow exponentially and no efficient exact algorithm is known. The methods that emerged form a ladder:

ChannelMethodCost
BECexact ZZ recursionsO(N)O(N), easy
General B-DMCno efficient exact algorithm known
General, approximateMonte-Carlo estimation of ZZ via an SC decoder with all inputs frozenO(NlogN)O(N\log N) per sample
General, approximateTal-Vardy density quantization: degrade/upgrade the channel to a bounded alphabet, then propagate exactlyO(Nμ2logμ)O(N\mu^2\log\mu), the standard method
BI-AWGNGaussian approximation tracking a single mean parameterO(N)O(N), fast, slightly lossy
5G NRone fixed, nested reliability sequence of 1024 indices, tabulated in the standardO(1)O(1)

Two consequences follow. First, a polar code is channel-specific: the information set depends on the design SNR, and a set designed for one operating point loses performance at another. One code per design SNR is an awkward property for a standard, and the 5G row of the table is precisely the engineering compromise that resolves it: a single tabulated nested ordering that is good enough across operating points. Second, polarization itself helps construction: as NN grows, most ZZ values cluster so close to 00 or 11 that even a coarse estimate classifies them correctly.

Arıkan closes his paper with a comparison that carries a design lesson bigger than polar codes. Reed-Muller codes have exactly the same matrix structure GN=FnG_N = F^{\otimes n}; they differ only in choosing A\mathcal{A} by row Hamming weight instead of by ZZ. His §X proves that the RM rule is asymptotically unreliable under SC decoding. The reason is that the RM rule is channel-blind: it maximizes minimum distance, which is the right criterion for maximum-likelihood decoding, but SC is not ML. SC is a sequential procedure whose error probability is governed by iAZ(WN(i))\sum_{i\in\mathcal{A}} Z(W_N^{(i)}), a channel-dependent reliability sum that the row-weight rule ignores entirely. (Amusingly, at N=8N = 8 on BEC(1/21/2) the two rules happen to select the same set {4,6,7,8}\{4,6,7,8\}; the agreement is a small-NN coincidence, and the RM choice degrades as NN grows.) The general lesson: the right design criterion depends on the decoder you will actually use. RM optimizes for ML, polar optimizes for SC, and this is the same lesson the turbo story taught when the effective free distance and the error coefficient, not dfreed_{\text{free}}, turned out to govern iteratively decoded performance. Codes are not good or bad in the abstract; they are good or bad for a decoder.

Check Yourself

  1. What quantity is conserved by the polarization transform, and what exactly does that conservation buy in the proof of Theorem 1?
  2. Why does W2(2)W_2^{(2)} end up better than WW, while W2(1)W_2^{(1)} ends up worse?
  3. State the two ZZ recursions. Which one is exact only for the BEC, and why does that matter in practice?
  4. For BEC(0.20.2), compute all four Z(W4(i))Z(W_4^{(i)}), verify the conservation check, and choose the information set for rate 1/21/2.
  5. Reed-Muller codes use the same FnF^{\otimes n} structure. Why does choosing A\mathcal{A} by row weight fail under SC decoding when it is the right rule under ML decoding?
  6. Why is SC decoding hard to parallelize, at the level of the tree recursion?
Answers
  1. Total symmetric capacity: I(W)+I(W+)=2I(W)I(W^-) + I(W^+) = 2I(W), which is nothing but the chain rule applied through the invertible XOR transform. It makes the capacity process {In}\{I_n\} along a random path a bounded martingale, hence almost surely convergent; combined with the extremization inequality (branches keep separating unless the channel is extremal), the limit must lie in {0,1}\{0,1\}, and the martingale mean then fixes Pr[I=1]=I(W)\Pr[I_\infty = 1] = I(W): the fraction of perfect channels equals the capacity.

  2. Because it is decoded second and receives u1u_1 as known side information on top of both channel outputs; adding observations cannot decrease mutual information, so I(W2(2))I(W)I(W_2^{(2)}) \ge I(W). W2(1)W_2^{(1)}, decoded first, must treat u2u_2 as unknown interference, and conservation then forces it below I(W)I(W) by exactly the amount W2(2)W_2^{(2)} gained. On the BEC: the second channel is erased only if both copies erase (ϵ2\epsilon^2), the first if either erases (2ϵϵ22\epsilon - \epsilon^2).

  3. Z(W+)=Z2Z(W^+) = Z^2, always exact; Z(W)2ZZ2Z(W^-) \le 2Z - Z^2, with equality if and only if WW is a BEC. The equality is why the BEC admits exact O(N)O(N) code construction, while every other channel needs approximation: Monte-Carlo estimation, Tal-Vardy density quantization, or the Gaussian approximation for the AWGN channel.

  4. Start from Z=0.2Z = 0.2. At N=2N = 2: Z=0.40.04=0.36Z^- = 0.4 - 0.04 = 0.36 and Z+=0.04Z^+ = 0.04. At N=4N = 4: from 0.360.36 come 0.720.1296=0.59040.72 - 0.1296 = 0.5904 and 0.12960.1296; from 0.040.04 come 0.080.0016=0.07840.08 - 0.0016 = 0.0784 and 0.00160.0016. So (0.5904,0.1296,0.0784,0.0016)(0.5904, 0.1296, 0.0784, 0.0016), summing to 0.8=4×0.20.8 = 4 \times 0.2, as conservation requires. For rate 1/21/2, A={3,4}\mathcal{A} = \{3, 4\} with Pe0.0784+0.0016=0.08P_e \le 0.0784 + 0.0016 = 0.08.

  5. Row weight maximizes minimum distance, which governs ML performance. SC is not ML: it is a sequential, greedy procedure whose block error probability is governed by iAZ(WN(i))\sum_{i\in\mathcal{A}} Z(W_N^{(i)}), a channel-dependent reliability quantity that the channel-blind row-weight rule ignores. Arıkan’s §X proves RM index selection is asymptotically unreliable under SC. The design criterion must match the decoder actually used.

  6. Because the input to every right subtree is computed by gg, which needs the left subtree’s decoded partial sums; the right half of any node cannot start until the left half has fully finished. This dependency chain spans the entire code, so the decoder is forced through the bits essentially one at a time, unlike LDPC belief propagation, which updates all nodes of its graph in parallel.

Where This Leaves Us

Arıkan’s 2009 paper closed the question this series opened: a fully explicit, deterministic code with O(NlogN)O(N\log N) encoding and decoding and a one-page martingale proof that it achieves capacity, no random ensemble anywhere. Along the way it quietly unified the field, since the SC decoder’s ff and gg are exactly the check-node and variable-node primitives of LDPC message passing, running on a different graph under a different schedule. But the same paper is candid about its two weaknesses: polarization is slow, so at practical block lengths the plain SC-decoded polar code loses to LDPC; and SC decoding is greedy and irrevocable, so one early wrong decision corrupts everything after it, with no mechanism for repair. A provably capacity-achieving code that ships worse performance than its theoretically ungrounded rivals: that is where 2009 left polar codes, and it took until Tal and Vardy to change it. Their fix, keeping a list of candidate decoding paths and letting a CRC pick the winner, not only rescued the decoder but exposed and then repaired a deeper weakness of the code itself, and it carried polar codes all the way into the 5G standard. That story, list decoding, the CRC trick, and the hardware endgame, is the finale.

← LDPC Reborn: Density EvolutionIndexList Decoding and the Road to 5G →