Good error-correcting codes based on very sparse matrices
David Mackay · IEEE Transactions on Information Theory · 1999
We study two families of error-correcting codes defined in terms of very sparse matrices. "MN" (MacKay-Neal (1995)) codes are recently invented, and "Gallager codes" were first investigated in 1962, but appear to have been largely forgotten, in spite of their excellent properties. The decoding of both codes can be tackled with a practical sum-product algorithm. We prove that these codes are "very good", in that sequences of codes exist which, when optimally decoded, achieve information rates up to the Shannon limit. This result holds not only for the binary-symmetric channel but also for any channel with symmetric stationary ergodic noise. We give experimental results for binary-symmetric channels and Gaussian channels demonstrating that practical performance substantially better than that of standard convolutional and concatenated codes can be achieved; indeed, the performance of Gallager codes is almost as close to the Shannon limit as that of turbo codes.
Cited by 3,684 works. doi:10.1109/18.748992
Authors
References
- A Mathematical Theory of Communication
- A Mathematical Theory of Communication
- Elements of Information Theory
- Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
- The Theory of Error-Correcting Codes
- Error bounds for convolutional codes and an asymptotically optimum decoding algorithm
- Low-density parity-check codes
- Optimal decoding of linear codes for minimizing symbol error rate (Corresp.)
- Optimal decoding of linear codes for minimizing symbol error rate
- Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1