Simulador de Sistema PLC Utilizando OFDM Adaptativo com

Transcription

Simulador de Sistema PLC Utilizando OFDM Adaptativo
com Códigos LDPC
José Manuel Gouveia de Miranda Ribeiro Vaz
Dissertação para obtenção do Grau de Mestre em
Engenharia Electrotécnica e de Computadores
Júri
Presidente: Doutor Marcelino Santos
Orientador: Doutor José António Beltran Gerald
Members: Doutor Francisco António Bucho Cercas
Novembro de 2010
PLC Simulator System using Adaptive OFDM with LDPC
codes
José Manuel Gouveia de Miranda Ribeiro Vaz
Dissertation Submitted for obtaining the degree of Master in
Electrical and Computer Engineering
Jury
President: Doutor Marcelino Santos
Supervisor: Doutor José António Beltran Gerald
Members:
Doutor Francisco António Bucho Cercas
November 2010
Abstract
Despite LDPC codes performing admirably for large block sizes - being mostly resilient to low
levels of channel SNR and errors in channel equalization - small and medium sized codes tend to
be affected by these two factors. For these small to medium codes, a new method is presented
that helps to reduce the dependency of correct channel equalization, without changing the inner
workings or architecture of existing LDPC decoders. This goal is achieved by using LDPC only
decoder-side information - gathered during standard LDPC decoding - that is used to improve the
estimation of channel parameters, thus improving the reliability of the error code correction, while
reducing the number of required iterations for a successful decoding. This scheme is suitable for
application to multicarrier communications, such as OFDM. A PLC-like environment was chosen
as the basis for presenting the results.
Keywords
OFDM, PLC, LDPC, LDPC encoding, Multi-channel estimation
i
Resumo
Apesar de os códigos LDPC terem um excelente desempenho para tamanhos de código
grandes - códigos de um modo geral resistentes a situações de ruído de canal elevado e tolerando
falhas na equalização de canal - códigos com tamanhos pequenos e médios tendem a ser afectados por esses dois factores. Para esses dois casos, é apresentado um método que ajuda a
reduzir a dependência de correcta equalização de canal, sem modificar a arquitectura interna
do descodificador LDPC. Este objectivo é atingido coligindo métricas e parâmetros do funcionamento habitual do descodificador LDPC. Estes dados são usados de forma a reduzir os erros
na estimação das propriedades do canal, aumentando assim a fiabilidade do sistema de correcção de erros, reduzindo também o número de iterações necessárias para descodificar com
sucesso uma palavra de código. Este esquema proposto é adequado a sistemas de comunicação
multi-portadora, tais como OFDM. Um ambiente do tipo PLC foi o escolhido para apresentar os
resultados.
Palavras Chave
OFDM, Desempenho LDPC, codificação LDPC, Estimação multi-canal
ii
Contents
1 Introduction
1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
1.2 LDPC notation and symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
1.3 LDPC and OFDM trends and applications . . . . . . . . . . . . . . . . . . . . . . .
4
1.3.1 PLC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
1.3.2 DVB-S2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
1.3.3 10GBASE-T Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
1.3.4 WiMAX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1.3.5 Space Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1.3.6 Hard drives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1.3.7 G.hn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1.4 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
1.5 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
1.6 Dissertation Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
2 Multi-carrier Modulation and Modeling Transmission Channels
7
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
2.2 Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
2.3 Noise Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
2.4 OFDM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
2.5 OFDM Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
2.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
3 Low-Density Parity-Check Coding
15
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
3.2 LDPC codes defined . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
3.3 LDPC encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18
3.3.1 Fast encoding
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
3.4 Decoding linear block codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21
3.4.1 LDPC decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
iii
Contents
3.4.2 Noisy channel decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
3.4.3 Iterative decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
3.4.4 Node update equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24
3.4.5 Iterative decoding example . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28
4 Building LDPC Codes
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
4.2 Performance analysis of LDPC parameters . . . . . . . . . . . . . . . . . . . . . .
31
4.2.1 Equivalence of ensembles of LDPC codes . . . . . . . . . . . . . . . . . . .
32
4.2.2 Girth and small-cycles removal . . . . . . . . . . . . . . . . . . . . . . . . .
33
4.2.3 Regular vs Irregular codes . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
4.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
5 Multi-channel parameter estimation
39
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
40
5.2 Typical LDPC decoding architecture . . . . . . . . . . . . . . . . . . . . . . . . . .
40
5.3 Channel noise parameter estimation . . . . . . . . . . . . . . . . . . . . . . . . . .
41
5.3.1 Estimation via Bayesian inference . . . . . . . . . . . . . . . . . . . . . . .
42
5.3.2 Channel error probability distribution function . . . . . . . . . . . . . . . . .
43
5.4 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
44
5.5 System performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
5.6 Conclusions and Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
46
6 Conclusions and Future Work
49
6.1 Conclusions and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . .
50
6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
50
A Appendix
53
A.1 Conditional probability and Likelihood . . . . . . . . . . . . . . . . . . . . . . . . .
54
A.2 Intrinsic Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
54
A.2.1 AWGN Channel with BPSK modulation . . . . . . . . . . . . . . . . . . . .
54
A.2.2 Binary symmetric channel . . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
A.2.3 Binary erasure channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
A.3 Estimating BER via Importance Sampling . . . . . . . . . . . . . . . . . . . . . . .
56
A.4 Fast addition rule for check-node update equations . . . . . . . . . . . . . . . . . .
58
Bibliography
iv
29
61
List of Figures
1.1 Noisy channel coding schematic . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
2.1 Channels used for simulation several types of noise . . . . . . . . . . . . . . . . .
8
2.2 Frequency-selective noise BSC bank channel . . . . . . . . . . . . . . . . . . . . .
10
2.3 OFDM vs FDM spectral efficiency
. . . . . . . . . . . . . . . . . . . . . . . . . . .
11
2.4 OFDM modulation/demodulation functional block . . . . . . . . . . . . . . . . . . .
12
2.5 Simple OFDM modulation/demodulation functional block . . . . . . . . . . . . . . .
12
3.1 Example H matrix and associated Tanner graph representation
. . . . . . . . . .
17
3.2 Codeword x = [0 1 0 1 1 0 1 1] parity-checking . . . . . . . . . . . . . . . . . . . . .
18
3.3 Graphic representation of a generator matrix in systematic form . . . . . . . . . . .
18
3.4 Generator matrix graphical representation for H . . . . . . . . . . . . . . . . . . . .
19
3.5 Parity-check matrix in systematic and fast encoding forms . . . . . . . . . . . . . .
19
3.6 Two example of noisy channel models . . . . . . . . . . . . . . . . . . . . . . . . .
22
3.7 Edge numbering scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
3.8 Message flow for bit-nodes and check nodes update process . . . . . . . . . . . .
27
4.1 Flowchart for Finding high-performant LDPC codes . . . . . . . . . . . . . . . . . .
31
4.2 Coding performance of 3 identical sized codes . . . . . . . . . . . . . . . . . . . .
32
4.3 Thick solid: A 4-loop defined by vertexes v3 c1 v7 c3 , in both tanner graph (left) and
parity-check matrix forms (right). Dashed: a 6-loop defined by vertexes v3 c1 v5 c2 v1 c3 33
4.4 H1 vs H2 - with girth removal step applied . . . . . . . . . . . . . . . . . . . . . . .
35
4.5 Coding performance of regular vs irregular codes . . . . . . . . . . . . . . . . . . .
36
5.1 LDPC decoder block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
40
5.2 Subchannel bit accounting at LDPC decoder side . . . . . . . . . . . . . . . . . . .
41
5.4 Overall System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
44
5.5 Inner block structure of an LDPC decoder with the herein proposed addition of
“Intrinsic LLR estimator” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A.1 Memoryless system H(xi ), where xi is a signal + noise sample, with pdf fX (x).
.
45
56
v
List of Figures
A.2 Plot of F (x) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vi
59
List of Tables
1.1 Notation for LDPC codes used throughout this document . . . . . . . . . . . . . .
3
2.1 List of present-day communication systems employing OFDM . . . . . . . . . . . .
10
3.1 Complexity of online part of the encoding process . . . . . . . . . . . . . . . . . .
21
3.2 Decoding evolution of the message passing algorithm . . . . . . . . . . . . . . . .
27
3.3 Evolution of soft and hard decoding results at the decoder output x̂
. . . . . . . .
28
4.1 Girth histogram for the (N = 140, M = 35, j = 3, k = 12) ensemble . . . . . . . . .
32
4.2 Listing of each code n-cycles before and after the removal of small cycles . . . . .
35
4.3 Listing n-cycles for a regular and irregular codes . . . . . . . . . . . . . . . . . . .
36
4.4 Average number of iterations taken to decode a codeword, for regular and irregular
code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
5.1 Number of LDPC decoder iterations taken, uncorrected errors, with and without
LLR estimator block . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
46
vii
List of Tables
viii
List of Acronyms
10GbE
10 Gigabit Ethernet
ADSL
Asymmetric Digital Subscriber Line
APP
a posteriori probability
ARQ
Automatic Repeat Request
AWGN
Additive White Gaussian Noise
BEC
Binary Erasure Channel
BER
Bit Error Rate
BPA
Belief-Propagation Algorithm
BPL
Broadband over Power Line
BPSK
Binary Phase-Shift keying
BSC
Binary Symmetric Channel
COFDM
Coded Orthogonal Frequency-Division Multiplexing
DAB
Digital Audio Broadcast
DSL
Digital Subscriber Line
ECC
Error-Correction Code
ESA
European Space Agency
FDM
Frequency Division Multiplexing
FEC
Forward Error Correction
INESC-ID Instituto de Engenharia de Sistemas e Computadores - Investigação e
Desenvolvimento
IS
Importance Sampling
ix
List of Acronyms
ISI
Inter Symbolic Interference
ITU
International Telecommunication Union
LDPC
Low-Density Parity-Check
LLR
Log-Likelihood Ratio
MAP
Maximum a posteriori probability
MC
Monte Carlo
MoCA
Multimedia over Coax Alliance
MPA
Message-Passing Algorithm
MPEG
Moving Picture Experts Group
NASA
National Aeronautics and Space Administration
OFDM
Orthogonal Frequency-Division Multiplexing
pdf
probability density function
PEG
Progressive Edge Growth
PCM
Parity-Check Matrix
PLC
Power Line Communication
QAM
Quadrature Amplitude Modulation
RS
Reed-Solomon
SMPS
Switched-Mode Power Supply
SPA
Sum-Product Algorithm
TC
Turbo Code
UWB
Ultra-Wide Band
VA
Viterbi Algorithm
VLSI
Very-Large-Scale Integration
WiMAX
Worldwide Interoperability for Microwave Access
x
1
Introduction
Contents
1.1 Introduction . . . . . . . . . . . . . . . . .
1.2 LDPC notation and symbols . . . . . . . .
1.3 LDPC and OFDM trends and applications
1.3.1 PLC . . . . . . . . . . . . . . . . . .
1.3.2 DVB-S2 . . . . . . . . . . . . . . . .
1.3.3 10GBASE-T Ethernet . . . . . . . . .
1.3.4 WiMAX . . . . . . . . . . . . . . . .
1.3.5 Space Communications . . . . . . .
1.3.6 Hard drives . . . . . . . . . . . . . .
1.3.7 G.hn . . . . . . . . . . . . . . . . . .
1.4 Motivation . . . . . . . . . . . . . . . . . .
1.5 Objectives . . . . . . . . . . . . . . . . . .
1.6 Dissertation Outline . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2
3
4
4
4
4
5
5
5
5
6
6
6
1
1. Introduction
1.1
Introduction
The design and project of any contemporary communication system requires trade-offs between system performance, power consumption, reliability and cost. Forward Error Correction
(FEC) is one of the available tools for performing such trade-offs. FEC is a method that adds ancillary data as redundancy to a given message. This enables the receiving end to use this ancillary
data to detect and/or perform recovery in the event the transmitted message is corrupted.
The addition of any sort of FEC always increases overall system complexity and reduces
usable bandwidth, but at the same time, it lowers transmission error rates. Channel noise and
interference from other communications or electronic devices are not the only sources that push
for the usage of FEC: worldwide telecommunication regulation agencies impose restrictions on
the amount of power allowed for certain classes of communications systems. Lower power levels
means higher transmission errors, thus, at the cost of increasing system complexity, FEC can also
be used to reduce transmission errors, while complying with local power level regulations.
Several modulation schemes can be used to transmit data over the Power Line Communication
(PLC) channel, but Orthogonal Frequency-Division Multiplexing (OFDM) is well suited to lessen
most of the channel’s conditions such as interference, attenuation and fading. Adding the ability
of FEC to an OFDM system gives rise to what is usually called Coded Orthogonal FrequencyDivision Multiplexing (COFDM). This document deals with improvements made to an OFDM
system when using a type of FEC called Low-Density Parity-Check (LDPC) coding.
LDPC codes are a very successful family of FEC codes. LDPC codes are a class of linear
block codes that can achieve near channel capacity. They were firstly proposed in the 1960’s by
Gallager [1], in his doctoral dissertation. However, for the most part of the next 30 years, those
codes remained as forgotten. In the mid 90’s these codes were “rediscovered” independently
by different researchers [2], who noticed the very desirable properties of linear block codes with
Low-Density (i.e. sparse) Parity-Check matrices. The current maximum error correction capacity
for LDPC codes currently falls within 0.0045dB of the Shannon limit [3], although the feasibility
of such codes (and associate decoders) for real-time applications can be questioned due to the
excessive requirements of computational resources.
The need for error correcting
Shannon’s noisy-channel coding theorem [4] establishes that it’s possible to receive an errorfree message, sent over an unreliable channel, as long as the information rate does not exceed
the capacity of the channel. The theorem also assures there is an optimal Error-Correction Code
(ECC) scheme, which just adds a minimal redundancy to the original message.
Unfortunately, Shannon’s theorem is non-constructive: it does not state how to build such an
encoder/decoder for (nearly) error-free communications. The notation for the noisy channel cod2
1.2 LDPC notation and symbols
Figure 1.1: Noisy channel coding schematic
ing is as follows (figure 1.1): a message u is subject to an optimal encoder that produces an
encoded representation, x, of the original message. This codeword is sent through a noisy channel, where errors, e, are added. At the receiver-end, the decoder tries to recreate the original
message from the channel output y, establishing a most likely representation x̂ for the encoded
message. After discarding the redundancy part in the estimate, the original message u is recovered.
This thesis focus on LDPC coding, a class of FEC codes. Although building such system is not
a trivial task, it can be shown that effective trade-offs can be made between decreasing the channel capacity (by virtue of adding error-correcting redundancy), and message delivery reliability.
All of this while trying to keep the system computationally feasible for real-time applications.
1.2
LDPC notation and symbols
Before starting to introduced LDPC codes, the parameters (and its symbols) that define an
LDPC code are introduced in table (1.1), which provides a convenient reference for all LDPC
used symbols.
Symbol
Meaning
H
C
u
x
y
Parity-Check Matrix (PCM)
linear block code
source message vector - user message/user bits
transmitted codeword vector - encoded sequence of user bits
received codeword vector: original codeword x after being sent through a
noisy channel
Generator matrix
girth - length of G shortest cycle of H
H column weight - number of H non-zeros per column
H row weight - number of H non-zeros per row
number of H columns, variable nodes and codeword length
number of H rows and number of check nodes
source message size
code rate
check-node m
variable-node n
G
g
j
k
N
M
K
R
cm
vn
Table 1.1: Notation for LDPC codes used throughout this document
3
1. Introduction
1.3
LDPC and OFDM trends and applications
LDPC codes are currently employed in communication systems standards such as DVB-S2,
10 Gigabit Ethernet (10GbE), Worldwide Interoperability for Microwave Access (WiMAX). The
majority of deep space missions, have being using LDPC instead of Reed-Solomon (RS) or other
coding mechanisms, since 2000. The hard-drive storage industry is also considering replacing
the current RS coding, as a way to increase storage density, and achieve lower error floors.
OFDM Most contemporary communication systems use OFDM as the basis for communications. Technologies such as Asymmetric Digital Subscriber Line (ADSL), Wi-Fi, DVB-S2, etc . . . all
employ OFDM due to its spectral efficiency, and resilience towards channel errors. See table 2.1
for a broad listing of systems and technologies that employ OFDM.
1.3.1
PLC
Power Line Communication (PLC) is a class of systems where data is carried over the power
transmission grid. Broadband over Power Line (BPL) systems deliver broadband services over
the typical household power line. The PLC is an extremely noisy and time changing medium
where any transmission is subject to the influence of random burst noise, propagation delays,
and selective erasures. The resilience of OFDM to harsh channel conditions makes it an evident
choice as the standard for modulation in PLC systems. HomePlug AV is one of the available PLC
consumer products that uses ECC in the form of Turbo Coding.
1.3.2
DVB-S2
DVB-S2 [5] was the first large scale system deployment to use LDPC as basis of a FEC
system. Although DVB-S2 is typically used for satellite television/broadcasting, it is suitable for all
kinds of high data rate transmission, because DVB-S2 can be used to transmit bit-stream as well
as packetized data, which audio and video applications (MPEG) are just a subset. DVB-S2 uses
large code lengths (16200 bits). As with all Linear Block Codes, larger code block lengths, result in
better LDPC coding performs as compared to other coding methods, such as Turbo Codes (TCs)
that tend to be used with smaller code-lengths. Some near-earth space missions also use the
DVB-S2 transmission standard as the basis for their communication systems, due to the ease of
obtaining off-the-shelf DVB-S2 equipment, taking advantage of the hardware commoditization.
1.3.3
10GBASE-T Ethernet
The 802.3an standard for 10Gb/s Ethernet, requires LDPC [6] coding as FEC mechanism
because the noise levels and attenuation are quite significant for twisted-pair copper cabling at
200MHz transmission frequencies. The particular code used, has an error-floor of 10−13 . It is a
(6,32)-regular RS-LDPC code length 2048 and dimension 1723. The reference encoder boasts a
4
1.3 LDPC and OFDM trends and applications
gate count of ≈27k gates, and produces a codeword within 30ns latency, for a 1MHz clock. The
iterative decoder has a latency of 0.32 µs at 450MHz.
1.3.4
WiMAX
The IEEE 802.16e standard, WiMAX, also includes the usage of LDPC codes - among others
- as FEC mechanism. The stated maximum protocol bandwidth is ≈70Mbit/s, although for non
line-of sight environments, typical speeds of 10Mbit/s for a 2km distance can be achieved. The
reference encoder has a gate count of ≈20k gates, while the iterative reference decoder has a
gate count of ≈125K gates.
1.3.5
Space Communications
Most of National Aeronautics and Space Administration (NASA) and European Space Agency
(ESA) probes use LDPC instead of RS or TC as coding mechanisms [7], since 2000. These
missions are mostly power constrained and thus transmission bandwidth is not a major bottleneck.
Employing LDPC coding lowers the amount of power required for successful transmissions, at the
residual cost of slightly increasing bandwidth.
1.3.6
Hard drives
In other to cope with the increasing bit error levels, consequence of higher hard-disk recording
densities [8], LDPC codes have being proposed as an ECC scheme [9]. Given the low complexity/latency and high throughput expected of an hard-disk drive, these LDPC implementations have
several constrains: code lengths are short (i.e. code parameters (N = 30, M = 5, j = 1, k = 6))
and decoder structures are unfolded instead of being fully iterative, being usually just a few levels
deep (4-6 iterations). In the first generation of hard drives with LDPC as FEC mechanism, LDPC
was used as an inner code while the outer coding was a RS code. Current generations of hard
drives employ a single LDPC code instead of the concatenation of these two separate codes [8].
1.3.7
G.hn
The G.hn/G.9960 are International Telecommunication Union (ITU) draft standards for nextgeneration networking. One of the chief advantages of G.hn specification is unified Physical and
Data Link layers, that can operate over multiple wire types, such as power lines, phone lines
and coaxial cable. The physical layer is based on OFDM as modulation method and LDPC as
the Forward Error Correction (FEC) layer. Besides employing strong FEC for handling with noisy
mediums, Selective ARQ and methods for dealing with line AC cycle are also employed. Expected
throughput values are in the range of 800 MBit/s to 1Gbit/s, for ethernet-link data-links.
5
1. Introduction
1.4
Motivation
Consumer grade PLC systems have been plagued by slow market adoption from its inception,
due to lower apparent performance, and lower installation easiness than its main competitors
technologies (Digital Subscriber Line (DSL), cable). The main factor contributing to the PLC
lower performance is the usage of the power distribution grid, as the communication medium.
The power line medium is a time-changing high noise environment, which heavily hinders the
propagation of the communication signal, degrading performance. Although through the years
several improvements have been made to the transmission apparatus, such as new modulations
and channel equalization techniques, these have topped their usage potential, and no longer
provide an effective efficiency increase in system performance. A different approach must be
taken in order achieve the reliability (and tentatively surpass) of competing technologies.
1.5
Objectives
The present work purpose, is to show that an high level of transmission reliability can be
achieved for PLC systems, using OFDM with LDPC coding. That work is substantiated by the
development of simulation benchmarks, testing several implementation options.
The presented work builds upon previous work developed at Instituto de Engenharia de Sistemas e Computadores - Investigação e Desenvolvimento (INESC-ID), namely a basic communication hardware system with just has plain OFDM, and no ECC ability.
1.6
Dissertation Outline
The dissertation is split into two major parts: The first part (chapters 2 and 4), detail the usage
systems with OFDM and with LDPC as a FEC mechanism, trade-offs, and decisions made in their
implementation. Chapter 3 details the theory behind LDPC codes.
The second part of the thesis (chapters 5 and 6) deals with the application of the methods
developed in the first part to an OFDM communication system in a PLC environment and its result
and considerations. At the end some hints for future work are pointed out.
6
2
Multi-carrier Modulation and
Modeling Transmission Channels
Contents
2.1
2.2
2.3
2.4
2.5
2.6
Introduction . . . . . .
Channels . . . . . . . .
Noise Scenario . . . . .
OFDM . . . . . . . . . .
OFDM Implementation
Conclusions . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. 8
. 8
. 9
. 10
. 12
. 13
7
2. Multi-carrier Modulation and Modeling Transmission Channels
2.1
Introduction
This chapter introduces several channels and modulation schemes used. Channel modeling
of Frequency Division Multiplexing (FDM) modulation is detailed, including simplifications and
assumptions.
The most significant advantage of FDM modulation schemes regarding frequency-selective
noise, is that in the event of a portion of the frequency spectrum being rendered useless, transmission is still possible using the parts of the spectrum not affected by such noise. The PLC
environment is heavily affected by this type of noise, where a single home-appliance, perhaps a
simple electric motor or a Switched-Mode Power Supply (SMPS), can pollute bands of the electromagnetic spectrum.
2.2
Channels
In evaluating the response of a given system to noise/perturbations, it is possible to use different types of noisy channel, where each type of model usually highlights some specific kind of
real-world perturbation.
Usually, for communication systems that employ LDPC codes, the following three channels
tend to be used in modeling the various kinds of errors (figure 2.1) a communication system is
subjected to:
(a) Model and parameters of
BSC channel
(b) Model and parameters of
BEC channel
(c) Model
and parameters of
BPSK-AWGN channel
Figure 2.1: Channels used for simulation several types of noise
The first channel (figure 2.1a) is the Binary Symmetric Channel, a channel that models the
kinds of noise that appear as bit flips. The BSC is a channel where a sent bit has a probability
pc (probability of cross-over) of being flipped, and a 1 − pc probability of being transmitted without
being affected by noise.
Figure 2.1b depicts the Binary Erasure Channel. The BEC models the types of noise that
erase transmitted bits - that is, the receiver is unable to reach a conclusion whether the original
bit was either a 1 or 0. The probability of a given bit being erased is pe , while the probability of a
bit being transmitted without being erased is 1 − pe .
8
2.3 Noise Scenario
The last channel (2.1c) is a Binary Phase-Shift keying (BPSK) modulated channel with Additive
White Gaussian Noise (AWGN). This channel models the case of AWGN, one of the most common cases of “real-world” noise, where the original bits are disturbed by an amount of noise
proportional to σ 2 .
Of these three, only the BSC and the BPSK-AWGN channels will be amply used in this thesis.
The Binary Erasure Channel (BEC) will not the used, since for small-to-medium size of LDPC
codes - the target of this thesis - small LPDC codes don’t recover efficiently erasures, especially
burst erasures [5, 9]. Real-world systems that employ small-to-medium LDPC codes, tend to use
an outer code such as Reed-Solomon (RS) code - which encapsulates the LDPC payload [8] - to
deal with burst erasures. The inner/outer split is done because different families of code, perform
differently in the presence of different kinds of noise. Large LDPC codes can deal admirably well
with burst erasures, but those large codes aren’t the target of this thesis (see chapter 6).
2.3
Noise Scenario
As previously stated, it is assumed that the proposed communication system operates in a
frequency-selective noise environment, typical of PLC or DSL environments. The best way to
deal with frequency-selective noise, is to discard all sub-bands affected by noise, while using the
remainder sub-bands. By design OFDM systems already split modulated data through several
subcarriers / sub-bands. In this thesis, it is introduced a modification for the Binary Symmetric
Channel (BSC) that takes into account frequency-selective noise. The standard BSC channel is
composed into a bank-like structure, where each subcarrier, is a single standard BSC - figure 2.2.
Each subchannel 1,2, . . . , N, has its own probability of cross over, pc1 , pc1 . . . pcN . In this stacked
model, each single BSC is intended to model a particular frequency of frequency -selective noise.
Here lies the major assumption in this thesis: for the modifications made to the LDPC decoder,
it is irrelevant where in the electromagnetic spectrum the first and subsequent subcarriers are
located [Hz], how each one is spaced, interleaved, what was the modulation type, etc. The
additions and modifications done to the LDPC decoder result that the decoder is only concerned
with the provenance of every bit and through what channel did a particularly bit come through.
The reasoning is that each subcarrier is essentially sampling the noise of that band of spectrum.
Thus, all uncorrected bits coming from that subcarrier, are sampling the same sub-band of the
spectrum, regardless of the modulation scheme. As long as such bit vs channel accounting is
possible the work is this thesis is applicable.
9
Figure 2.2: Frequency-selective noise BSC bank channel
2.4
OFDM
Orthogonal Frequency-Division Multiplexing is a digital multi-carrier modulation scheme, firstly
proposed in 1966 [10]. Regarding other FDM systems, OFDM boasts advantages such as improved spectral efficiency, and robustness against channel interference, though these advantages
come at the expense increased system complexity. Table 2.1 lists present-day communication
systems and services employing OFDM.
Usage
System / Service
Broadband
ADSL and PLC (both wired)
Computer networks
WiFi, IEEE 802.11a/n and WiMAX
Digital radio
DAB, DAB+, Digital Radio Mondiale, HD Radio, T-DMB
and ISDB-TSB
Terrestrial TV
DVB-T, DVB-H, T-DMB, ISDB-T and MoCA (wired)
Satellite TV
DVB-S2
Cellular communication
Flash-OFDM
Personal Area Networks
UWB
Mobile broadband
3GPP and HSOPA
Table 2.1: List of present-day communication systems employing OFDM
The basic principle of OFDM, like all FDM schemes, is to assign several frequency ranges
to different signals. However, the orthogonality principle allows to go beyond the tight packing
of non-overlapping subcarriers of FDM: with OFDM it is possible to achieve reliable transmission
with overlapping adjacent subcarriers, thus reducing the required bandwidth (figure 2.3). These
10
2.4 OFDM
subcarriers are modulated with standard modulations such as BPSK, Quadrature Amplitude Modulation (QAM), etc.
Figure 2.3: OFDM vs FDM spectral efficiency
As can be seen from the figure 2.3, as the number of subcarriers, Nc , grows, OFDM allows
for tighter and tighter packing of subcarriers, since it is possible to overlap adjoining subcarriers.
For the bottom example, where Nc = 3, for OFDM the spectral usage is BW = 4R/3 versus the
BW = 2R for FDM. Thus more subcarriers can be fitted into the same bandwidth window, BW ,
which means an higher spectral efficiency for OFDM vs FDM.
Another major feature of OFDM is splinting the original bitstream into lower-rate subcarriers.
This lower bitrate of each subcarrier allows using guard intervals between symbols, reducing
propagation delay errors, lowering Inter Symbolic Interference (ISI).
Any frequency mis-synchronization between transmitter and receiver degrades the performance of an OFDM receiver. This is a consequence of the detection method, time-integration
of the incoming signal. Any jitter between both ends, may cause an incoming symbol to fall outside the integration time window of the decoder, thus increasing the symbol error rate.
An overview of a complete OFDM modulator/demodulator system is shown in figure 2.4. At
the transmitter side, the modulation process starts with a serial-to-parallel transform applied to the
input bitstream, which is split among N substreams. Next, each substream is individually mapped
with a standard modulation such as BPSK. Then, the FFT is computed for each substream. The
OFDM symbol is created by simply summing all the N FFT outputs. This symbol is then converted
into the analog domain where it is shifted into an appropriate RF carrier.
At the receiver end, the reverse process takes places: After downmixing the analog signal
from its RF carrier, the baseband signal is converted back into the digital domain. The digital
samples of the received OFDM symbol are then demodulated via a FFT into the same number, N,
11
Transmitter side
Original
message
0101 1001 1100
Serial to
parallel
conversion
011
101
000
110
Modulated
message
OFDM
Mod.
Noisy
Channel
Receiver side
De-modulated
message
Noisy
Channel
OFDM
Demod.
011
101
000
110
Received
message
Parallel to
serial
conversion
0101 1001 1100
Figure 2.4: OFDM modulation/demodulation functional block
of subcarriers. Once these individual lower rate subcarriers are demodulated, they are regrouped
into an output bitstream via a parallel-to-serial transformation.
2.5
OFDM Implementation
The intended OFDM system used in this thesis, is a simple one, firstly because the thesis
focus is coding theory, and secondly because the features of OFDM that are the attention of this
work - its wide-spread adoption and its serial-to-parallel transmission architecture, that allows to
perform the bit vs carrier accounting mentioned earlier - are already built in the basis of OFDM.
Features such as cyclic prefix, training pilots, changing modulation type according to channel SNR
are omitted.
The implemented OFDM system has the general architecture presented in figure 2.4), which
is a standard OFDM modem, but implemented as in figure 2.5.
The modulator block is simply serial-to-parallel converter, followed by BPSK modulation step,
followed by the last step where an IFFT block that generates the OFDM symbol.
The demodulator performs the reverse transformations: IFFT, followed by BPSK demodulation, and finally performs parallel-to-serial conversion, but retaining the information of which bits
traveled through which carrier.
Figure 2.5: Simple OFDM modulation/demodulation functional block
12
2.6 Conclusions
2.6
Conclusions
This chapter presented various channels typically used in Coding Theory, and also the type of
channels employed in this thesis. The major idea and assumption of this thesis was introduced:
That from the standpoint of coding theory, the LDPC decoder is only concerned with the provenance of every bit and through what subchannel/subcarrier did a particularly bit come through no other aspect of communication systems theory is taken into account.
The implemented OFDM solution was presented and the assumptions regarding the coupling
to the LDPC decoder were introduced.
13
14
3
Low-Density Parity-Check Coding
Contents
3.1 Introduction . . . . . . . . . . .
3.2 LDPC codes defined . . . . . .
3.3 LDPC encoding . . . . . . . . .
3.3.1 Fast encoding . . . . . . .
3.4 Decoding linear block codes . .
3.4.1 LDPC decoding . . . . . .
3.4.2 Noisy channel decoding .
3.4.3 Iterative decoding . . . . .
3.4.4 Node update equations . .
3.4.5 Iterative decoding example
3.5 Conclusions . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
16
16
18
19
21
22
22
23
24
25
28
15
3. Low-Density Parity-Check Coding
3.1
Introduction
Low-Density Parity-Check (LDPC) codes are a class of linear block codes that can achieve
near channel capacity [3]. They were first proposed in the 1960’s by Gallager [1], in his doctoral
dissertation.
This chapter starts by presenting the definition, parameters and structure of LDPC codes, then
details the encoding and decoding processes, their performance and shortcomings.
3.2
LDPC codes defined
As a linear-block code, an LDPC code has a parity-check matrix representation, denoted by
H, although to be considered as a LDPC code, it has to satisfy some specific construction rules
and parameters. All LDPC code parameters are linked to several properties of the (M × N ) H
matrix. The LDPC parameters are: (N, M, j, k), where N is code block-length, M is the number
of constraints - or check nodes - j and k are respectively H column and row weight (weight: the
number of non-zero elements in a row/column). The codeword size, N , is related to the message
size K, by the expression N = M + K. The code rate (the portion that is used for message data)
is given by R = (N − M )/N = 1 − M/N
A code is said sparse or low-density (hence its name) when k N and j M . This is the
first main “feature” of LDPC codes: the H matrix sparseness/low-density.
An example of an (N = 8, M = 4, j = 2, k = 4) LDPC code (non-sparse for demonstration
only purposes), is the following 4 × 8 matrix:

1 0 1
1 0 0
H=
0 1 1
0 1 0
0
1
0
1
1
0
0
1
0
1
1
0
1
0
1
0

0
1

0
1
(3.1)
The rate for this code is R = 1 − 4/8 = 0.5 which is a code rate of 50%. Thus for every 4
message bits another 4 bits are added as redundancy.
The code presented in the previous matrix is regular since every row has the same number of
1’s, and every column has also the same number of 1’s:
X
H= 2
2
2
2
2
columns
2
2
2
 
4
X
4

H=
4
rows
4
Regular codes verify the equation N · j = M · k. Irregular codes are codes that have non-uniform
distribution in the number of 1’s per row/column.
Besides the parity-check matrix representation shown above, LDPC codes have a undirected
bipartite graph representation, as introduced by Tanner in [11]. This type of graphs, which are
particular useful for studying the decoding process of LDPC codes, are called Tanner Graphs in
16
3.2 LDPC codes defined
his honor. The graph is called bipartite because there are two different types of nodes connected
by the edges: variable-nodes depicted as circles (#), and check-nodes depicted as boxes (2)
(following the notation in [12] henceforth referred as v-nodes and c-nodes). The Tanner graph for
the previous example of H is shown in the figure 3.1:
Figure 3.1: Example H matrix and associated Tanner graph representation
Each H non-zero element is an edge in a Tanner graph. Each edge connection from check
node cm to a variable node vn (or vice-versa) corresponds to a non-zero H entry, Hm,n 6= 0.
As can be seen from figure 3.1, there are M = 4 rows (4 check nodes) and N = 8 columns
(8 variable nodes). Notice the correspondence between the graph and the H matrix: From the
graph, it can be seen that the first check node (top-left) is connected to the 0th, 2nd, 4th and 7th
check-nodes. This corresponds exactly to H’s first row non-zero elements. From a variable node
perspective, the first v-node (bottom-left) is connected the 0th and the 1st c-node. This matches
the first column of H non-zero elements. Thus, the M rows of H define the connections of each
cm and the N columns of H define the connections of each vn .
Check nodes receive their name because each node can be seen as an check equation that
detects the even-parity of the codeword bits present at the adjacent v-nodes. These v-nodes
contain the values depicted in figure 3.2, which has the following check-node equations, where ⊕
is the binary XOR operator:
c0 = v0 ⊕ v2 ⊕ v4 ⊕ v6
c1 = v0 ⊕ v3 ⊕ v5 ⊕ v7
(3.2)
c2 = v1 ⊕ v2 ⊕ v5 ⊕ v6
c3 = v1 ⊕ v3 ⊕ v4 ⊕ v7
This is the second main “feature” of LDPC codes: the parity-checking ability.
As an example, the codeword x = [0 1 0 1 1 0 1 1], present at the v-nodes (figure 3.2) verifies
17
Figure 3.2: Codeword x = [0 1 0 1 1 0 1 1] parity-checking
all check equations (equation 3.2):
c0 = 0 ⊕ 0 ⊕ 1 ⊕ 1 = 0
c1 = 0 ⊕ 1 ⊕ 0 ⊕ 1 = 0
(3.3)
c2 = 1 ⊕ 0 ⊕ 0 ⊕ 1 = 0
c3 = 1 ⊕ 1 ⊕ 1 ⊕ 1 = 0
This example codeword also satisfies the linear block-code check equation xHT = 0, because
any LDPC code is also a linear block-code.
3.3
LDPC encoding
(a) H in sparse representation
(b) H in systematic form
Figure 3.3: Graphic representation of a generator matrix in systematic form
Because LDPC codes are block codes, the same encoding methods apply. The textbook
approach for encoding linear block codes is a standard vector-matrix operation, x = uG. The
generator matrix usually has the form G = [Ik |P], where P is a dense matrix containing the parity
information, obtained by reducing the original H matrix to the systematic form H = −PT |In−k
through Gaussian elimination. Figure 3.3b is graphical representation of an example parity check
matrix in systematic form, where black pixels represent non-zero elements. The original H in
sparse representation is depicted in 3.3a.
The generator matrix G of the example code is presented in figure 3.4. Most of the generator
matrix is a largely an identity matrix. Generating a codeword, x = uG, is thus an inexpensive
18
3.3 LDPC encoding
Figure 3.4: Generator matrix graphical representation for H
n-m
n-m
m
1
1
g
m-g
1
1
1
1
1
0
1
1
m
1
1
1
1
1
1
B
A
T
m
C
D
1
1
0
1
E
1
m-g
1
g
1
n
n
(a) Parity-check matrix in lower triangular systematic
form
(b) Parity-check matrix in approximate lower triangular
form
Figure 3.5: Parity-check matrix in systematic and fast encoding forms
operation (software or hardware wise) for the Ik part of G. The expensive part is the dense
right-side block.
For dense matrices, an n sized vector-matrix multiplication has complexity O(n2 ). Thus, for
large values of n the computational effort required for encoding via generator matrix can be quite
large.
To achieve linear encoding, methods such as restricting the structure of the code, or forcing
the parity-check matrix to have a lower triangular form (figure 3.5a) have been proposed since
the appearance of LDPC codes, but these simplifications and/or restrictions in the code structure
always result in loss of coding performance [3].
3.3.1
Fast encoding
After the re-discovery [13] of LDPC codes, one of the major drawbacks towards their widespread
usage was high encoder complexity, specially compared to Turbo-Codes, which boasted lineartime encoding. Richardson & Urbanke in 2001 [14] introduced a method that allowed efficient
encoding of LDPC codes in linear time, with the provable maximum complexity:
C(n) = 0.0172 n2 + O(n)
(3.4)
19
Because the constant affecting the quadratic term is very small, their method can be used even
for large codes. The algorithm has two parts: an offline (prepossessed) and an online part.
In the prepossessed part of the algorithm, the matrix H is firstly rearranged into an approximate triangular form (figure 3.5b) suitable for fast encoding. This is achieved by iteratively permuting the row/columns until a suitable partition of H is found (the limiting criterion is given in the
explanation of equation 3.9). The partitioning process splits H into six sub-matrices:
A B T
H=
C D E
(3.5)
The sub-matrices sizes (figure 3.5) are: A is (m − g, n − m), B is (m − g, g), T is (m − g, m − g),
C is (g, n − m), D is (g, g) and E is (g, m − g). Left multiplying H by
I
0
−ET−1 I
achieves:
A
B
H=
−ET−1 A + C −ET−1 B + D
T
0
(3.6)
Splitting a codeword into x = [u, p1 , p2 ], where p1 and p2 are intermediary parity vectors, the
parity check equation HxT = 0 results into:
AuT + BpT1 + TpT2 = 0
(3.7)
(−ET−1 A + C)uT + (−ET−1 B + D)pT1 = 0
(3.8)
p1 can be easily obtained from the last equation (3.8):
pT1 = φ−1 (−ET−1 A + C)uT
(3.9)
where φ = (−ET−1 B+D). In the event of φ being singular, another partition for H must be found,
possibly with another value for g.
The intermediary parity vector p1 , can be rewritten as
pT1 = Moff uT
(3.10)
where the Moff matrix can be fully pre-computed. This greatly reduces the number of computations required in the online part of the encoding algorithm.
In a similar manner, pT2 can be obtained from equation 3.8:
pT2 = −T−1 (AuT + BpT1 )
(3.11)
In the online part of the algorithm (i.e. the actual encoding), a LDPC codeword vector is
obtained by the concatenation of the user bits (n − m row vector) and the intermediate p1 and p2
parity vectors.
20
3.4 Decoding linear block codes
Operation
T
Moff · u
A · uT
B · pT1
−1
T · A · uT
Comments
Complexity
dense matrix multiplication
sparse matrix multiplication
sparse matrix multiplication
sparse matrix multiplication, performed by backsubstitution, because T−1 is lower triangular and
also sparse
O(g × (n − m))
O(n)
O(n)
O(n)
Table 3.1: Complexity of online part of the encoding process
In the previous equations, A and B are sparse matrices, while Mof f is a dense (g, n − m)
matrix. The computation of Moff is the most complex part of the whole encoding process. Table
3.1 details the online encoding process complexity namely, equations 3.10 and 3.11.
In conclusion, the “efficient encoding” algorithm [14] goal is to provide a fast encoding method
for LDPC codes. This is achieved by rearranging and partition the H matrix with a gap parameter,
g, as small as possible, because, as can be seen from table 3.1, the most expensive step is the
dense matrix multiplication with complexity O(g × (n − m)). Thus, as stated in equation 3.4, the
encoding process complexity is set by g.
Another advantage of this encoding process is storage savings: There is no need to store
the generator matrix G, because a full encoder/decoder system can perform both operations with
only the parity-check matrix H.
3.4
Decoding linear block codes
A message (a sequence of K user bits, u = [u0 u1 . . . uK−1 ]) in encoded through linear block
coding into a N bit sequence, called a codeword, x = [x0 x1 . . . xN −1 ]. The encoding transform
adds redundancy to the user bits, with the purpose of performing error detection/recovery upon
reception. The process of linear block encoding can be mathematically expressed as x = u G
where G is a specially crafted matrix (generator matrix) which has the rules for adding redundancy
to u. Unreliable and/or noisy channels can introduce errors in the transmitted codeword. However,
the receiver must be able to i) check if the received codeword is valid and error-free, ii) perform
its decoding in order to retrieve the original user bits, and iii) in case of errors, perform some kind
of error correction.
From a generator matrix, a parity-check matrix H can be derived, where H = −PT |In−k ,
if and only if the G matrix has a decomposition such as G = [Ik |P]. The H matrix has all the
verification rules that a given codeword must satisfy in order to be declared error free [1]. For linear
block codes, satisfying the condition x HT = 0 proves that a codeword is valid and error-free. This
property, however, allows only detecting errors, not correcting them.
In order to correct errors, one can apply methods such as the algebraic syndrome decoding
method, or a probabilistic method such as the Viterbi Algorithm (VA) [15], which minimizes the
21
probability of decoding error in a given codeword. Although the VA decoding was conceived for
convolutional decoding, the work by [16] showed that soft decoding techniques of convolutional
decoding could be applied to block codes. This brought the power of maximum likelihood decoding
to block codes, because the algebraic decoding, although quite powerful, only performs well in
low-noise/linear channels.
3.4.1
LDPC decoding
One of the major challenges in implementing LDPC coding system is the complexity or feasibility of the decoder. Although LDPC decoding is a NP problem, Gallager also introduced a
near-optimal decoding [17] algorithm. The decoding algorithm has different names, since it was
“invented” several times in distinct domains: Sum-Product Algorithm (SPA), the Belief-Propagation
Algorithm (BPA), and the Message-Passing Algorithm (MPA).
3.4.2
Noisy channel decoding
The working basis of every decoder placed at the receiving end of a noisy communication
channel is “infer the original input, given the channel output”. In the probability domain, this can
be restated as: “for a given channel input x, from a probability ensemble X, the output y is related
to x by the following joint ensemble XY ”:
P (x, y) = P (y|x) P (x)
(3.12)
Using Bayes’ Theorem, the probability of a received codeword y, given the input, x can be
computed as:
P (y|x) =
(a) Binary Erasure Channel
P (y|x) P (x)
P (y)
(3.13)
(b) Binary Symmetric Channel
Figure 3.6: Two example of noisy channel models
The BEC, the BSC and the AWGN channel are the most used models of noisy channels,
because most problems in communication theory can be reduced to analyzing these models.
22
In the BEC (figure 3.6a), a binary symbol X has a probability pe of being “erased”, that is the
receiver has no idea what the original bit was. Bits not erased are assumed to be error free.
In the BSC (figure 3.6b), a binary symbol X has a probability pc of “crossing over”, that is
flipping from 1 → 0 or from 0 → 1.
The AWGN channel the input symbols are impaired with the addition of wide band white noise
having a Gaussian distribution of amplitude. The channel parameters are the mean and the
variance of such noise (µ, σ).
3.4.3
Iterative decoding
The purpose of iterative decoding algorithms applied to the noisy channel problem is to compute and estimate of original codeword x given the sampled channel output y, while minimizing
the decoded sequence’s bit error rate, x̂. This is achieved by computing the Maximum a posteriori
probability (MAP) for P (x|y).
For LDPC decoding, the most efficient known method for computing each of the xn bits is the
a posteriori probability (APP), P (xn = 1|y). Because the decoding is performed on graphs, the
particular algorithm used for computing the APP is the Message-Passing Algorithm (MPA). This
method iteratively computes the probability of a transmitted codeword bit (xn ) being 1, given all
received codeword’s bits y = [y0 y1 . . . yN −1 ], that is P (xn = 1|y).
Instead of having to deal separately with the P (xn = 1|y) or P (xn = 0|y), the APP ratio can
be used:
. P (xn = 1|y)
l(xn ) =
P (xn = 0|y)
(3.14)
Or, for better numerical stability, the log APP ratio, Log-Likelihood Ratio (LLR), can be used:
λ xn
.
≡ ln (xn ) = ln
P (xn = 1|y)
P (xn = 0|y)
(3.15)
The MPA is an iterative algorithm based on the code’s graph for the computation of P (x = 1|y),
l(xn ) or λxn . The decoder operates as follow: i) iteratively, the decoder evaluates the codeword’s
check-constraints, ii) notifies neighboring nodes of the confidence level of that bit being correct.
iii) Each node, given the new incoming confidence level, recomputes his own confidence levels,
and again iv notifies neighboring nodes. This confidence propagation occurs through all edges,
from c-nodes to v-nodes, and back again, several times, until a predefined number of iteration is
reached, or the codeword is valid.
The estimate of a given bit be 0 or 1 in the log-likelihood domain is given by:
(
λxn > 0 ⇒ x̂n = 1
λxn < 0 ⇒ x̂n = 0
(3.16)
Thus the estimate that a received bit being 0 or 1 is depends only on the sign of λxn , x̂n =
sign(λxn ). Establishing the value of x̂n only using the sign information is quite a coarse evaluation.
23
The magnitude of the λxn carries the reliability of the estimate of x̂n being towards 0 or 1. For
example, for λxn = 0, x̂n has the same likeliness of being a 0 or 1.
Because the computation of the log-MAP via the MPA (computing the estimate of each xn
value), occurs on a graph, two sources of reliabilities are available: the reliability of xn given yn ,
and the reliability of xn given all the bits of y except yn , that is: y\n . Taking into account these
two sources of reliabilities, and applying Bayes’ theorem to both the numerator and denominator
of equation 3.15:
P (xn = 1|yn , y\n )
=
P (xn = 0|yn , y\n )
P (yn , bn = 1, y\n )
P (yn , y\n )
= ln
· ln
=
P (yn , y\n )
P (yn , bn = 0, y\n )
P (yn |bn = 1, y\n ) · P (xn = 1, y\n )
P (yn |y\n ) · P (y\n )
= ln
·
P (yn |y\n ) · P (y\n )
P (yn |bn = 0, y\n ) · P (xn = 0|y\n )
λxn = ln
(3.17)
The received channel’s sample yn is independent of the set of received samples (y\n ). Thus,
the previous expression simplifies to:
λxn = ln
P (yn |xn = 1) · P (xn = 1|y\n )
P (yn |xn = 0) · P (xn = 0|y\n )
(3.18)
Which in turn can be split in order to highlight the contributions of each xn and y\n towards
computing λxn :
λxn = ln
|
P (yn = 1|y\n )
P (yn |xn = 1)
+ ln
P (yn |xn = 0)
P (yn = 0|y\n )
{z
} |
{z
}
Intrinsic
(3.19)
Extrinsic
The first term of (3.19), contains the intrinsic information, that is the reliability of xn based on
the channel sampling yn . The second term contain the extrinsic information, produced by the
decoder using all the channel samples except the current bit n.
The intrinsic information is dependent of the type of channel and modulation used. For the
AWGN channel with BPSK modulation, the intrinsic information is (see equation A.4, section
A.2.1 for details):
λxn ,int. =
3.4.4
2yn
σ2
Node update equations
Having introduced the LLR for evaluation the MAP algorithm in equation 3.19 hereafter the
extrinsic information is given by
λxn ,extr. , ln
24
P (bn = 1|y\n )
P (bn = 0|y\n )
(3.20)
3.4.5
Iterative decoding example
Consider a BPSK modulated noisy codeword symbol, z, present at the input of the decoder,
after traveling through an AWGN channel with noise vector e (see section A.2.1 for the derivation
details). A BPSK z symbol is given by:
z = (2x − 1) + e
(3.21)
The example codeword (notice it’s a valid codeword, because it satisfies xHT = 0)
x = [0, 1, 0, 1, 1, 0, 1, 1]
is sent through an AWGN channel with error vector e:
e = [−0.1077, −0.3408, −0.6059, 0.2683, 0.0531, −0.2545, −1.0363, 0.1343]
which has an AWGN parameter σ 2 = 0.20 ⇔ Eb /N0 = 5 dB (see equation A.5).
The noisy symbol vector at the channel output, z, is thus:
z = [−1.1077, 0.6592, −1.6059, 1.2683, 1.0531, −1.2545, −0.0363, 1.1343]
The channel reliabilities, that is the intrinsic information for this channel type with AWGN (equation A.4) is:
λx,int. = [−11.0770, 6.5923, −16.0592, 12.6834, 10.5309, −12.5455, −0.3630, 11.3425]
(3.22)
Performing hard decoding (equation 3.16) on the previous reliability will produce a wrong result
for the estimate of the original codeword:
x̂ = λx,int. > 0 ⇔
⇔ x̂ = [ 0, 1 , 0 , 1 , 1 , 0 , 1 , 0 ]
The underlined element is an hard-decoding error: bit x̂7 = 0 is ill-decoded, because the original
codeword bit value was x7 = 1.
Notice the dissimilar log-magnitudes of the channel’s output reliabilities (equation 3.22): as
high as λx2 ,int. = 16.0592 and, as low as λx6 ,int. = 0.3630. Low log-magnitudes mean low reliability/confidence that a particular x̂n trustworthy represents the original bit value. For the bit-error in
x̂7 , the corresponding reliability is low: λx7 ,int. = 1.1343.
A decoder has two equivalent methods for detecting errors: One is checking if the decoded
estimate x̂ verifies x̂HT = 0 (which for this example fails, x̂HT = [0, 1, 0, 1]T 6= 0). Another is to
check the LDPC code parity equations (equation 3.23):
c0 = v 0 ⊕ v 2 ⊕ v 4 ⊕ v 6 = 0 ⊕ 0 ⊕ 1 ⊕ 1 = 0
c 1 = v0 ⊕ v3 ⊕ v5 ⊕ v 7 = 0 ⊕ 1 ⊕ 0 ⊕ 0 = 1
(3.23)
c 2 = v1 ⊕ v2 ⊕ v 5 ⊕ v 6 = 1 ⊕ 0 ⊕ 0 ⊕ 1 = 0
c 3 = v1 ⊕ v3 ⊕ v4 ⊕ v7 = 1 ⊕ 1 ⊕ 1 ⊕ 0 = 1
25
Choosing one method over the other is a matter of picking the method most suited for the
decoder implementation. This is however only error detection. In order to perform error correction,
decoding via the MPA must be performed on the code’s graph.
The decoding process takes place by exchanging node reliability messages through the graph’s
edges. Edge numbering schemes can have a v-node or c-node perspective. In the current work,
v-node perspective numbering was chosen, as presented in figure 3.7.
c0
0
1
v0
c1
2
5
3 4
v1
v2
c3
c2
9 10 11 12
7 8
6
v3
v4
v5
13 14
v6
15
v7
Figure 3.7: Edge numbering scheme
(0)
When the decoding process starts, the messages present at the check nodes, λcm ,vn , are 0,
which means an equal likeliness of the input being a 0 or a 1. Thus on the first iteration, the
variable nodes only spread the channel reliabilities (equation 3.22) through the graph, along each
corresponding edge (figure 3.8a).
On the next iteration, the check-nodes send back to all neighboring nodes the update mes(1)
sages λcm ,vn (figure 3.8a).
Each time the variable node equations are re-evaluated, the reliability estimation of the bits
presented at the input of the decoder increases. As can be seen from table 3.3, the decoding
algorithm is able to remove the error in the codeword presented at its inputs. Further iterations
would only strengthen the reliabilities, without changing the output codeword. Thus, the algorithm
stops as soon as it finds a codeword that satisfies all parity check equations, but only after running
for at least for 2 iterations, in order to assure that enough reliability messages were exchanged
between check and variables nodes.
26
v0
v0
-1
1.0
7
-11.0
770
70
v1
c0
2
.059
- 16
30
9
10
.5
-0
.3 6
c1
12
20
v3
25
11.34
12
.40
10
v6
v7
(a) Bit-nodes updates
c2
6
0
-0.362 6.589
-
-10
.07
17
.53
09
-10
.7 1
94
30
-0.
36
11
.3
v7
v5
c3
42
5
v6
10
6
64
.5
-6
-0.
5
-12.545
v5
v4
c2
0
363
c1
-10.3856
923
4
83
.6
5
45
2 .5
-1
c0
59
.08
- 10
v4
30
0
12.6834
-0.36
58
630
-0.3
v2
6.5
v3
v1
63
0.3
92
3
6.0
5
92
-1
6.5
v2
10
.89
-6
.58
14
c3
08
-6.57
(b) Check-nodes updates
Figure 3.8: Message flow for bit-nodes and check nodes update process
Edge
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
(0)
(0)
(1)
λcm ,vn
λvn ,cm
λcm ,vn
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
−11.0770
−11.0770
6.5923
6.5923
−16.0592
−16.0592
12.6834
12.6834
10.5309
10.5309
−12.5455
−12.5455
−0.3630
−0.3630
11.3425
11.3425
−0.3630
10.8958
0.3630
−10.0859
−0.3630
−0.3620
−10.3856
−6.5646
0.3630
−6.5814
10.4012
−0.3620
−10.0717
−6.5896
−10.7194
−6.5708
(1)
λvn ,cm
−0.1812
−11.4400
−3.4936
6.9553
−16.4212
−16.4222
6.1188
2.2978
3.9495
10.8939
−12.9075
−2.1443
−6.9526
−10.4347
4.7717
0.6231
(2)
λcm ,vn
−3.9010
4.5404
2.1441
−0.5039
−0.1740
1.9170
−4.7702
−0.6219
0.1809
−0.5029
4.5396
3.4927
−0.1744
1.9172
−6.1129
−2.2883
(2)
λvn ,cm
−6.5366
−14.9780
6.0884
8.7364
−14.1422
−16.2332
12.0615
7.9132
10.0280
10.7118
−9.0528
−8.0059
1.5542
−0.5374
9.0542
5.2296
Table 3.2: Decoding evolution of the message passing algorithm
27
(0)
Node
0
1
2
3
4
5
6
7
λvn
−11.0770
6.5923
−16.0592
12.6834
10.5309
−12.5455
−0.3630
11.3425
x̂(0)
0
1
0
1
1
0
0
1
(1)
λ vn
−0.5442
−3.1306
−16.7842
−4.2667
4.3124
−2.5063
−17.0243
−5.9476
x̂(1)
0
0
0
0
1
0
0
0
(2)
λ vn
−10.4376
8.2325
−14.3162
7.2914
10.2089
−4.5132
1.3799
2.9414
x̂(2)
0
1
0
1
1
0
1
1
Table 3.3: Evolution of soft and hard decoding results at the decoder output x̂
3.5
Conclusions
In this chapter the LDPC codes were introduced. The details and code generation were presented, as well as the steps takes in order to assure that the generated code is good-performing.
Fast methods for real time encoding LDPC codes were presented. The LDPC decoding algorithm was introduced and detailed.
28
4
Building LDPC Codes
Contents
4.1 Introduction . . . . . . . . . . . . . . . . . . . .
4.2 Performance analysis of LDPC parameters . .
4.2.1 Equivalence of ensembles of LDPC codes
4.2.2 Girth and small-cycles removal . . . . . .
4.2.3 Regular vs Irregular codes . . . . . . . . .
4.3 Conclusions . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
30
31
32
33
35
37
29
4. Building LDPC Codes
4.1
Introduction
Several parameters dictate the performance of LDPC codes regardless any other implementation details or the system’s operating conditions. But even a code designed for the utmost coding
performance must also have an efficient run-time encoding and decoding methods (section 3.3.1),
else it isn’t suitable for real-time and/or low-latency usage.
Dozens of LPDC building methods or improvements to existing methods continue to be proposed annually. Cyclic geometries, finite-geometries [18], progressive edge growth and its many
variants [19], combinatorial constructions [20], and so on. The list of methods is truly extensive. Although most of these proposals introduce gains in the efficiency of LDPC error correcting, these gains are at the expense of run-time performance and/or system complexity. Successful applications of LDPC systems tend to prefer simpler codes with simple encoder/decoder
structure (whether regular or irregular) though throughly tested, such as the LPDC code used in
10GBASE-T Ethernet, where the validation and optimization of that code took 2 weeks in a 256
node computer cluster [6].
In this thesis, all the methods employed in the construction of LDPC are built upon simple
algebraic operations over an parity-check matrix and/or its adjacency matrix. These are mostly
simple operations such as 1’s insertion, and rows/column swaps, governed by simple heuristics.
The flowchart in figure 4.1 presents the order of the steps required to achieve a well performing
code: Given the inputs (N, R, j, k) - (blocklength, code rate, column weight, row weight) - the
LDPC code generator algorithm starts by creating an empty M by N matrix, where the number
of constraints M is derived from the input parameters N and R (block size and code rate). For
irregular codes, j and k contain the desired distribution of 1’s by column and row respectively. For
regular codes, j and k cardinality is 1.
Several algorithms can be employed from distributing the 1’s throughout the parity-check matrix. Two different algorithms can be used, a i) random placement of 1’s while complying with
the code (j, k) parameters or ii) a random opportunistic greedy filling algorithm than given a
target row/column weight distribution (j, k) or (λl , λr ), tries on each iteration to fill still vacant
rows/columns, while avoiding violating weighting distribution rules. This last method allows for
fast encoding of LDPC codes, and is the method introduced in [3]. After the filling process ends,
and because such process is non-deterministic, a code must be checked for compliance with the
input parameters. Non-compliant codes are discarded and the process restarts.
The next phase in the process tries to break small loops into larger loops. The smallest loop
in a LDPC code/graph is a loop with 4 cycles: It is the minimum path from a given node back to
itself (figure 4.3). See section 4.2.2 for more details. If most of the small loops were removed, the
process continues into the next phase. Else, the process restarts with a new empty matrix.
The next step in the building process, generate a fast encoding representation from the current
30
4.2 Performance analysis of LDPC parameters
Figure 4.1: Flowchart for Finding high-performant LDPC codes
code, is responsible for the existence of a fast encoding process, implementing the ideas of [14].
One question that might arise regarding the filling algorithm is why an almost brute-force (NP
complexity) approach is used instead of other approaches such as Progressive Edge Growth
(PEG) algorithms (which is still an NP hard problem) that have better changes in producing codes
with large girths. The reasoning is that the partitioning process is an extremely selective process,
rejecting a very large number of codes, thus being the bottleneck (resource-wise). Implementing
any of the PEG methods, yielding perhaps above-than-good-average codes, might have been a
sensible thing to do regarding completeness, but it can also be seen as a waste in resources
since most codes would be rejected because they would not have a fast-encoding representation.
4.2
Performance analysis of LDPC parameters
The ultimate benchmark for a LDPC code usually is its ECC ability. In [17], is stated on of
the most basic results of coding theory: The larger the code, the better it performs as an error
correction construct. Regardless any other considerations larger codes (for the same code rate)
perform better than smaller codes.
31
Other parameters and properties can also affect a code performance. These, girth size, and
code structure. These parameters are detailed in the following sub-sections.
4.2.1
Equivalence of ensembles of LDPC codes
Consider the set ensemble of (N = 140, M = 35, j = 3, k = 12) LDPC codes, with code rate
R = 1 − 35/140 = 0.75. There is an almost infinite number of codes that satisfy these constrains.
Let Ha , Hb and Hc be the PCMs of three different codes from the previous ensemble. Ha and
Hb are randomly generated while Hc is generated while having a fast enconding structure.
Figure 4.2 details the results of the three codes performance over the AWGN channel.
Figure 4.2: Coding performance of 3 identical sized codes
As can be seen from the figure, all three codes have similar ECC performance in the waterfall
region, although the Hc code has slightly lower performance. As stated previously, finding a code
with structure amenable to easy encoding usually mean that can has degraded performance
compared to other codes of the same size. This is specially true for small codes.
For reference, the code’s girth histogram is detailed in table 4.1. The girth histogram is quite
similar for all three codes.
Code
Number of n-cycles
4
6
8
10
Ha
294
3273
17490
2
Hb
262
3337
17226
0
Hc
256
3347
17190
0
Table 4.1: Girth histogram for the (N = 140, M = 35, j = 3, k = 12) ensemble
32
4.2.2
Girth and small-cycles removal
The girth is define as the smallest loop/cycle (the path from a given node back to itself) within
a code’s graph representation. A code’s minimum girth is the parameter that mostly affects performance, particularly for small-sized codes.
Because codes have finite block-length, any cycles on its graph are bounded. Thus at every
node there are many possible paths that revert back to the original node. Messages over larger
cycles take more iterations to travel back to a given starting node, but have the opportunity to
gather reliability updates from more distant nodes. For small cycles the reverse is true: spanning the entire cycle takes fewer iterations, and fewer nodes are reached. Thus, smaller cycles
biases incoming messages of all its spanning nodes towards the reliability updates of the cycles
members.
c1
c0
c3
c2

1
1
H=
0
0
v0
v1
v2
v3
v4
v5
v6
0
0
1
1
1
0
1
0
0
1
0
1
1 0 1
0 1 0
0 1 1
1 0 0

0
1

0
1
v7
Figure 4.3: Thick solid: A 4-loop defined by vertexes v3 c1 v7 c3 , in both tanner graph (left) and
parity-check matrix forms (right). Dashed: a 6-loop defined by vertexes v3 c1 v5 c2 v1 c3
Smaller cycles, (see figure 4.3) mean that less reliability for a given bit can be gathered by the
decoder. Consider the (dashed) small loop (4-cycle) depicted in the figure, spanning the vertexes
c1 , v3 , c3 and v7 : At variable node v3 , there can be only reliability updates coming from c1 and
c3 . After 2 iterations (two “up” and two “down” half-iterations), v3 will receive updated reliabilities
weighting the whole v3 c1 v7 c3 cycle. There are of course larger cycles that also have v3 as a
vertex (see the dashed 6-cycle v3 c1 v5 c2 v1 c3 ). But the consequence is that every 2 iterations v3
will receive through c1 /c3 similar updates, weighting mostly c1 v7 c3 . Messages from distant nodes
will arrive at a slower place: reliabilities over the whole dashed 6-cycle will arrive every 3 iterations.
Thus, the reliability updates received at v3 will be heavily biased towards the information updates
that happen over the small 4-cycle.
This is a limiting result, because the condition of convergence of the MPA assumes that the
cycles spanning at each node are infinite.
Most authors [21] acknowledge the desirability of removing small loops in LDPC graphs, due
to the loss of error coding performance caused by such small cycles. The typical algorithm for
generating LDPC codes usually makes a minimal effort to discard at least 4-cycles, because they
33
are easy to spot in a parity-check matrix: search for columns with two 1s in identical positions
(thus forming a rectangle of four 1s in the matrix - figure 4.3). With some shuffling and row/column
swapping it is usually possible to remove a 4-cycle. Longer cycles, aren’t as easily removed as
4-loops. Approaches such as given in [13] which just delete small cycles, tend to deeply change
the properties of a LDPC code.
An effective small loop removal process must perform and assure the following two conditions:
i) be able to locate all nodes with a girth of size n. ii) to be able to remove a n-loop without
introducing smaller n-loops.
In order to efficiently locate loops of any length, the removal process is performed over a code’s
adjacency matrix. Because the parity-check matrix of LDPC codes represents a bipartite graph,
the adjacency matrix A is simply:
A=
0
HT
H
0
(4.1)
where 0 is the all-zero matrix.
As presented in [22], the following observations regarding A stand:
Theorem 1. The (i, j)-entry of An is the number of paths of length n from vi to vj
Theorem 2. In a graph with girth n, two nodes vi and vj are opposite each other (i.e. connected
via a 3rd node) iff:
n/2
Aij
(n/2)−2
Aij
(4.2)
≥2
=0
(4.3)
With theorem 2, its possible to identify which nodes have a particular girth. Once an unwanted
loop is found, the next step is to remove it from the graph. This is performed on a edge-by-edge
basis. Care must be taken in order to avoid creating smaller loops. The details and the needed
cautionary steps required to successfully perform a loop removal are detailed in [22]. Succinctly,
the removal process is as follows: in order to efficiently remove a small loop, a random edge from
that loop is selected, with nodes vi and vj . From the set of the remainder adjacent nodes of vj ,
a node vk is selected giving rise to a new edge e = vj vk . Then, from the set of all nodes with a
distance greater than n − 1 from both vj and vk (set Se ), a random node vc is selected. From the
previous set Se , an edge e0 = vl vm with both ends in that set is selected. The actual loop removal
step, deletes both e and e0 from the graph, and replaces them with e2 = vj vm and e02 = vl vk . See
the appendix, for a graphical explanation of this process.
In order to show the detrimental effect of small cycles, a random LDPC code (N = 140, N =
35, j = 3, k = 12) with PCM H1 is introduced. H2 is the PCM of the same code but with most
of the small cycles removed, which was obtained from the first PCM by applying the small-girth
34
removal process described previously. Table 4.2 details the girth removal algorithm results. For
each code (first column), the number of {4, 6, 8}-cycles in that code are detailed.
Code
Number of n-cycles
4
6
8
H1
276
1252
984
H2
10
2098
56
Table 4.2: Listing of each code n-cycles before and after the removal of small cycles
Its easily noticeable from table 4.2 that the number and type of cycles of H2 has changed.
There are less much less 4-cycles than in the original PCM, H1 , but the number of 6-cycles
almost doubles in H1 . Notice that for a given code, the total number of cycles before and after the
removal of small cycles is not constant. This is due to the way how the removal process works,
merging smaller cycles into larger cycles, thus decreases the total number of cycles in a graph.
The next figure represents the Bit Error Rate (BER) vs SNR for a simulation performed over
the AWGN channel.
Figure 4.4: H1 vs H2 - with girth removal step applied
4.2.3
Regular vs Irregular codes
One of the improvements (with associated tradeoffs) that can be done to systems using LDPC,
is to employ irregular codes. This increases ECC performance, but at the expense of decoder
complexity.
LDPC codes can be of two different classes regarding the code graph representation: regular
35
and irregular. In regular codes, all check nodes have the same number of incident edges, and
also, all variable nodes have the same number of incident edges. In irregular codes none of these
assumptions stands, so the number of incident edges at each node can vary.
Regarding hardware implementations, for irregular codes, where different nodes can have a
varying number of incident edges, this "extra" variety increases the number of hardware resources
need at the decoder, and, it also puts extra demands on the scheduling algorithms that sequences
the MPA. A regular code only needs to have one type of hardware block to compute check
equations, while an irregular code needs to have distinct hardware blocks to perform all different
types of check equations. For example, consider a k = 4 regular code. The decoder needs to have
(at least one) hardware block that implements the function cm = fk=4 (va , vb , vc , vd ). For a irregular
code, with some check nodes having k = 3, others k = 5 and yet others with k = 2, the decoder
needs to have hardware blocks for computing all the different check equations: cm = fk=2 (va , vb ),
cm = fk=4 (va , vb , vc , vd ) and cm = fk=5 (va , vb , vc , vd , ve ).
In order to demonstrate the performance differences between these two classes of LDPC
codes, two codes are introduced: Hreg , exactly the same code as H1 - a (N = 140, M = 35, j =
3, k = 12) LDPC code - and an irregular code, Hirr , which was created by just deleting and
merging all 4-cycles from Hreg into other loops, increasing the number of 6-cycles, and almost
tripling the number of 8-cycles. The codes girth histogram is presented in table 4.3.
Code
Number of n-cycles
4
6
8
Hreg
276
1252
984
Hirr
0
1308
3738
Table 4.3: Listing n-cycles for a regular and irregular codes
Figure 4.5: Coding performance of regular vs irregular codes
As can be seen in figure 4.5 (depicting the performance of Hreg vs Hirr ), the irregular code
36
4.3 Conclusions
is better performing in regards of ECC compared to the regular code. Another advantage of
the irregular code is that it takes on average less cycles to successfully decode a codeword,
as detailed on table 4.4. Although this last result holds true for software implementations, the
hardware scheduling of the MPA in decoding irregular codes might offset this decoding speed-up.
Code
Iterations taken
Hreg
20 20 20 20 20 20 20 20 6
4
4
4
4
4
5
5
5
4
4
4
Hirr
20 20 20 20 20 8
5
4
4
4
5
4
4
4
4
4
4
8
8
4
Table 4.4: Average number of iterations taken to decode a codeword, for regular and irregular
code
These last results were obtained with the maximum number of LDPC decoding iterations
bounded to 20, and the minimum to 4 (where 4 iterations is the minimum number of iterations
that allows the MPA to travel along a 4-cycle - the shortest cycle possible)
4.3
Conclusions
This chapter presented methods for improving an LDPC code through changes in its structure.
A workflow for generating good codes was presented, along with all the methods and algorithms
for performing such generation.
Regarding LDPC codes parameters, and how those parameter affect their ECC performance,
the following conclusions apply:
• A random generated (N, M, j, k) code performs similar to other random generated (N, M, j, k).
• For the same block-length, irregular codes perform better than regular codes.
• Larger girth translates into better code performance, and is one of the major parameters
that dictates the ECC performance of a LDPC code.
37
38
5
Multi-channel parameter estimation
Contents
5.1 Introduction . . . . . . . . . . . . . . . . . . . . .
5.2 Typical LDPC decoding architecture . . . . . . .
5.3 Channel noise parameter estimation . . . . . . .
5.3.1 Estimation via Bayesian inference . . . . . .
5.3.2 Channel error probability distribution function
5.4 System Architecture . . . . . . . . . . . . . . . .
5.5 System performance . . . . . . . . . . . . . . . .
5.6 Conclusions and Remarks . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
40
40
41
42
43
44
45
46
39
5. Multi-channel parameter estimation
5.1
Introduction
This chapter introduces the final goal of the developed work, introducing herein an architecture
that:
Delivers improvements to the decoding stage of multicarrier LDPC-coded systems, without
resorting to high-complexity LDPC implementations, or increasing the complexity of existing
ones.
As previously stated, LDPC decoding can be made into a near-Shannon process [3], although
this comes at the cost of a massive increase in decoder complexity and run-time. Improvements
to LDPC systems can be split into two areas: i) LDPC code structure (chapters 3,4) and ii) LDPC
encoding/decoding algorithms. This chapter deals with improvements to the LDPC decoding
process, specifically to channel noise estimation.
5.2
Typical LDPC decoding architecture
The block diagram and signal flow of a typical LDPC decoder is as follows:
Figure 5.1: LDPC decoder block diagram
From the figure 5.1 (and as detailed in section 3.4.1) at the noisy channel end, each bit measure, yn plus associated noise, en , is assigned a log-probability ratio, LLR, λxn ,int. . This intrinsic
LLR quantity is the only information/measurement available at the end of the noisy channel, and
it is this information that enters the LDPC decoder. Inside the decoder, an iterative process called
the Message-Passing Algorithm (MPA) takes place, which will check and infer whether the reliability of a given bit (given all other bits measured at the channel’s exit) represents accurately a given
bit. At each iteration the degree of confidence increases by successively harvesting all extrinsic
LLRs. After a preset number of iterations, hard-decisioning is performed yielding the estimated
decoded codeword ŷn .
The intrinsic LLR quantity is a function of two parameters: a channel bit observation yn ∈
{0, 1}, and channel noise conditions. As an example, for the AWGN channel (see section A.2.1)
40
5.3 Channel noise parameter estimation
the intrinsic LLR for bit xn is:
2yn
λxn ,intrinsic yn , σ 2 = 2
σ
(5.1)
As evidenced by 5.1, both the bit observation yn and the AWGN channel noise parameter, σ 2 ,
are the parameters that define the intrinsic LLR.
Different channels types have different expressions for the LLRs (as detailed in section A.2)
which mirrors that particular channel noise model. Thus any uncertainty in the estimate of channel
noise parameter will also be reflected as uncertainty in the intrinsic LLR value.
As shown in [23], intrinsic LLR values that result from estimating the channel noise as greater
than the real noise value, don’t affect the ECC ability of the LDPC decoder. However, if the LDPC
decoder is feeded with intrinsic LLRs computed over a more optimistic noise parameter, lower
than the real noise value, the efficiency of the LDPC decoder and/or its ECC ability decreases.
5.3
Channel noise parameter estimation
The generic parameter estimation approach is to take a set of measured data and try to infer
some unknown model parameter from the measured data.
The unknown parameter of interest here is the channel noise. The available/measured data
is the data at the input and output of the decoder. Herein is introduced the hypothesis that by
comparing how many bits are corrected between the LDPC decoder input and output, it is possible
to infer the channel noise, and thus compute a better estimate for the intrinsic LLR.
The problem of estimation is introduced as follows: given the number of bits corrected by the
LDPC decoder, estimate the value of channel (or subchannel) noise. For the BSC this means estimating pc - the probability of cross-over, and, for the BPSK-AWGN channel this means estimating
the variance, σ 2 .
The method introduced here is suitable to single channel and multichannel systems. For
generalization, the multichannel approach is the one detailed here, presented in figure 5.2.
Figure 5.2: Subchannel bit accounting at LDPC decoder side
41
For the multichannel case, in order to perform multichannel estimation, any bits corrected
by the decoder must be accounted to a specific subchannel. For instance, from figure 5.2 for
computing the noise on the first subchannel, only the bits 1, 4 and 7 would be used. For the
second subchannel only bits 2 and 5, and so on. Thus the whole accounting process records how
many bits were corrected by the LDPC decoder per subchannel.
5.3.1
Estimation via Bayesian inference
Estimating a channel noise parameter uses the accounting of bits corrected between the input
and the output of the LDPC decoder, as detailed above. Several methods could be used, such as
a simple frequentist interpretation as in equation 5.2, where P (·) denotes a probability, and the #
symbol the cardinality of the given event:
P (error) =
#bits corrected
#total number of bits
(5.2)
The frequentist method has the disadvantage that a large number of samples are required, and
also, since this method does not require any knowledge of the underlying probability distribution,
the method converges slowly to the expected result.
Since the underlying probability distribution process is known (see section 5.3.2), methods
such as Bayesian inference can be used to better (and faster) estimate the channel noise parameter.
The Bayes theorem, used herein as the method for estimation is as follows [24]:
P (X|Y, I) =
P (Y |X, I) × P (X|I)
P (Y |I)
(5.3)
Replacing X by “hypothesis” and Y by “data”, and where I is the information (or experiment/sampling context), the data analysis and inference capability becomes apparent:
P (hypothesis|data, I) ∝ P (data|hypothesis, I) × P (hypothesis|I)
(5.4)
Regarding iterative estimation - which is used herein - the general estimation procedure for
Bayesian estimation is the Bayes theorem in the following form:
P (new hypothesis) =
P (previous hypothesis) × P (new data)
normalization f actor
(5.5)
Equation 5.5 is just an informal representation of Bayes Theorem [24]. P (new hypothesis) is
a pdf, where the x-axis value of the maximum is the desired estimated value of the channel noise
parameter.
In this application - where the only concern is the x-axis value of the maximum, the normalization factor can be omitted, so there is no need to keep the normalization factor.
Figure 5.3 details the iterative unfolding of an example estimation process – As can be seen
from figure 5.3a - just displaying one subcarrier for simplicity - the estimation process starts with
42
5.3 Channel noise parameter estimation
a prior (initial P (previous hypothesis)) showing “full ignorance”, that is the estimated parameter
could be uniformly distributed in the whole 0 to 1 probability interval. As the process iterates and
new information is available, it starts to bend and narrow the range were the estimated parameter
might lie. By the last three iterations (from a total of nine), figures 5.3 (g) (h) and (i), the maximum
of the pdf is already located near the real pc = 0.30.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
Figure 5.3: Posteriori evolution of an error process with a pc = 0.30
5.3.2
Channel error probability distribution function
Regarding the source of P (new hypothesis), in the Binary Symmetric Channel (BSC), there
are only two possible outcomes for a received bit y, given a channel input x. This is Bernoulli trial,
which has an binomial distribution model. The pmf the binomial distribution is:
f (k, n, p) =
n k
n−k
p (1 − p)
k
(5.6)
where n is the number of trials, k successes (in this case "success" is the event of cross-over),
and p ≡ pc , which is the parameter being estimated.
43
For the BPSK with AWGN channels the problem might seem intractable. The binomial distribution maps Bernoulli trials with associate parameters into directly into probabilities, but the AWGN
has no probability value in its parameters, only (µ, σ) - the mean, and variance - while the binomial
has the probability p ≡ pc which can be estimated through the maximum of its associated pdf.
A solution is to use the Normal approximation to the binomial distribution [24], where instead
of the usual N µ, σ 2 there is the approximation N µ = np, σ 2 = np(1 − p) - see this chapter
conclusion for remarks on this approximation. This broad interpretation means that the BPSKAWGN channel, from the point of view of the LDPC decoder is also a BSC bank channel, as
presented in section 2.3.
5.4
System Architecture
Figure (5.4) presents the overall system architecture, encoders, decoder, modulator, demodulators, channels, but without the subchannel error estimation block.
Figure 5.4: Overall System Architecture
The operation of this system is as follows: consider the five bit message u (top left corner
- represented as numbers inside circles). These are encoded into an LDPC codeword, x (bits
1 to 7) where the boxes 6 and 7 are the redundancy bits introduced by the LDPC encoder.
These bits then are introduced into a multi-channel modulator - while being conveyed from serial
to parallel ordering. The multi-channel modulator can be an OFDM modulator or a BSC channel
bank. Notice how different bits are transported through different subchannels/subcarriers, where
each subchannel can have different noise profiles (modulates bits represented by triangles). Then
the bits are demodulated and serialized into and estimated codeword y, which is sent to the LDPC
decoder, yielding at the output an estimate for the original message u.
The LDPC decoder presented in chapter 3 was introduced in a graph-like perspective. A blockoriented representation is depicted in figure 5.5. Again, the core of the LDPC decoding algorithm
is the Message-Passing Algorithm (MPA). The MPA, iteratively tries to i) gather and ii) increase
the assurance that a particular bit is either a one or zero.
44
5.5 System performance
Figure 5.5: Inner block structure of an LDPC decoder with the herein proposed addition of “Intrinsic LLR estimator”
In the starting decoding iteration, the MPA is supplied with an intrinsic Log-Likelihood Ratio
(LLR) that measures the reliability of a given bit being a one or zero given the channel conditions
yn + en . This LLR is then added (or composed) to the extrinsic LLR computed by the MPA in
an iteratively fashion. After a preset number of iterations, a hard-decisioning is performed on the
extrinsic LLR values yielding a decoded (and/or corrected) codeword ŷn .
The proposed addition is an “Intrinsic LLR estimator”, that through the account of which bits
were modified on a given subchannel, tries to infer subchannel noise parameters, and thus compute a more reliable value for the intrinsic LLR that enters the LDPC decoder in order to improve
the decoder ECC performance.
5.5
System performance
Figure 5.6 is the result of a simulation test with two subcarriers (two, for display simplicity) with
channel noises pe1 = 0.1 and pe2 = 0.4. The evolution of the estimation for pe in shown.
As can be seen by the graphs in figure 5.6, the estimator block is able to find the subchannels
noise parameters, although, as can be seen from the last iteration, pˆe1 = 0.02 and pˆe2 = 0.47,
almost matching the real values of pe1 = 0.1 and pe2 = 0.4.
To measure any performance gains of the estimator block, two runs were performed. Run
1 with the augmented LLR estimator - figure 5.2 - and run 2 with just a plain multicarrier LDPC
coded system, as figure 5.4. In order to dismiss any performance differences caused by the
random generation of source messages u and the random generation of channel noise, the two
runs were performed with a pre-generated batch of source messages, and pre-generated BSC
noise, and these two batches were replayed for each test run. The LDPC code employed was an
(n = 140, m = 35, j = 3, k = 12), Ha from subsection 4.2.1.
As can be seen from table 5.1 in Run 1 the total number of iterations and total number of
errors is lower than Run 2, where no LLR estimator was used.
45
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
Figure 5.6: Posteriori evolution of an error process with a pe1 = 0.1 (dashed line) and pe2 = 0.4
(solid line)
Test
Iterations
Errors
Run 1
268242
6726
Run 2
287480
9978
Table 5.1: Number of LDPC decoder iterations taken, uncorrected errors, with and without LLR
estimator block
5.6
Conclusions and Remarks
This chapter introduced the overall system architecture, and introduced the addition of an
intrinsic LLR estimator to a LDPC decoder system - in order to better estimate each subchannel
noise parameters. The concept of estimating and then applying that estimation was detailed. The
assumptions and rules that govern the estimation process via subcarrier corrected-bit accounting
were presented, and the Bayesian inference estimation process was detailed.
A performance run, of two systems, one with the LLR estimator, and another without this block,
showed that it is possible to reduced the number of iterations taken by the LDPC decoder.
46
5.6 Conclusions and Remarks
As a final remark, regarding using the Gaussian approach of the binomial distribution, better
methods for estimating the variance could have been derived, by finding the inverse Gaussian-like
distribution, or even find a closed form expression via the Bayes Theorem, by taking into account
the methods presented in [24]. But due to the sheer size of that undertaking, both are tasks for
future work.
47
48
6
Conclusions and Future Work
Contents
6.1 Conclusions and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . 50
6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
49
6. Conclusions and Future Work
6.1
Conclusions and Recommendations
This work introduced several types of methods to improve the performance of LDPC coded
systems. These improvements range from offline methods that deal with code structure and its
representation - chapter 4 - trying thus to improve code performance trough structural parameters
optimization.
Chapter 3 present methods for fast encoding, and decoding of LDPC codes, highlighting the
importance of having code that allow for fast encoder/decoder performance in order to assure low
encoding/decoding latency with maintaining high throughput.
Chapter 5 presents a novel way, of using just the bit information present at the end of an LDPC
decoder, to try to build an accurate model of the channel noise, thus improving the reliabilities
(LLRs) entering a LDPC decoder.
When building a communication system from the ground up which will employ LDPC as the
FEC method, if not constrained by i) hardware process; ii) maximum number of gates; iii) memory;
iv) power or v) area/density, then adopt an LDPC code with the largest block code length, N , as
possible, with an irregular structure.
But this is a “perfect world scenario”, without any sort of financial and/or engineering constraints. Although the larger the code, the closer it is to the optimal code - as defined by Hamming
and referenced by Gallager [1] - this code, and its associated decoder tends to be exponentially
complex, memory and/or implementation wise. The present-day major hurdle for larger block
LDPC codes is the sheer amount of memory required to i) store the Parity-Check Matrix, even
with a sparse representation; ii) store and move all message-updates of the Message-Passing
Algorithm and iii) store/buffer in-flight data, while trying to maintain high-throughput and latency.
Usually a compromise is required. Small to medium LDPC codes tend to perform “just good
enough”, while keeping the system complexity envelop under check. It is this kind of codes that
were targeted in this thesis. As shown in chapter 5 its is possible to reduced the number of
decoding iterations taken for decoding an valid LDPC codeword, while slightly increasing the
error recovery rate of existing system.
For high SNR environments, the present work becomes transparent, since the FEC capability
of the LDPC code (or most kinds of ECC) can just simply correct all errors, without any further
assistance.
6.2
Future Work
Regarding chapter 4, for completeness, the Progressive Edge Growth (PEG) method for building low-density parity-check matrices could also have been implemented. The proposed advantage of such method is building in one single procedure H matrices that are guaranteed to have
50
6.2 Future Work
a fast encoding representation while having high girths. Disadvantages, include heavy computational burden upon code generation (possibly weeks [6]), and, the inability to prevent deviation
from the code’s desired input parameters.
The probability formulation and estimation presented in chapter 5 should be the subject of
more extensive study, through better pdf modeling and/or finding closed form expressions for alternative pdf s. Regarding the same chapter, some of the presented contents could be formulated
under the framework of stochastic optimization theory.
Another avenue for future work is the realization of the proposed architecture, first in a Hardware Description Language - including all the steps of simulation, validation, etc - followed by
a physical hardware realization, integrating the proposed system in a full-blown communication
system, for performing testing and performance measuring in real world scenarios.
51
6. Conclusions and Future Work
52
A
Appendix
Contents
A.1 Conditional probability and Likelihood . . . . . . . .
A.2 Intrinsic Information . . . . . . . . . . . . . . . . . . .
A.2.1 AWGN Channel with BPSK modulation . . . . .
A.2.2 Binary symmetric channel . . . . . . . . . . . .
A.2.3 Binary erasure channel . . . . . . . . . . . . . .
A.3 Estimating BER via Importance Sampling . . . . . .
A.4 Fast addition rule for check-node update equations
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
54
54
54
55
55
56
58
53
A. Appendix
A.1
Conditional probability and Likelihood
Given two observations assigned to the variables a and b, the conditional probability of a given
b, P (a|b), can have different names according to the types of a and b:
If P (a|b = const), where b is a constant value, it’s called conditional probability. The probability
is a function of first argument, a, with b held fixed.
If P (a = const|b), it’s called likelihood. The probability is a function of the second argument,
b, with a held fixed. Likelihood can be interpreted as the ability to perform reasoning (inference)
backwards, from a given set of data.
A.2
A.2.1
Intrinsic Information
AWGN Channel with BPSK modulation
For BPSK modulation, a symbol zn is given by:
(A.1)
zn = (2xn − 1)
where xn is the nth encoded bit.
At the output of the Additive White Gaussian Noise (AWGN) the received symbol yn expression
is:
(A.2)
yn = (2xn − 1) + nn
where nn is a normal random variable with zero mean and variance σ 2 :
P (yn |xn ) = √
1
2πσ 2
· e−
(yn −xn )2
2σ 2
(A.3)
With the pdf it’s now possible to calculate the intrinsic information for bit xn :
λxn ,int. ≡ ln
P (yn |xn = 1)
P (yn |xn = 0)
= ln
P (yn |xn = 1)
P (yn |xn = 0)

= ln
−
e
e−
(yn −1)2
2σ 2
(yn +1)2
2σ 2

=
(yn + 1)2
(yn − 1)2
(yn2 − yn2 ) + (2yn + 2yn ) + (1 − 1)
−
=
=
2σ 2
2σ 2
2σ 2
2yn
= 2
σ
=
(A.4)
The relationship between the σ 2 and the channel noise is:
Eb /N0 =
1
2σ 2 ηM R
[dB]
where ηM is the bit/symbol ratio (modulation efficiency) and R the code rate.
54
(A.5)
A.2 Intrinsic Information
A.2.2
Binary symmetric channel
From figure 3.6b, the conditional probabilities for the BSC channel are:
P (yn = 0|xn = 0) = 1 − pc
P (yn = 1|xn = 0) = pc
P (yn = 0|xn = 1) = pc
P (yn = 1|xn = 1) = 1 − pc
where pc is the cross-over probability. The pdf for the channel, given all channel output symbols
xn = {0, 1} is:
(
pc
P (yn |xn = 1) =
1 − pc
for yn = 0
for yn = 1
(A.6)
(
1 − pc
P (yn |xn = 0) =
pc
for yn = 0
for yn = 1
(A.7)
Thus, the log-domain intrinsic information for the BSC comes as:
 
pc
ln 1−p
for yn = 0
P (yn |xn = 1)
c
≡ ln
=
1−p
c
ln
P (yn |xn = 0)
for yn = 1
pc
 ln pc
for yn = 0
1−p
c =
− ln pc
for yn = 1
1−pc
pc
y
= (−1) n ln
1 − pc
λxn ,int.
A.2.3
(A.8)
Binary erasure channel
The conditional probabilities for the BEC (figure 3.6a) are:
P (yn = 0|xn = 0) = 1 − pe
P (yn = 1|xn = 0) = 0
P (yn = |xn = 0) = pe
P (yn = 1|xn = 1) = 1 − pe
P (yn = 0|xn = 1) = 0
P (yn = |xn = 1) = pe
where pe is the probability of occurring an erasure. The channel pff, given the channel output
symbols xn = {0, 1, } is:


0e
P (yn |xn = 1) = 1 − pe


1/2
for yn = 0
for yn = 1
for yn = (A.9)
55
A. Appendix


1 − pe
P (yn |xn = 0) = 0


1/2
for yn = 0
for yn = 1
for yn = (A.10)
The intrinsic information for the BEC comes as:
λxn ,int. ≡ ln
P (yn |xn = 1)
P (yn |xn = 0)

e
ln 1−p

+

0
for yn = 0

0
for yn = 1
= ln 1−pe


1/2
ln
for yn = 1/2


ln (+∞) for yn = 0
= ln (0)
for yn = 1


ln (1)
for yn = 

+∞ for yn = 0
= −∞ for yn = 1


0
for yn = A.3
(A.11)
Estimating BER via Importance Sampling
The standard method for estimating BER in communication systems, assumes a Laplacian
probability approach: An experiment has only two possible outcomes, success or failure, and
there is no a priori knowledge of the outcomes’s plausibilities.
P (x) =
ℵS
ℵN
(A.12)
where ℵS is the number of successful outcomes of event x, and ℵN is the total number of outcomes.
Typical assays of BER values, range from 100 to 10−15 . In order to have a 10% BER confidence
level, the ℵN must be:
ℵN ≥
10
BER
(A.13)
For example, a typical communication system test at BER = 10−9 , a total of ℵN = 1010 assays
have to be performed.
xi
yi
H ( xi )
Figure A.1: Memoryless system H(xi ), where xi is a signal + noise sample, with pdf fX (x).
From [25], for a memoryless receiver model (figure A.1) the probability of a sequence of output
bits y, falling inside an error region E is:
Pe ≡ BER = P (y ∈ E) =
Z
fY (y) dy
E
where the decoding operation is H(·).
56
(A.14)
A.3 Estimating BER via Importance Sampling
The integral over the error region,
R
E
, where E can be a fuzzily defined region, can be replaced
by an error indicator function 1E (τ ):
(
1
1E (τ ) =
0
if τ ∈ E
if τ ∈
/E
Using the indicator function, equation A.14 becomes:
Z
+∞
Pe =
1E (y)fY (y) dy
−∞
Z +∞
=
(A.15)
1E (x) (H(x)) fX (x) dx
−∞
Thus, for each output, the indicator function 1E (x) (H(x)) is also evaluated.
Monte Carlo method
Evaluating the integral in A.15 over the full probability space [−∞, +∞] , can be a difficult or
even impossible task. An estimator for the value of Pe is thus required. A standard estimator for
Bernolli trials, is the MC estimator is:
P̂M C =
1
N
MC
X
NM C
i=1
1E (xi ) (H(xi ))
(A.16)
where P̂M C is the BER-MC estimator and NM C is the number of Bernoulli trials in the experiment.
Importance Sampling method
The purpose of the Importance Sampling (IS) is to modify the statistics of the Monte Carlo (MC)
estimator, with the goal of making success outcomes more frequent. This of course biases the MC
estimator, but because the biasing function is known, its possible to adjust the estimator, resulting
in an unbiased estimator.
In the IS method, the biasing is performed by multiplying the system’s output, H(x), by a
∗
biased probability density function (pdf) fX
:
Z
+∞
Pe =
1E (x) (H(x))
−∞
+∞
Z
=
∗
fX
(x)
∗ (x) fX (x) dx
fX
(A.17)
∗
1E (x) (H(x)) w(x)fX
(x) dx
−∞
where the weighting function w(x) is defined as:
w(x) =
fX (x)
∗ (x)
fX
(A.18)
57
A. Appendix
∗
Notice that Pe is now inferred from the biased pdf, fX
, instead of the original one (equation
A.17)
This biasing is reversed in the estimator expression, by weighting the modified system output
by w(x):
P̂IS =
A.4
NIS
1 X
1E (xi ) (H(xi )) w(x)
NIS i=1
(A.19)
Fast addition rule for check-node update equations
The check-node update equation isn’t well suited for machine computation, because it contains
two computationally expensive trigonometric functions, tanh and tanh−1 . A more efficient method
is needed for Very-Large-Scale Integration (VLSI) implementations [26]. To this aim, the LLRs are
rewritten in the following form:
(A.20)
λ = sgn(λ) · |λ|
Thus, the check-node update equation A.21 becomes:
(l)
tanh −
λcm ,bn
(l−1)
!
2
=
Y
tanh −
i ∈ Nm , i6=n

λ(l)
c
,b
m n
(l)
=
tanh sgn −λcm ,bn ·
2
λbi ,cm
!
2


Y
(l−1)
tanh sgn −λbi ,cm
i ∈ Nm , i6=n
(l−1) 
λbi ,cm 
−
2
(A.21)
Because tanh is odd (tanh(x) = − tanh(−x)) the previous equation can be restated as:
 (l) 
λcm ,bn (l)
=
sgn −λcm ,bn · tanh 
2
 (l−1) 
λbi ,cm (l−1)

sgn −λbi ,cm · tanh 
2
Y
(A.22)
i ∈ Nm , i6=n
It is possible to decouple the sign and magnitude components, obtaining two equations, one
for the sign update rule, and another for the magnitude update rule. The decoupled sign update
equation is:
(l)
sgn −λcm ,bn =
Y
i ∈ Nm , i6=n
(l−1)
sgn −λbi ,cm =
⊕
i ∈ Nm , i6=n
(l−1)
sgn −λbi ,cm
(A.23)
Here the ⊕ operator is the exclusive-or (XOR). Thus, the sign of the LLR message leaving the
check-node is just the XOR sum of all incoming messages signs. The magnitude update equation
is:
58
A.4 Fast addition rule for check-node update equations
 (l) 
λcm ,bn =
tanh 
2
Y
i ∈ Nm , i6=n
 (l−1) 
λbi ,cm 
tanh 
2
(A.24)
Applying − log(·) to both sides of (A.26):
 (l) 
λcm ,bn  = −
− log tanh 
2

X
i ∈ Nm , i6=n
 (l−1) 
λbi ,cm 
tanh 
2
(A.25)
which yields:


(l) λcm ,bn = F −1

X
i ∈ Nm , i6=n


(l−1) F λbi ,cm 
(A.26)
where F (·) is defined as:
x x = log coth
=
F (x) = − log tanh
2
2
ex/2 + e−x/2
ex + 1
= log x/2
=
log
ex − 1
e
− e−x/2
(A.27)
F (·), represented in figure A.2, is monotonically decreasing, and has the following properties:
F (x) =

 lim F (x) = 0
(A.28)
x→∞
 lim F (x) = ∞
x→0
Both sign (A.23) and magnitude (A.26) equations are much simpler to compute, than the full
update equation (A.21).
7
6
5
F(x)
4
3
2
1
00
1
2
3
x
4
5
6
7
Figure A.2: Plot of F (x)
59
A. Appendix
60
Bibliography
[1] R. G. Gallager, “Low-Density Parity-Check Codes,” Ph.D. dissertation, M.I.T., 1963.
[2] D. J. C. MacKay and R. M. Neal, “Good codes based on very sparse matrices,” Cryptography
and Coding 5th IMA Conference, no. 1025, pp. 100–111, 1995.
[3] S.-Y. Chung, G. D. Forney, T. J. Richardson, and R. Urbanke, “On the Design of Low-Density
Parity-Check Codes within 0.0045 dB of the Shannon Limit,” IEEE Communications Letters,
vol. 5, no. 2, February 2001.
[4] D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms.
Cambridge Uni-
versity Press, 2003.
[5] European Telecommunications Standards Institute, “Digital Video Broadcasting (DVB); Second generation framing structure, channel coding and modulation systems for Broadcasting,
Interactive Services, News Gathering and other broadband satellite applications (DVB-S2),”
August 2009.
[6] AIST. (2004, Sep.) Error correcting code for 10 gb/s twisted pair wire ethernet accepted
as ieee standards. [Online]. Available: http://www.aist.go.jp/aist_e/latest_research/2004/
20040906/20040906.html
[7] Consultive Committee for Space Data Systems, “Low density parity check codes for use in
near-earth and deep space applications,” National Aeronautics and Space Administration,
Office of Space Communication (Code M-3), NASA, Washington, DC 20546, USA, Tech.
Rep. 1, August 2006, ccsds 131.1-O-1.
[8] R. Galbraith and T. Oenning, “Iterative detection read channel technology in hard disk drives,”
White paper, Hitachi Global Storage Technologies, November 2008.
[9] T. Morita, M. Ohta, and T. Sugawara, “Efficiency of Short LDPC Codes Combined With Long
Reed-Solomon Codes for Magnetic Recording Channels,” IEEE Transactions on Magnetics,
vol. 40, no. 4, July 2004.
[10] R. W. Chang, “Synthesis of band-limited orthogonal signals for multichannel data transmission,” Bell Systems Technology Journal, vol. 45, pp. 1775–1796, December 1966.
61
Bibliography
[11] R. M. Tanner, “A recursive approach to low complexity codes,” IEEE Trans. Information
Theory, vol. 27, no. 5, pp. 533–547, September 1981.
[12] W. E. Ryan,
CRC Handbook for Coding and Signal Processing for Recording Systems.
CRC Press, 2004, ch. An Introduction to LDPC Codes.
[13] D. MacKay and R. Neal, “Near Shannon limit performance of low density parity check codes,”
Electronic Letters, vol. 32, no. 18, pp. 1645–1646, August 1996.
[14] T. J. Richardson and R. Urbanke, “Efficient Encoding of Low-Density Parity-Check Codes,”
IEEE Transactions on Information Theory, vol. 47, no. 2, February 2001.
[15] G. D. F. Jr, “The Viterbi algorithm,” Proceedings of the IEEE, vol. 61, no. 3, pp. 268–278,
March 1973.
[16] G. Solomon and H. C. A. Tilborg, “A connection between block and convolutional codes,”
SIAM Journal of Applied Mathematics, vol. 37, no. 2, pp. 358–369, October 1979.
[17] R. G. Gallager, “Low-Density Parity-Check Codes,” IRE Transactions on Information Theory,
vol. 8, no. 1, pp. 21–28, January 1962.
[18] L. Lan, L. Zeng, T. Y.Y., L. Chen, S. Lin, and Abdel-Ghaffar, “Construction of Quasi-Cyclic
LDPC Codes for AWGN and Binary Erasure Channels: A Finite Field Approach,” IEEE
Transactions on Information Theory, vol. 53, no. 7, pp. 2429–2458, July 2007.
[19] G. Richter, “An Improvement of the PEG Algorithm for LDPC Codes in the Waterfall Region,”
EUROCON, 2005.
[20] K. M. Krishnan, R. Singh, L. S. Chandran, and P. Shankar, “A Combinatorial Family of Near
Regular LDPC Codes,” IEEE International Symposium on Information Theory, pp. 761–765,
June 2007.
[21] D. J. C. MacKay, “Good Error-Correcting Codes Based on Very Sparse Matrices,” IEEE
Transactions on Information Theory, vol. 45, pp. 399–431, March 1999.
[22] J. A. McGowan and R. C. Williamson, “Removing Loops from LDPC Codes,” Australian
Communication Theory Workshop Proceedings, 2003.
[23] D. J. C. MacKay and C. P. Hesketh, “Performance of Low Density Parity Check Codes as
a Function of Actual and Assumed Noise Levels,” Electronic Notes in Theoretical Computer
Science, no. 74, 2003.
[24] D. S. Sivia, Data analysis: A Bayesian tutorial.
Oxford University Press, 1996.
[25] A. Bohdanowicz, “On Efficient BER Evaluation of Digital Communication Systems via Importance Sampling,” Ubiquitous Communications Program, 2001.
62
Bibliography
[26] M. Rovini, “Low-Density Parity-Check Codes: a Tutorial,” ESA/ESTEC, The Netherlands,
Tech. Rep. ESA STR-245, May 2004.
63

Simulador de Sistema PLC Utilizando OFDM Adaptativo com

Transcription

Similar documents

Erasure Codes for Storage Applications

Short Block Length LDPC Codes for TC Synchronization

Pantallas Acusticas Pared Verde (EN)

Step # 13 Open up TMPGEnc 4.0 Express and click on “Start a new

quietzone® noise control solutions

Distortion, 0.01 Full Power Input - Electro

Mathematical Games

A Progress Report on Encoding an Old Map in Japan

Build An Alphanumeric Pager Decoder

June-July11 - Mulhern Advertising, Inc.