An investigation on digital signal processing for

coherent optical communication system with

advanced modulation format
line 1: Simiao Huang
line 2: communication engineering
line 3: Xidian University
line 4: Xi’an, China
line 5:hsm1455806487@163.com

Abstract—In recent years, many new applications such as production of glass fiber made of quartz (optic fiber). In 1970,
cloud computing, Internet of Things (IoT), artificial intelligence Corning developed a quartz optical fiber with a loss of 20
(AI) are developing rapidly, which put large capacity dB/km and a length of 30m, marking the initial realization of
requirements on communication network. Coherent optical optical fiber. In 1976, Bell Labs established an experimental
communication systems based on advanced modulation format line in Atlanta and Washington, with a transmission rate of
have been extensively studied to improve transmission capacity. 45Mb/s. The goal of second generation of optical fiber
High-order modulation format requires a better signal-to-noise communication was aimed at improving transmission rate and
ratio (SNR) for signal receiving. However, the various damages increasing transmission distance from 1976 to 1986.
in transmission links would decrease SNR. In this paper, digital
The working wavelength was developed from a short
signal processing (DSP) techniques are investigated to
wavelength of 850nm to a long wavelength of 1300nm. The
compensate signal damages, including Time-domain equalization
(TDE) algorithm to compensate chromatic dispersion (CD), development of the third generation optical fiber
constant mode algorithm (CMA) and direct decision least mean communication system, from 1986 to 1996, took super large
square (DD-LMS) algorithm to compensate polarization mode capacity and long distance as the research goal. In 1981,
dispersion (PMD), fourth power algorithm to correct frequency the researchers developed an indium gallium arsenide
deviation, Viterbi & Viterbi and Blind Phase Search (BPS) photodiode to implement a 1.55um dispersion shifted single
algorithms to estimate phase noise. Moreover, constellation mode fiber (SMF) optical communication system. The fourth
shaping technology is also discussed to make the transmission generation optical fiber communication system was in the era
capacity closer to Shannon limit. of synchronous digital system optical fiber transmission
network from 1996 to 2009. Optical amplifiers were introduced
Keywords—Coherent optical communication, advanced to reduce the requirements for repeaters.
modulation format, digital signal processing, constellation shaping.
The optical communication system can be divided into
I. INTRODUCTION coherent detection and direct detection. In direct detection
Nowadays, the rate of social informatization is increasing system, only intensity information can be detected. However,
gradually. Many new Internet services like cloud computing, optical intensity, frequency and phase information can be
Internet of Things (IoT), artificial intelligence (AI) are simultaneously detected in coherent detection system. Then the
developing rapidly. High-speed, long-distance and stable data processing can be performed in electrical domain to
communication systems are our long-term development goals. improve receiver sensitivity. Moreover, coherent detection can
Optical fiber communication system that uses light waves as also be combined with advanced modulation format to improve
carrier and optical fiber as transmission media has become the spectrum utilization and increase system capacity. Coherent
main transmission method in the world due to its advantages detection puts higher requirements for receiver side. In
of big bandwidth, high anti-interference and small signal coherent optical communication system, there are many factors
attenuation. Optical fiber communication has gone through which would cause signal distortion, such as conventional
several development stages. In 1966, Charles Kao proposed the chromatic dispersion (CD), polarization mode dispersion

978-1-7281-9797-5/20/$31.00 ©2020 IEEE 154

Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 04:11:08 UTC from IEEE Xplore. Restrictions apply.
(PMD), carrier frequency deviation, and the phase shift modulation formats and polarization division multiplexing at
introduced by the laser line width. the transmitter. At the same time, coherent detection also has
better receiving sensitivity.
In this paper, the digital signal processing (DSP)
technology in coherent optical communication system with III. DSP ALGORITHMS
advanced modulation format is investigated. DSP is the key
technology to compensate the linear and nonlinear damages Fig. 2 shows the process of DSP to recover the distorted
and improve system performance, including Finite Impulse signal at the receiver side. The digital filter module is used to
Response (FIR) filter to compensate CD, Constant Modulus compensate the linear damages caused by dispersion and
Algorithm (CMA), Cascaded Multimode Algorithm nonlinear damage introduced by the nonlinearity. The carrier
(CMMA) and direct decision Least Mean Square (DD-LMS) recovery module is used to compensate the damage caused by
algorithm to compensate for PMD, fourth power algorithm to frequency offset and phase offset.
correct frequency deviation, Viterbi & Viterbi and Blind Phase Backw ard Transm ission
Com pensation
Polarizatio n M ode D ispersion
Com pensation
C arrier
Recovery
Perform ance A nalysis

Search (BPS) algorithms for estimating phase noise

and constellation shaping to make communication capacity D ispersion
C o m pensatio n
N o n-lin ear
C o m pensatio n H xx + × D eco din g

closer to Shannon's limit.

II. COHERENT OPTICAL COMMUNICATION SYSTEM H yx
exp( jϕ )
B ER
ARCHITECTURE Perform ance
C alculation
exp( jϕ )
H xy
The coherent optical communication system architecture is
generally composed of a transmitter, a fiber channel, and a
receiver. The transmitting part includes an electrical transmitter D ispersion
C o m pensatio n
N o n-lin ear
C o m pensatio n H yy + × D eco din g

and a RTO up-converter, and the receiving part includes an

OTR down-converter and an electrical receiver. There are two Fig.2. The process of DSP to recover the distorted signal.
ways to realize the transmitter, including direct up-conversion A. CD compensation
and intermediate frequency conversion, which are shown in
Fig.1. In direct up-conversion method, baseband in-phase (I) The cause of CD is that optical signals with different
and quadrature (Q) signals are first generated by the electric wavelengths propagate at different speeds in the optical fiber.
transmitter, and then directly modulated to the optical carrier The signal will go away after a period of transmission,
through an optical I/Q modulator. Intermediate frequency resulting in group delay difference and optical pulse
conversion means that the baseband signal is first modulated to broadening. When this phenomenon is serious, adjacent pulses
an intermediate frequency by an electrical I/Q modulator, and will overlap with each other, causing inter-symbol interference
then up-converted by an optical intensity modulator (IM). (ISI), resulting in an increase in the bit error ratio at the
Compared with intermediate frequency conversion, direct up- receiving side, affecting the signal performance. With the
conversion has a higher cost. However, it is simpler to development of DSP, it is a more efficient choice to realize CD
implement and the electrical signal occupies a smaller compensation through the electric domain equalization
bandwidth. Therefore, most experimental systems adopt direct algorithm, in which the fiber transfer function is processed in
up-conversion method. the time or frequency domain to construct a corresponding
time-domain equalizer (TDE) or frequency-domain equalizer
(FDE).
In TDE, the impulse response of the fiber transfer function
is first inverted. Then a FIR filter based on the truncated
impulse response is formed. The dispersion can be removed by
convolution calculation between the received distortion signal
and the filter tap coefficients.
Because CD is a linear effect, the nonlinear effect of the
optical fiber can be ignored and the optical fiber can be
regarded as a linear system. The effect of linear dispersion on
the optical pulse envelope can be expressed by the following
formula [1]：
Fig.1. A CO-OFDM system in (a) direct up/down conversion architecture,
and (b) intermediate frequency (IF) architecture.
Coherent detection is to generate a local oscillator (LO) ∂A ( z, t ) Dλ 2 ∂ A ( z , t )
2

with a stable frequency at the receiving side, and then the

= j (1)
∂z 4π c ∂t 2
power is split into two channels. One passes through a 90-
degree phase modulator and the other channel remains In the formula, z represents the signal transmission
unchanged. Then the two channels are mixed with the received distance, λ represents the wavelength of the emitting laser,
signal. Finally, baseband I and Q paths are obtained by balance c represents the speed of light, and D represents the fiber
detectors. Compared with the intensity detection, the dispersion coefficient.
coherently detected signal can recover the amplitude and phase
information, which is beneficial to the usage of high-order

155
Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 04:11:08 UTC from IEEE Xplore. Restrictions apply.
 By performing Fourier transform and solving the above communication systems, the performance of FDE algorithm is
formula, we can get the frequency domain transfer function of more excellent [3]. FDE algorithm is mainly realized by
the fiber: frequency domain product. The tap coefficient of the filter can
be directly designed according to the fiber frequency domain
 Dλ 2 z 2  transfer function, which improves the disadvantage of time
G ( z , ω ) = exp  − j ω  (2) domain convolution. To a certain extent, it reduces the amount
 4π c  of calculation and reduces the complexity of the system. On the
other hand, the conversion between time domain and frequency
⍵ means angular frequency
domain also increases some complexity.
Through the above formula, we can get the impulse
B. PMD compensation
response function of fiber dispersion:
In SMF, two mutually orthogonal polarization modes form
c  πc 2  the fundamental mode. In theory, the two polarization modes
g ( z, t ) = exp  j t  (3) are completely degenerate and have the same propagation
jDλ z
2
 Dλ z 
2
speed. The x-direction mode will not couple with the y-
By changing the sign of the fiber dispersion coefficient, the direction mode. However, in the actual application process,
time-domain impulse response function of the compensation due to the production process and mechanical external forces,
filter can be obtained the actual shape of the optical fiber is not completely
cylindrical. The unexpected change in the shape of the core
jc πc 2 and the existence of anisotropy make the effective refractive
gc ( z, t ) = exp ( − jφ ( t ) ) φ ( t ) = t (4) indexes be different in different directions, which leads to the
Dλ z 2
Dλ 2 z birefringence of the fiber, resulting in the difference of the
At this time, the time-domain impulse response function of propagation constants of the two orthogonal polarization
the compensation filter is a non-causal infinite impulse modes.
response, and a feedback loop is required to implement it. When the optical signal is transmitted in the optical fiber,
However, the feedback loop will increase the operating burden the birefringence of the optical fiber and the group velocity
of the system, so we can truncate and use non-recursive FIR change are random, so the pulse broadens, resulting PMD [4].
filtering to implement it [2]. At present, the CMA is usually used to compensate the PMD
in the electrical domain [5]. Four butterfly adaptive filters are
Assuming that the number of filter taps is N and the
used to achieve polarization recovery, demultiplexing, PMD
system sampling interval is T , then the tap coefficient can be compensation and residual dispersion compensation. Fig.4.
expressed as: shows the principle of polarization equalizer.

ak =
jcT 2
Dλ 2 z
exp
 π cT 2 2 
− j
 Dλ z 
2
k 
xin
h xx + xout

N  N   Dλ 2 z  h xy
−  ≤ k  , N = 2 2 
+1 (5) Polarization
2 2  2cT  Equalization
h yx
  means round down.

x(k)
Tim e
D elay T
Tim e
D elay T
Tim e
D elay T
yin h yy + yout
Fig.4. The schematic diagram of polarization equalizer
CMA is suitable for constant amplitude modulation
formats, such as MPSK [5]. The CMA algorithm mainly
× h0 × h1 × h2 ... × hM
obtains the tap coefficients of the filter array by solving the
2
minimum value of the error function ε x = 1 − xout and
+ + + y(k)
ε y = 1 − yout
2
.
Fig.3. The principle of TDE dispersion compensation scheme.
Suppose the input signal in the two polarization states are
Fig. 3 shows the principle of TDE. The design of the filter
length and tap coefficients is important in TDE scheme. The xin ( k ) and yin ( k ) , the output signal are xout ( k ) and
disadvantage of TDE is that more multipliers and adders are
required when performing convolution calculations, which yout ( k ) , then the relationship between input and output is:
increases the amount of calculation of the entire system. When   M −1

the transmission rate increases or the length of the signal to be xout ( k ) = hxxT x in + hxyT y in =  hxx ( m ) xin ( k − m ) + hxy ( m ) yin ( k − m ) (6)
m=0
processed becomes longer, the system needs more storage
units. Therefore, for long-distance high-speed optical

156
Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 04:11:08 UTC from IEEE Xplore. Restrictions apply.
   M −1
yout ( k ) = hTyx x in + hTyy y in =  hyx ( m ) xin ( k − m ) + hyy ( m ) yin ( k − m ) (7) μ dε x 2
m=0 hxx → hxx − = hxx + με x xin∗
2 dhxx
Among them, hxx , hxy , hyx , hyy respectively represent four
adaptive filter arrays obtained from the Anti-Jones matrix, and μ dε x2
hxy → hxy − = hxy + με x yin∗
the length is M. Because the minimum value of the error 2 dhxy
function is required:
μ dε y
2
d ε x2 d ε x2 d ε y2 d ε y2 hyx → hyx − = hyx + με y xin∗
=0 =0 =0 =0 (8) 2 dhyx
dhxx dhxy dhyx dhyy
μ dε y
2
Get the corrected tap coefficient [4]: hyy → hyy − = hyy + με y yin∗ (13)
hxx = hxx + με x xout ⋅ x ∗
hxy = hxy + με x xout ⋅ y ∗
in
2 dhyy
in
The error function at this time can be expressed as:
hyx = hyx + με y yout ⋅ xin∗ hyy = hyy + με y yout ⋅ yin∗ (9)
ε x = d x − xout , ε y = d y − yout (14)
μ represents the convergence parameter, xin∗ is the complex
Among them, d x , d y represents the sample decision
conjugate of xin , yin∗ is the complex conjugate of yin . After
value that is closest to the input values of xout and yout .
the CMA converges, the resulting signal can obtain the phase
offsets ϕ x and ϕ y through the phase offset estimation module C. Frequency deviation compensation
and the sampling module. Ideally, the LO and the transmitted laser should maintain
Suppose the output in the X, Y polarization directions are: the same oscillation frequency. However, the frequency of
laser would deviate with time and temperature, and the
X = exp ( − jϕ x ) xout , Y = exp ( − jϕ y ) yout frequency deviation is inevitable. With the extension of the
Get the update formula of tap coefficient: communication time, the frequency deviation brings a phase
shift, which causes the rotating of received signal, and the
hxx = hxx + με x xin∗ hxy = hxy + με x yin∗ original information cannot be correctly restored. Therefore,
hyx = hyx + με y xin∗ hyy = hyy + με y yin∗ (10) it is important to eliminate the frequency deviation. Most
experimental systems adopt fourth power algorithm [6] to
At this time the error function is: eliminate frequency deviation, which is shown in Fig.5.
 c sgn ( X )  ( ⋅) *
ε x = exp ( jφx ) 
Z −1

 2 − X  xk
× ( ⋅)
4
 arg (⋅) / 4
Δϕ

 c sgn (Y )
N


ε y = exp ( jφ y )  (11) Fig.5. The schematic diagram of fourth power algorithm
 2 − Y  If only the symbol phase is considered, the sample value
among them, of the k-th symbol can be set as:

 1+ j Re ( x ) > 0, Im ( x ) > 0 (
S ( k ) = exp j (θ s ( k ) + Δω kT + θ L ( k ) + θ n ( k ) ) ) (15)

 1− j Re ( x ) > 0, Im ( x ) < 0 θ S ( k ) represents the modulation phase of the k-th
c sgn ( x ) =  (12)
symbol, Δω kT represents the phase error due to frequency
 −1 + j Re ( x ) < 0, Im ( x ) > 0
 −1 − deviation, θ L ( k ) represents the phase noise caused by the
 j Re ( x ) < 0, Im ( x ) < 0
It is relatively simple to use CMA to implement adaptive
laser linewidth, θn ( k ) represents the phase error caused by
tracking of the source. However, the convergence is slower amplifier noise.
and the residual error is also larger. It is necessary to select a First the phase noise θL introduced by the laser linewidth
smaller step size, otherwise the equalizer cannot work stably.
is removed. Suppose that in a sufficiently short period of time,
At this time, DD-LMS [5] algorithm can be utilized to
continue processing. θ L remains unchanged, then the phase difference between the
The DD-LMS algorithm has a fast convergence speed and preceding and following symbols is:
a small residual error. DD-LMS needs decision guidance. The S ( k ) S ∗ ( k − 1) = exp ( j ( Δθ s + ΔωT + θ n ) ) (16)
eye diagram of the input signal needs to be opened to a certain
degree to ensure the correct decision. CMA can be used for So θL is eliminated.
pre-convergence, then DD-LMS can be used to calculate the
error.

157
Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 04:11:08 UTC from IEEE Xplore. Restrictions apply.
 Second the modulation phase difference Δθ S is eliminated Let Δθ ( k ) = Δω ' kT + θ L ( k ) , then:
the modulated phase difference of QPSK is constant, with the

value of 0,
π
,π ,
3π
or
π 3π 5π 7π
, , , .
(
S ' ( k ) = exp j (θ s ( k ) + Δθ ( k ) + θ n ) ) (21)
2 2 4 4 4 4 The first step is to eliminate the modulation phase error
( S ( k ) S ( k − 1) ) = exp ( j ( 4Δθ s + 4ΔωT + 4θn ) ) (17) Δθ S ( k ) . It can be eliminated by taking its fourth power:
∗ 4

So after performing the fourth power operation, the value

( S '( k )) (
= exp j ( 4Δθ s ( k ) + 4Δθ ( k ) + 4θ n ) )
4
of 4Δθ S ( k ) is 0, 2π , 4π , 6π , (22)

then Δθ S is eliminated. The next step is to eliminate θn . Because the phase noise
Next, eliminate the noise phase error θn . In high-speed offset is 0, so we can average M symbols:
optical transmission systems, the laser frequency deviation is a
 ( S '( k ))
4
slow change compared to the symbol rate. The phase
difference value caused by the frequency deviation between
N consecutive symbols can be regarded as constant, the noise
M
M
= exp j 4Δθ ( ) (23)

phase error θn can be eliminated after averaging it : Phase deviation component can be obtained with the
following operation :
 ( S ( k ) S ( k − 1) ) ∗ 4

N
N
= exp ( j 4ΔωT )
(18)
{ (
arg exp j 4Δθ )} = Δθ
(24)
The frequency deviation component can be obtained by
4
taking the amplitude: Finally, the phase compensate can be realized by following
arg {exp ( j 4ΔωT )} operation compensate for each symbol:
= ΔωT (19)
4
S '' ( k ) = S ' ( k ) exp − j Δθ ( ) (25)

D. Phase offset compensation Viterbi & Viterbi algorithm has very low computational
As mentioned in the previous section, the frequency offset complexity. However, the higher-order modulation formats
will cause a phase shift. The phase shift is time-varying, and have much phase values and are more easily affected by phase
the rate of change is generally faster than the frequency noise. In this scenario, the Viterbi & Viterbi algorithm cannot
deviation. When the phase offset exceeds a certain range, it recover the phase.
will cause a decision error and affect the demodulation of the 2) BPS algorithm
signal. The carrier phase recovery algorithms can estimate and
The BPS algorithm can effectively recover the phase of the
remove the phase offset. Commonly used algorithms include
high-order modulation format, but it is very complex and still
Viterbi & Viterbi and Blind Phase Search (BPS) [7].
has certain limitations for larger laser linewidths.
1) Viterbi & Viterbi algorithm
|d k+ N ,0 |2 ŷ k ,0
( ⋅)
4
 ( ⋅) M arg (⋅) 4 exp − j ( ⋅)
M
- d k,0 |d k,0 |2 ŷk ,1
×
M UX
V erdict Circuit | |2 ⁿ
|d k-N ,0 |2
.
.
.
+
yˆ k , B −1
S '(k ) S '' ( k ) exp {jφ0 }

Z −1
× |d k+ N ,0 |2
rk
Fig.6. The principle of Viterbi & Viterbi algorithm × V erdict Circuit
- d k,1
| |2
|d k,0 |2
ⁿ
Fig. 6 shows the principle of Viterbi & Viterbi. The input +
|d k-N ,0 |2

signal after the previous frequency offset compensation can be exp {jφ1 }

expressed as : .
.
.
|d k+ N ,0 |2 Sk ,0

(
S ' ( k ) = exp j (θ s ( k ) + Δω ' kT + θ L ( k ) + θ n ) (20) ) × V erdict Circuit
-

+
d k,B -1
| |2
|d k,0 |2
|d k-N ,0 |2
ⁿ
Sk ,1
.
.
.
M IN (.)

S k , B−1

Δω ' = Δω − Δω ' , which represents the residual exp{jφB-1 }

Fig.7. The principle of BPS algorithm

frequency offset, Δω ' kT represents the phase error caused Fig. 7 shows the flow chat of BPS. The principle of BPS is
by the residual frequency offset, θ L ( k ) represents the phase to rotate the phase of the input signal rk B times, the phase
shift caused by the laser linewidth. angle of each rotation is ϕb :

158
Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 04:11:08 UTC from IEEE Xplore. Restrictions apply.
 b π B B information rate, bit error ratio. The heuristic algorithm or
ϕb = ⋅ − ≤ b < −1 (26) optimization algorithm is used to obtain the optimal solution
B 2 2 2 of the objective function. Finally, the optimized constellation
Sending the rotated signal to the decider, it can get the distribution is obtained. There are many types of geometric
constellation point coordinate yˆ k ,b with the smallest Euclidean shaping optimization design, such as exhaustive method
nonlinear constrained optimization algorithms [11-12],
2
distance, and get the square distance d k ,b between it and the Simulated Annealing [13], Pair Optimization (PO) algorithm
[14] and differential evolution algorithm [15].
input signal. To further eliminate the sudden effect, we can Probability shaping can make the transmission capacity
2
average d k ,b and the square distance between 2 N input approach the Shannon limit by adjusting the occurrence
probability of different constellation points to make them
symbols and yˆ k ,b : follow a Gaussian distribution [10]. The key factor of
probability shaping is to design independent binary
N distribution bit sequences as symbol sequences with target

2
S k ,b = d k − n ,b (27) distribution [16-17]. Distribution matching can be divided into
n =− N two types, one is from fixed length to variable length, such as
Select the minimum value of the sum of B squared distribution matching based on Huffman coding [18], the other
is fixed-length to fixed-length, such as distribution matching
distances, and take the corresponding yˆ k ,b as the final based on arithmetic coding-Constant Composition
judgment result. Distribution Matching(CCDM) [17].
In order to improve the efficiency of the algorithm, second- IV. SUMMARY
order BPS can be used for phase recovery. First, a smaller B
From transferring texts to transferring pictures and videos,
is adopted for coarse BPS phase estimation. Then a larger B live broadcast and future cloud services and Internet of
is selected for fine estimation. Finally, maximum likelihood is
Things, the urgent need of communication capacity is
adopted for the final phase recovery.
increasing. Technologies like high-order modulation format
E. Constellation shaping and coherent detection are adopted to increase transmission
Shannon's theorem shows that when the information capacity. In optical fiber transmission system, link
transmission rate is not greater than the channel capacity, there transmission damage is the key factor limiting the
is always a coding method that makes the error probability of transmission performance. In this paper, the approaches based
the channel output arbitrarily small, and the information on DSP to compensate the damages in transmission link is
transmission rate is infinitely close to the channel capacity. investigated. Time-domain equalization algorithm designed
High-order modulation techniques have been adopted to with FIR filters and frequency-domain equalization with less
increase transmission capacity and spectral efficiency. complexity are discussed to compensate CD. CMA and DD-
However, there is still a certain gap between the information LMS algorithms are utilized to compensate the PMD.
rate and the capacity limit. This gap can be further narrowed Generally, the CMA algorithm with slow convergence speed
by adopting constellation shaping technology. and large residual error is used for initial convergence, and the
Constellation shaping technology can be divided into DD-LMS algorithm with fast residual convergence and small
geometric shaping and probabilistic shaping. Without residual error is used to continue convergence. The fourth
introducing redundancy, geometric shaping maintains equal power estimation algorithm is used to correct the frequency
probability, but the interval becomes unequal between offset. Viterbi & Viterbi and BPS algorithms are adopted to
different constellation points. On the contrary, the probability estimate phase noise. The Viterbi & Viterbi algorithm is less
shaping with redundancy remains at equal intervals, but each complex and suitable for low-order modulation formats. The
constellation point has a different probability. BPS algorithm is suitable for higher-order modulation formats
In optical fiber communications, long-distance fiber but more complex. Finally, constellation shaping technology
channels can be regarded approximately as additive white is also discussed to reduce the gap between transmission
Gaussian noise (AWGN) channels [8]. In the AWGN channel capacity rate and Shannon’s capacity limit.
model, under the condition of high signal-to-noise ratio
REFERENCES
(SNR), the uniform discrete equidistant constellation signal
[1] Savory, Seb J,"Digital filters for coherent optical receivers," Optics
differs from Shannon's capacity by 1.53dB [9]. Express, Vol.16 Issue 2, pp.804-817.2008
Geometric shaping can achieve Shannon's limit capacity [2] S.J.Savory, G.Gavioli, R.I.Killey and P.Bayvel,"Electronic
under the AGWN channel when the number of constellation compensation of chromatic dispersion using a digital coherent receiver,"
points tends to be infinite [10]. Original constellation Opt. Express 15, 2120-2126 (2007).
distribution is used as the initial constellation, and restrictions [3] M.Kuschnerov et al., "DSP for Coherent Single-Carrier Receivers," in
based on actual needs and application environment are added. Journal of Lightwave Technology, vol. 27, no. 16, pp. 3614- 3622, Aug.
15, 2009.
Then the target function is selected to optimize system
[4] Agrawal G.P., "Principles and Applications of Nonlinear fiber Optics (
performance, such as mutual information, achievable 3rd edition )",Electronics Industry Press, 2002, 10-11,155-156

159
Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 04:11:08 UTC from IEEE Xplore. Restrictions apply.
[5] D.Godard, "Self-Recovering Equalization and Carrier Tracking in Two- [14] S. Zhang, F. Yaman, E. Mateo, T. Inoue, K. Nakamura, Y. Inada, “A
Dimensional Data Communication Systems," in IEEE Transactions on generalized pairwise optimization for designing multi-dimensional
Communications, vol. 28, no. 11, pp.1867-1875, November 1980. modulation formats,” Optical Fiber Communications Conference and
[6] Morelli, M., & Mengali, U. (1998). “Feedforward frequency estimation Exhibition (OFC), 2017, p.W4A.6.
for PSK: A tutorial review,” European Transactions on [15] F. Steiner, G. Böcherer, "Comparison of geometric and probabilistic
Telecommunications, 9(2), 103-116. shaping with application to ATSC 3.0", SCC 2017; 11th International
[7] A. Ghazisaeidi, "A Theory of Nonlinear Interactions Between Signal ITG Conference on Systems Communications and Coding, pp. 1-6, Feb
and Amplified Spontaneous Emission Noise in Coherent Wavelength 2017.
Division Multiplexed Systems," in Journal of 'Lightwave Technology, [16] Amjad, R. A., & Bocherer, G. (2013). Fixed-to-variable length
vol. 35, no. 23, pp. 5150-5175, 1Dec.1, 2017. distribution matching. 2013 IEEE International Symposium on
[8] Cisco System, Inc. Cisco Visual Networking Index: Forecast and Information Theory.
Trends, 2017-2012[OL]. [17] Schulte, P., & Bocherer, G. (2016). Constant Composition Distribution
https://www.cisco.com/c/en/us/solutions/collateral/service- Matching. IEEE Transactions on Information Theory, 62(1), 430-434.
provider/visual-networking-index-vni/white-paper-c11-741490.html, [18] T. Liu and I. B. Djordjevic, "LDPC-coded BICM-ID based nonuniform
November 26,2018. signaling for ultra-high-speed optical transport," Optical Fiber
[9] SHANNON, C. E. “A mathematical theory of communication,” The Communications Conference and Exhibition (OFC), 2016, pp. 1-3.
Bell System Technical Jourmal, 1948, 27(3): 379-423.
[10] F.-W. Sun and H. C. A. van Tilborg,"Approaching capacity by
equiprobable signaling on the Gaussian channel," IEEE Trans. Inf.
Theory,vol. 39, no.5, pp.1714 -1716, Sep. 1993.
[11] M. F. Barsoum, c. Jones, and M. Fitz, "Constellation Design via
Capacity Maximization," in Proc. IEEE Int. Symp. Inf. Theory (ISIT),
Jun.2007, pp. 1821-1825.
[12] J. Zoellner and N. Loghin, "Optimization of high-order non-uniform
QAM constellations," in Proc. IEEE Int. Symp. Broadband Multim.
Syst. Broadc. (BMSB), Jun. 2013, pp.1-6.
[13] F. Kayhan and G. Montorsi," Constellation design for transmission over
nonlinear satellite channels," in Proc. IEEE Global Telecommun. Conf.
(GLOBECOM), Dec.2012, pp.3401-3406.

160
Authorized licensed use limited to: Carleton University. Downloaded on May 29,2021 at 04:11:08 UTC from IEEE Xplore. Restrictions apply.