UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Machine learning aided non-linear interference compensation for optical fiber communication systems Jain, Prasham


The capacity of optical fiber communication systems needs to be increased to meet the ever-growing demand for reliable high-speed data transmission. Nonlinear impairments induced by the Kerr effect are the primary bottleneck limiting the capacity of optical fibers. Since an analytical solution to nonlinearity compensation has not been found, machine learning based solutions to overcome this algorithm deficit have gained considerable traction. It has been demonstrated that neural network (NN) based methods can provide performance and complexity benefits over conventional digital signal processing techniques. Moreover, unlike conventional nonlinearity compensation (NLC) schemes, machine learning solutions do not require accurate knowledge of the fiber parameters. In this thesis, we investigate the numerous NN based NLC techniques proposed in the literature and classify them based on key characteristics of their design. For a dual-polarized single mode fiber, we demonstrate that a distributed compensation scheme designed based on a conventional digital signal processing (DSP) solution provides the best performance at the lowest computational complexity. We note that this is due to the simplification of the nonlinear effect and its interplay with linear effects for short sub-sections of the fiber. Based on this, we propose a novel deep convolutional recurrent neural network (DCRNN) with distributed compensation of polarization mode dispersion (PMD). Based on numerical results, we show that the proposed learned NLC method outperforms all previous solutions, learned and deterministic, in both performance and complexity. The proposed neural network model achieves a 1.43 dB Q-factor gain over state of the art learned NLC schemes for a 960 Km 64-QAM single channel dual polarized transmission at 32 Gbaud/s. For practical fibers, PMD may drift over time, resulting in arbitrary performance loss. In this thesis, we propose a transfer learning based selective online training method to adapt the learned model to continuous evolution of PMD in real-time. Based on numerical results using the proposed online training method, we show that the learned model maintains its superiority at various levels of PMD drift. Furthermore, the model shows fast convergence even for random abrupt change in the PMD realization of the fiber.

Item Media

Item Citations and Data


Attribution-NonCommercial-NoDerivatives 4.0 International