Mode-division multiplexing (MDM) technology has received intensively attention due to its potential to multiply the transmission capacity of the fiber-optic communication system [1,2]. It utilizes the orthogonal spatial modes in the multi-mode fiber (MMF) or the few-mode fiber (FMF) as independent channels to transmit the multi-channel data in parallel [3,4]. In practical scenarios, affected by mechanical stress or manufacturing defects, the introduced mode-coupling causes the signal interference between different spatial channels [5,6]. Therefore, besides optical signal-to-noise ratio (OSNR), the crosstalk strength (XT) caused by mode-coupling is also supposed to be a crucial indicator for optical performance monitoring (OPM) in MDM-based optical networks.

The monitoring methods of OSNR have been fully studied for both direct detection and digital coherent system in conventional single-mode fiber (SMF)-based fiber-optic networks [[7], [8], [9]]. Since OSNR is related to the features of the transmitted signal, OSNR monitoring schemes could be categorized as time-, frequency-, and spatial-domain schemes according to the source domain of the exploited features. Besides the time and frequency information, the features in the spatial-domain schemes are extracted from the signal constellations. In order to achieve the accurate OSNR prediction, machine learning (ML) algorithms, which are capable to solve the nonlinear classification and regression problems, are favorable for mapping the signal features to OSNR [10,11]. Among the ML algorithms, the deep neural network (DNN) consisted of multiple neural layers is more suitable for the complex modeling with stronger scalability to further improve the prediction accuracy of OSNR as well as other indicators [12,13].

Currently, although the significance of MDM system in terms of transmission capacity has been extensively proven, there is few research on OPM in MDM systems. Accurate monitoring of OSNR and XT is helpful to improve the stability of MDM systems. An active learning (AL)-based OSNR monitoring scheme was proposed to reduce the required training dataset in FMF transmission system [14]. However, there was a lack of XT prediction and corresponding analyzation. A transfer learning (TL) and DNN-assisted OPM scheme was proposed in FMF transmission systems [15], realizing the recognition for five modulation formats. But the discussion and description on mode-coupling as well as the relative interference, which is an inevitable behavior in MDM system, was still insufficient. The 1-D and 2-D in-phase quadrature histogram (IQH) features-based scheme was proposed and verified in predicting OSNR, XT and CD with different ML algorithms in MDM systems [16]. But the IQH features were extracted from the constellation diagrams and processed through the complex CNN networks.

In this paper, we propose a time-frequency dual-domain DNN (TF-DD-DNN)-based OPM scheme to jointly monitor the OSNR and XT for the few-mode channel without any demodulation process. The DNN-utilized features in time and frequency domains are respectively extracted from the filtered waveform amplitude histogram (FWAH) and the spectrum amplitude histogram (SAH) of the collected signals with low complexity and without complex digital signal processing (DSP) algorithms. In the 3-mode multiplexed simulation system, we sequentially predict the OSNR and XT for the 25 GB/s QPSK and 16QAM signals with TF-DD-DNN, to verify the scheme performance. In the results, the OSNR prediction accuracies are achieved more than 0.99 and 0.98 for QPSK and 16QAM signals, and the XT accuracies are achieved up to 1 with the root mean square error (RMSE) of 0.11 dB. We have also investigated the impact of CD and the low-pass filter bandwidth on OSNR and XT prediction performance. It's proved that the SAH-based OSNR prediction effectively shielded the interference of CD and XT. And the smaller low-pass bandwidth is helpful to preserve the effective features for improving the XT prediction performance. Besides that, the validated performance of the proposed OPM scheme is not affected by the multiplexed number of spatial modes.