The utilization of laser as a carrier in fiber-based and free-space optical (FSO) based communications has demonstrated considerable potential and impressive advantages, owing to the substantial communication capacity and rapid data transmission [1]. Compared to fiber, FSO technology eliminates the requirement for optical fiber, which is suitable for deployment when fiber installation is either unfeasible or impractical. One of the recent advances in the optical communications field particularly in FSO is using orbital angular momentum (OAM) structured light as a multiplexing technique to increase the communication system capacity [2]. Structured light shapes include Laguerre Gaussian (LG), Bessel Gaussian (BG), and Hermite Gaussian (HG) [3] mode families. These shapes have replaced the conventional Gaussian waveforms to utilize space as an extra domain for data multiplexing in optical wireless communication [2]. Also, the various patterns of spatial light modes can be exploited as information carriers and used to build M-ary pattern coding schemes [3].

The inherent property of OAM is that its topological charge which represents the number of twists the light undergoes within a single wavelength can take on an integer value. Consequently, beams that carry OAM possess a vast number of orthogonal eigenstates, thereby endowing them with higher-dimensional attributes. Due to this, OAM has found many applications as a modulation or multiplexing technique [2]. Moreover, when OAM is multiplexed with other multiplexing techniques such as wavelength division multiplexing (WDM) and polarization multiplexing, the communication system data rate can be boosted to tens of terabits per second [4].

On the other side, one of the challenges in OAM-based systems is detecting the OAM mode on the receiver side for correctly demodulating the signal. The OAM modes’ wavefronts are very sensitive to channel impairments such as atmospheric turbulence, smoke, dust, and fog [3]. These impairments introduce distortion to the phase front of the OAM mode. Such distortion complicates the mode detection process and results in inaccurate detection [5]. Therefore, it is better to consider the intensity distribution as a feature for correctly detecting the OAM modes at the receiver. Among the various sources of channel impairments, fog is considered as a main limitation for FSO systems. Fog introduces attenuation that can reach hundreds of dBs per km. According to Ref. [6], fog is responsible for most of the visibility range degradation of less than 2 km in Europe.

Another route to increase the communication system capacity is to exploit the new class of InAs/InP quantum dash (QD) broadband laser diodes [7,8], with QD nanostructures exhibiting wide wavelength tunability covering C-, L- and U-bands. The multi-wavelength feature of this InAs/InP QD laser (QDL) has been exploited to demonstrate an aggregate data capacity of 10.8 Tb/s 48 × 224 Gb/s dual polarization 16-quadrature amplitude modulated multiplexed system employing a single C-band source [7]. Moreover, very recently, a 0.19 Gb/s QDL based multiplexed system has also been demonstrated in the challenging L-band wavelength window [8], thus strengthening this source potential in ultrawideband and high system capacity networks. Therefore, integrating InAs/InP QDL and OAM techniques could substantially increase the network capacity and, thus, is a promising solution for future communication infrastructures.

Optical instruments can be used at the receiver side to identify the different OAM modes such as the Mach–Zehnder interferometer as reported in Ref. [9] and elliptical nanopore array in Ref. [10]. However, using optical instruments complicates the receiver design and reduces its flexibility. In the last few years, deep learning (DL) has gradually become the most used computational approach in the machine learning area, where it outperformed many machine learning techniques in various domains [11]. It has been used widely to address many applications in different areas including speech signal processing, visual data processing, etc. It is also used in communication engineering for signal classification [12], performance monitoring [13], nonlinearity compensation [14], etc. Recently, the research community has exploited DL in identifying the OAM modes at the receiver instead of using optical instruments.

Using DL in OAM modes detection was reported in Refs. [5,[15], [16], [17], [18]]. Convolutional neural network (CNN) models were used in Refs. [[15], [16], [17], [18]] for identifying various numbers of OAM modes and the achieved accuracy was >94%. In Ref. [5], the authors used a deep feedforward neural network (FNN) to detect five superimposed modes. The detection accuracy was 98% under a moderate atmospheric turbulent medium. To improve the training speed of the DL model, the authors in Ref. [19] proposed using transfer learning. All the discussed works considered atmospheric turbulence as the source of OAM mode distortion. In Refs. [20,21], the authors considered smoke as the source of channel impairment. The tiny particles in the smoke can cause power scattering and absorption leading to serious power loss which affects OAM modes detection at the receiver. Using the DenseNet-121 CNN model, an identification accuracy of 99% was achieved [21]. Also, identification accuracies of >96% and >98% were reported in Ref. [21] using CNN and UNET algorithms, respectively, under smoke conditions. Similar to smoke conditions, the authors in Ref. [3] studied OAM modes identification under the effect of dust storm conditions. Dust is considered the most severe condition for FSO signals and therefore its effect on FSO links is of high importance in dry cities. The authors were able to achieve a 99% classification accuracy using a CNN model. Table 1 reviews DL algorithms considered in identifying OAM mode patterns under various FSO channel impairments.

In this work, we consider for the first time by experimental demonstration, OAM mode pattern generation using QDL source and OAM modes identification under foggy channel conditions. To achieve this.

goal, eight modes have been formed from the LG mode family. A 4-ary LG pattern coding scheme is constructed by varying the LG topological charge parameter [2] from 1 to 4, this generates LG01, LG02, LG03, and LG04 mode patterns. Also, a 4-ary MuxLG pattern coding scheme is constructed by superpositions of opposite topological charge LG modes, which generates LG0±1, LG0±2, LG0±3, and LG0±4 mode patterns. We exploit a controlled chamber environment to emulate the outdoor fog condition. The foggy channel causes the optical beam attenuation, which distorts the OAM mode wavefront and hence complicates identifying the transmitted mode. Both CNN and UNET DL methods were used on the receiver side as classifiers and regressors for mode identification and channel condition prediction, respectively. In addition, we incorporate a data balancing strategy for the experimental dataset prior to training via data augmentation. The results show that the visibility parameter can be estimated with 17 and 10 m root mean square error (RMSE) using CNN and UNET techniques, respectively. Besides, mode recognition has been achieved with an average accuracy of ∼94% under various fog conditions.

The remainder of this paper is organized as follows. In Section 2, we discuss the experimental setup, data acquisition, dataset generation, and the DL models’ architectures. In Section 3, we analyze the performance of the system and discuss the obtained results. Finally, we conclude in Section 4.