We characterized the olfactory receptor gene repertoire, including OR, TAAR, OlfC, and ORA genes, for 185 species of ray-finned fishes selected based on high genome completeness (Fig. 1, Additional file 1: Fig. S1, Additional file 2: Supplementary Data 1).

The mean size of the total olfactory gene repertoire for actinopterygians was 224 genes. The largest (1317 genes) was found in the polypteriform Erpetoichthys calabaricus (reedfish) and the smallest (28 genes) in the tetraodontiform Mola mola (ocean sunfish) (Fig. 1). ORA is a small and stable family typically comprising six genes (ORA1 to ORA6) in teleosts [15]. Nevertheless, we found up to three ORA genes have been lost in several lineages, and, surprisingly, this gene family is much larger in some lineages, particularly Polypteriformes, which have nearly 50 functional ORA genes (Fig. 1, Additional file 1: Fig. S1A). Two genes, ORA7 and ORA8, were present in the last common ancestor of ray-finned fishes; ORA7 was lost in the common ancestor of teleosts, while ORA8 was lost in clupeocephalans [15] (Additional file 1: Fig. S2).

The evolution of the other three gene families (OR, TAAR, OlfC) has been more dynamic. For example, we identified an average of 126 functional OR genes in ray-finned fishes, but the variance is large, with 623 and 606 OR genes in the Polypteriformes Erpetoichthys calabaricus and Polypterus senegalus, respectively, and only 15 OR genes in the ocean sunfish Mola mola and broad-nose pipefish Syngnathus typhle (Fig. 1, Additional file 1: Fig. S1B). The OR family is split into seven monophyletic subfamilies, α, β, γ, δ, ε, ξ, and η [21]. In tetrapods, α and γ families expanded and other subfamilies are relictual or absent. In contrast, in teleosts, the α family is absent and only one copy of a γ family gene occurs in Zebrafish Danio rerio [21]. Our analysis shows that α family genes occur in all non-teleost actinopterygians but that the α family was lost in the common ancestor of teleosts (Additional file 1: Fig. S1B). The γ family is well represented in non-teleost actinopterygians whereas only a few copies are scattered in the teleost phylogeny (Additional file 1: Fig. S1B). This suggests that the γ family was present in the common ancestor of teleosts but lost in most teleost lineages. The number of genes in the TAAR and OlfC repertoires is smaller than in the OR repertoire, with an average of 51 and 40 genes per species, respectively. For these two gene families, the variance is also large. For example, Erpetoichthys calabaricus (Polypteriformes) has 486 TAAR and 161 OlfC genes. At the opposite extreme, only three TAAR genes were found in Callionymus lyra (Syngnathiformes) and two OlfC genes in Mola mola (Tetraodontiformes) (Fig. 1 and Additional file 1: Fig. S1C, D).

To analyze the evolutionary dynamics of the olfactory receptor gene families, we computed birth and death rates along branches of the phylogeny for the four families using the gene tree—species tree reconciliation method [22]. The mean birth and death rates were similar in OR, TAAR, and OlfC families, 0.0071/0.0071, 0.0101/0.0079, and 0.0059/0.0069 per gene per million years, respectively, but lower in the ORA family, 0.0018/0.0047 (Additional file 1: Fig. S3). Whereas birth and death rates are similar along most branches, we observed concomitant high death rates of OR, TAAR and OlfC genes in the common ancestor of two sampled species of Siluriformes (Bagarius yarrelli and Tachysurus fulvidraco), in the common ancestor of Lophiiformes and Tetraodontiformes, and in the common ancestor of Kurtiformes and Syngnathiformes. We also observed concomitant high birth rates of OR, TAAR, and OlfC genes in the common ancestor of Labriformes and Cyprinodontiformes and in the common ancestor of Perca + Sander (Fig. 1, Additional file 1: Fig. S3).

Despite variation in the number of genes in a family, we did not find evidence that contraction of one gene family is compensated by expansion of others. On the contrary, there is a correlation between the number of functional genes in each family (phylogenetic generalized least squares (PGLS); R2 = 0.50 between OR and TAAR, R2 = 0.56 between OR and OlfC, R2 = 0.40 between TAAR and OlfC, all p-values < 2e−16, Fig. 3). Moreover, in most species, the number of OR genes is greater than the number of TAAR or OlfC genes, and often, the number of TAAR genes is greater than the number of OlfC genes (Fig. 3). Although the number of ORA genes is less dynamic, particularly in teleosts, species with a high number of OR, TAAR, and OlfC genes, such as Polypteriformes or Anguilliformes, tend to have more ORA genes, whereas species with few genes in these three families, such as Mola mola, tend to have fewer ORA genes (Fig. 1).

Coevolution of number of OR, TAAR and OlfC genes in ray-finned fishes. A OR and TAAR families. B OR and OlfC families. C TAAR and OlfC families. Coefficient of determination (R2), p-value (P), and regression line (solid line) of PGLS analyses are reported. Dashed line shows slope = 1. Dot color code: red, Polypterus senegalus; brown Erpetoichthys calabaricus; light blue Polyodon spathula; dark blue Acipenser ruthenus; yellow Amia calva; dark green Atractosteus spatula; light green Lepisosteus oculatus; black teleosts

The coevolution of the three dynamic receptor gene families (OR, TAAR, OlfC) is also supported by the correlation between the number of gene losses along the branches of the phylogenetic tree (Pearson’s r = 0.8 between OR and TAAR, 0.52 between OR and OlfC and 0.62 between TAAR and OlfC, all p-values < 2e−16) and gene gains (r = 0.79 between OR and TAAR, 0.78 between OR and OlfC and 0.69 between TAAR and OlfC, all p-values < 2e−16) (Additional file 1: Fig. S4). The coevolution of the OR, TAAR, and OlfC receptor gene families is further supported by a correlation of the number and proportion of pseudogenes, which agrees with similar gene death rates in the three dynamic gene families (Additional file 1: Fig. S5).

Together, these results suggest that dramatic changes in evolutionary constraints on the size of the olfactory repertoire occurred several times, with periods of expansion or contraction affecting OR, TAAR, and OlfC olfactory receptor families the same way. Hence, they do not constitute independent evolutionary units in ray-finned fishes.

Using data for 72 species, 66 teleosts and 6 non-teleost ray-finned fishes (Additional file 2: Supplementary Data 1), we confirmed the correlation between the number of OR genes and the number of lamellae in the olfactory organ (PGLS; R2 = 0.57, p = 1.38e−14, Fig. 4A) reported recently for a smaller sample of 35 teleosts and two non-teleost ray-finned fishes [10]. While no significant correlation was found between the number of lamellae and the number of TAAR genes (PGLS; R2 = 0.00177, p = 0.726, Fig. 4B), a correlation was found with the number of OlfC genes (PGLS; R2 = 0.21, p = 4.55e−05, Fig. 4C) and the total number of olfactory receptor genes (PGLS; R2 = 0.13, p = 0.00176, Fig. 4D). The smallest olfactory repertoires occur in ocean sunfish Mola mola (28 genes) and broad-nosed pipefish Syngnathus typhle (35 genes). These extreme reductions of olfactory receptor diversity evolved independently and in parallel with the simplification of the olfactory organ, which is a small, flat olfactory epithelium in both species [10, 23] (Fig. 2A). Moreover, M. mola has greatly reduced olfactory nerves and reduced olfactory bulbs [24]. Limited data suggests that ocean sunfish are highly visual predators of gelatinous organisms [25,26,27]; however, more research on molid ecology is essential to determine if this is an example of a sensory tradeoff. At the other extreme is the unique organization of the olfactory organ of Polypteriformes. In both species studied, the olfactory organ consists of six sectors, each with a rosette-like structure [11, 12], resulting in many more olfactory lamellae than any other ray-finned fishes (Fig. 2A). Polypteriformes also have a much larger olfactory gene repertoire with many more genes in all four gene families than in any other ray-finned fishes (Polypterus senegalus: 1237 olfactory receptors, 300 olfactory lamellae; Erpetoichthys calabaricus: 1317 olfactory receptors, 150 olfactory lamellae; Fig. 1). The two other species studied that had the most olfactory receptor genes also had many olfactory lamellae (Anguilla anguilla: 658 olfactory receptors and 99 olfactory lamellae; Mastacembelus armatus: 677 olfactory receptors, 68 olfactory lamellae; Fig. 1). Interestingly, P. senegalus, E. calabaricus, A. anguilla, and M. armatus are nocturnal [28, 29], perhaps making them more reliant on olfaction. They also have other specializations of the olfactory system, such as prominent, anteriorly directed incurrent narial tubes (Additional file 1: Fig. S6). Such tubes direct water flow into the olfactory organ, which allows the fish to sample water above its boundary layer and thus more rapidly detect odors [30].

Coevolution of the olfactory gene repertoire and number of olfactory lamellae. A OR genes. B TAAR genes. C OlfC genes. D Total olfactory receptor genes. The coefficient of determination (R2), the p-value (P), and regression line (solid line) of PGLS analyses are reported. Dot color code: red, Polypterus senegalus; brown Erpetoichthys calabaricus; light blue Polyodon spathula; dark blue Acipenser ruthenus; yellow Amia calva; dark green Atractosteus spatula; light green Lepisosteus oculatus; black teleosts

After an extreme contraction of the olfactory gene repertoire and simplification of the olfactory epithelium, a secondary expansion in the gene repertoire occurred in parallel with the reacquisition of a multilamellar epithelium in the Tetraodontiformes genus Takifugu. The genomes of Takifugu rubripes, T. flavidus, and T. bimaculatus have more olfactory genes (156, 124, and 140 respectively) than other Tetraodontiformes with a flat olfactory epithelium, Dichotomyctere nigroviridis (70 genes) and Mola mola (28 genes). The increased number of genes in the three species of Takifugu is due to duplications of OR, TAAR, and OlfC genes (Additional file 1: Fig. S1B-D). We dissected a specimen of T. rubripes and found a non-rosette, but multilamellar, organization of parallel lamellae on the floor of the olfactory chamber that continues on the ventral surface of the nasal bridge between the incurrent and excurrent nares (Fig. 2A and Additional file 1: Fig. S7). This novel organization supports the hypothesis of a reacquisition of a multilamellar olfactory epithelium in association with secondary expansion of the olfactory receptor gene repertoire.

Together, our results indicate a functional link between the number of receptors and the number of lamellae in the olfactory organ of ray-finned fishes (Fig. 2B). In the most extreme cases, this leads to the loss of the rosette (e.g., Mola mola and Syngnathus typhle) or anatomical innovations with several rosettes (e.g., Polypteriformes) or a novel organization of olfactory lamellae (e.g., species of Takifugu). This link limits the morpho-genomic space occupied by ray-finned fishes (Fig. 2B). Accordingly, we did not observe ray-finned fishes with many olfactory genes and few olfactory lamellae or fishes with few olfactory genes and many olfactory lamellae (Figs. 2 and 4). Because many olfactory neurons expressing each olfactory receptor are necessary for efficient olfaction, there is a functional limit to the number of olfactory receptor genes that can be expressed on a given area of olfactory epithelium. This would explain why there are no species with many olfactory receptor genes and few olfactory lamellae. We did not find any examples of macrosmatic fishes with a low number of olfactory receptor genes, which would favor high sensitivity for a small set of odorants. There is probably no functional limit moderating the evolution of such a specialization, and perhaps cartilaginous fishes, which have few olfactory receptor genes and large multilamellar olfactory organs [31], may occupy this area of the morpho-genomic space.

No significant correlations were found between other morphological characters (maximum length of the fish, relative eye size (eye diameter/standard length of the fish)), ecological parameters (trophic level, preferred temperature, maximum depth), or genome size and the number of functional OR [10] or TAAR or OlfC genes (present study, data not shown).

Our analysis of 185 highly complete genomes of ray-finned fishes highlights the diversity of the olfactory receptor repertoire. The number of genes is highly dynamic for three (OR, TAAR, OlfC) of the four gene families, but the reasons for large gene gains or losses are still unknown. In marine tetrapods, including cetaceans and sea snakes, extreme reductions in the number of olfactory genes occurred likely because air-adapted olfactory systems were not useful in marine environments [32]. No such major ecological transition is associated with gene losses of similar magnitude in Syngnathiformes and Tetraodontiformes, and it remains unknown why their olfaction degenerated at both morphological and genomic levels. The complexity of the olfactory organ and large olfactory gene repertoire in Polypteriformes is also surprising. These fishes have a high olfactory sensitivity [33]. An olfactory organ with a large olfactory epithelium surface is probably involved in high sensitivity; however, a link between sensitivity and gene repertoire size is less obvious. For example, some Astyanax mexicanus cavefish have a higher sensitivity (105) to some molecules than surface conspecifics [34], while their olfactory gene repertoires are very similar (present study). To date, few olfactory receptor genes have been de-orphanized, and such functional information, combined with behavioral studies, may shed light on the dynamics of losses and specializations. Together, our analyses of the olfactory gene repertoire and morphology of the olfactory epithelium show that olfaction is a heterogeneous sensory modality in ray-finned fishes. Our identification of non-model species with particularly poorly developed olfaction (e.g., Mola mola) or exceptionally well-developed sense of smell (e.g., Erpetoichthys calabaricus) opens new possibilities for comparative and functional research on olfaction.