To test whether EVM analysis of IRT imagery can be broadly applied as a non-invasive tool to measure animal vital signs, 58 individuals across 36 families (28 mammals, 4 birds, 4 reptiles) and 52 species (39 mammals, 7 birds, 6 reptiles) were imaged at the Cincinnati Zoo and Botanical Garden in Cincinnati, OH (n = 44), the Louisville Zoo in Louisville, KY (n = 11), the Columbus Zoo and Aquarium in Columbus, OH (n = 2), and the Salisbury Zoo in Salisbury, MD (n = 1) (Table 1). Infrared images and videos were recorded using a FLIR T540 camera (30 Hz image frequency, 464 × 348 pixel IR resolution) with a 24° lens (Teledyne FLIR, Wilsonville, OR) placed on a tripod. A GoPro Hero 4 (GoPro, San Mateo, CA) was attached to the tripod and recorded red–green–blue (RGB) color video simultaneously. To determine which point locations on the subjects’ bodies would be most useful for non-invasive vital rate measurements, multiple videos were taken across the body at areas with relatively high thermal signatures and the least amount of movement.

Table 1 Species imaged, their physical characteristics, successful RR and HR measurementsRGB & IRT video analysis

First, FLIR Research Studio (Teledyne FLIR, Wilsonville, OR) software was used to identify IRT videos of adequate quality to analyze. A scoring system of 0–8 was developed to reflect video quality (see Methods), with 8 being the highest quality videos, so videos of low quality could be excluded from analysis.

To magnify small changes in thermal energy associated with blood flow, Eulerian video magnification (EVM) was performed using Lambda Vue (Quanta Computer, Taiwan) software that uses amplification algorithms developed in Wu et al. 2012 and was adapted from Lauridsen et al., 2019 (Fig. 1). Nine second segments of video were used to reduce unmanageable processing, as recommended by Lauridsen et al. 2019. First, a wide passband encompassing 0.1 – 3.5 Hz was used to amplify changes in colored pixels (at 40 × magnification), and extracted signals were normalized. Fourier transformation was used to decompose the signal from each video into its component frequencies. A normalized intensity plot was used to identify the dominant peak intensity, which always corresponded closely with ‘true’ RR, determined by the observation of ribcage expansion and/or nostril flaring from the (RGB) color video.

Fig. 1

A representative example of Eulerian Video Magnification image processing of a gray seal (Halichoerus grypus) infrared video. Ai. In the raw infrared video without magnification, there is no visible temporal variation in thermal signatures, as demonstrated by the spatiotemporal slices (Aii.). Aiii. Signal intensity did not vary over time nor was there a peak frequency intensity. Bi. When the infrared video was magnified 40 × with a 0.1–3.5 Hz passband, there was substantial variation in signal intensity through time (Bii.). Biii. The frequency domain had a clear peak at 0.63 Hz which is assumed to be RR, and 0.63 Hz = 38 breaths per minute (brpm). The ‘true’ RR was 40 brpm. To ensure the narrow passband is not dominated by the RR peak, the narrow passband will be chosen to exclude 0.63 Hz. Ci. The infrared video was magnified 40 × with a 1–2 Hz passband, resulting in variation across spatiotemporal slices (Cii.), the frequency domain had an obvious peak at 1.64 Hz which is assumed to be HR (Ciii.), and 1.64 Hz = 98.4 beats per minute (bpm). The ‘true’ HR via stethoscope was 104 bpm

On the same video, EVM analysis was then repeated. The dominant frequency (taken to be RR) was excluded and a narrower passband ranging 1 Hz in width was used for EVM, to focus on secondary peak intensities. For example, if the dominant peak of the wide passband showed RR was 0.6 Hz, then a narrow passband of 1–2 Hz was used to focus on secondary intensities (as in Fig. 1). The peak frequency from the narrow passband was taken to be HR and was compared to ‘true’ values obtained using a stethoscope (3 M Littmann CORE digital stethoscope, Eko Health, Oakland, CA), ultrasound, manual palpation, or veterinarian’s electrocardiogram (ECG). These true measurements were taken near-simultaneously (within ~ 30 s), as animal movement and other logistics sometimes prevented the stethoscope measurement and infrared imaging to occur at the exact same time.

The analysis workflow developed during this study resulted in two normalized signal intensity plots and peak frequencies (wide passband corresponding with RR, and narrow passband corresponding with HR) for each video (see narrow passband in Fig. 1). An imaging session was considered successful if the video analysis produced a clear peak frequency and that peak frequency was comparable to the ‘true’ measurement of either HR or RR.

Imaging sessions

‘True’ RR and/or HR were successfully measured in 44 imaging sessions out of 58, which included 44 individuals and 40 species (30 mammals, 6 birds, 4 reptiles) and were used for comparison to IRT-derived measurements. Eighteen of these imaging sessions occurred while the animal was immobilized (45%) and 26 imaging sessions were conducted while the animal voluntarily remained still (65%). Seven individuals were imaged through barriers causing some obstruction via bars or mesh grates, while 37 were imaged with no obstruction.

Use of infrared thermography versus RGB for vital rate measurements

To identify when IRT was superior to RGB imagery for obtaining vital measurements, EVM analysis was also conducted on recorded color video. A dominant peak for RR could also be extracted using EVM of RGB videos in 27 of the 40 species (67.5%) imaged. Peak frequencies could not be identified after EVM analysis to identify HR in any species using RGB video. This demonstrates that using IRT was necessary to measure animal HR, and this could not be accomplished using RGB video.

Accuracy and precision of IRT-derived physiological measurements

Non-invasive IRT provided an accurate means with which to measure animal vital rates. Of the 40 different species, broad bandpass frequency EVM analysis of IRT video yielded a prominent peak associated with RR in 36 individuals (81.8%) and 32 species (80%) (see Table 1). This included all species that RR was observed via ribcage expansion or nostril flaring from the RGB videos, and RR could be measured in an additional 5 species with IRT by measuring the change in temperature around the nostrils: Long tailed chinchilla (Chinchilla lanigera), harbor seal (Phoca vitulina), minilop rabbit (Oryctolagus cuniculus minilop), California sea lion (Zalophus californianus), Tawny frogmouth (Podargus strigoides); see Table 1. Using temperature changes around the nostrils facilitated RR measurements in these additional species, either because animal movement had made it difficult to observe ribcage expansion or the animal had significant subcutaneous fat, fur or plumage. Image analysis provided accurate measurements of RR (from ‘true’ measurements mean absolute error: 1.9 brpm; average percent error: 4.4%), and there was no significant difference between RR values obtained using IRT and ‘true’ RR measurements (t = -0.810, p = 0.424).

In 24 individuals (54.5%) and 22 species (55%), the narrow bandpass frequency analysis yielded a prominent peak representative of HR (see Table 1) and were also highly accurate (from ‘true’ measurements mean absolute error: 2.6 bpm; average percent error: 1.3%); these were statistically indistinguishable from ‘true’ values (t = 1.068, p = 0.297). The most common point locations on the body with high thermal signatures for HR measurement were temples and inner legs. Figure 2 demonstrates the importance of measuring HR at areas with high thermal signatures.

Fig. 2

Video analysis outputs from one imaging session of one orangutan (Pongo pygmaeus x Pongo abelii) focusing on three different locations. All videos were magnified 40 × and had the narrow bandpass (1 – 2 Hz) applied. A is the output from analyzing a spot on the chest (marked with the white box) with more fur, which produces no signal. The dominant peak here is probably due to animal movement. The shoulder (B) and wrist (C) produce the same output, taken to be HR. The stethoscope reading was 102 bpm

Precision of vital rate measurements derived from non-invasive imagery

To demonstrate that the IRT-derived measurements are precise, RR and HR were measured in different parts of the individual in videos from a subset of imaging sessions. RR was measured in more than one location on an animal’s body (nostrils, abdomen, chest) in ten videos and HR was measured in more than one location in seven videos (Tables 2, 3). Vital rate measurements were statistically similar across the body (paired t-test—RR: t = 0.190, p = 0.8534; HR: t = 1.162, p = 0.2894; Table 1).

Table 2 Successful respiration rate and heart rate measurements differed by sedation status, taxa, integument and fat thickness, and video quality. Percentages refer to successfully extracting a signal for vital rate measurements for that group. Percentages labeled with the same letter are not significantly different from one another, while different letters denote significant differences (p < 0.05)Table 3 Best-fit general linear mixed-effect (GLMM) models showing the relationship between ‘true’ and IRT-derived RR and HR values with species ID as a random effect to account for any species where multiple individuals were imaged. Fixed effects were added to investigate the role of physical characteristics in IRT-EVM errors. Models are ordered by AICc, with best models at the top of the table; the base model is included. See Additional file 1, Table S1 for additional model informationCharacteristics that make an animal a good candidate for using IRT

Video quality, immobilization status, taxa, thickness of integument, and subcutaneous fat influenced the success of the IRT-derived measurement while animal color, ambient temperature, and humidity did not impact measurements (Table 1). Accurate RR measurements were more robust to animal movement, with no differences in measured RR from immobilized or voluntary animals (\(\) 2 = 1.024, p = 0.312), and physical features of the animal (fur, scales, or feathers) (\(\) 2 = 3.902, p = 0.142). However, HR measurements (i.e., a peak frequency was identified after EVM analysis) were more likely to be obtained when imagery was collected from immobilized animals (\(\) 2 = 4.860, p = 0.027) and from mammals compared to birds and reptiles (\(\) 2 = 6.525, p = 0.038).

Successful extraction of physiological signals (RR: \(\) 2 = 6.200, p = 0.013; HR: \(\) 2 = 4.385, p = 0.036) was more likely in animals with thin than thick integument. Similarly, animals without significant subcutaneous fat were more likely to have a successful RR (\(\) 2 = 4.141, p = 0.042) and HR validation (\(\) 2 = 25.615, p < 0.00001). Video quality also significantly affected the ability to obtain RR (\(\) 2 = 13.974, p = 0.0002) and HR measurements (\(\) 2 = 13.424, p = 0.0003), with high quality videos (score of 6–8) more likely to produce a clear RR and HR signal.

Effects of species physical characteristics on accuracy of IRT measurements

IRT-derived vital rate measurements and ‘true’ values were highly correlated for both RR (Table 3, Fig. 3A; all taxa combined (n = 36): y = 1.0146x + 0.0386, R2 = 0.96; mammals only (n = 29): y = 1.0494x – 0.6931; R2 = 0.9349) and HR (Table 3, Fig. 3B; all taxa combined (n = 25): y = 0.856x – 10.431, R2 = 0.93; mammals only (n = 23): y = 1.0018x – 0.7602, R2 = 0.9917), and the slopes did not differ from one. However, some of the species’ physical features contributed to errors in IRT-derived physiological metrics (Fig. 4). The errors in IRT-derived RR relative to ‘true’ RR measurements were higher in animals with thick integument, fur, or scales compared to animals with thinner integument/pelage (Table 3). The accuracy of IRT-derived HR also differed among taxa, with greatest accuracy in mammals (Table 3, Fig. 4).

Fig. 3

Linear regressions showing the relationship between ‘true’ and IRT-derived A respiration rate (RR) and B heart rate (HR). Points are color coded by taxa (blue = mammal; yellow = bird; purple = reptile). The dashed black line shows a 1:1 relationship; the solid black regression line shows the relationship between IRT-derived and ‘true’ values for all taxa combined; and the blue regression shows the relationship for mammals only

Fig. 4

RR and HR residuals by taxa (A, B), thick or thin integument (C, D), immobilized or voluntary imaging (E, F), and presence of significant subcutaneous fat (G, H). All residuals were taken from the regression lines encompassing all imaging sessions (RR: y = 1.0146x + 0.0386; HR: y = 0.8559x + 10.431)

Successful vs. failed use of IRT and characteristics that played a part in errors in RR and HR

Because the majority of animals participated voluntarily, not all stayed still long enough to capture videos ≥ 9 s, and as a result IRT did not provide a clear enough signal for RR in 8 of 40 species or HR in 18 of 40 species (Table 4). There were 6 species for which neither IRT-derived RR nor HR could be measured (African crested porcupine (Hystrix cristata), prehensile-tailed porcupine (Coendou prehensilis), King penguin (Aptenodytes patagonicus), blue penguin (Eudyptula minor), Magellanic penguin (Spheniscus magellanicus), slender-tailed meerkat (Suricata suricatta)). These imaging sessions produced videos of low quality due to movement of the animal or background ‘noise’ (i.e., movement) that interfered with EVM analysis and provided no clear peak frequency.

Table 4 Failed validations. Species for which there were no peak frequencies extracted related to RR and/or HR are listed here. An ‘X’ in the Validation Failed column indicates which measurement was not obtained via IRT. A suspected reason for failure is listed for each species

For a subset of animals, ‘true’ HR measurements could not be obtained due to the difficulty of using a stethoscope on animals with thick scales or skin (African elephant (Loxodonta africana), hippopotamus (Hippopotamus amphibius), gopher tortoise (Gopherus polyphemus), ostrich (Struthio camelus), radiated tortoise (Astrochelys radiata)), eating while imaging (causing the stethoscope to pick up mastication and/or deglutition and not heart rate), or logistical issues in placing the stethoscope on the animal through the enclosure (brown bear (Ursus arctos), lesser kudu (Tragelaphus imberbis)) (see Additional file 2, Table S2). While these cannot be directly compared to true values, IRT analyses yielded RR and HR values comparable to previous studies in a subset of these animals (Table 5).

Table 5 Species for which IRT-derived RR and HR were measured, but no true values could be measured to allow for comparison

The successful validation of measuring vital rates with IRT allows for its use to measure RR and HR in a range of species, with the potential to address larger ecological questions. For example, the non-invasive IRT-derived measurements had similar relationships with animal body mass, when compared with ‘true’ measurements that required animal training or immobilization, demonstrating the applicability of this method in comparative studies (Fig. 5; A. ‘true’ RR: y = -1.659ln(x) + 28.749; IRT RR: y = -1.916ln(x) + 30.323; ‘true’ HR: y = -16.4ln(x) + 154.32; IRT HR: y = -7.907ln(x) + 120.82).

Fig. 5

The negative log–log relationship between mass and A RR and B HR. Squares are ‘true’ values and circles show IRT-derived vital rate values, with all points colored by taxa (blue = mammal; yellow = bird; purple = reptile). The solid black line is the regression of ‘true’ values with body mass, and the dashed black line shows the regression IRT-derived vital rates with body mass