ORIGINAL ARTICLE Year : 2023  |  Volume : 9  |  Issue : 4  |  Page : 415-418

Investigation of the test–retest reliability and inter-rater agreement of traditional Chinese medicine-based pulse diagnosis among Indian traditional Chinese medicine practitioners

Mamta Jagwani1, Gita Sharma1, Hemanshu Sharma2, Tara K Kafle3, Gulab Rai Tewani4, Pradeep M K. Nair5
1 Department of Acupuncture and Energy Medicine, Sant Hirdaram Medical College of Naturopathy and Yogic Sciences, Bhopal, Madhya Pradesh, India
2 Department of Community Medicine, Sant Hirdaram Medical College of Naturopathy and Yogic Sciences, Bhopal, Madhya Pradesh, India
3 Department of Community Medicine, Birat Medical College and Teaching Hospital, Tankisinuwari, Nepal
4 Department of Naturopathy, Sant Hirdaram Yoga and Nature Cure Hospital, Bhopal, Madhya Pradesh, India
5 Department of Research, Sant Hirdaram Medical College of Naturopathy and Yogic Sciences, Bhopal, Madhya Pradesh, India

Date of Submission30-Jun-2022Date of Acceptance12-Jul-2022Date of Web Publication30-Oct-2023

Correspondence Address:
Prof. Pradeep M K. Nair
Sant Hirdaram Medical College of Naturopathy and Yogic Sciences, Sant Hirdaram Nagar, Bhopal - 462 020, Madhya Pradesh
India

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/2311-8571.388728

Objective: The objective of this study was to assess the test–retest reliability and inter-rater agreement between the traditional Chinese medicine (TCM) pulse diagnosis between two Indian TCM practitioners. Methods: A total of 300 participants were evaluated for their bilateral pulse based on TCM independently by two investigators with similar years of experience but who graduated from different medical schools. The investigators who were blinded diagnosed the participants as per TCM based on the depth, rate, and strength of their pulse. Cronbach α and weighted kappa were used for assessing the reliability and repeatability. Results: The data of all the 300 participants were included in the analysis. The test–retest reliability computed using Cronbach's alpha indicated high reliability between the investigators (α =0.963). There were a total of 37 different types of diagnoses identified by both investigators. The investigators diagnoses matched for 116 participants out of 300 participants; whereas 184 cases were not matched between the investigators. The inter-rater agreement for the TCM diagnosis tested by weighted Kappa did not returned any significant agreement (Kappa = 0.370). Discussion: The present study shows higher reliability between the pulse diagnoses between the investigators. However, the inter-rater agreement for TCM diagnosis based on pulse diagnosis was not significant probably due to the variation in subjective assessments. This calls for the standardization of training in TCM diagnostic methods among practitioners. Future studies should include the differences in training, medical education, and experience among the practitioners while determining the reliability of pulse diagnosis.

Keywords: Oriental medicine, pulse diagnosis, reliability, traditional Chinese medicine, validity

How to cite this article:
Jagwani M, Sharma G, Sharma H, Kafle TK, Tewani GR, K. Nair PM. Investigation of the test–retest reliability and inter-rater agreement of traditional Chinese medicine-based pulse diagnosis among Indian traditional Chinese medicine practitioners. World J Tradit Chin Med 2023;9:415-8
How to cite this URL:
Jagwani M, Sharma G, Sharma H, Kafle TK, Tewani GR, K. Nair PM. Investigation of the test–retest reliability and inter-rater agreement of traditional Chinese medicine-based pulse diagnosis among Indian traditional Chinese medicine practitioners. World J Tradit Chin Med [serial online] 2023 [cited 2023 Dec 23];9:415-8. Available from: https://www.wjtcm.net/text.asp?2023/9/4/415/388728   Introduction 

Traditional Chinese medicine (TCM) is one among the popular complementary systems of medicine across the globe. TCM utilizes various diagnostic methods such as pulse diagnosis and tongue diagnosis which are aligned with its basic principles.[1] Even though these diagnostic techniques are empirical, they are often termed as subjective,[2] as these assessments are influenced by numerous factors such as assessors experience, change in patients emotions, timing of pulse measurement, and training.[3],[4],[5] There is a growing interest among scientists in assessing the intra-rater and inter-rater reliability of TCM diagnostic methods, especially pulse diagnosis, which is widely popular among TCM practitioners.

Numerous studies earlier have analyzed the intra-rater and inter-rater reliability of pulse diagnosis and have reported pulse diagnosis to be a reliable diagnostic method. These studies have used varying statistical methods and have reported low-to-moderate levels of inter-rater agreement for pulse diagnosis ranging from 57% to 80%.[3],[6],[7] Besides this, in an attempt to quantify pulse diagnosis, numerous artificial/computerized models of pulse diagnosis were utilized which also has shown pulse diagnosis to be a dependable diagnostic method.[1],[8],[9],[10]

TCM even though it has originated from China, presently, it is a global phenomenon where numerous practitioners across the globe integrate TCM regimes into their mainstream practice. India is one among the lead runner in integrating and licensing TCM practices, especially acupuncture.[11],[12] Indian physicians extensively use pulse diagnosis as a primary method for point selection in acupuncture. However, till date, there are no reports on the inter-rater agreement of pulse diagnosis among Indian TCM practitioners. It will be interesting to understand the inter-rater and test–retest reliability of pulse diagnosis among Indian TCM practitioners considering their diverse educational and training backgrounds.

The present study examined the inter-rater and test–retest reliability of pulse diagnosis among two Indian TCM practitioners who graduated from two different medical schools but having with similar experiences in TCM practice.

  Methods 

Study setting

The study was conducted at the acupuncture and energy medicine department of a private medical college hospital after obtaining the institutional ethics committee approval (12/SHMCNYS-IEC/P23/2020-2021). All the participants signed a written informed consent before participating in the study.

Study design

The study was a double-blinded experimental study at a real-time setting, where both the investigators examining the pulse of the patients were blinded from each other's findings. The investigators were licensed yoga and naturopathy physicians with a postgraduate degrees in acupuncture and energy medicine. Both investigators examined the same patient at a 30-min interval on the same day to avoid the diurnal variation in pulse. The schematic representation of the flow of pulse diagnosis is represented in [Figure 1]. The recorded data were then analyzed by another investigator (statistician) for determining the level of agreement between the two investigators and its reliability.

Participants

The participants were patients with diverse medical conditions enrolled for a residential yoga and naturopathy-based lifestyle program at the study settings. Both male and female patients aged 18 or above who did not have a previous consultation/pulse diagnosis session with the investigators were included in the study. Patients with symptoms of any acute illness such as flu and common cold were excluded from the study as it may interfere with pulse.

Sample size calculation

The sample size was calculated using the following formula: n = Z2pq/e2 where 'n' stands for sample size, “Z” stands for confidence level at 95% (standard value of 1.96), “pq” stands for the variance of population, and “e” stands for allowable error. Allowing an attrition rate of 15%, the final sample size was calculated by n = 300.

Method of pulse examination

The pulse was examined in sitting position where the patients arm, wrist, and hand were relaxed and well-supported with a pillow. The investigators positioned their middle three fingers at the cun, guan, and chi positions along the course of radial artery. The same procedure was repeated in the other hand as well. The pulse was assessed for its depth (vertical position of a pulse), rate (the rhythm of the pulse), and strength (forcefulness of a pulse in response to a change of applied pressure). Further, the depth, rate, and strength were classified into three levels as + 1, +2, and +3. The detailed descriptions of the levels are tabulated in [Table 1].[13] Based on these levels, a TCM diagnosis was made by each investigator separately.

According to TCM, there are two types of pulses: (i) deep pulse and (ii) superficial pulse. The deep pulse in the right hand corresponds to lung, spleen, and pericardium, similarly, the superficial pulse in the right hand corresponds to large intestine, stomach, and triple warmer. The deep pulse in the left hand corresponds to heart, liver, and kidney, whereas small intestine, gall bladder, and urinary bladder represent superficial pulse.[13],[14]

Statistical analysis

The data were tabulated and analyzed using Statistical Package for the Social Sciences software Version 23, IBM, New York, United States of America. The inter-rater reliability test was done using Cronbach's alpha test. An alpha value ranging from 0.70 to 0.95 is considered an indication of higher reliability.[15] Kappa is considered an appropriate test of reliability for categorical data such as diagnosis or interpretation of investigations.[16] The reliability of TCM-based diagnosis rated by the two different investigators was tested using the weighted kappa test.

  Results 

The data of all the 300 participants were included in the analysis. Mean, standard deviation, and reliability of various parameters of the pulse were rated by two different raters. The test–retest reliability computed using Cronbach's alpha showed that the average measure of reliability rate as 0.963. All the enrolled patients were initially diagnosed based on the International Classification of Disease and further based on the pulse characteristics, each patient was given a TCM-based diagnosis. All the categorical variables (TCM diagnoses) tested for reliability using weighted kappa have shown that there is no significant match between the diagnoses of the two investigators (Kappa = 0.370). The results are tabulated in [Table 2].

Table 2: Inter-rater reliability test of traditional Chinese medicine between the investigators using weighted kappa

Click here to view

Further, we have cross-matched the diagnosis between two investigators. There were a total of 37 different diagnosis categories. Among 300 participants in total, the findings of 116 participants' matched (38.67%) but 184 cases were not matched between the investigators.

  Discussion 

This study reports the levels test–retest and inter-rater reliability of the TCM-based pulse diagnosis between two TCM practitioners. Our results suggest that the assessment of pulse in TCM is reliable as it returned a Cronbach's alpha score of 0.963 that indicates optimum consistency. Both investigators have demonstrated identical expressions in the measurement of the characteristics of the pulse such as depth, rate, and strength. Fewer studies earlier have also attempted to demonstrate the reliability and validity of pulse diagnosis. Bilton et al. suggested that an acceptable level of reliability in pulse diagnosis can be achieved by having an operational definition for pulse diagnosis.[17] Pulse diagnosis is also shown to have high sensitivity and specificity in diagnoses of hypertension.[1] Instrument-based pulse diagnosis on the elderly population has shown pulse diagnosis to be 62% accurate among the elderly population.[18]

Inter-rater reliability measures the degree of agreement between two investigators examining the same phenomenon. The present study did not observed significant agreement between the investigators. This is in agreement with an earlier review which suggested TCM diagnoses such as pulse diagnosis and tongue diagnosis have low levels of inter-rater agreement. The low levels of inter-rater agreement are attributed to subjective changes in assessment/approaches in interpretation between the investigators.[19] This subjective variability in TCM pulse diagnosis may be surmounted by introducing robust standards in pulse diagnosis.

Numerous studies have been reported to investigate inter-rater reliability earlier; however, most of these studies are reported to have serious flaws such as inappropriate statistical tests being used, lack of supporting data, lack/difference of experience between the assessors, and lack of uniform standards in pulse diagnosis methods.[3],[4],[6],[17] Further, it was also reported that the experience of the assessors to be directly proportional to the accuracy of pulse diagnosis.[4] In this study, even though both the investigators have similar levels of experience, they have completed their undergraduate programs from two different medical schools which may have a role in lesser inter-rater agreement. Therefore, standardization of training among Indian TCM practitioners is warranted for better reliability and agreement in pulse diagnosis.

The present study is the first attempt from India to identify the reliability and repeatability of TCM-based pulse diagnosis on a larger sample. TCM-based therapies, especially acupuncture, are immensely popular in India, which are imparted as a medical therapy through licensed yoga and naturopathy physicians.[20] However, health being a state subject in India, the curriculum of acupuncture varies from medical school to medical school. This may have some implications in the understanding of pulse diagnosis. Further research should be undertaken to include practitioners from the same medical schools/region considering the possible diversity in approach on pulse diagnosis. Nevertheless, the insights gained from this study may be helpful in furthering the understanding on pulse diagnosis and add more value to TCM practices. Future studies may consider disease-specific pulse diagnosis as it can widen the diagnostic spectrum of TCM pulse diagnosis.

  Conclusion 

The present study contributes to the growing body of knowledge about the reliability and repeatability of TCM pulse diagnosis. While the present data suggests high reliability for test–retest scores, the inter-rater reliability scores were not significant. This merits wider study addressing the differences in training, medical education, and experience among the practitioners.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

 

  References 
1.Tang AC, Chung JW, Wong TK. Validation of a novel traditional Chinese medicine pulse diagnostic model using an artificial neural network. Evid Based Complement Alternat Med 2012;2012:685094.  
    2.Yoon SH, Koga Y, Matsumoto I, Ikezono E. Clinical study of objective pulse diagnosis. Am J Chin Med 1986;14:179-83.  
    3.King E, Cobbin D, Walsh S, Ryan D. The reliable measurement of radial pulse characteristics. Acupunct Med 2002;20:150-9.  
    4.O'Brien KA, Birch S. A review of the reliability of traditional East Asian medicine diagnoses. J Altern Complement Med 2009;15:353-66.  
    5.Hammer L. Contemporary pulse diagnosis: Introduction to an evolving method for learning an ancient art-part 1. Am J Acupunct 1993;21:1-16.  
    6.O'Brien KA, Abbas E, Zhang J, Guo ZX, Luo R, Bensoussan A, et al. Understanding the reliability of diagnostic variables in a Chinese medicine examination. J Altern Complement Med 2009;15:727-34.  
    7.Walsh S, Cobbin D, Bateman K, Zaslawski C. Feeling the pulse: Trial to assess agreement level among TCM students when identifying basic pulse characteristics. Eur J Orient Med 2001;3:25-31.  
    8.Yoon SH, Koga Y, Matsumoto I, Ikezono E. An objective method of pulse diagnosis. Am J Chin Med 1987;15:147-53.  
    9.Wang NY, Yu YH, Huang DW, Xu B, Liu J, Li TD, et al. Pulse diagnosis signals analysis of fatty liver disease and cirrhosis patients by using machine learning. ScientificWorldJournal 2015;2015:859192.  
    10.Lee SH, Niu TL, Yang XZ, Li HY, Zhu QW, Niu X. A quantitative investigation of pulse and tongue features in post-stroke depressive patients and healthy volunteers: An observational pilot study. Complement Med Res 2015;22:292-7.  
    11.Bakshi D, Mukherjee B, Basu S, Pal S, Chatterjee J. Historical introduction of acupuncture in India. Bull Indian Inst Hist Med Hyderabad 1995;25:216-25.  
    12.Elahee SF, Mao HJ, Shen XY. Traditional Indian medicine and history of acupuncture in India. World J Acupunct Moxibustion 2019;29:69-72.  
    13.BIng Z, Wang HC, editors. Diagnostics of Traditional Chinese Medicine. 1st ed. Philadelphia, Pennsylvania: Singing Dragon, People's Military Medical Press; 2010.  
    14.Beijing College of Traditional Chinese Medicine. Essentials of Chinese Acupuncture. 1st ed. Saratoga, California: Bamboo Delight Company; 1980.  
    15.Bland JM, Altman DG. Cronbach's alpha. BMJ 1997;314:572.  
    16.Sim J, Wright CC. The kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Phys Ther 2005;85:257-68.  
    17.Bilton K, Smith N, Walsh S, Hammer L. Investigating the reliability of contemporary Chinese pulse diagnosis. Aust J Acupunct Chin Med 2010;5:3-13.  
    18.Zhou S, Ogihara A, Nishimura S, Leng Z, Jin Q. Analysis of pulse diagnosis data for the elderly by using two analytical methods. Int J Soc Humanist Comput 2019;3:158.  
    19.Eigenschink M, Dearing L, Dablander TE, Maier J, Sitte HH. A critical examination of the main premises of traditional Chinese medicine. Wien Klin Wochenschr 2020;132:260-73.  
    20.Government Yoga and Naturopathy Medical College. MD Acupuncture Eligibility. Available from: https://www.gynmc.com/acadamics.php. [Last accessed on 2022 Jan 24].  
    
  [Figure 1]
 
 
  [Table 1], [Table 2]