Introduction

Achieving the desired therapeutic outcomes with antiseizure medication (ASM) therapy is of paramount importance in the management of epilepsy. The integral component of the outcomes is mainly related to the effectiveness and safety of the therapy. Researchers worldwide have been putting vast efforts into finding solutions to overcome poor or adverse therapeutic outcomes, which aligns with the International League Against Epilepsy (ILAE) aspirations. In practice, healthcare providers need to make decisions regarding optimizing the pharmacotherapy, which includes initiating and/or deferring the medication therapy, switching and altering the drug regimen and decisions on employing surgical interventions.1 In order to aid the healthcare providers in executing the above workup, a clinical prediction tool, also known as a prognostic model, clinical prediction rule or risk score, is often used to estimate the probability of the patient experiencing a particular outcome, requiring further interventions as mentioned above.

For epilepsy management, ILAE has outlined the definition of the outcome of therapeutic intervention, which includes the component of seizure freedom, treatment failure and undetermined outcome. An undetermined outcome is defined as an inadequate trial of the drug therapy or a lack of information in assessing the outcome.2 To date, there is only one study that developed a tool to estimate the prognostic outcome of epilepsy (Scale to Estimate Prognostic Outcomes of Epilepsy — SEPE) with ASM therapy in an outpatient setting. This study used regression analysis of data retrospectively collected from a specialized epilepsy center and subsequent telephonic follow-up of the patient’s seizure occurrence. The tool was able to predict whether a patient will become seizure-free without ASM therapy, the possibility of achieving long-term remission or whether the patient will experience pharmacoresistance.3

The most common method used to identify predictor variables in quantitative studies is regression analysis. The combination of multiple predictors would provide an understanding of the comparative influence of the predictors in the model.4 This method has been used by many other researchers investigating clinical outcomes in epilepsy and other neurological disorders.3,5,6 Developing a prediction tool from regression analysis requires meticulous examination of the model fit as overfitting models may hinder the applicability of the tool in other centers of similar population.4,7,8 Furthermore, establishing the tool’s validity is not less important and should not be taken for granted. Researchers need to select validity and reliability assessments appropriate for the method used in the tool’s development.

Apart from the prediction model derived from regression analysis, methods utilizing machine learning applications are also increasingly widespread, especially in developed nations. Regardless of the approach, a risk assessment tool for predicting the therapeutic outcomes of ASM therapy is highly warranted given the high rates of adverse outcomes as evident in previous studies. Thus, this study aims to develop a risk assessment tool for predicting the adverse therapeutic outcomes of ASM therapy and subsequently validate the developed tool.

Materials and Methods Study Design

This cross-sectional study involves adults diagnosed with epilepsy who were being monitored by a specialized clinic. Patients’ variables and information collected from medical records, along with face-to-face assessments using established validated tools, constituted the dataset for model development. The scope of development involves the identification of outcome measurement and predictor variables as well as analysis, verification and validation of the prediction model. The study was conducted between September 2022 to August 2023 at Tengku Ampuan Rahimah Hospital, a tertiary, government-funded healthcare center located in the central region of Malaysia.

Outcome Definition

The therapeutic outcomes of ASM therapy involve the outcomes that can be measured by retrieving the data from patients’ medical records and direct assessment during patient encounters. The finalized tool was sub-categorized into three main domains: seizure control (domain 1), seizure severity (domain 2) and adverse effect (AE) of ASM (domain 3). Seizure control (domain 1) is defined as freedom from any form of seizure activities (including auras) for at least one year from the last clinic visit. The information regarding the frequency and occurrence of the patient’s seizures was collected based on the doctor’s assessment documented in the patient’s medical records. For the seizure severity domain, the patients were assessed using a validated tool developed by Cramer et al which is called the Seizure Severity Questionnaire (SSQ).9 This tool contains 24 items and is made up of three subscales, gauging the patient’s or carer’s perspectives on the severity of the seizure within the past four weeks. The subscales include the phases of seizure activity, namely: i) pre-ictal (aura), ii) ictal and iii) post-ictal phases. The final SSQ score, which can range between 0 to 7, represents the severity of the seizure. The lowest score indicates the least severe seizure, while the highest score reflects the most severe seizure experienced by the patient.

For domain 3, the Liverpool Adverse Events Profile (LAEP)10 tool was employed to assess the incidence and intensity of adverse effects within the past four weeks.10 The tool consists of 19 items, each corresponding to a common adverse effect of ASM overall. Each item is tagged with a four-point Likert scale with the lowest score of 1 – no AE within four weeks, 2 – AE appeared for three to four days within four weeks, 3 – AE appeared for 15 days within four weeks and 4 – AE occurred almost every day within four weeks. The lowest score, which is 19, indicates that the patient never experienced any adverse effects within the past four weeks. Conversely, higher scores correlate with higher prevalence and intensity of adverse effects with a maximum score of 76.10 Permission to use the tools was granted by the corresponding authors via email. These three main outcomes served as the main domains in developing the assessment tool. While other outcomes such as quality of life and cognitive and psychosocial function are as important, these will not be included in this study.

Source of Data

Patients who had been on ASM for at least three months and had at least two years of clinic follow-up were eligible for inclusion in the study. For seizure control outcome, all patients (n = 397) from retrospective data collection were included as the data on seizure frequency were available in patients’ medical records. According to Kwan et al,2 counting the seizure frequency as a measure of the effectiveness of ASM therapy is appropriate. Thus, the most recent seizure control (seizure frequency within a certain period of time) was recorded. For the seizure severity and adverse effect domain, the patients’ outcomes were measured by a self-administered tool during the patients’ follow-up appointments (n = 105). This was conducted during the scheduled follow-up visits of pre-randomized patients within the study period (Supplementary material 1). Those who attended the clinic appointments were called and directed to a designated room before being seen by the medical officer. Once the informed consent form was signed by the patient or carer, they were requested to complete the SSQ and LAEP tools. The researcher assisted the patients throughout the assessment process. All other variables were retrieved directly from patients during the encounter or noted from patients’ medical records.

Data Analysis, Model Development and Validation

There were two analysis methods used for the development of final model in this study. For binary outcomes (seizure control), multivariate logistic regression was utilized. For outcomes involving continuous parameters, ie SSQ and LAEP scores, multivariate linear regression was applied. The patients’ demographic and clinical variables and ASM factors that may impact therapeutic outcomes were initially subjected to univariate analysis with a significance threshold set at p ≤ 0.2511 to identify potential predictor variables. Subsequently, the significant variables identified through this univariate analysis were further scrutinized using multivariate logistic (for seizure control outcome) and linear (seizure severity and adverse effects outcomes) regressions. In this multivariate analysis, continuous data variables were examined and classified based on established evidence and findings from statistical analysis with the rationale discussed earlier.

Variables with a significant value of p ≤ 0.05 were taken into account as predictors and retained for scoring system generation.12 Repeatedly, the variables in the final model were checked for multicollinearity and any possible interactions. The distribution of the residuals and homoscedasticity of the data was also checked. For the generation of the scoring system, the beta coefficient values of the predictors were used as the score value.1,3,13 For the development of the items’ score, the value of the beta coefficient of each predictor was divided by the smallest beta coefficient and the resulting value was rounded.1,14 All statistical analyses were performed using Statistical Packages for Social Sciences (SPSS Inc., Chicago, IL) version 27. Validity assessments were done using the criterion (concurrent) validity method where scores obtained from the developed tool were compared with the patients’ actual outcomes.

Results Identification and Verification of Predictor Variables

All of the relevant independent variables for therapeutic outcomes of epilepsy management with ASM therapy have been pre-identified from literature. Each of the potential variables were analyzed using univariate logistic regression (against seizure control status) and linear regression (for SSQ and LAEP scores). Patients’ descriptive and univariate analysis findings for the seizure control domain are described in Table 1, whereas results for multivariate regression analysis for the domain is shown in Table 2.

Table 1 Descriptive and Univariate Analysis of Patients’ Variables with Seizure Control

Table 2 The Final Predictive Model of Seizure Control Status Using Multivariate Analysis

Finalization of Prediction Model

The predictor variables included in the final model were duration of epilepsy, type of seizure, adherence status and the number of ASM prescribed. The final model was checked for multicollinearity using bivariate correlation analysis and the result showed a small value of Pearson’s correlation (<0.3) across all variables. All the predictors were deemed necessary to be included in the final model based on clinical importance.

The interaction terms for each of the variables against the other were also checked and the result showed a p-value of >0.05, indicative of no interactions within all the variables. The goodness of fit for the final model was assessed using the Hosmer–Lemeshow test and reviewing the correctly classified percentage from the classification table From the analysis, the results showed that the model is able to establish that there was no significant difference between the observed probability and the predicted probability with the p-value of 0.900 and the degree of freedom (df) of 6. Also, the correctly classified percentage from the classification table demonstrated that the model is of good fit (81.1%).

For the seizure severity domain, two variables, seizure types and psychiatric comorbidities, were initially included in the multivariate analysis, but both showed insignificant results and were omitted to enhance the model fit. All assumptions of linear regression were evaluated and scrutinized. In this study, the outliers detected by Mahalanobis distance were retained due to the clinical complexity of seizure severity context and its measurement findings. There was no serial correlation present among the residuals as proven by the Durbin–Watson test, which yielded a value of 2.162 (values that lie between 1.5 and 2.5 indicate no serial correlation). Apart from that, the variance inflation factor (VIF) revealed values of less than 10 for all the variables, indicative of no multicollinear issues in the model. Both the histogram and P–P plot showed a normal distribution of the standardized residuals. Similar to the SSQ domain, the LAEP domain also adopted the same method to establish the final predictive model. Except for homoscedasticity assessment results, all the other linear regression assumptions for both the SSQ and LAEP domains were met (Table 3).

Table 3 The Results of Regression Analysis for SSQ Score and LAEP Score Against Predictor Variables Respectively

Devising the Weight of the Items for the Domain’s Score

For the seizure control domain, we assigned each outcome (good and poor seizure control) with a weighted integer value accordingly. Essentially, good seizure control carries 0 points whereas poor seizure control is valued at 1 point. Below is the regression equation (binary logistic regression) prorated with the smallest β coefficient:

Where the weighted integer values are as follows:

Disease duration: <10 years = 0≥10 years = 1

Seizure type: Unknown = 0Generalized = 1Focal = 1

ASM adherence: Yes = 0No = 1

No. of ASM: <3 = 0≥3 = 1

From the equation, it could be inferred that the predicted score is equivalent to the sum of all the predictors. We further reshuffled the equation as follows:

Given the assigned weight for good seizure control is 0, the right side of the equation is thus represented by “0”. Therefore, the cut-off point corresponding to good seizure control is 2, where:

Thus,

For good seizure control, the cut-off point = 0 + 2 = 2

For poor seizure control, the cut-off point = 1 + 2 = 3

From the above formula, good seizure control is calculated to be equivalent to 2. Hence, the risk of poor seizure control was predicted for patients with a total score of more than 2.

Similar to seizure control, the smallest values of the respective β coefficients for seizure severity domains became the denominator of the other variables, which are disease duration (β = 0.625) and neuro deficit status (β = 1.191) respectively. On the other hand, for the adverse effect domain, the coefficients were prorated so that the constant in the formula becomes a value of 19, which is the minimum possible score for LAEP. For patients with a negative finding (0 points) for all three variables, the total score will be 19, which indicates no adverse effects experienced. The finalized items and points of each domain are described in Table 4.

Table 4 Scoring System to Calculate Point Values for Each Item in the Risk Assessment for ASM Therapeutic Outcomes Tool (RAS-TO)

Finalized Tool Validity and Reliability Assessment

Measurement of agreement (interrater reliability) of the tool was performed using Kappa’s coefficient analysis and the result revealed a value of 0.220 (fair agreement). Further validity test was performed by determining the area under the ROC curve for seizure control as predicted by the developed assessment tool (Figure 1). The result showed that the AUC value was 0.711, which would translate the tool’s ability of discriminating the cases with 71.1% accuracy (p < 0.001). Apart from this, another predictive analysis performed was the determination of the specificity and sensitivity of the tool to accurately predict the seizure control outcomes. It was found the sensitivity of the tool was 82.1% whereas the specificity was 68.2%. The Bonferroni test from the ANOVA analysis was performed to determine the homogeneity of variance and compare the means of the actual and predicted scores for each of the possible scores in domain 2 (Figure 2) and domain 3 respectively. The mean for the actual score against the predicted scores was generated, and from the graph, with the emphasis on domain 3 (LAEP score), the scores obtained for the tool did not accurately parallel the corresponding mean of the actual scores.

Figure 1 The area under the Receiver Operating Characteristic (ROC) Curve exhibiting the proportion of the predicted seizure control in concordance with the actual outcome.

Figure 2 The mean of actual Seizure Severity Questionnaire (SSQ) score versus the predicted score.

For the seizure severity domain, the range of scores was determined by obtaining the mean of the interval between the lower and upper values (eg a domain score of 1 yielded 1.5 from the equation, and the lower value of the calculated score is 0, thus the mean is 0.75) and the mean was subsequently added and subtracted from the calculated score to establish the score range. A simple descriptive analysis was conducted to verify the proportion of patients fell under each actual score range. It was found that the largest proportions of patients from each domain score group fell under the corresponding score range. The same procedure was repeated for domain 3 in determining and verifying the corresponding scores for the tool. It was discovered that the predicted LAEP scores from the new tool was not clearly corresponding to the actual scores. This construed the unfitness of the predictive model for the adverse effect outcome in predicting the LAEP score. Therefore, it is appropriate for the domain to be omitted from the final tool.

Discussion

The therapeutic outcomes following ASM therapy have been one of the topics of interest of researchers in epilepsy management, particularly in developing countries. Evidence has shown that, with the optimal use of ASM, a greater proportion of patients have successfully achieved seizure remission.15 However, there are still a significant number of studies reporting prominent fractions of patients who were still suffering from poor seizure outcomes.16–19 Recognizing patients with the tendency to experience undesired therapeutic outcomes is a challenge that should be collaboratively addressed by healthcare professionals from multiple disciplines.

In 2009, Kwan and the ad hoc Task Force of the ILAE Commission on Therapeutic Strategies (2010)20 established a framework for the definition of drug-resistant epilepsy. In the report, the researchers developed a scheme for categorizing the outcome of therapeutic intervention in epilepsy that comprises two main dimensions, which are seizure control and the occurrence of adverse effects. These dimensions serve as the foundation of the outcome measures in this current study with the added measurement of seizure severity as recommended by Cramer et al (2002)9 and Todorova et al (2013).21 A recent study originating in China developed a scale for predicting the outcome of adults with epilepsy (SEPE scale) with a total of 141 subjects. The subjects were divided into three groups based on clinical outcomes during the follow-up period: seizure-free without ASM therapy, pharmacoresponsive epilepsy and pharmacoresistant epilepsy. The study utilized logistic regression analysis to determine the predictor variables and simply included all the predictors with a significance level of p < 0.05 from univariate analysis. These variables are family history of epilepsy, mental deterioration, sleep-wake circles of seizure occurrence, the number of seizure types, frequency of seizures and MRI findings. Integer values were allocated to each item of the variables based on the regression findings. The sum of the score indicated the estimation of epilepsy prognosis, where a score of ≤3 would predict that the patient will become seizure-free without ASM therapy, a score of ≤4 meant that the patient has the potential for long-term remission and a score of ≥6 indicates that the patient is more likely to have pharmacoresistant epilepsy (maximum score: 16.5). Predictive analysis was the only measure conducted of the scale’s validity and the mean scores among the three groups were compared using ANOVA (p < 0.001).3

There are various strategies with statistical tools available for model generation. The commonly used approaches are regression analyses (either logistic, linear or both) and Cox proportional hazard models, depending on the intended purpose of the model.14 As the prognostic or predictive models are intrinsically multivariable, multivariable regression modelling is the most frequently used approach.1 In this current study, both logistic and linear regressions were applied as a method of model generation. The steps involved in the prediction model development in regression analysis include identification of candidate variables, elimination of insignificant variables using either forward, backward or stepwise elimination approach, assessing the goodness of fit and other assumptions in regression analysis and finally, checking the model performance.1,8,14,22 Although the strength of linearity between the predictors and outcomes was modest (), many researchers have repeatedly emphasized that statistically significant predictors were not even a prerequisite criterion to be legitimately included in a prognostic/prediction model as long as the predictors have profound values within the subject matter context.14,22,23

In this study, the generation of a simplified model in the form of weighted scores was derived from the regression coefficient of the final model of each domain. This was carried out by converting the β coefficients to integer values, with the aim of the user-friendliness of the developed tool. For seizure control and severity domain, β coefficients of each variable were divided by the smallest β values of the respective model. As for adverse effects, the denominator was computed so that the constant value becomes the minimum score value of the adverse effect measurement (LAEP score). This generation of a simplified model approach has been verified by researchers and adopted in numerous studies related to prediction model development.1,3,8,14,24,25

Typically, the assessment of a model’s overall performance involves measuring the difference between the predicted outcome and the actual outcome. This difference is associated with the notion of a model’s “goodness of fit”, in which superior models exhibit narrow gaps between predicted and observed results.4 In this study, the model’s performance was assessed by the ability of the model to discriminate between positive and negative outcomes for the binary outcome prediction model. For the seizure control domain, the model was assessed by determining the c-index of the ROC curve, which is equal to the area under the curve. A model is considered to have a perfect discrimination ability if the c-index (AUC) is equal to 1. A value of 0.5 indicates that the model has no better discriminative ability than random predictions.4,13,22 From the finding of the area under the ROC curve (AUC = 0.711), it can be deduced that the model was able to discriminate cases within the seizure control outcome. Furthermore, the specificity of the seizure control domain was found to be 68.2% with an 82.1% sensitivity. This is comparable with Chen’s SEPE scale with 67–80% specificity and sensitivity ranging between 50–81% for the two main outcomes.

Validating a tool derived from a quantitative study involves assessing its reliability and validity to ensure its reproducibility and measuring what it is intended to measure with good accuracy and consistency.8,22,26,27 Given the nature of this study, the criterion validation method, which is a concurrent and predictive validity method, is the appropriate validation assessment to be applied. In both types of criterion validity, the correlation between the predicted outcome from the finalized model and the actual patients’ outcome was analyzed to determine the strength of the relationship.28 A high correlation coefficient indicates a strong relationship, suggesting good criterion validity.29–31 Although there was only a fair correlation between the domain and actual outcome (Kendall’s Tau-b ≤ 0.3), it is important to note that the strength of the correlation would depend on the seriousness of the outcome, the vulnerability of the population and the manipulability of the risk factors.32,33 Moreover, in the context of concurrent validity assessment, the correlation analysis employed in this study was mainly to establish the relationship between the predicted and actual outcome without the means to explore the causal relationship. Nevertheless, a diagnostic or mortality prediction of an acute disease might require a higher correlation coefficient in validating the prediction model owing to the seriousness of the outcome.

As the final step of the tool development, the corresponding range of actual scores with the predicted score was determined and verified for the finalized tool’s interpretation. Initially, the means plots, which were derived from the Bonferroni (ANOVA) test, were assessed to determine the consistency of each of the predicted scores to resemble the actual scores.34 By visual observation of the plot, the seizure severity domain yields good consistency between the domain and actual scores while the adverse effect domain fails to match the actual score consistently. Both actual and predicted scores were subsequently regressed and the actual range was determined by substituting the predicted score into the regression formula. Further descriptive analysis to gauge the patients’ proportions of each score revealed inconsistency between the patients’ proportions across the domain scores with the corresponding actual scores for the adverse effect domain. A plausible explanation for this discrepancy is the model was generated from a heterogeneous dataset which denotes unequal variances within the variables. To prevent inaccurately predicting a patient’s LAEP score, the developed domain from the adverse effect prediction model is excluded as part of the domain in the final tool.

Internal validation is a process to assure the relationship between the predictors and the outcome did not occur by chance. Internal validity does not resolve issues such as selection bias within the recruitment, measurement errors and missing data, as the validation process is executed within the study population.13,35 The most important value of internal validity is that it addresses the stability of the predictors’ selection, hence adjudicating the quality of the predictions.4 Although internal validation could not replace the role of external validation in establishing the generalizability of the model, performing either of the approaches would nevertheless heighten the confidence in employing the model in comparable settings.

This new tool; Risk Assessment tool for ASM Therapeutic Outcomes (RAS-TO) possesses an important feature which is simple and easy to use based on variables that are readily available. It can be applied in clinic settings where clinicians can begin the patient’s consultation using the tool as a guide. Alternatively, other healthcare professionals such as pharmacists or epilepsy care nurses can play an active role in assessing the patients at the clinic beforehand. This may also lead to more efficient patient management as the risk of poor outcomes would have been acknowledged in advance. This allows the doctors to plan the best treatment strategies while upholding the interest of patients’ waiting time at the clinic.

It is important to acknowledge the limitations of the study. The method of universal sampling of the subjects for both outcome assessments, seizure severity and adverse effect outcomes, may have contributed to the heterogeneity of variances. Nevertheless, the findings of this study could notably contribute to the appropriate pharmacotherapy management of adults with epilepsy.

Conclusion

From this study, the two important therapeutic outcomes of ASM therapy, seizure control and severity, can be predicted using the assessment tool developed from the regression analysis of patients’ data. Identifying and recognizing the patients’ therapeutic outcomes are the responsibility of not only the treating physicians but also all members of the healthcare team alike. Therefore, the use of a practical and convenient tool may aid the decision in optimizing the ASM therapy in the management of epilepsy.

Data Sharing Statement

All data generated or analyzed during this study are included in this article. Further enquiries can be directed to the corresponding author.

Ethics Approval and Consent to Participate

Ethical approval was obtained from the Medical Research and Ethics Committee (MREC) of the Ministry of Health, Malaysia (NMRR-21-1087-59121), and Universiti Kebangsaan Malaysia (JEP-2022-040). Permission to conduct the study from the site of investigation was obtained from the Director of Tengku Ampuan Rahimah Hospital. The study was conducted in accordance with the Declaration of Helsinki Ethical Principles. Written informed consent was obtained from all participants who participated in the SSQ and LAEP assessment. All information about the participants will be kept strictly confidential.

Acknowledgments

The authors would like to thank the Director General of the Ministry of Health for the approval of publication and studentship support for this study.

Disclosure

The authors report no conflicts of interest in this work.

References

1. Han K, Song K, Choi BW. How to Develop, Validate, And Compare Clinical Prediction Models Involving Radiological Parameters: Study Design And Statistical Methods. Korean J Radiol. 2016;17(3):339–350. doi:10.3348/kjr.2016.17.3.339

2. Kwan P, Hao X-T. Update and Overview of the International League Against Epilepsy Consensus Definition of Drug-resistant Epilepsy. Eur Neurol Rev. 2011;6(1):57–59.

3. Chen X, Ma XB, Zhang Q, Yin Q, Li XH. A Scale for Predicting the Outcomes of Patients with Epilepsy: a Study of 141 Cases. Int J Gen Med. 2021;14:1565–1574. doi:10.2147/IJGM.S302735

4. Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35(29):1925–1931. doi:10.1093/eurheartj/ehu207

5. Roberg LE, Monsson O, Kristensen SB, et al. Prediction of Long-term Survival After Status Epilepticus Using the ACD Score. JAMA Neurol. 2022;79(6):604–613. doi:10.1001/jamaneurol.2022.0609

6. Van Diessen E, Lamberink HJ, Otte WM, et al. A Prediction Model to Determine Childhood Epilepsy After 1 or More Paroxysmal Events. Pediatrics. 2018;142(6). doi:10.1542/peds.2018-0931

7. Chen L. Overview of clinical prediction models. Ann Translat Med. 2019;8(4):71. doi:10.21037/atm.2019.11.121

8. Moons KGM, Kengne AP, Woodward M, et al. Risk prediction models: i. Development, internal validation, and assessing the incremental value of a new (bio)marker. Heart. 2012;98(9):683–690. doi:10.1136/heartjnl-2011-301246

9. Cramer JA, Baker GA, Jacoby A. Development of a new seizure severity questionnaire: initial reliability and validity testing. Epilepsy Res. 2002;48(3):187–197. doi:10.1016/S0920-1211(02)00003-7

10. Baker GA, Frances P, Middleton E, Dafalla B, Jacoby A. Initial development, reliability and validity of a patient-based adverse drug event scale [Abstract]. Epilepsia. 1994;35(7):20. doi:10.1111/j.1528-1157.1994.tb02907.x

11. Hosmer DW, Lemeshow S, Sturdivant RX. Model-Building Strategies and Methods for Logistic Regression. Applied Logistic Regression. Third edition ed. New York: John Wiley & Sons Inc.; 2013:89–151.

12. Hosmer D, Lemeshow S. Applied Logistic Regression. Hoboken; 2000.

13. Shipe ME, Deppen SA, Farjah F, Grogan EL. Developing prediction models for clinical use using logistic regression: an overview. J Thorac Dis. 2019;11(Suppl 4):S574–S84. doi:10.21037/jtd.2019.01.25

14. Lee YH, Bang H, Kim DJ. How to Establish Clinical Prediction Models. Endocrinol Metab. 2016;31(1):38–44. doi:10.3803/EnM.2016.31.1.38

15. Niriayo YL, Mamo A, Kassa TD, et al. Treatment outcome and associated factors among patients with epilepsy. Sci Rep. 2018;8(1):17354. doi:10.1038/s41598-018-35906-2

16. Ahmed M, Nasir M, Yalew S, Getahun F, Getahun F. Assessment of Treatment Outcome and Its Associated Factors among Adult Epileptic Patients in Public Hospitals in the Southern Ethiopia: a Multi-center Cross-sectional Study. Ethiop J Health Sci. 2023;33(2):327–336. doi:10.4314/ejhs.v33i2.18

17. Dubale M, Gobena K, Aklog A, Ababu Y, Bose L. Treatment Outcome and Associated Factors among Adult Epileptic Patients at Hawassa University Specialized Hospital, Southern Ethiopia. J Bioanal Biomed. 2018;12:7.

18. Yazie TS, Kefale B, Molla M. Treatment Outcome of Epileptic Patients Receiving Antiepileptic Drugs in Ethiopia: a Systematic Review and Meta-Analysis. Behav Neurol. 2021;2021:5586041. doi:10.1155/2021/5586041

19. Zewudie A, Mamo Y, Feyissa D, Yimam M, Mekonen G, Abdela A. Epilepsy Treatment Outcome and Its Predictors among Ambulatory Patients with Epilepsy at Mizan-Tepi University Teaching Hospital, Southwest Ethiopia. Neurol Res Int. 2020;2020:8109858. doi:10.1155/2020/8109858

20. Kwan P, Arzimanoglou A, Berg AT, et al. Definition of drug resistant epilepsy: consensus proposal by the ad hoc Task Force of the ILAE Commission on Therapeutic Strategies. Epilepsia. 2010;51(6):1069–1077. doi:10.1111/j.1528-1167.2009.02397.x

21. Todorova KS, Velikova VS, Kaprelyan AG, Tsekov ST. Seizure Severity as an Alternative Measure of Outcome in Epilepsy. J IMAB. 2013;19(3):433–437. doi:10.5272/jimab.2013193.433

22. Cowley LE, Farewell DM, Maguire S, Kemp AM. Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature. Diagn Progn Res. 2019;3(1):16. doi:10.1186/s41512-019-0060-y

23. Chowdhury MZI, Turin TC. Variable selection strategies and its importance in clinical prediction modelling. Fam Med Community Health. 2020;8(1):e000262. doi:10.1136/fmch-2019-000262

24. Lee YH, Bang H, Park YM, et al. Non-laboratory-based self-assessment screening score for non-alcoholic fatty liver disease: development, validation and comparison with other scores. PLoS One. 2014;9(9):e107584. doi:10.1371/journal.pone.0107584

25. Steyerberg EW. Applications of Prediction Models. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. Cham: Springer International Publishing; 2019:15–36.

26. Cai QC, Yu ED, Xiao Y, et al. Derivation and validation of a prediction rule for estimating advanced colorectal neoplasm risk in average-risk Chinese. Am J Epidemiol. 2012;175(6):584–593. doi:10.1093/aje/kwr337

27. Steyerberg EW. Validation of Prediction Models. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. Cham: Springer International Publishing; 2019:329–344.

28. Taherdoost H. Validity and Reliability of the Research Instrument; How to Test the Validation of a Questionnaire/Survey in a Research. SSRN Electron J. 2016. doi:10.2139/ssrn.3205040

29. Akoglu H. User’s guide to correlation coefficients. Turk J Emerg Med. 2018;18(3):91–93. doi:10.1016/j.tjem.2018.08.001

30. Gust L, D’journo XB. The use of correlation functions in thoracic surgery research. J Thoracic Dis. 2015;7(3):E11–E5. doi:10.3978/j.issn.2072-1439.2015.01.54

31. Arndt S, Turvey C, Andreasen NC. Correlating and predicting psychiatric symptom ratings: spearman’s r versus Kendall’s tau correlation. J Psychiatr Res. 1999;33(2):97–104. doi:10.1016/S0022-3956(98)90046-2

32. Brossart DF, Laird VC, Armstrong TW. Interpreting Kendall’s Tau and Tau-U for single-case experimental designs. Cogent Psychol. 2018;5(1):1518687. doi:10.1080/23311908.2018.1518687

33. Kraemer HC. Correlation coefficients in medical research: from product moment correlation to the odds ratio. Stat Methods Med Res. 2006;15(6):525–545. doi:10.1177/0962280206070650

34. Armstrong RA. When to use the Bonferroni correction. Ophthalmic Physiol Opt. 2014;34(5):502–508. doi:10.1111/opo.12131

35. Harrell FE. Resampling, Validating, Describing, and Simplifying the Model. Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY: Springer New York; 2001:87–103.