Assessing the validity of Society of Thoracic Surgeons (STS) score in predicting stroke risk among patients undergoing cardiothoracic surgery at a tertiary hospital in Pakistan: a retrospective cohort study

Abstract

Background

Postoperative stroke is one of the most serious consequences of cardiac surgery. Morbidity risk assessment is critical for preoperative risk assessments and resource allocation. In this article, we aim to investigate the predictive value of Society of Thoracic Surgeons (STS) score’s effectiveness in stroke risk in cardiothoracic surgery patients in our population.

Methods

This retrospective cohort study was conducted at Aga Khan University Hospital (AKUH) using a consecutive sampling technique. The study included all eligible patients aged 18 years or older who underwent cardiac surgical procedures between January 2010 and December 2016. Of the 3,898 patients initially identified, 814 records were excluded due to incomplete data or pre-existing conditions. Statistical analyses, including chi-square tests, t-tests, and logistic regression, were performed to identify significant predictors of stroke. Prediction accuracy was assessed using a Receiver Operating Characteristic (ROC) curve, with Youden’s J statistic employed to determine optimal sensitivity and specificity thresholds.

Results

Out of 3,084 patients, 52 (1.7%) experienced a postoperative stroke. Stroke patients were significantly older (mean age 62.8 years vs. 57.7 years). They also had higher white blood cell count (10.7 ± 5.1 vs. 9.3 ± 3.3) and a longer history of myocardial infarction (9.2 ± 9.3 years vs. 6.9 ± 7.6 years). Cardiovascular interventions (15.4% vs. 7.7%), postoperative congestive heart failure (21.1% vs. 7.3%), and use of inotropes (5.8% vs. 1.6%) were more prevalent in stroke patients. Emergent surgical status (19.2% vs. 13.4%) and complications such as dialysis, prolonged ventilation, and intra-aortic balloon pump use were also significantly higher. The predictive model demonstrated strong accuracy in predicting postoperative stroke (AUC: 0.841, CI: 0.794–0.888). The ROC analysis for the STS stroke model showed high sensitivity (90.4%) and negative predictive value (99.7%), with moderate specificity (64.3%) and overall accuracy (64.8%), indicating excellent performance in ruling out stroke but moderate reliability in identifying positive cases.

Conclusion

The STS risk score demonstrated strong predictive accuracy for postoperative stroke risk in cardiothoracic surgery patients, effectively incorporating clinical factors already accounted for in the comprehensive set of 70 variables used in its calculation.

Introduction

Stroke is defined as any confirmed neurological deficit of abrupt onset caused by disrupted blood supply to the brain which remains unresolved after 24 h [1]. It is one of the leading causes of death and morbidity worldwide, both in developed and, increasingly, in low- and middle-income countries (LMICs) [2]. Ischemic or hemorrhagic perioperative stroke that happen during or within 30 days after surgery—can be a fatal consequence. The incidence ranges from 0.1 to 0.7% for those having non-cardiac and non-neurological surgery, from 1 to 5% for those having cardiac surgery, and from 1 to 10% after neurological surgery [3]. Despite advancements in procedural safety, cardiac surgeries continue to carry a high perioperative and postoperative risk [4].

To combat this, a myriad of risk stratification models has been designed over the years to develop patient risk profiles for evaluating intraoperative risk. These models can predict post-operative mortality and morbidity prior to surgery [5]. Among these, the Society of Thoracic Surgeons (STS) risk score calculator is widely used due to its validated accuracy, which is based on a large sample size of 774,881 and comprehensive clinical data [6]. This model works on multiple main categories: isolated coronary artery bypass grafting (CABG), isolated aortic valve replacement (AVR), isolated mitral valve replacement or repair (MVRR), AVR + CABG, and MVRR + CABG to predict post-operative outcomes for numerous complications as defined by STS. Thus, STS calculator serves as a useful way to foresee certain serious complications [7]. Early and delayed post-operative strokes have different risk factors such as age, gender, diabetes, hypotension and decreased cardiac output, some of which often are not accurately assessed clinically for the risk of stroke prior to surgery. Subsequently, this can lead to an underestimation of the risk of postoperative stroke. The STS calculator offers a more accurate risk assessment, which is crucial for surgical planning to help reduce the incidence of stroke [8]. A retrospective study performed using data from the STS adult cardiac database on 11,190 indexed cardiac operations done in University of Pittsburgh reported post-operative mortality in 2.2% (246 patients), with stroke accounting for 2.4% of these deaths [9].

The use of risk stratification models can assist in this process; however, their performance may vary across different populations. For example, the EURO score was predominantly used in European populations for a long time. When it was compared to the STS score in a U.S. population by the University of Virginia, it was concluded that the STS score was more effective for the U.S. population [10]. The STS risk score has proven to be the most effective stratification tool within the Pakistani population [11]. Thus, the STS score can be a valuable tool for predicting the risk of comorbidities, such as stroke, which require timely recognition and efficient allocation of healthcare resources. This may be especially beneficial in financially and resource-constrained countries like Pakistan [12]. To the best of our knowledge, there is limited literature evaluating the validity of the STS score specifically for predicting stroke in any population. Therefore, the aim of our study is to explore the validity of the STS calculator as a tool for predicting the risk of postoperative stroke in cardiac surgery patients within the Pakistani population.

Methods

Data source

This retrospective cohort study was conducted at Aga Khan University Hospital in Karachi, Pakistan, to evaluate the validity of the Society of Thoracic Surgeons (STS) Risk Calculator in predicting stroke morbidity in patients undergoing cardiac surgery. A consecutive sampling technique was employed, including all eligible patients aged 18 years or older who underwent any cardiac surgical procedure between 2010 and 2016.

The data was retrieved from the computerized database of the Cardiothoracic Surgery (CTS) department. Included procedures encompassed isolated surgeries such as Coronary Artery Bypass Grafting (CABG), Aortic Valve Replacement (AVR), and Mitral Valve Replacement (MVR), as well as combined procedures like AVR + CABG, MVR + CABG, and MV repair + CABG. Independent variables such as patient demographics, medical history, procedure details, and clinical characteristics were gathered. For this study, in-hospital data was collected with a primary focus on preoperative test results, including all the laboratory values necessary for calculating the STS score. Postoperative data included creatinine levels, discharge medications, the duration of hospital stay, and 30-day mortality following cardiac surgery. The STS score, an algorithmic learning-based risk prediction model, was calculated using an online risk calculator [13].

The actual results were stated as qualitative variables. The study was granted an exemption by the Ethical Review Committee of Aga Khan University.

Patient populations

A consecutive sampling technique was employed, including all eligible patients aged 18 years or older who underwent any cardiac surgical procedure between 2010 and 2016. The analysis was conducted based on the type of surgical procedure, including all patients regardless of their survival status during hospitalization, resulting in an initial dataset of 3,898 records. However, records with missing data necessary to calculate STS mortality, as well as those of patients with pre-existing renal failure or a history of stroke (n = 814 records), were excluded from the analysis. Consequently, the final study cohort comprised 3,084 records.

Statistical analysis

Statistical analyses were conducted using Python version 3.5 and R version 4.1.3, focusing on descriptive statistics, including mean, standard deviation, minimum, maximum, frequencies, and percentages for continuous variables. The study aimed to examine the relationships between demographic, clinical, and laboratory variables and stroke morbidity. Categorical variables were analyzed using chi-square or Fisher’s exact tests, while continuous variables were evaluated using either an independent sample t-test or a Mann-Whitney U test.

Univariate analysis identified significant variables for further investigation. These variables, with p-values less than 0.05, were included in a binary logistic regression model. Multivariate analysis employed stepwise backward selection, retaining only those variables with p-values below 0.05 in the final model. The stroke morbidity prediction was assessed using a Receiver Operating Characteristic (ROC) curve, and the area under the AUC curve was calculated with a 95% confidence interval.

Youden’s J statistics were utilized to determine the optimal cutoff point for the prediction probability score. At various cutoff points, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated, each with a 95% confidence interval. The results were presented as relative risk (odds ratio, OR) with a 95% confidence interval, and findings were displayed through charts, tables, and figures.

Results

The study population comprised 3,084 patients, of whom 3,032 did not experience postoperative stroke, while 52 patients did. The demographic and clinical characteristics of all patients are summarized in Table 1 while a matched cohort analysis of clinical and demographic factors is presented in Table 2. Patients who experienced postoperative stroke were significantly older (p < 0.001). The mean age of patients without postoperative stroke was 57.7 years (± 12.1), compared to 62.8 years (± 11.2) for those with postoperative stroke.

Table 1 Patient demographics and clinical factors with postoperative stroke incidence

Table 2 Matched cohort analysis of clinical and demographic factors in patients with and without postoperative stroke

Gender, hypertension, diabetes, family history of coronary artery disease (CAD), and obesity were found to be statistically insignificant factors. Specifically, none of the patients without postoperative stroke were receiving immunosuppressive therapy, while 1.9% of the patients with postoperative stroke were on immunosuppressive treatment.

Regarding hematological parameters, the creatinine levels, platelet counts, and hematocrit (HCT) levels in our patients did not show significant difference. WBC counts were significantly higher in postoperative stroke patients (10.7 ± 5.1) compared to non-stroke patients (9.3 ± 3.3).

Past medical history of coronary artery bypass grafting, valvular procedures, and myocardial infarction were not significant. However, the previous CV intervention, duration since myocardial infarction and post-operative congestive heart failure (CHF) were found to be significant. A history of CV intervention was seen in 15.4% of patients who developed a postoperative stroke, as compared to 7.7% of patients who did not develop a postoperative stroke. In addition, the time since myocardial infarction was significantly longer in postoperative stroke patients (9.2 ± 9.3 years vs. 6.9 ± 7.6 years, p < 0.05). Similarly, postoperative congestive heart failure was observed in 21.1% of stroke patients, compared to 7.3% of non-stroke patients (p < 0.001).

When assessing angina types, stable angina did not appear to influence postoperative stroke (Table 1). However, both the rest of the angina and new class angina were significantly associated with increased postoperative stroke risk (p < 0.001). Cardiogenic shock and arrhythmias aren’t significant, but the use of inotropes is significantly higher in postoperative stroke patients —5.8% of postoperative stroke patients were using inotropes before surgery as compared to 1.6% of non-postoperative stroke patients.

The use of beta blockers, ADP inhibitors, ACE inhibitors, steroids, and Glycoprotein IIb/IIIa inhibitors, as well as resuscitation, did not significantly affect postoperative stroke risk. There is a significantly higher number of patients with elective status among non-postoperative stroke patients (79.9% for non-postoperative stroke patients versus 61.5% for postoperative stroke patients). Similarly, emergent status was more common among postoperative stroke patients (19.2% for postoperative stroke, as compared to 13.4% for non-postoperative stroke, p < o.oo1).

Among patients without postoperative stroke, 6.5% were on preoperative intra-aortic balloon pump (IABP), while 13.5% of patients with postoperative stroke were on preoperative IABP (p = 0.047). This indicates a significant difference. Before the surgery our patients had normal kidney functions. After cardiac surgery, dialysis is more commonly required in postoperative stroke patients (5.8% for postoperative stroke versus 1% for without postoperative stroke, p < 0.001). Additionally, 23.1% of postoperative stroke patients required prolonged ventilation (24 h) after surgery, compared to 9.1% of without postoperative stroke patients(p < 0.001).

Figure 1 illustrates the ROC curve, where an AUC of 0.841 shows the model’s predictive performance. Additionally, Fig. 2 shows a low positive predictive value (PPV) of 4.2% and an exceptionally high negative predictive value (NPV) of 99.7%. The ROC analysis of the STS stroke model, as presented in Fig. 3 and the supplementary table, demonstrates high sensitivity at 90.4% (95% CI: 78.8–96.2%) and moderate specificity at 64.3% (95% CI: 62.6–66.0%). The model’s overall accuracy is 64.8%. A statistically significant p-value of 0.01, along with an AUC of 0.88, underscores the model’s strong predictive performance.

Fig. 1

Receiver Operating Characteristic (ROC) curve for STS stroke prediction, showing an Area Under the Curve (AUC) of 0.841

Fig. 2

Diagnostic performance metrics at various STS stroke thresholds

Fig. 3

Sensitivity and specificity of STS stroke prediction

Discussion

This study evaluates the predictive potential of STS risk score in assessing postoperative stroke risk among patients undergoing cardiothoracic surgery at a tertiary care hospital in Pakistan. Although the STS score is a well-established and reliable tool for predicting morbidity and mortality in this patient population, our findings further validate its accuracy in stroke risk prediction [11, 13, 14]. The key findings of our study identified 1- age, 2- WBC count, 3- specific cardiac history elements, and 4- the status of surgery (elective, urgent, emergent, or salvage) as significant predictors of postoperative stroke.

Age has been considered a critical predictor of postoperative complications. Our study highlights that patients who experienced post-operative stroke were, on average, older (62.8 years) compared to non-stroke patients (57.7 years). Older patients undergoing cardiothoracic surgery are more vulnerable to postoperative stroke due to several age-related factors which include the presence of comorbidities such as hypertension and diabetes and vascular changes like arterial stiffening. Together, these factors increase the risk of cerebrovascular events during and after surgery as has been demonstrated in existing literature [15, 16].

Among the comorbidities linked to postoperative stroke, including hypertension, immunosuppressive therapy, family history of coronary artery disease, diabetes, and chronic lung disease, only notable parameters, the WBC count, emerged as a significant predictor in evaluating stroke risk. Elevated WBC counts are associated with inflammatory processes that can predispose patients to thrombotic events, a finding consistent with current literature [17, 18].

Patients with a history of cardiovascular interventions had a higher incidence of postoperative stroke, reinforcing the idea that previous cardiac interventions might indicate underlying vascular pathology that predisposes these patients to cerebrovascular events during subsequent surgeries. A few studies have recognized that the history of CV interventions is a significant predictor of ischemic stroke after cardiac surgery [19, 20].

Postoperative congestive heart failure (CHF) emerged as a significant predictor of postoperative stroke, reflecting its contribution to hemodynamic instability and potential embolic events following surgery. Literature reviews indicate that CHF is associated with an increased risk of thrombus formation, with studies reporting a 2- to 3-fold higher likelihood of stroke [21]. Additionally, unstable angina, was strongly linked to an elevated risk of postoperative stroke. This increased risk may result from factors such as heightened myocardial ischemia, the potential for atrial fibrillation, and hemodynamic instability. Moreover, the use of inotropes, often required for patients with compromised cardiac function, was identified as a significant risk factor for stroke, highlighting the necessity of careful hemodynamic management in this population.

Our findings also indicated that patients undergoing emergency surgical procedures had a greater risk of stroke-related complications compared to those undergoing elective surgeries. This underscores the importance of thorough preoperative assessment and optimization to mitigate stroke risk, especially in patients with unstable angina and those requiring urgent interventions. Furthermore, the incidence of intra-aortic balloon pump (IABP) use was significantly higher among patients who experienced postoperative strokes. The need for additional postoperative interventions, such as prolonged ventilation and dialysis, was also markedly elevated in stroke patients, suggesting that these individuals are more likely to encounter complex postoperative courses.

The STS score is one of the most effective predictive tools for assessing high-risk patients, with an area under the curve (AUC) of 0.841, indicating strong predictive power. Additionally, a p-value of 0.01 for an AUC of 0.88 further underscores the model’s clinical utility, suggesting it can reliably differentiate between patients at risk for stroke and those not at risk. A study conducted in Brazil found that the STS score outperformed Euro SCORE II in predicting mortality (0.90 vs. 0.76 and 0.77) and any morbidity, including stroke (0.80 vs. 0.65 and 0.64), with a significance of p < 0.001.

Limitations

Although our study offers valuable insights into stroke rates and influencing factors among cardiothoracic surgery patients, it has some limitations. Primarily, it was conducted at a single center, which may restrict the generalizability of the findings to the broader population of cardiothoracic surgery patients in Pakistan. Future studies should involve multiple centers across diverse regions of Pakistan.

Moreover, the retrospective design of the study limits our ability to establish causality between the identified factors and mortality rates. Prospective, multi-center research incorporating a wider array of variables could offer a more robust understanding of stroke risk factors in cardiothoracic surgery patients. This approach would strengthen our findings and enhance the risk assessment framework.

Conclusion

Our study provides a comprehensive evaluation of the STS stroke risk score’s effectiveness in predicting postoperative stroke morbidity among cardiothoracic surgery patients at a tertiary care hospital in Pakistan. Various populations have assessed the validity of the STS score against other scoring systems to determine the most suitable tool for their specific contexts. In low- and middle-income countries (LMICs) like ours, the STS score has proven to be an accurate measure of mortality risk. In this study, we specifically utilized the STS score to predict stroke morbidity in our population. This study represents a novel contribution to the literature by validating the use of the STS score for predicting stroke morbidity in Pakistani population.