Introduction

The COVID-19 pandemic has tested the response capacity of health systems, with more than 529 million confirmed cases and more than six million deaths as of early June 2022, but the lessons learned offer us the opportunity for improvement when facing future threats1,2. One of the main reasons for excess mortality is an overload of critically ill patients in the health system due to lack of space, technical resources and qualified personnel in intensive care units (ICUs)3,4. The distribution of excess mortality throughout the pandemic was heterogeneous depending on the country, suggesting that each individual health system adapted differently5. The areas with fewer ICU resources located far from the main urban centers are more vulnerable6. The guidelines for dealing with the response to a mass critical care surge have focused on balancing the overload of the health system by creating “regional transfer hubs” to avoid altering regular patient care, but there is a lack of specific triage strategy at this level 7,8,9. It has been described experiences of a regional transfer coordination in United States during COVID-19 with no triage process10.

In Catalonia, a European region of 7.5 million inhabitants, a regional command center was created for the pandemic surge. This structure was housed within an emergency dispatch center and managed the inter-hospital transfer of 2,697 severe COVID-19 patients during the first four waves of the pandemic, a figure that corresponds to one third of all ICU admissions due to COVID-19. The coordination center facilitated the distribution of patient flows, allowed for the monitoring of available resources and improved communication between the different healthcare system participants11. To help determine the severity and priority of critical respiratory patients, our research group developed during the first pandemic wave and validated during the second wave the TIHCOVID score based on a multivariate mortality predictive model12. This was the first description of a triage tool used at the regional level to facilitate the flow of patients in mass critical care surges. Its use improved transfer management times, especially for the most seriously ill cases13.

The creation of a prehospital research network in our environment has allowed the TIHCOVID score to be widely used in other regions giving the opportunity for improvement14. The aims of this study were to fine-tune and validate a predictive model of mortality in COVID-19 patients with severe respiratory involvement who were candidates for inter-hospital transfer and to create a regional mass triage (MATER) score.

Results

During the study period, 8,724 patients were transferred in the studied regions. A total of 1,018 transferred severe COVID-19 patients were included in the study, as shown in Fig. 1.

Fig. 1
figure 1

Flow chart.

Full size image

Clinical-epidemiological characteristics of the patients

The patients had a mean age of 62.3 years (SD 12) and 65.1% were male. 60.2% of these patients required invasive mechanical ventilation prior to transfer, while the rest required some type of oxygenation support. 24.6% of patients had a history of respiratory disease and 17.9% had comorbidities as shown in Table 1. No adverse events were recorded during the transfer. 89.6% were admitted to an ICU. The in-hospital mortality rate was 23.0%.

Table 1 Clinical characteristics of patients transferred and comparison of groups in relation to survival.
Full size table

Calibration and discrimination of the predictive model

The final predictive model shown in Table 2 included six variables. The model had an area under the curve (AUC) of 0.79 (95%CI 0.77–0.81), a sensitivity of 0.74, a specificity of 0.75, a positive predictive value of 0.46 and a negative predictive value of 0.91. Regarding the calibration of the model, a Hosmer and Lemeshow test was performed, obtaining a p = 0.86 and a Brier score of 0.135. Figure 2 shows the discrimination and calibration of the predictive model.

Table 2 Final predictive model of the variables associated with mortality.
Full size table
Fig. 2
figure 2

Discrimination and calibration of the predictive model. (a) Discrimination plot. (b) Calibration plot. (a) Shows area under the receiver operating characteristic curve (AUC) and his 95% confidence interval in blue. (b) Shows the calibration between the predicted and the observed risk of death. The bars at the bottom show the distribution of the number of cases.

Full size image

Regarding the risk classification established by the MATER score, 316 patients (34.1%) were classified as low risk, 337 (36.3%) as intermediate risk and 274 (29.6%) as high risk. The observed mortality rate for each group was 7.3%, 16.0% and 44.5%, respectively. Of the patients who died, 73.8% were classified as high risk by the MATER score. Figure 3 shows the details of each variable of the MATER score and the analogue nomogram.

Fig. 3
figure 3

The score and nomogram of the MATER score. (a) Score of each variable. (b) Analogue nomogram.

Full size image

Discussion

The MATER Score showed good discrimination and calibration for predicting mortality in this COVID-19 series of critically ill transferred patients. The fine tuning of the model has reduced the number of the score variables needed respect the original score, has allowed the PaO2/FiO2 ratio to be used as a continuous variable and has improved its applicability and discrimination power (12). It is a single score composed by six clinical variables widely used and easily available with a high negative predictive value that facilitate the detection of the most critically ill patients, rapid decision making and information transfer in situations requiring agile triage as in other scales applied in mass casualty situations15.

The coordination centers manage the demand for transfers when a hospital is unable to admit the critical patient, mainly because it does not have an ICU or does not have sufficient capacity in its ICU. This score therefore allows the risk assessment of patients who, due to the severity and complexity of their condition, are candidates for ICU admission. The participation of a critical care specialist at the regional coordination center facilitates decision-making regarding the appropriateness of the therapeutic effort, especially for those hospitals that do not have an ICU service. Only 5.4% of all the requests for transfers were considered inappropriate16.

It is important to underline that the MATER score is the first triage tool helping prioritize transfers from a regional coordination center during a mass critical surge. As an alternative the SOFA score has been proposed for previous mass triage plans8,17. SOFA score is commonly used in mass critical care surges at the hospital level but offer lower discrimination when predicting mortality than MATER score18,19. The SOFA score was not designed to predict mortality but rather organ dysfunction in the context of sepsis20. A prospective study analysed the effectiveness of various risk scores for critically ill patients requiring inter-hospital transfer. They found that APACHE II and SAPS II were more useful than RAPS or REMS21. APACHE II or SAPS II are applicable to all types of critically ill patients and combine 12 and 17 variables respectively which makes them difficult to use as triage during a mass critical surge22,23. An observational study conducted in a tertiary hospital prior to ICU admission compared the use of several scores in COVID-19 patients. The APACHE II performed better in predicting mortality than SAPS II, 4 C Mortality Score and SOFA24. The AUC of the APACHE II for predicting mortality in COVID-19 patients ranges from 0.62 to 0.77 which is similar or even lower than the MATER score24,25,26.

In situations in which resources are scarce, most recommendations share the ethical principles of maximising benefits by saving in the first place as many lives and as many years of life as possible3. There is an active debate regarding how triage scores based on prognosis can be less favourable for socially marginalised population19,27,28. With regard to this issue, firstly, our score does not specify exclusion criteria and its initial objective is to prioritize the most critically ill patients. Secondly, its application is directed by a regional coordination center that aims to improve the situational awareness of health authorities in order to increase the necessary resources and redirect critical patient flows to hospitals with greater resource flexibility11. Therefore, load-balancing in this way may be the best way to address socio-economic inequalities and minimise possible injustices due to the variations in triage policies between hospitals17.

Regarding the variables in our predictive model, the PaO2/FiO2 ratio has the highest weight in the MATER score, followed by age, which is in contrast to our previous score, probably due to a change in patient profiles between the first two waves and the third and fourth waves12. A multivariable model for predicting the in-hospital mortality rate of COVID-19 patients found a similar OR for the PaO2/FiO2 ratio29. Another multicenter study that included 63% of mechanically ventilated patients also employed the PaO2/FiO2 ratio on admission in its mortality predictive model30. In contrast, the PaO2/FiO2 ratio at ICU admission was not an independent factor associated with prognosis in a series including only mechanically ventilated patients31. In our sample there is a certain heterogeneity in this regard which would justify our results, since there are patients who did not undergo invasive mechanical ventilation. Age and comorbidities are the next most important variables in the score calculation and these are variables that provide a strong indication of a poor outcome in COVID-19 patients32,33,34. With respect to epidemiological variables such as sex or age or the background prevalence of hypertension, diabetes and smoking, our series does not differ from other COVID-19 ICU multicenter series in our setting35. The mortality rate in our series is similar to that of other series of patients admitted to the ICU, supporting the safety of transfers36.

This study has important clinical implications. The MATER score would be applicable to pandemic crisis situations from a regional coordination center in which there are more patients with severe respiratory disease than the health care system can cope with. In our environment, the regions most affected by COVID-19 are the most densely populated ones, suggesting that this tool is exportable to other countries, especially those with a similar distribution of ICU resources 11,37. The consultations received by emergency coordination centers have proven useful for predicting ICU admissions in advance38. Further studies are needed to provide more evidence regarding the transfers safety, and to design new tools to predict health system overloads39.

The main limitation of this study is that this predictive model was developed during the COVID-19 pandemic. It would be advisable to adjust the model to other epidemic critical care surges in the future. In a small number of patients, the score was not used prospectively in the case assessment and was calculated a posteriori. Since there are differences between the coordinating centers involved due to the internal organisation of each region, external validation in other countries would be desirable. The presence of selection bias due to the lack of randomization could be considered; however, in an emergency scenario involving a surge of critically ill patients, randomization is impeded for ethical reasons.

Finally, the MATER score successfully predicts mortality in severe COVID-19 patients requiring inter-hospital transfer and can support decision making for triage and prioritization at regional level in situations marked by a mismatch between demand and intensive care unit resources available in the healthcare system. Although future studies are needed to confirm the findings this is the first triage tool created for this type of scenario and will be useful for this and future pandemics.

Methods

Design and context

A multicenter prospective observational study, carried out in the prehospital emergency services of eight autonomous communities in Spain. This study is published in accordance with the TRIPOD guidelines40. The study protocol was approved on April 2020 by the Ethics and Clinical Research Committee of the Pere Virgili Health Research Institute (IISPV) (Reference number: 107/2020) who gave a waiver of informed consent. This study was conducted in accordance with the Helsinki Declaration of 1975.

The public emergency medical services (EMS) of the autonomous communities of Andalusia, Asturias, the Balearic Islands, the Basque Country, Catalonia, La Rioja, Madrid and Navarra participated in this study. These EMS are part of the public health system, offering a 100% coverage of their regions, with a total population of 28,367,251 inhabitants41. The EMS participating in the study manage and coordinate requests for inter-hospital transfers at their respective dispatch centers. Each autonomous community manages its own health system so there are a number of organizational differences, as specified in the Online Data Supplement Table S1. The management of all of the inter-hospital transfers is carried out by a team of physicians, nurses and health technicians who receive the request from an issuing hospital via a dedicated telephone line. After evaluating the clinical information provided and consulting the digital medical record, this team is responsible for providing clinical advice if necessary, prioritizing transfers, assigning the most appropriate resources for the transfer and selecting the destination hospital. The team is led by a physician specialized in the management of critically ill patients. During the pandemic, all of the dispatch centers required the inclusion of new professionals to respond to the increase in demand. The transfers of the severe COVID-19 patients were carried out by an advanced life support ambulance staffed by a physician, a nurse and a healthcare technician.

Patients and the collection of variables

Consecutive hospitalized patients with severe COVID-19 transferred to other hospitals and managed by the coordination center were included. Patients with a COVID-19 positive test and transferred due to a non-respiratory disease were excluded. The criteria for severe COVID-19 cases were respiratory failure requiring high-concentration oxygen therapy or ventilation support (invasive or non-invasive), a situation of sustained shock, or target organ failure. The study period was from 1 January to 30 April 2021 (third and fourth pandemic waves).

The following variables were collected. Two demographic variables: age and sex; five variables referring to personal history: hypertension, diabetes, obesity (body mass index > 30), smoking status, chronic respiratory pathology (chronic obstructive pulmonary disease (COPD) GOLD A-B, asthma or obstructive sleep apnoea syndrome); the presence of severe comorbidity (COPD GOLD C-D, pulmonary fibrosis, stroke with residual neurological deficit, chronic heart failure functional class III-IV, neurodegenerative diseases, active cancer and Child-Pugh B-C liver cirrhosis); baseline situation by the Clinical Frailty Scale (CFS); type of oxygen therapy and ventilation support received in the sending hospital; clinical variables at the time of the transfer request (PAO2/FIO2 ratio, need for pronation, acidosis/shock or lactate > 3 mmol/L and acute renal failure); and finally one structural variable: presence of ICU in the sending hospital. These variables were collected at the time of the initial assessment by the coordinating center and the probability of death of each patient based on original TIHCOVID score was used for prioritization. The primary outcome variable was all-cause mortality during hospital stay.

Model development and validation

Initially, the original TIHCOVID score consisted of seven clinical variables: age, comorbidity, need for pre-transfer pronation, acute renal failure, use of high flow therapy prior to invasive mechanical ventilation, active smoking and PaO2/FiO2 < 5012. The validation of this original score in the new multicenter series did not provide good enough discrimination with an AUC = 0.69 (95% CI 0.63–0.76), as is shown in the Online Data Supplement Figure S1. High-flow therapy was not available in the entire territory studied, so this variable of the original model was not applicable in the entire multicenter series. In order to improve its applicability and the discrimination power, the model was fine-tuned through a rederivation process. Rederivation of a predictive model can improve its performance with respect to recalibration42. Priority was given to constructing a robust model by minimising missing data and being parsimonious with as few variables as possible43. Given the sample size, rederivation using data of the multicentre series was chosen to improve the predictive model rather than other strategies44,45.

First, to fine-tune the logistic regression model, the variables associated with mortality were determined using the “Lasso” (least absolute shrinkage and selection operator) technique, a regression analysis method that performs variable selection and regularisation to improve the accuracy and interpretability of the resulting model. Automatically selected variables were included in the model. The performance of the model was assessed by Leave-one-out cross-validation (LOOCV). LOOCV is an iterative method that starts by using all available observations except one as a training set, which is then excluded for validation purposes. By using all available data for both training and validation, this method reduces the variability caused by randomly dividing the observations into two groups. Subsequently, to reflect the weight of each variable, a logistic regression model was performed and the odds ratio (OR) with its 95% confidence interval (CI) was calculated.

The area under the curve (AUC) was used to assess the model’s ability to distinguish between patients who died in hospital and those who were discharged. AUC scores range from 0.5 (no discriminatory ability) to 1.0 (perfect discriminatory ability). Calibration measures the agreement between predicted and observed risk and was assessed using the Brier score, which ranges from 0.0 (perfect calibration) to 1.0 (poor calibration).

Three risk groups were defined, low risk (up to 100 points), intermediate risk (between 101 and 124 points) and high risk (from 125 points) based on categorisation by terciles. Finally, a nomogram was constructed to facilitate the use of the MATER score. This nomogram represents the scores obtained for the different predictor variables of the model, which allows the predicted probability of death to be calculated and the risk group to which the patient belongs to be obtained. The MATER score is also available online (https://semscores.shinyapps.io/materscore/).

Statistical analysis

Qualitative variables were described in terms of number of cases and percentages. Quantitative variables were described as mean and standard deviation (SD), if they followed a normal distribution, or as median and interquartile range (IQR) otherwise. Comparison of groups according to mortality was performed for qualitative variables with the chi-square test or Fisher’s exact test, and for quantitative variables with Student’s t-test or the Mann-Whitney U-test. The difference was considered significant if the bilateral p-value was less than 0.05 or if the 95% CI of the OR excluded the value 1. The statistical analysis was performed using SPSS version 24.0 for Windows (SPSS Inc, Chicago, USA) and R version 4.1.2.