BACKGROUND

By 2030 around 1 in 4 people in Europe will be aged over 65 [1], an age group currently accounting for over two thirds of inpatient hospital admissions in the UK [2]. Around half of older hospital inpatients have frailty, a condition characterised by loss of biological reserves across multiple organ systems and vulnerability to physiological decompensation after a stressor event [3]. It is an established prognostic factor in a range of conditions and outcomes, including mortality, nursing home admission, length of hospital stay, delirium and falls [4,5]. Around 66.5% of inpatients over 65 have been estimated to be vulnerable and 14% as having severe frailty [6]. Other studies have found the prevalence of frailty to be as high as 81% in hospitalised patients aged over 85 [7].

Although many frailty assessment instruments are available, relatively few have been validated in the acute inpatient setting because acute illness can conflate frailty when a performance-based tool, such as gait speed, is used. For this reason, UK guidelines on frailty assessment have recommended avoiding performance-based frailty measures in the context of acute illness [8]. One instrument that has been validated in the inpatient setting is the Clinical Frailty Scale (CFS) (Figure 1), which is a simple face to face clinician assessment of frailty status [9]. The CFS can be rapidly scored within a clinical environment and has demonstrated high inter-rater reliability in a range of hospital settings, including the emergency department, intensive care and geriatric acute care [9–12]. The CFS has been found to predict clinical outcomes including mortality and dependence on health care in a range of settings [13–17]. Some studies have found that the CFS can be reliably assigned prospectively using a standardised algorithm and by a telephone assessment [18,19].

Routinely collected information contained with inpatient hospital records contain significant information regarding functional assessments, residential status, activities of daily living and other information which could potentially be used to derive an individual’s score on the clinical frailty scale. Validating estimates of CFS from information in hospital records would enable analysts to use retrospective measures of the prognostic effect of frailty for research and large national retrospective service improvement audits of hospital records such as patient outcome enquiries [20] and clinical coding for renumeration.

FIGURE 1

Figure 1. The Clinical Frailty Scale (Reproduced with permission from [21], copyright ©2007–2009 Geriatric Medicine Research, Dalhousie University).

A recent Canadian study found good inter-rater reliability (IRR) between CFS derived from face to face clinical assessment and score derived from case note assessment (k = 0.64) in a tertiary care outreach service in older patients [22]. However, this study was conducted in an outpatient setting and therefore cannot be applied to an inpatient context. An Australian study in an intensive care setting found no clinically important difference between CFS scores based on bedside chart review, family interview or patient interview, and no significant differences in CFS scores based on chart reviews between different assessors (researcher, occupational therapist, geriatrics resident) [23]. Information contained in intensive care records may differ from that of the general hospital inpatient population.

However, no studies have assessed the validity of CFS scores derived from hospital case notes in general medical inpatient settings. Additionally, no studies have assessed the validity of CFS scores derived from hospital case notes in the NHS, and the two previous studies may not be generalisable to a routine UK inpatient setting.

The aim of this study was to determine whether CFS scores derived from the retrospective review of UK inpatient hospital record have good agreement with scores from face to face assessment.

The objectives were:

METHODS

A prospective cross-sectional study was conducted on the acute older people’s medical admissions unit at Bradford Royal Infirmary between March and August 2019 to test agreement between a CFS score derived from face to face assessment and a score derived solely from information contained within inpatient electronic health records (EHRs).

Bradford Royal Infirmary is a large teaching hospital in an urban setting in West Yorkshire, England.

All patients aged 65 or over with capacity to consent, or consultee available for assent admitted as inpatients with available EHR were eligible for inclusion within 72 h of admission to elderly inpatient medical wards.

Patients admitted for palliation, too acutely unwell to participate (including patients who were unconscious) or who were unable to speak English where translation was not possible were ineligible.

CFS scores were assigned by core medical and higher specialist medical trainees in geriatric medicine (the assessors) who had been trained in scoring the CFS. Assessors were asked to estimate the frailty of eligible patients two weeks prior to their acute admission or equivalent to their baseline function when not affected by acute illness. One assessor completed a face to face assessment, collected demographic information within 72 h of admission, and assigned a CFS score for each participant. The face to face assessment was performed independently to routine clinical care as part of a general patient assessment. The assessment took no more than 5 min.

Two different assessors who were blinded to the original CFS score then separately assigned a CFS score for each patient solely using information recorded in the EHR. Information in the EHR is collected in a standardised way for all acute admissions using a proforma. This contains information collected by medical staff, nursing staff and other allied health professionals as part of a comprehensive geriatric assessment (CGA) process. This includes relevant detail on medical, functional, social and psychological problems, and includes a cognitive assessment (4-item abbreviated mental test (AMT-4) score) and delirium screening using the single question in delirium (SQiD) test. Data were collected by assessors using a standardised data collection tool and assessors had access to all completed health care records. Information was primarily extracted from the initial inpatient clerking with a focus on patients’ social circumstances and pre-hospital functional status.

We estimated that a sample size of 40 participants was required, on the basis of detecting an inter-rater reliability (IRR) using a quadratically weighted Cohen’s Kappa of at least 0.6 and the use of a 9-point scale (with a power of 90%, significance 0.05, k₁ = 0.3 and k₂ = 0.6) [24]. A value of 0.6 was chosen as this demonstrates good strength of agreement and the previous Canadian study reported an IRR of 0.64 between face to face and case note measurement of the CFS [22].

A quadratically weighted Cohen’s kappa was used to estimate agreement between assessors (IRR) [25,26]. Kappa represents the proportion of agreement greater than that expected by chance. The quadratic weighting places a greater emphasis on larger discrepancies between assigned CFS scores, which are penalised more than smaller differences.

The IRR between the CFS scores assigned using inpatient records was estimated first to check the reproducibility of CFS scoring using this source of information. The IRR between the CFS score assigned from face to face assessment and the first assessor completing CFS scoring using information solely from the hospital records was then estimated.

Analysis was conducted on the full 9 point CFS (Figure 1) and also on the basis of 4 categories derived from the 9 point CFS: 1–3 (not frail), 4–5 (vulnerable-mildly frail), 6–8 (moderately to severely frail) and 9 (terminally ill) which has been used in some recent studies [27].

A subgroup analysis on agreement (IRR) between the face to face assessment and assessment using hospital records alone, split by the number of admissions in the last 12 months. This was to assess whether increased availability of routinely collected information due to frequent admissions improved the accuracy of retrospectively assigning a CFS score.

All analyses were undertaken on Stata/SE 14.

Informed consent was gained from all participants and NHS Research Ethics Committee Approval was granted by Yorkshire & The Humber-Bradford Leeds Research Ethics Committee reference: 18/YH/0374 (the date of approval: 30 October 2018).

RESULTS

40 patients were recruited: 21 (53%) were female, 36 (90%) were white, the majority lived in their own home (37 (93%)) and 2 (5%) had a known diagnosis of dementia (see Table 1).

TABLE 1

Table 1. Patient Characteristics.

Table 2 shows that there was good agreement on the 9 point CFS (Figure 1) between the two assessors who assigned frailty scale using scores hospital records (k = 0.82). There was also good agreement between the frailty scale scores derived from face to face assessment and those using inpatient records on the same patients (k = 0.84).

TABLE 2

Table 2. IRR on 9 point and on collapsed 4-category Clinical Frailty Scale.

There was slightly lower but still good to moderate agreement when using the collapsed scale.

A subgroup analysis of patients with only one admission (Table 3) shows that there was good agreement between the 2 assessors assigning a frailty scale score using inpatient records and between those assigning a frailty scale score on the full scale from a face to face assessment and the first clinician assigning a frailty scale score from case note review. The same was true for patients with >1 admission (n = 18, Table 3).

TABLE 3

Table 3. IRR 9 point Clinical Frailty Scale by number of admissions.

DISCUSSION

This study demonstrates that the CFS may be accurately determined retrospectively from information routinely collected in routine inpatient notes. High levels of agreement were observed between CFS scores derived from face to face assessment compared to clinical notes review as well as high IRR between independent reviews of the clinical notes. We believe we have conducted the first study to do this internationally. This agreement was unaffected by the number of recent hospital admissions. Therefore, the information in the case notes from a single inpatient admission appears sufficient to reliably assign a frailty scale score. Collapsing the frailty scale reduced agreement between assessors assigning scores based on inpatient records and between ratings derived from inpatient records and face to face assessment.

We are aware of only two other studies which have assessed whether a CFS score can be reliably assigned from information in inpatient hospital records and these were both conducted in different settings [22,23]. One study investigated retrospective CFS assignment in a Canadian geriatric community outreach service, not an inpatient setting, and reported a Kappa of 0.64, indicating moderate to good agreement between these methods [22]. Darvall et al. conducted a similar study within an ICU setting and found a similar level of agreement (k = 0.67) [23]. Our study found a higher level of agreement between frailty scale scores assigned through face to face assessment and case note review than previously reported (k = 0.84).

As has been previously reported, collapsing of the CFS, reduced the IRR [27]. This may mean that boundaries between the categories in the collapsed scale may not adequately distinguish between distinct frailty category severities. This was particularly a problem in distinguishing between categories 4–5 (vulnerable-mildly frail) and 6–8 (moderately to severely frail). We would therefore discourage collapsing of the frailty scale.

We have used an adequate sample size based upon the use of appropriate statistical techniques to assess agreement between assessors assigning frailty scale scores. Blinding of clinicians to previous frailty scores assigned to patients was maintained throughout. This is the first study to also assess whether the number recent admissions affected the reliability of frailty scale scores assigned using information in inpatient records. The study was conducted in a general inpatient medical care for the elderly ward and therefore may be applicable to multiple inpatient settings.

Data were collected at a single site therefore the amount of information contained in inpatient records regarding the functional assessment of patients may not be representative of all hospitals in the UK, or internationally. All patients were admitted under a geriatric medicine inpatient team and the amount of information available to assign retrospectively a CFS score from inpatient records may vary if patients are admitted under different inpatient specialties. The assessors were geriatric medicine specialists and therefore scoring may be less accurate if performed by health care professionals with different medical training backgrounds. Although the CFS is intended to represent a baseline level of frailty measured in the community, it is possible that [9] any measure of frailty in an inpatient hospital assessment may be influenced by the effects of acute illness. Assessors were asked to estimate the level of frailty patients had 2 weeks prior to their acute admission (when they were assessed) which is an accepted convention for assigning frailty scale scores in an acute setting [27].

Ninety percent of frailty scale scores assigned by the face to face assessment ranged between 3 and 6. Therefore, agreement between the face to face assessment and values assigned from information in the inpatient case records outside this range was not as robustly assessed and may not be generalisable to patients at the extremes of the scale. The prevalence of dementia in those aged over 65 in an inpatient setting has been found to range between 13% and 63% [28]. The low prevalence in our population probably reflects barriers in obtaining consultee assent and the results may not be as applicable to cohort with a higher prevalence of dementia. Data were extracted from an electronic inpatient record system and the information in paper based records may differ, however in the UK NHS hospital records contain similar routinely collected data, irrespective of whether electronic or paper based systems are used.

Although retrospective assignment of a CFS score from the information available in inpatient records may be of use for service improvement projects and research using retrospective inpatient data, this will be less directly useful for improving current inpatient care.

This study provides robust evidence on the validity of retrospective assessment of frailty from clinical records using the CFS. This means that frailty scores retrospectively assigned from such records may be used for the purposes of research and service evaluation. In settings where information routinely recorded is not as extensive or the population demographics are not comparable to this study, the IRR may not be as high. Further studies may be required in different care setting in the UK to confirm that a frailty score can be universally accurately scored using information from inpatient records.

CONCLUSION

Frailty status can be accurately assessed by retrospective review of the EHR of older acute medical inpatients, using the CFS. This method of retrospectively assigning a CFS score can be applied to a range of retrospective service improvement audits and research aimed at improving care for older patients.

DATA AVAILABILITY

The dataset of the study is available from the authors upon reasonable request.

AUTHOR CONTRIBUTIONS

The idea for the study was conceived by CM, TS and AC. VT drafted the study protocol and supported the initial stages of planning the study. Data collection was completed by JB, AW and HC. VA provided specialist statistical advice and completed the statistical analysis. All authors read and approved the final manuscript.

CONFLICTS OF INTEREST

No competing financial interests exist. Matthew Hale is supported by a National Institute for Health Research Academic Clinical Fellowship.

Carl Marincowitz is funded by a National Institute for Health Research Doctoral Fellowship (DRF-2016-09-086). This study presents independent research funded by the National Institute for Health Research (NIHR). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.