INTRODUCTION
The Short Physical Performance Battery (SPPB) is an established tool used to assess lower extremity function.1) Low SPPB scores, indicative of poor physical performance, are associated with adverse outcomes such as increased fall risk,2) functional impairment,3,4) cognitive impairment,5) hospital readmission,6) and all-cause mortality.7) The SPPB also showed good diagnostic accuracy for detecting frailty in community-dwelling older persons.8,9) The SPPB, which is one of the recommended tests for physical performance, forms an integral part of the diagnostic algorithm for sarcopenia in both the European Working Group on Sarcopenia in Older People 2 (EWGSOP2)10) and Asian Working Group for Sarcopenia (AWGS) 2019 consensus definitions.11) In a recent study, the high sensitivity of the SPPB when using a cutoff score of ≤8 suggests that it may be a favorable screening tool for sarcopenia in clinical settings where muscle mass measurements are not available.12) Additionally, the SPPB has been suggested as a relevant tool to measure outcomes for interventions targeting frailty and sarcopenia in older adults.4,7,10,13)
Generally, the SPPB is administered manually by trained personnel in accordance with the standardized protocol. The tool shows good inter-rater and test-retest reliability when conducted by trained experienced personnel.14,15) However, training can be resource-intensive, and limitations in manpower may limit the scalability and widespread application of the SPPB in clinical settings.7,16)
To address these potential barriers to clinical application, a multi-sensor-based kiosk using modern sensor technologies and computer algorithms to use the SPPB in an automated fashion (eSPPB) was first developed in Korea.16) A pilot study of a small sample comprising older adults attending a rehabilitation clinic demonstrated the feasibility of the eSPPB in a clinical setting.16) The eSPPB also showed a good correlation with the manual SPPB (mSPPB) performed by a physical therapist.16) However, to date, no prospective study has examined the reliability and agreement of the eSPPB in frailer older adults with poorer physical performance and increased risk of falls. Additionally, previous eSPPB studies conducted in geriatric clinical settings were of retrospective nature.17,18)
A growing body of literature supports the early detection and intervention of sarcopenia and frailty in at-risk older adults.19) The SPPB is a useful screening and assessment tool as well as a relevant outcome measure for interventions that target sarcopenia and frailty. Therefore, it is important to find ways to adapt the SPPB for widespread clinical application in frailer older adult populations. Thus, we aimed to examine the construct validity, reliability, and agreement of the eSPPB in relation to those of mSPPB among patients attending a tertiary fall clinic. The results of our exploratory study in a real-world clinical setting of frailer older adults with an increased risk of falls will shed light on the future scalability of the eSPPB.
MATERIALS AND METHODS
Study Setting
This study was conducted in community-dwelling older adults recruited from the Falls and Balance Clinic at Tan Tock Seng Hospital, a tertiary hospital in Singapore. We recruited participants from July 2020 to July 2021. The inclusion criteria included patients aged ≥65 years attending the Falls and Balance Clinic who were able to walk ≥100 m independently (with or without aid); those who scored ≥4 on the Abbreviated Mental Test (AMT), which corresponded to the cutoff for mild-moderate dementia in a local validation study20); and those who could understand instructions and adhere to the study protocol. The exclusion criteria were (1) chair or bed-bound status, (2) AMT ≤3, and (3) inability to understand simple commands or provide consent. Informed written consent was obtained from the participants or their legally appointed representative, where appropriate, in the presence of a trained research assistant. Ethical approval was obtained from the Institutional Review Board of the National Healthcare Group (No. 2020/00038).
Manually Measured SPPB (mSPPB)
A trained physiotherapist conducted the mSPPB. In the balance test, the participants were asked to maintain three positions (side-by-side, semi-tandem, and tandem stances) for 10 seconds. To measure gait speed, the participants were timed with a stopwatch (Casio Model HS-3, with a measurement accuracy of up to 1/100-seconds) as they walked 4 m from a standing start. For the chair-stand test, the participants were timed for five consecutive sit-to-stand repetitions with their arms folded across their chest and ending with a fifth sit. We employed a sitting stop, as this was the prevalent practice for the timing of the chair stand test. This also allowed for comparability with the chair sensor of the eSPPB kiosk.16) The cutoff points were based on previously published norms for SPPB scoring.1)
The eSPPB Kiosk
The eSPPB kiosk prototype was developed by Dyphi (Daejeon, Korea). We used the eSPPB setup described in the original validation study.16) In brief, the three SPPB components can be semi-automatically estimated. Balance was measured with a load cell array that could detect the location of the participant’s foot and measure the force applied to it. Gait speed was measured with a one-dimensional light detection and ranging (LiDAR) sensor that could measure the distance between the sensor and the participant.21) The chair stand test was performed five times using a combination of two sensors: a load cell-embedded chair to measure the weight of the participants and a LiDAR sensor to measure the distance between the participant and the chair. The three components were input directly into a program that allowed the conduction of SPPB in a standardized manner with graphic and voice instructions. The eSPPB kiosk algorithm was used to calculate the score for each component based on previously published cutoff points.
Data Collection
Protocol for mSPPB and eSPPB
The participants performed the mSPPB and eSPPB simultaneously to optimize the reliability evaluation and avoid the need for repeated assessments. While the physical therapist instructed the participants and manually recorded timings for individual test sections of the mSPPB, the research assistant recorded their performance using the eSPPB. The research assistant aided with the setup of the eSPPB components and operated the eSPPB software during the assessment.
Other variables
We collected data on demographics, height, weight, and body mass index. The functional ability, basic activities of daily living (ADL), and physical activity were evaluated using the modified Barthel Index (MBI),22) Lawton and Brody's instrumental ADL (iADL),23) and Frenchay Activities Index (FAI),24) respectively. Frailty status was measured using the FRAIL (fatigue, resistance, ambulation, illness, loss of weight) scale, a self-reported five-item scale that assesses the domains of fatigue, resistance, ambulation, illnesses, and loss of weight.25) Individuals who scored 1–2 were considered as pre-frail, whereas those who scored 3–5 were classified as frail. Fall efficacy and balance performance were measured using the Falls Efficacy Scale International (FES-I)26) and Berg Balance Scale (BBS),27) respectively.
Statistical Analysis
The sample size was calculated based on the evaluation of the intraclass correlation coefficient (ICC) between the mSPPB and eSPPB. Using an a priori postulated ICC of 0.80, a study power (1−β) of 0.80, and a half-width 95% confidence interval (CI) of ICC <0.15, we required 35 participants for the study.28) Based on an anticipated dropout rate of 10%, we recruited 39 participants.
Descriptive and inferential statistics were analyzed using IBM SPSS Statistics for Windows (version 27.0; IBM, Armonk, NY, USA) and MedCalc for Windows (version 20.013; MedCalc Software, Ostend, Belgium). Two-sided p < 0.05 was considered statistically significant.
First, we performed descriptive statistics to assess the demographic and clinical characteristics of the study participants. Next, we examined the construct validity of the mSPPB and the eSPPB in two ways. First, for convergent validity, we evaluated the correlations with common geriatric parameters using a partial correlation coefficient adjusted for age and sex. Second, using a cutoff of ≤8 to denote sarcopenia,12) we performed independent samples t-tests to ascertain the ability to discriminate physical function, physical activity, frailty, balance performance, fear of falling, and physical performance between the ≤8 and >8 subgroups.12) We then assessed the reliability of the mSPPB and eSPPB based on the ICCs of the total and component-specific scores. ICC values were indicated as: poor reliability (<0.5), moderate reliability (0.5–0.75), good reliability (0.75–0.9), and excellent reliability (>0.9).29) We also performed paired t-tests to compare the mean differences between the readings. Lastly, we constructed Bland-Altman plots to determine the agreements between the total and component-specific scores. Systematic bias was calculated as the mean difference between readings, and 95% limits of agreement were calculated as the bias ±2 standard deviation for the differences between readings. Proportional bias was ascertained by inspecting the regression line and Pearson correlation to quantify the degree of bias.
RESULTS
Recruitment Flowchart
Of the 88 eligible participants, 35 (39.8%) declined further participation due to fear of technology, lack of time, or the coronavirus disease 2019 (COVID-19) situation. Of the 53 recruited participants, 16 were excluded from the study as they had incomplete data owing to technical issues that resulted in their data not being properly captured by the computer or them not being recognized by the sensors. Thus, the final sample comprised 37 participants who underwent both mSPPB and eSPPB assessments. Age, anthropometry, physical function, activity, and frailty status did not differ significantly between the included (n=37) and excluded (n=16) participants (Fig. 1).
Baseline Characteristics
The participants were predominantly female (62%) with a mean age of 78.5±6.8 years. The AMT scores ranged from 6 to 10; while one participant had mild cognitive impairment, none had dementia. The mean FRAIL score was 1.2±1.0; thus, most of the participants were pre-frail (65%). The mean mSPPB score was 6.6±3.3, which is lower than the cutoff of ≤8 used to denote sarcopenia. The female participants were lighter, shorter, and had lower MBI and BBS scores. Our participants from the Falls and Balance Clinic were slightly older and appeared to be frailer than those of original validation study16) derived from a rehabilitation clinic. This is evidenced by the lower BBS, mSPPB, and eSPPB and higher FRAIL scores obtained in our study (Table 1).
Convergent Validity
Not surprisingly, we observed a strong correlation between the mSPPB and eSPPB total scores (r=0.933, p<0.01). Both mSPPB and eSPPB total scores showed strong correlations with the BBS (r=0.900 and r=0.869, respectively, p<0.01), moderate correlation with the MBI (r=0.507 and r=0.508, p<0.01) and iADL (r=0.465 and r=0.530, p<0.05), and weak to moderate correlations with the FRAIL scale (r=-0.441 and r=-0.383, p<0.05). Neither mSPPB nor eSPPB was correlated with the FAI or FES-I. The subdomains of the mSPPB and eSPPB showed similar results as the total scores, with a moderate to strong correlation with the BBS (lowest r=0.460 for eSPPB gait speed, and highest r=0.831 for eSPPB balance), moderate correlations with the MBI and iADL, and weak to moderate correlations with the FRAIL scale. The FAI was only moderately correlated with eSPPB gait speed but was not correlated with the rest of eSPPB subdomains or with mSPPB. None of the subdomains in either the mSPPB or eSPPB were correlated with the FES (Table 2).
Discriminatory Ability for Outcomes
Using a cutoff of ≤8 to denote sarcopenia, both mSPPB and eSPPB were associated with significantly higher FRAIL scores (mSPPB: 1.5±1.0 vs. 0.4±0.5, p=0.013; eSPPB: 1.5±1.0 vs. 0.6±0.6, p=0.010), lower BBS scores (mSPPB: 38.2±10.0 vs. 50.8±2.9, p=0.008; eSPPB: 36.6±9.8 vs. 50.2±2.8, p=0.005), and lower SPPB total scores (mSPPB: 5.0±2.4 vs 10.5±1.0, p=0.003; eSPPB: 4.4±2.2 vs. 10.1±1.3, p=0.001). The domain-specific scores for balance, gait speed, and chair stand were also significantly lower in the ≤8 subgroups for both the mSPPB and eSPPB (Table 3).
Total/Domain Scores and ICCs
We observed no significant differences in the mean mSPPB and eSPPB total and domain scores. Relative to the mSPPB, the eSPPB showed excellent reliability for the total score (ICC=0.94; 95% CI, 0.88–0.97) and gait speed domain (ICC=0.94; 95% CI, 0.89–0.97), and good reliability for the balance and chair stand domains (ICC 0.86–0.89). The confidence intervals were wider for the balance and chair stand domains (95% CI, 0.75–0.93 and 0.81–0.94, respectively) than those of the total score and gait speed domain. Our reliability results were similar to those of the original study, which also showed good to excellent reliability for total SPPB scores and its domains16) (Table 4).
Bland-Altman Plots
The Bland-Altman plot showed good agreement for the eSPPB and mSPPB total scores, with most values within the limits of agreement. We observed no evidence of systematic (mean difference=-0.2; 95% CI, -3.2–2.9) or proportional (r=0.102, p=0.546) biases. Similarly, the domain-specific scores showed good agreement for balance (mean difference=0.1; 95% CI, -1.3–1.5; r=-0.225, p=0.181), gait speed (mean difference=-0.2; 95% CI, -1.7–1.3; r=-0.100, p=0.555), and chair stand (mean difference=-0.1; 95% CI, -1.6–1.3; r=0.019, p=0.912), with most values within the limits of agreement (91.9%, 97.3%, and 94.6%, respectively). We observed no evidence of systematic or proportional biases in the domain-specific scores (Fig. 2).
DISCUSSION
Since the original validation study of the automated multi-sensor-based kiosk for SPPB, emerging evidence from retrospective analyses of clinical populations supports the application of the eSPPB in real-world clinical settings.16-18) To our knowledge, this exploratory study is the first to compare the validity, reliability, and agreement of the eSPPB with those of the mSPPB among predominantly pre-frail older adults attending a fall and balance clinic. Similar to the original validation study involving a less frail population, we observed no significant differences in the total and domain scores of the eSPPB and mSPPB. In addition, our study builds upon the body of evidence by corroborating the construct validity, good-to-excellent reliability, and good agreement without systematic or proportional biases between the eSPPB and mSPPB readings for total and domain scores.
As the criteria for referral to our Falls and Balance Clinic is recurrent falls and/or unsteady gait, it was unsurprising that the overall mean score for both the eSPPB and mSPPB was <8, suggesting that a significant number of our participants were already in the sarcopenia category. This was consistent with the results of studies that reported an association between sarcopenia and an increased risk of falls.29,30) In contrast, the higher SPPB scores (mean, 9.9–10.8) in the original validation study16) suggested a more robust population with a lower prevalence of sarcopenia.
Our study results supported the construct validity of the eSPPB in at-risk older adults by demonstrating its convergent validity in terms of balance performance, physical function, and physical activity. Notably, the correlation coefficient with the BBS score was higher in our study than that in the original validation study. Although the exact reason remains to be elucidated, one possibility is the closer relationship between the balance sub-domain and BBS in frailer older adults at increased risk of falls, which further affirms the convergent validity of the eSPPB in this at-risk population. Our results also corroborated the discriminatory ability of frailty, balance performance, and SPPB scores using a cutoff of ≤8 to denote sarcopenia. In support of this, participants scoring ≤8 on both the eSPPB and mSPPB were also in the pre-frail range, based on the FRAIL scale. This was consistent with previous studies that showed that SPPB score of ≤8 was a useful measure for identifying sarcopenia10) and physical frailty phenotype9) in community-dwelling older adults.
Our study also demonstrated excellent reliability for total scores and good to excellent reliability for domain scores between the eSPPB and the mSPPB. In relation to the original validation study, it was reassuring that the ICC in our study was comparable to that of the total score and higher for the balance and gait speed domain scores, albeit lower for the chair stand despite the frailer population. However, caution is needed when interpreting ICCs, as the values can be affected by samples heterogeneity, which exemplifies the concept of signal-to-noise ratio, wherein the proportion of variance is due to differences between subjects instead of the assessments performed.31) Despite the good reliability, balance and chair stand showed the lowest ICC and widest confidence intervals among the domains, suggesting the need to address technical issues related to sensors in frailer older adult populations.
In addition, we observed good agreement between the SPPB total score, with almost all data points lying within the 95% limits of agreement. The absolute difference in mean scores of 0.2 for SPPB total score in our study was lower than the minimally significant change of 0.3–0.8 points reported in the LIFE-P study.32) We also observed no evidence of systematic or proportional biases. Examination of the domain scores showed the highest number of outliers in the balance and chair stand assessments, with 8.1% and 5.4% of data points, respectively, beyond the limits of agreement. These outliers could have resulted from technical challenges in the sensing of balance and chair pads.
Our study has several limitations. Due to the cross-sectional design, we were unable to assess the test-retest reliability of the eSPPB or evaluate its predictive validity via longitudinal outcomes. As an exploratory study, our sample size was small, precluding comparisons between sexes or other subgroups. In addition, our results pertain to a predominantly pre-frail at-risk patient group attending the Falls and Balance Clinic and may not be generalizable to a wider population of frail older persons. As our study sample included no individuals with dementia, our results cannot be extrapolated to patients with dementia. We employed the sitting stop for the chair-stand test to allow comparability with the chair sensor of the SPPB. A recent study indicated that the timings for standing versus sitting stop in the chair stand test may not be comparable; therefore, our results may not be generalizable to settings where a standing stop is the prevalent practice.33) We also did not collect data pertaining to the feasibility and user acceptability of the eSPPB from the participants’ perspective. Future studies in larger populations with greater proportions of frail older adults are needed to examine the feasibility and acceptability of the eSPPB for widespread use in clinical settings.
In summary, the results of our exploratory study corroborated the construct validity, reliability, and agreement of the eSPPB with the mSPPB in a small sample of predominantly pre-frail older adults with increased fall risk. In addition, the balance and chair stand domains were associated with potential technical issues that need to be addressed to improve the reliability and agreement of the readings. This study paves the way for future studies examining the scalability and feasibility of the widespread use of eSPPB for frailty and sarcopenia detection in the clinical setting.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Full text via PMC
Download Citation
