-
The International Journal of Behavioral... Jun 2023Wearable technology is used by consumers and researchers worldwide for continuous activity monitoring in daily life. Results of high-quality laboratory-based validation... (Review)
Review
BACKGROUND
Wearable technology is used by consumers and researchers worldwide for continuous activity monitoring in daily life. Results of high-quality laboratory-based validation studies enable us to make a guided decision on which study to rely on and which device to use. However, reviews in adults that focus on the quality of existing laboratory studies are missing.
METHODS
We conducted a systematic review of wearable validation studies with adults. Eligibility criteria were: (i) study under laboratory conditions with humans (age ≥ 18 years); (ii) validated device outcome must belong to one dimension of the 24-hour physical behavior construct (i.e., intensity, posture/activity type, and biological state); (iii) study protocol must include a criterion measure; (iv) study had to be published in a peer-reviewed English language journal. Studies were identified via a systematic search in five electronic databases as well as back- and forward citation searches. The risk of bias was assessed based on the QUADAS-2 tool with eight signaling questions.
RESULTS
Out of 13,285 unique search results, 545 published articles between 1994 and 2022 were included. Most studies (73.8% (N = 420)) validated an intensity measure outcome such as energy expenditure; only 14% (N = 80) and 12.2% (N = 70) of studies validated biological state or posture/activity type outcomes, respectively. Most protocols validated wearables in healthy adults between 18 and 65 years. Most wearables were only validated once. Further, we identified six wearables (i.e., ActiGraph GT3X+, ActiGraph GT9X, Apple Watch 2, Axivity AX3, Fitbit Charge 2, Fitbit, and GENEActiv) that had been used to validate outcomes from all three dimensions, but none of them were consistently ranked with moderate to high validity. Risk of bias assessment resulted in 4.4% (N = 24) of all studies being classified as "low risk", while 16.5% (N = 90) were classified as "some concerns" and 79.1% (N = 431) as "high risk".
CONCLUSION
Laboratory validation studies of wearables assessing physical behaviour in adults are characterized by low methodological quality, large variability in design, and a focus on intensity. Future research should more strongly aim at all components of the 24-hour physical behaviour construct, and strive for standardized protocols embedded in a validation framework.
Topics: Humans; Adult; Adolescent; Wearable Electronic Devices; Fitness Trackers; Monitoring, Physiologic; Posture; Sedentary Behavior
PubMed: 37291598
DOI: 10.1186/s12966-023-01473-7 -
Aging and Disease Jul 2022Osteoporotic fractures (OF) are a global public health problem currently. Many risk prediction models for OF have been developed, but their performance and... (Review)
Review
Osteoporotic fractures (OF) are a global public health problem currently. Many risk prediction models for OF have been developed, but their performance and methodological quality are unclear. We conducted this systematic review to summarize and critically appraise the OF risk prediction models. Three databases were searched until April 2021. Studies developing or validating multivariable models for OF risk prediction were considered eligible. Used the prediction model risk of bias assessment tool to appraise the risk of bias and applicability of included models. All results were narratively summarized and described. A total of 68 studies describing 70 newly developed prediction models and 138 external validations were included. Most models were explicitly developed (n=31, 44%) and validated (n=76, 55%) only for female. Only 22 developed models (31%) were externally validated. The most validated tool was Fracture Risk Assessment Tool. Overall, only a few models showed outstanding (n=3, 1%) or excellent (n=32, 15%) prediction discrimination. Calibration of developed models (n=25, 36%) or external validation models (n=33, 24%) were rarely assessed. No model was rated as low risk of bias, mostly because of an insufficient number of cases and inappropriate assessment of calibration. There are a certain number of OF risk prediction models. However, few models have been thoroughly internally validated or externally validated (with calibration being unassessed for most of the models), and all models showed methodological shortcomings. Instead of developing completely new models, future research is suggested to validate, improve, and analyze the impact of existing models.
PubMed: 35855348
DOI: 10.14336/AD.2021.1206 -
BMJ (Clinical Research Ed.) Oct 2021To assess the methodological quality of studies on prediction models developed using machine learning techniques across all medical specialties.
OBJECTIVE
To assess the methodological quality of studies on prediction models developed using machine learning techniques across all medical specialties.
DESIGN
Systematic review.
DATA SOURCES
PubMed from 1 January 2018 to 31 December 2019.
ELIGIBILITY CRITERIA
Articles reporting on the development, with or without external validation, of a multivariable prediction model (diagnostic or prognostic) developed using supervised machine learning for individualised predictions. No restrictions applied for study design, data source, or predicted patient related health outcomes.
REVIEW METHODS
Methodological quality of the studies was determined and risk of bias evaluated using the prediction risk of bias assessment tool (PROBAST). This tool contains 21 signalling questions tailored to identify potential biases in four domains. Risk of bias was measured for each domain (participants, predictors, outcome, and analysis) and each study (overall).
RESULTS
152 studies were included: 58 (38%) included a diagnostic prediction model and 94 (62%) a prognostic prediction model. PROBAST was applied to 152 developed models and 19 external validations. Of these 171 analyses, 148 (87%, 95% confidence interval 81% to 91%) were rated at high risk of bias. The analysis domain was most frequently rated at high risk of bias. Of the 152 models, 85 (56%, 48% to 64%) were developed with an inadequate number of events per candidate predictor, 62 handled missing data inadequately (41%, 33% to 49%), and 59 assessed overfitting improperly (39%, 31% to 47%). Most models used appropriate data sources to develop (73%, 66% to 79%) and externally validate the machine learning based prediction models (74%, 51% to 88%). Information about blinding of outcome and blinding of predictors was, however, absent in 60 (40%, 32% to 47%) and 79 (52%, 44% to 60%) of the developed models, respectively.
CONCLUSION
Most studies on machine learning based prediction models show poor methodological quality and are at high risk of bias. Factors contributing to risk of bias include small study size, poor handling of missing data, and failure to deal with overfitting. Efforts to improve the design, conduct, reporting, and validation of such studies are necessary to boost the application of machine learning based prediction models in clinical practice.
SYSTEMATIC REVIEW REGISTRATION
PROSPERO CRD42019161764.
Topics: Bias; Clinical Decision Rules; Data Interpretation, Statistical; Humans; Machine Learning; Models, Statistical; Multivariate Analysis; Risk
PubMed: 34670780
DOI: 10.1136/bmj.n2281 -
Clinical Microbiology and Infection :... Apr 2015Bacteraemia is associated with high mortality. Although many models for predicting bacteraemia have been developed, not all have been validated, and even when they were,... (Review)
Review
Bacteraemia is associated with high mortality. Although many models for predicting bacteraemia have been developed, not all have been validated, and even when they were, the validation processes varied. We identified validated models that have been developed; asked whether they were successful in defining groups with a very low or high prevalence of bacteraemia; and whether they were used in clinical practice. Electronic databases were searched to identify studies that underwent validation on prediction of bacteraemia in adults. We included only studies that were able to define groups with low or high probabilities for bacteraemia (arbitrarily defined as below 3% or above 30%). Fifteen publications fulfilled inclusion criteria, including 59 276 patients. Eleven were prospective and four retrospective. Study populations and the parameters included in the different models were heterogeneous. Ten studies underwent internal validation; the model performed well in all of them. Twelve performed external validation. Of the latter, seven models were validated in a different hospital, using a new independent database. In five of these, the model performed well. After contacting authors, we found that none of the models was implemented in clinical practice. We conclude that heterogeneous studies have been conducted in different defined groups of patients with limited external validation. Significant savings to the system and the individual patient can be gained by refraining from performing blood cultures in groups of patients in which the probability of true bacteraemia is very low, while the probability of contamination is constant. Clinical trials of existing or new models should be done to examine whether models are helpful and safe in clinical use, preferably multicentre in order to secure utility and safety in diverse clinical settings.
Topics: Bacteremia; Blood; Decision Support Techniques; Humans; Microbiological Techniques
PubMed: 25677625
DOI: 10.1016/j.cmi.2015.01.023 -
American Journal of Obstetrics &... Oct 2023Valid and reliable maternity patient-reported experience measures are critical to understanding women's experiences of care. They can support clinical practice, health... (Review)
Review
OBJECTIVE
Valid and reliable maternity patient-reported experience measures are critical to understanding women's experiences of care. They can support clinical practice, health service and system performance measurement, and research. The aim of this review is to identify and critically appraise the risk of bias, woman-centricity (content validity), and psychometric properties of maternity patient-reported experience measures published in the scientific literature.
DATA SOURCES
MEDLINE, CINAHL Plus, PsycINFO, and Embase were systematically searched for relevant records between January 1, 2010 and July 10, 2021.
STUDY ELIGIBILITY CRITERIA
We searched for articles describing the instrument development of maternity patient-reported experience measures and measurement properties associated with instrument validity and reliability testing. Articles that described patient-reported experience measures developed outside of the maternity context and articles that did not contribute to the instruments' development, content validation, and/or psychometric evaluation were excluded.
METHODS
Included articles underwent risk of bias, content validity, and psychometric properties assessments in line with the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) guidance. Patient-reported experience measure results were summarized according to language subgroups. An overall recommendation for use was determined for each patient-reported experience measure language subgroup.
RESULTS
A total of 54 studies reported on the development and psychometric evaluation of 25 maternity patient-reported experience measures, grouped into 45 language subgroups. The quality of evidence underpinning the instruments' development was generally poor. Only 2 (4.4%) patient-reported experience measures reported sufficient content validity, and only 1 (2.2%) received a level "A" recommendation, required for real-world use.
CONCLUSION
Maternity patient-reported experience measures demonstrated poor-quality evidence for their measurement properties and insufficient detail about content validity. Future maternity patient-reported experience measure development needs to prioritize women's involvement in deciding what is relevant, comprehensive, and comprehensible to measure. Improving the content validity of maternity patient-reported experience measures will improve overall validity and reliability and facilitate real-world practice improvements. Standardized patient-reported experience measure implementation also needs to be prioritized to support advancements in clinical practice for women.
PubMed: 37517609
DOI: 10.1016/j.ajogmf.2023.101102 -
Zeitschrift Fur Gerontologie Und... Dec 2022Life-space mobility (LSM), as the extent of mobility within one's environment, is a key for successful aging and has become a relevant concept in gerontology and... (Review)
Review
BACKGROUND
Life-space mobility (LSM), as the extent of mobility within one's environment, is a key for successful aging and has become a relevant concept in gerontology and geriatric research. Adequate assessment instruments are needed to identify older persons with LSM restrictions, and to initiate, adapt or evaluate intervention strategies.
OBJECTIVE
To systematically identify, describe and analyze the psychometric properties of LSM questionnaires, with a special focus on their availability in the German language.
METHODS
A systematic literature search was conducted in PubMed, PsycINFO, Cochrane Library, CINAHL, and Web of Science. Studies that examined at least one psychometric property of LSM questionnaires published up to August 2021 were included and evaluated based on the consensus-based standards for the selection of health measurement instruments (COSMIN) guidelines.
RESULTS
This study included 37 validation studies describing 13 different LSM questionnaires. Methodological quality and comprehensiveness of validations were heterogeneous. Based on comprehensive and high-quality results, four LSM questionnaires stood out: the University of Alabama at Birmingham life-space assessment (UAB-LSA), life-space assessment in persons with cognitive impairment (LSA-CI), interview-based and proxy-based versions of the life-space assessment in institutionalized settings (LSA-IS), all of them available in the German language.
CONCLUSION
This systematic review provides a concise overview of available LSM questionnaires and their psychometric properties to facilitate the selection for use in clinical practice and research. The UAB-LSA and LSA-CI for community settings and the interview-based or proxy-based LSA-IS for institutional settings were found to be the most appropriate LSM questionnaires.
Topics: Humans; Aged; Aged, 80 and over
PubMed: 35244765
DOI: 10.1007/s00391-022-02035-5 -
Infant Behavior & Development May 2023Interactions between newborns and their parents/primary caregivers are characterized by asymmetric and dependent relationships. This systematic review mapped,... (Review)
Review
Interactions between newborns and their parents/primary caregivers are characterized by asymmetric and dependent relationships. This systematic review mapped, identified, and described the psychometric parameters, categories, and items of instruments used to assess mother-newborn interaction. Seven electronic databases were accessed in this study. Furthermore, this research included neonatal interaction studies describing instruments' items, domains, and psychometric properties while excluding studies that focused on maternal interactions and lacked items for assessing newborns. Additionally, studies validated with older infants that did not have a newborn in the sample were used for test validation, which is a criterion used to decrease the risk of bias. Fourteen observational instruments from 1047 identified citations were included that addressed interactions using varying techniques, constructs, and settings. Particularly, we focused on observational settings that assessed interactions with communication-based constructs in the context of proximity or distance as influenced by physical, behavioral, or procedural barriers. These tools are also used to predict risk behaviors in a psychological context, mitigate feeding difficulties, and conduct neurobehavioral assessments of mother-newborn interactions. The elicited imitation was also an observational setting. This study found that the most described properties in the included citations were inter-rater reliability followed by criterion validity. However, only two instruments reported content, construct, and criterion validity, as well as a description of an internal consistency assessment and inter-rater reliability. Finally, the synthesis of the instruments reported in this study can guide clinicians and researchers in selecting the most appropriate one for their own application.
Topics: Infant; Female; Humans; Infant, Newborn; Mothers; Reproducibility of Results; Parents; Psychometrics; Communication
PubMed: 36863246
DOI: 10.1016/j.infbeh.2023.101825 -
The Journal of Pediatrics Jul 2023To review systematically and assess the accuracy of prediction models for bronchopulmonary dysplasia (BPD) at 36 weeks of postmenstrual age. (Meta-Analysis)
Meta-Analysis
OBJECTIVE
To review systematically and assess the accuracy of prediction models for bronchopulmonary dysplasia (BPD) at 36 weeks of postmenstrual age.
STUDY DESIGN
Searches were conducted in MEDLINE and EMBASE. Studies published between 1990 and 2022 were included if they developed or validated a prediction model for BPD or the combined outcome death/BPD at 36 weeks in the first 14 days of life in infants born preterm. Data were extracted independently by 2 authors following the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (ie, CHARMS) and PRISMA guidelines. Risk of bias was assessed using the Prediction model Risk Of Bias ASsessment Tool (ie, PROBAST).
RESULTS
Sixty-five studies were reviewed, including 158 development and 108 externally validated models. Median c-statistic of 0.84 (range 0.43-1.00) was reported at model development, and 0.77 (range 0.41-0.97) at external validation. All models were rated at high risk of bias, due to limitations in the analysis part. Meta-analysis of the validated models revealed increased c-statistics after the first week of life for both the BPD and death/BPD outcome.
CONCLUSIONS
Although BPD prediction models perform satisfactorily, they were all at high risk of bias. Methodologic improvement and complete reporting are needed before they can be considered for use in clinical practice. Future research should aim to validate and update existing models.
Topics: Infant; Infant, Newborn; Humans; Infant, Premature; Bronchopulmonary Dysplasia
PubMed: 37059387
DOI: 10.1016/j.jpeds.2023.01.024 -
Revista Chilena de Pediatria Dec 2017According to the International Classification of Functioning, Disability and Health (ICF), participation is defined as "the involvement of the subject in situations of... (Review)
Review
INTRODUCTION
According to the International Classification of Functioning, Disability and Health (ICF), participation is defined as "the involvement of the subject in situations of life" and is fun damental in the development of children and adolescents. In case of children with disabilities, participation allows a better understanding of the possible impact of deficiencies in daily life.
OBJECTIVE
To evaluate measurement scales of participation in children and adolescents with and without disabilities.
METHOD
Systematic review. Validation studies of measurement scales of parti cipation in children and adolescents with and without disabilities without language restriction were included. The search was performed in Pubmed, EMBASE, CINAHL, The Cochrane Library, Health Virtual Library, Opengrey and Google Scholar. The data were extracted and analyzed in Microsoft Excel. Protocol Register PROSPERO 2015: CRD42015020644.
RESULTS
1689 articles were collected through electronic search, 9 scales were selected for analysis. Diversity in size and application bet ween the scales selected was found. The number of patients included in the original validations was variable, as the percentage of children and/or adolescents with disabilities included in the validation studies.
CONCLUSIONS
There is great variability in the psychometric properties and characteristics of the scales included in this review, mainly for which the participation construct differs according to culture, so the selected scales require transcultural adaptations for their use.
Topics: Adolescent; Child; Community Participation; Disability Evaluation; Disabled Children; Humans; Psychometrics
PubMed: 29546934
DOI: 10.4067/S0370-41062017000600812 -
Frontiers in Public Health 2023Increasing attention on workplace wellbeing and growth in workplace wellbeing interventions has highlighted the need to measure workers' wellbeing. This systematic...
INTRODUCTION
Increasing attention on workplace wellbeing and growth in workplace wellbeing interventions has highlighted the need to measure workers' wellbeing. This systematic review sought to identify the most valid and reliable published measure/s of wellbeing for workers developed between 2010 to 2020.
METHODS
Electronic databases Health and Psychosocial Instruments, APA PsycInfo, and Scopus were searched. Key search terms included variations of AND . Studies and properties of wellbeing measures were then appraised using Consensus-based Standards for the selection of health Measurement Instruments.
RESULTS
Eighteen articles reported development of new wellbeing instruments and eleven undertook a psychometric validation of an existing wellbeing instrument in a specific country, language, or context. Generation and pilot testing of items for the 18 newly developed instruments were largely rated 'Inadequate'; only two were rated as 'Very Good'. None of the studies reported measurement properties of responsiveness, criterion validity, or content validity. The three instruments with the greatest number of positively rated measurement properties were the Personal Growth and Development Scale, The University of Tokyo Occupational Mental Health well-being 24 scale, and the Employee Well-being scale. However, none of these newly developed worker wellbeing instruments met the criteria for adequate instrument design.
DISCUSSION
This review provides researchers and clinicians a synthesis of information to help inform appropriate instrument selection in measurement of workers' wellbeing.
SYSTEMATIC REVIEW REGISTRATION
https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=79044, identifier: PROSPERO, CRD42018079044.
Topics: Humans; Mental Health; Health Personnel; Language; Workplace; Working Conditions
PubMed: 37293618
DOI: 10.3389/fpubh.2023.1053179