-
Work (Reading, Mass.) 2022Developing reliable tools to tap into all the behavioral dimensions of individual job performance and identifying the right sub-dimensions is necessary for both research...
BACKGROUND
Developing reliable tools to tap into all the behavioral dimensions of individual job performance and identifying the right sub-dimensions is necessary for both research and practice.
OBJECTIVE
This study aimed at developing and validating an IJPQ that addresses shortcomings of existing questionnaires.
METHODS
After a comprehensive systematic literature review, a framework consisting of four dimensions, including task performance (TP), contextual performance (CP), counterproductive work behavior (CWB), and adaptive performance (AP) was structured for measuring IJP. As well, 45 sub-dimensions were identified for measuring IJP's dimensions. Content and face validity were evaluated, and item impact score (IS), content validity index (CVI), Kappa, and content validity ratio (CVR) were calculated. For reliability and confirmatory factor analysis (CFA), 525 workers completed the validated questionnaire and Cronbach alpha and goodness of fit indexes were determined, respectively.
RESULTS
Of the 62 items generated to measure dimensions, 53 were approved. Based on item-level CVI, of the 53 items, only 45 items were accepted. Finally, the results of item level CVR led to the extraction of 27 questions to evaluate IJP. The obtained scale level CVI and scale level CVR were 0.91 and 0.68, respectively. Based on the results obtained from 525 Iranian workers, values of Cronbach's Alpha, X2/df, RMSEA, and P-value were in the acceptable range.
CONCLUSIONS
Conclusively, a questionnaire containing 20 items was developed and validated for measuring IJP of Iranian worker's culture. The four dimensions of TP, CO, CWB, and AP consisted of 6, 5, 5, and 4 items each, respectively. Overall, IJPQ is a theory-based, reliable, and valid instrument for assessing job performance.
Topics: Humans; Iran; Psychometrics; Reproducibility of Results; Surveys and Questionnaires; Work Performance
PubMed: 35871384
DOI: 10.3233/WOR-211004 -
International Journal of Nursing Studies Aug 2017Valid observational pain scales are needed to assess pain and ensure sufficient treatment of pain in children that lack the verbal ability to self-report pain. Published... (Review)
Review
BACKGROUND
Valid observational pain scales are needed to assess pain and ensure sufficient treatment of pain in children that lack the verbal ability to self-report pain. Published reviews attempt to synthesize results from primary studies validating these scales and based on the findings recommendations may be given, for example which pain scales are the most appropriate for use in different pediatric populations.
OBJECTIVES
The aims of this review were to describe how systematic reviews have evaluated and recommended observational pain scales for use in children aged 0-18 years and appraise the evidence underlying these recommendations.
DESIGN
Systematic review of reviews.
DATA SOURCES
The Cochrane Library, PubMed/MEDLINE, CINAHL, Web of Science, and PsychINFO were searched from inception to September 2016. Reference lists and gray literature were searched for additional studies.
REVIEW METHODS
Study selection and data extraction were performed by two reviewers independently with a disagreement procedure in place. Methodological quality or study validity was measured using the Assessment of Multiple Systematic Reviews checklist and risk of bias or internal validity was measured using the Risk of Bias in Systematic Reviews tool. The review protocol was registered with PROSPERO: registration number CRD42016035264.
RESULTS
Twelve reviews met the inclusion criteria. Together; they included 65 different observational pain scales for use in children, of which 28 were recommended at least once. Face, Legs, Activity, Cry, Consolability/revised version of Face, Legs, Activity, Cry, Consolability, COMFORT/COMFORT behavioral scale and Children's Hospital of Eastern Ontario Pain Scale were evaluated and recommended most frequently. Few of the included reviews assessed the methodological quality of the studies included in the review. The narrative analysis consisted mostly of a reiteration of the results from the primary studies. In general, more recent reviews showed a lower risk of bias than older ones.
CONCLUSIONS
Included reviews exhibited low quality of evidence; thus, their recommendations regarding pain scales for use in clinical practice or research with children that lack the verbal ability to self-report pain should be interpreted with caution.
Topics: Child; Humans; Observation; Pain Measurement
PubMed: 28558342
DOI: 10.1016/j.ijnurstu.2017.05.010 -
Colorectal Disease : the Official... Nov 2020Haemorrhoidal disease can severely affect a patient's quality of life. Its classification is commonly based on morphology of the degree of prolapse; however, this does... (Review)
Review
AIM
Haemorrhoidal disease can severely affect a patient's quality of life. Its classification is commonly based on morphology of the degree of prolapse; however, this does not take into account the symptoms and impact on the quality of life. The aim of this systematic review was to determine the most appropriate instruments that classify the severity of disease according to symptoms.
METHOD
A PRISMA-compliant search was conducted in December 2019 to identify studies that described the validation of a haemorrhoidal symptom score. The measurement properties of the scoring systems were assessed based on the consensus-based standards for the selection of health status measurement instruments (COSMIN) methodology for systematic reviews for patient-reported outcome measures.
RESULTS
A total of 5288 articles were identified, with five articles included. Three studies developed a scoring system based on a set of core symptoms for a cohort of patients and validated the score against treatment outcomes. One study developed a disease-specific quality of life questionnaire based on symptoms to evaluate disease burden. One study combined both quality of life and symptom measures and tested measurement properties on two cohorts of patients. Only one study demonstrated satisfactory valid, reliable and responsive measurement criteria.
CONCLUSION
A single study demonstrated sufficient quality in measurement properties to be recommended for clinical use. Further studies in this area should utilize consensus-based standards for designing and reporting validation research to ensure that the appropriate evidence base is acquired if any further patient-reported outcome measures are to be recommended.
Topics: Hemorrhoids; Humans; Patient Reported Outcome Measures; Quality of Life; Reproducibility of Results; Surveys and Questionnaires
PubMed: 32639663
DOI: 10.1111/codi.15253 -
Best Practice & Research. Clinical... Jun 2016The diversity, technical skills required, and risk inherent to advanced endoscopy techniques all contribute to complex training curricula and steep learning curves.... (Review)
Review
INTRODUCTION
The diversity, technical skills required, and risk inherent to advanced endoscopy techniques all contribute to complex training curricula and steep learning curves. Since trainees develop endoscopy skills at different rates, there has been a shift towards competency-based training and certification. Validated endoscopy performance measures for trainees are, therefore, necessary. The aim of this systematic review was to describe and critically assess the existing evidence regarding measures of performance for trainees in advanced endoscopy.
METHODS
A systematic review of the literature from January 1980 to January 2016 was carried out using the MEDLINE, EMBASE, CENTRAL, and ISI Web of knowledge databases. MeSH terms related to 'advanced endoscopy' and 'performance' were applied to a highly sensitive search strategy. The main outcomes were face, content, and construct validity, as well as reliability.
RESULTS
The literature search yielded 1,662 studies and 77 met the inclusion criteria after abstract and full-text review (endoscopic retrograde cholangiopancreatography (ERCP)=23, endoscopic ultrasound (EUS)=30, colonoscopic polypectomy (CP)=11, balloon-assisted enteroscopy (BAE)=7, luminal stenting=3, radiofrequency ablation (RFA)=2, and endoscopic muscosal resection (EMR)=1). Good validity and reliability were found for measurement tools of overall performance in ERCP, EUS and CP, with applications for both patient-based and simulator training models. A number of specific technical skills were also shown to be valid measures of performance. These include: selective biliary cannulation, sphincterotomy, biliary stent placement, stone extraction and procedure time for ERCP; pancreatic solid mass T-staging, EUS-guided fine needle aspiration (EUS-FNA) procedure time, number of EUS-FNA passes and puncture precision for EUS; procedure time and en bloc resection rate for CP; retrograde fluoroscopy time for BAE; and mean number of endoscopy sessions required to achieve complete eradication of intestinal metaplasia (CIEM) for RFA. The evidence for EMR and luminal stenting is of insufficient quality to make recommendations.
CONCLUSIONS
We have identified multiple valid and readily available performance measures for advanced endoscopy trainees for ERCP, EUS, CP, BAE and RFA procedures. These tools should be considered in advanced endoscopy training programs wishing to move away from apprenticeship-based training and towards competency-based learning with the help of patient-based and simulator tools.
Topics: Balloon Enteroscopy; Cholangiopancreatography, Endoscopic Retrograde; Clinical Competence; Colonoscopy; Education, Medical, Graduate; Educational Measurement; Endoscopy; Endosonography; Humans; Pulsed Radiofrequency Treatment; Reproducibility of Results
PubMed: 27345650
DOI: 10.1016/j.bpg.2016.05.003 -
Journal of Community Psychology Sep 2021The Communities That Care Youth Survey (CTCYS) is an assessment for youth risk and protective factors focused on antisocial behaviors, drug abuse, violence, and...
AIMS
The Communities That Care Youth Survey (CTCYS) is an assessment for youth risk and protective factors focused on antisocial behaviors, drug abuse, violence, and delinquency. The systematic review aimed to analyze psychometric validations and cross-cultural adaptation processes of CTCYS to prepare its Brazilian validation.
METHODS
PubMed, SpringerLink, Scopus, and Web of Science were the databases included. The study followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) protocol.
RESULTS
Twenty articles were selected based on the inclusion/exclusion criteria. In general, the construct validity indexes were considered adequate, and the measures of risk and protective factors were equivalent between ethnic groups, gender, and populations at risk. Reliability values for most scales are good, averaging about 0.78. We found cultural adaptations from six different countries with some satisfactory results, but not so good as the original properties of CTCYS and the adjustments to other ethnic groups or subpopulations in the United States.
CONCLUSION
This survey may be a potentially reliable and valid instrument to assess risk and protection in different cultures and populations. However, there is still a gap in the instrument's cross-cultural adaptation processes.
Topics: Adolescent; Humans; Psychometrics; Reproducibility of Results; Social Change; Surveys and Questionnaires; United States
PubMed: 33634886
DOI: 10.1002/jcop.22529 -
Journal of Affective Disorders Sep 2020Grief reactions that become abnormally persistent and cause significant impairment in functioning have been introduced as new diagnostic entities within the... (Review)
Review
BACKGROUND
Grief reactions that become abnormally persistent and cause significant impairment in functioning have been introduced as new diagnostic entities within the classification systems DSM-5 and ICD-11 termed persistent complex bereavement disorder (PCBD) and prolonged grief disorder (PGD), respectively. In order to assess these conditions, reliable and valid assessment instruments are indispensable. This systematic review examines available assessment tools for disordered grief, reviews their psychometric properties and assesses the extent to which they reflect the diagnostic entities.
METHODS
A systematic search of PubMed, PsycINFO and Web of Science databases was conducted. English language articles describing either the development of a measure for assessing PGD in adults or the validation of such a measure were included.
RESULTS
Overall 2215 publications were screened, 29 of which met the eligibility criteria. Three of the eleven described assessment tools are structured clinical interviews, one tool is a carer-based assessment and the remaining tools are self-report questionnaires. Most instruments demonstrate excellent or good reliability and validity. No tool assesses the current diagnostic criteria for PGD fully. While three tools cover all diagnostic criteria for PCBD, only one (TGI-SR) provides an adequate, empirically tested diagnostic algorithm.
LIMITATIONS
The inclusion of only English-language publications may have led to omission of relevant measurement tools and/or validation studies in different languages.
CONCLUSIONS
The newly released ICD-11 criteria for PGD could serve as a gold standard for diagnosis and build a foundation for the development of more precise assessment tools. Recommendations for clinical practice and future research are provided.
Topics: Adult; Bereavement; Diagnostic and Statistical Manual of Mental Disorders; Grief; Humans; International Classification of Diseases; Reproducibility of Results
PubMed: 32663972
DOI: 10.1016/j.jad.2020.05.049 -
The International Journal of Behavioral... Jun 2023Wearable technology is used by consumers and researchers worldwide for continuous activity monitoring in daily life. Results of high-quality laboratory-based validation... (Review)
Review
BACKGROUND
Wearable technology is used by consumers and researchers worldwide for continuous activity monitoring in daily life. Results of high-quality laboratory-based validation studies enable us to make a guided decision on which study to rely on and which device to use. However, reviews in adults that focus on the quality of existing laboratory studies are missing.
METHODS
We conducted a systematic review of wearable validation studies with adults. Eligibility criteria were: (i) study under laboratory conditions with humans (age ≥ 18 years); (ii) validated device outcome must belong to one dimension of the 24-hour physical behavior construct (i.e., intensity, posture/activity type, and biological state); (iii) study protocol must include a criterion measure; (iv) study had to be published in a peer-reviewed English language journal. Studies were identified via a systematic search in five electronic databases as well as back- and forward citation searches. The risk of bias was assessed based on the QUADAS-2 tool with eight signaling questions.
RESULTS
Out of 13,285 unique search results, 545 published articles between 1994 and 2022 were included. Most studies (73.8% (N = 420)) validated an intensity measure outcome such as energy expenditure; only 14% (N = 80) and 12.2% (N = 70) of studies validated biological state or posture/activity type outcomes, respectively. Most protocols validated wearables in healthy adults between 18 and 65 years. Most wearables were only validated once. Further, we identified six wearables (i.e., ActiGraph GT3X+, ActiGraph GT9X, Apple Watch 2, Axivity AX3, Fitbit Charge 2, Fitbit, and GENEActiv) that had been used to validate outcomes from all three dimensions, but none of them were consistently ranked with moderate to high validity. Risk of bias assessment resulted in 4.4% (N = 24) of all studies being classified as "low risk", while 16.5% (N = 90) were classified as "some concerns" and 79.1% (N = 431) as "high risk".
CONCLUSION
Laboratory validation studies of wearables assessing physical behaviour in adults are characterized by low methodological quality, large variability in design, and a focus on intensity. Future research should more strongly aim at all components of the 24-hour physical behaviour construct, and strive for standardized protocols embedded in a validation framework.
Topics: Humans; Adult; Adolescent; Wearable Electronic Devices; Fitness Trackers; Monitoring, Physiologic; Posture; Sedentary Behavior
PubMed: 37291598
DOI: 10.1186/s12966-023-01473-7 -
Aging and Disease Jul 2022Osteoporotic fractures (OF) are a global public health problem currently. Many risk prediction models for OF have been developed, but their performance and... (Review)
Review
Osteoporotic fractures (OF) are a global public health problem currently. Many risk prediction models for OF have been developed, but their performance and methodological quality are unclear. We conducted this systematic review to summarize and critically appraise the OF risk prediction models. Three databases were searched until April 2021. Studies developing or validating multivariable models for OF risk prediction were considered eligible. Used the prediction model risk of bias assessment tool to appraise the risk of bias and applicability of included models. All results were narratively summarized and described. A total of 68 studies describing 70 newly developed prediction models and 138 external validations were included. Most models were explicitly developed (n=31, 44%) and validated (n=76, 55%) only for female. Only 22 developed models (31%) were externally validated. The most validated tool was Fracture Risk Assessment Tool. Overall, only a few models showed outstanding (n=3, 1%) or excellent (n=32, 15%) prediction discrimination. Calibration of developed models (n=25, 36%) or external validation models (n=33, 24%) were rarely assessed. No model was rated as low risk of bias, mostly because of an insufficient number of cases and inappropriate assessment of calibration. There are a certain number of OF risk prediction models. However, few models have been thoroughly internally validated or externally validated (with calibration being unassessed for most of the models), and all models showed methodological shortcomings. Instead of developing completely new models, future research is suggested to validate, improve, and analyze the impact of existing models.
PubMed: 35855348
DOI: 10.14336/AD.2021.1206 -
American Journal of Obstetrics &... Oct 2023Valid and reliable maternity patient-reported experience measures are critical to understanding women's experiences of care. They can support clinical practice, health... (Review)
Review
OBJECTIVE
Valid and reliable maternity patient-reported experience measures are critical to understanding women's experiences of care. They can support clinical practice, health service and system performance measurement, and research. The aim of this review is to identify and critically appraise the risk of bias, woman-centricity (content validity), and psychometric properties of maternity patient-reported experience measures published in the scientific literature.
DATA SOURCES
MEDLINE, CINAHL Plus, PsycINFO, and Embase were systematically searched for relevant records between January 1, 2010 and July 10, 2021.
STUDY ELIGIBILITY CRITERIA
We searched for articles describing the instrument development of maternity patient-reported experience measures and measurement properties associated with instrument validity and reliability testing. Articles that described patient-reported experience measures developed outside of the maternity context and articles that did not contribute to the instruments' development, content validation, and/or psychometric evaluation were excluded.
METHODS
Included articles underwent risk of bias, content validity, and psychometric properties assessments in line with the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) guidance. Patient-reported experience measure results were summarized according to language subgroups. An overall recommendation for use was determined for each patient-reported experience measure language subgroup.
RESULTS
A total of 54 studies reported on the development and psychometric evaluation of 25 maternity patient-reported experience measures, grouped into 45 language subgroups. The quality of evidence underpinning the instruments' development was generally poor. Only 2 (4.4%) patient-reported experience measures reported sufficient content validity, and only 1 (2.2%) received a level "A" recommendation, required for real-world use.
CONCLUSION
Maternity patient-reported experience measures demonstrated poor-quality evidence for their measurement properties and insufficient detail about content validity. Future maternity patient-reported experience measure development needs to prioritize women's involvement in deciding what is relevant, comprehensive, and comprehensible to measure. Improving the content validity of maternity patient-reported experience measures will improve overall validity and reliability and facilitate real-world practice improvements. Standardized patient-reported experience measure implementation also needs to be prioritized to support advancements in clinical practice for women.
PubMed: 37517609
DOI: 10.1016/j.ajogmf.2023.101102 -
Clinical Lung Cancer Mar 2016Many lung cancer risk prediction models have been published but there has been no systematic review or comprehensive assessment of these models to assess how they could... (Review)
Review
Many lung cancer risk prediction models have been published but there has been no systematic review or comprehensive assessment of these models to assess how they could be used in screening. We performed a systematic review of lung cancer prediction models and identified 31 articles that related to 25 distinct models, of which 11 considered epidemiological factors only and did not require a clinical input. Another 11 articles focused on models that required a clinical assessment such as a blood test or scan, and 8 articles considered the 2-stage clonal expansion model. More of the epidemiological models had been externally validated than the more recent clinical assessment models. There was varying discrimination, the ability of a model to distinguish between cases and controls, with an area under the curve between 0.57 and 0.879 and calibration, the model's ability to assign an accurate probability to an individual. In our review we found that further validation studies need to be considered; especially for the Prostate, Lung, Colorectal, and Ovarian (PLCO) Cancer Screening Trial 2012 Model Version (PLCOM2012) and Hoggart models, which recorded the best overall performance. Future studies will need to focus on prediction rules, such as optimal risk thresholds, for models for selective screening trials. Only 3 validation studies considered prediction rules when validating the models and overall the models were validated using varied tests in distinct populations, which made direct comparisons difficult. To improve this, multiple models need to be tested on the same data set with considerations for sensitivity, specificity, model accuracy, and positive predictive values at the optimal risk thresholds.
Topics: Decision Support Techniques; Early Detection of Cancer; Humans; Lung Neoplasms; Models, Statistical; Predictive Value of Tests; Prognosis; Risk Assessment; Risk Factors; Sensitivity and Specificity
PubMed: 26712102
DOI: 10.1016/j.cllc.2015.11.007