-
Infant Behavior & Development May 2023Interactions between newborns and their parents/primary caregivers are characterized by asymmetric and dependent relationships. This systematic review mapped,... (Review)
Review
Interactions between newborns and their parents/primary caregivers are characterized by asymmetric and dependent relationships. This systematic review mapped, identified, and described the psychometric parameters, categories, and items of instruments used to assess mother-newborn interaction. Seven electronic databases were accessed in this study. Furthermore, this research included neonatal interaction studies describing instruments' items, domains, and psychometric properties while excluding studies that focused on maternal interactions and lacked items for assessing newborns. Additionally, studies validated with older infants that did not have a newborn in the sample were used for test validation, which is a criterion used to decrease the risk of bias. Fourteen observational instruments from 1047 identified citations were included that addressed interactions using varying techniques, constructs, and settings. Particularly, we focused on observational settings that assessed interactions with communication-based constructs in the context of proximity or distance as influenced by physical, behavioral, or procedural barriers. These tools are also used to predict risk behaviors in a psychological context, mitigate feeding difficulties, and conduct neurobehavioral assessments of mother-newborn interactions. The elicited imitation was also an observational setting. This study found that the most described properties in the included citations were inter-rater reliability followed by criterion validity. However, only two instruments reported content, construct, and criterion validity, as well as a description of an internal consistency assessment and inter-rater reliability. Finally, the synthesis of the instruments reported in this study can guide clinicians and researchers in selecting the most appropriate one for their own application.
Topics: Infant; Female; Humans; Infant, Newborn; Mothers; Reproducibility of Results; Parents; Psychometrics; Communication
PubMed: 36863246
DOI: 10.1016/j.infbeh.2023.101825 -
Journal of Community Psychology Sep 2021The Communities That Care Youth Survey (CTCYS) is an assessment for youth risk and protective factors focused on antisocial behaviors, drug abuse, violence, and...
AIMS
The Communities That Care Youth Survey (CTCYS) is an assessment for youth risk and protective factors focused on antisocial behaviors, drug abuse, violence, and delinquency. The systematic review aimed to analyze psychometric validations and cross-cultural adaptation processes of CTCYS to prepare its Brazilian validation.
METHODS
PubMed, SpringerLink, Scopus, and Web of Science were the databases included. The study followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) protocol.
RESULTS
Twenty articles were selected based on the inclusion/exclusion criteria. In general, the construct validity indexes were considered adequate, and the measures of risk and protective factors were equivalent between ethnic groups, gender, and populations at risk. Reliability values for most scales are good, averaging about 0.78. We found cultural adaptations from six different countries with some satisfactory results, but not so good as the original properties of CTCYS and the adjustments to other ethnic groups or subpopulations in the United States.
CONCLUSION
This survey may be a potentially reliable and valid instrument to assess risk and protection in different cultures and populations. However, there is still a gap in the instrument's cross-cultural adaptation processes.
Topics: Adolescent; Humans; Psychometrics; Reproducibility of Results; Social Change; Surveys and Questionnaires; United States
PubMed: 33634886
DOI: 10.1002/jcop.22529 -
Aging and Disease Jul 2022Osteoporotic fractures (OF) are a global public health problem currently. Many risk prediction models for OF have been developed, but their performance and... (Review)
Review
Osteoporotic fractures (OF) are a global public health problem currently. Many risk prediction models for OF have been developed, but their performance and methodological quality are unclear. We conducted this systematic review to summarize and critically appraise the OF risk prediction models. Three databases were searched until April 2021. Studies developing or validating multivariable models for OF risk prediction were considered eligible. Used the prediction model risk of bias assessment tool to appraise the risk of bias and applicability of included models. All results were narratively summarized and described. A total of 68 studies describing 70 newly developed prediction models and 138 external validations were included. Most models were explicitly developed (n=31, 44%) and validated (n=76, 55%) only for female. Only 22 developed models (31%) were externally validated. The most validated tool was Fracture Risk Assessment Tool. Overall, only a few models showed outstanding (n=3, 1%) or excellent (n=32, 15%) prediction discrimination. Calibration of developed models (n=25, 36%) or external validation models (n=33, 24%) were rarely assessed. No model was rated as low risk of bias, mostly because of an insufficient number of cases and inappropriate assessment of calibration. There are a certain number of OF risk prediction models. However, few models have been thoroughly internally validated or externally validated (with calibration being unassessed for most of the models), and all models showed methodological shortcomings. Instead of developing completely new models, future research is suggested to validate, improve, and analyze the impact of existing models.
PubMed: 35855348
DOI: 10.14336/AD.2021.1206 -
Advances in Health Sciences Education :... May 2014Ongoing transformations in health professions education underscore the need for valid and reliable assessment. The current standard for assessment validation requires... (Review)
Review
Ongoing transformations in health professions education underscore the need for valid and reliable assessment. The current standard for assessment validation requires evidence from five sources: content, response process, internal structure, relations with other variables, and consequences. However, researchers remain uncertain regarding the types of data that contribute to each evidence source. We sought to enumerate the validity evidence sources and supporting data elements for assessments using technology-enhanced simulation. We conducted a systematic literature search including MEDLINE, ERIC, and Scopus through May 2011. We included original research that evaluated the validity of simulation-based assessment scores using two or more evidence sources. Working in duplicate, we abstracted information on the prevalence of each evidence source and the underlying data elements. Among 217 eligible studies only six (3 %) referenced the five-source framework, and 51 (24 %) made no reference to any validity framework. The most common evidence sources and data elements were: relations with other variables (94 % of studies; reported most often as variation in simulator scores across training levels), internal structure (76 %; supported by reliability data or item analysis), and content (63 %; reported as expert panels or modification of existing instruments). Evidence of response process and consequences were each present in <10 % of studies. We conclude that relations with training level appear to be overrepresented in this field, while evidence of consequences and response process are infrequently reported. Validation science will be improved as educators use established frameworks to collect and interpret evidence from the full spectrum of possible sources and elements.
Topics: Education, Medical; Educational Measurement; Humans; Prevalence; Reproducibility of Results; User-Computer Interface
PubMed: 23636643
DOI: 10.1007/s10459-013-9458-4 -
Thyroid : Official Journal of the... Oct 2022Molecular tests for thyroid nodules with indeterminate fine needle aspiration results are increasingly used in clinical practice; however, true diagnostic summaries of... (Meta-Analysis)
Meta-Analysis Review
Molecular tests for thyroid nodules with indeterminate fine needle aspiration results are increasingly used in clinical practice; however, true diagnostic summaries of these tests are unknown. A systematic review and meta-analysis were completed to (1) evaluate the accuracy of commercially available molecular tests for malignancy in indeterminate thyroid nodules and (2) quantify biases and limitations in studies that validate those tests. PubMed, EMBASE, and Web of Science were systematically searched through July 2021. English language articles that reported original clinical validation attempts of molecular tests for indeterminate thyroid nodules were included if they reported counts of true-negative, true-positive, false-negative, and false-positive results. We performed screening and full-text review, followed by assessment of eight common biases and limitations, extraction of diagnostic and histopathological information, and meta-analysis of clinical validity using a bivariate linear mixed-effects model. Forty-nine studies were included. Meta-analysis of Afirma Gene expression classifiers (GEC; = 38 studies) revealed a sensitivity of 0.92 (confidence interval: 0.90-0.94), specificity of 0.26 (0.20-0.32), negative likelihood ratio (LR-) of 0.32 (0.23-0.44), positive LR+ of 1.24 (1.15-1.35), and area under the curve (AUC) of 0.83 (0.74-0.89). Afirma Genomic Sequencing Classifier (GSC; = 10) had a sensitivity of 0.94 (0.89-0.96), specificity of 0.38 (0.27-0.50), LR- of 0.18 (0.10-0.30), LR+ of 1.52 (1.28-1.87), and AUC of 0.91 (0.62-0.92). ThyroSeq v1 and v2 ( = 10) had a sensitivity of 0.86 (0.82-0.90), specificity of 0.74 (0.59-0.85), LR- of 0.19 (0.13-0.26), LR+ of 3.52 (2.08-5.92), and AUC of 0.86 (0.81-0.90). ThyroSeq v3 ( = 6) had a sensitivity of 0.92 (0.86-0.95), specificity of 0.41 (0.18-0.69), LR- of 0.24 (0.09-0.62), LR+ of 1.67 (1.09-2.98), and AUC of 0.90 (0.63-0.92). Fourteen percent of studies conducted a blinded histopathologic review of excised thyroid nodules, and 8% made the decision to go to surgery blind to molecular test results. Meta-analyses reveal a high diagnostic accuracy of molecular tests for thyroid nodule assessment of malignancy risk; however, these studies are subject to several limitations. Limitations and their potential clinical impacts must be addressed and, when feasible, adjusted for using valid statistical methodologies.
Topics: Humans; Thyroid Nodule; Pathology, Molecular; Biopsy, Fine-Needle; Molecular Diagnostic Techniques; Bias; Thyroid Neoplasms
PubMed: 35999710
DOI: 10.1089/thy.2022.0269 -
Frontiers in Veterinary Science 2021In order to base welfare assessment of dairy cattle on real-time measurement, integration of valid and reliable precision livestock farming (PLF) technologies is needed....
In order to base welfare assessment of dairy cattle on real-time measurement, integration of valid and reliable precision livestock farming (PLF) technologies is needed. The aim of this study was to provide a systematic overview of externally validated and commercially available PLF technologies, which could be used for sensor-based welfare assessment in dairy cattle. Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, a systematic literature review was conducted to identify externally validated sensor technologies. Out of 1,111 publications initially extracted from databases, only 42 studies describing 30 tools (including prototypes) met requirements for external validation. Moreover, through market search, 129 different retailed technologies with application for animal-based welfare assessment were identified. In total, only 18 currently retailed sensors have been externally validated (14%). The highest validation rate was found for systems based on accelerometers (30% of tools available on the market have validation records), while the lower rates were obtained for cameras (10%), load cells (8%), miscellaneous milk sensors (8%), and boluses (7%). Validated traits concerned animal activity, feeding and drinking behavior, physical condition, and health of animals. The majority of tools were validated on adult cows. Non-active behavior (lying and standing) and rumination were the most often validated for the high performance. Regarding active behavior (e.g., walking), lower performance of tools was reported. Also, tools used for physical condition (e.g., body condition scoring) and health evaluation (e.g., mastitis detection) were classified in lower performance group. The precision and accuracy of feeding and drinking assessment varied depending on measured trait and used sensor. Regarding relevance for animal-based welfare assessment, several validated technologies had application for good health (e.g., milk quality sensors) and good feeding (e.g., load cells, accelerometers). Accelerometers-based systems have also practical relevance to assess good housing. However, currently available PLF technologies have low potential to assess appropriate behavior of dairy cows. To increase actors' trust toward the PLF technology and prompt sensor-based welfare assessment, validation studies, especially in commercial herds, are needed. Future research should concentrate on developing and validating PLF technologies dedicated to the assessment of appropriate behavior and tools dedicated to monitoring the health and welfare in calves and heifers.
PubMed: 33869317
DOI: 10.3389/fvets.2021.634338 -
Child Abuse & Neglect Oct 2021Child maltreatment is a complex and multidimensional construct that encompasses a great number of risk factors. The Childhood Trauma Questionnaire - Short Form, one of...
BACKGROUND
Child maltreatment is a complex and multidimensional construct that encompasses a great number of risk factors. The Childhood Trauma Questionnaire - Short Form, one of the most widely used and validated instruments to assess childhood maltreatment in the past ten years, is a retrospective instrument that assesses several types of childhood abuse and maltreatment which is divided into five dimensions.
OBJECTIVE
The objectives of this systematic review are to critically appraise, compare, and summarize the methodological quality and psychometric properties of published research articles validating the Childhood Trauma Questionnaire - Short Form utilizing the COSMIN checklist.
METHOD
Articles published in English or Spanish, in the past ten years in the databases of Scopus, Web of Science, and ProQuest, and which, directly or indirectly analyzed psychometric properties of the CTQ-SF were screened, examined, and assessed utilizing the COSMIN checklist.
RESULTS
Main results indicate that there is a general pattern of assessing the same three psychometric properties (internal consistency, structural validity, and hypothesis testing) in a variety of samples, but leaving unassessed the rest of properties examined by the COSMIN checklist. Additionally, there are some problems with the internal consistency of several factors.
IMPLICATIONS AND CONCLUSIONS
While replicability and internal consistency are good psychometric indicators of the CTQ-SF, there is a big scientific gap of information regarding some psychometric properties. It is suggested that future research should address the remaining psychometric properties, reliability, measurement error, content validity, cross cultural and criterion validity, as well as re-examining internal consistency of some dimensions, in order to advance in the knowledge on childhood maltreatment assessment.
Topics: Adverse Childhood Experiences; Child; Humans; Psychometrics; Reproducibility of Results; Retrospective Studies; Surveys and Questionnaires
PubMed: 34352686
DOI: 10.1016/j.chiabu.2021.105223 -
PloS One 2018Patients are at risk for harm when treated simultaneously by healthcare providers from different healthcare organisations. To assess current practice and improvements of... (Review)
Review
BACKGROUND
Patients are at risk for harm when treated simultaneously by healthcare providers from different healthcare organisations. To assess current practice and improvements of transitional patient safety, valid measurement tools are needed.
AIM AND METHODS
To identify and appraise all measurement tools and outcomes that measure aspects of transitional patient safety, PubMed, Cinahl, Embase and Psychinfo were systematically searched. Two researchers performed the title and abstract and full-text selection. First, publications about validation of measurement tools were appraised for quality following COSMIN criteria. Second, we inventoried all measurement tools and outcome measures found in our search that assessed current transitional patient safety or the effect of interventions targeting transitional patient safety.
RESULTS
The initial search yielded 8288 studies, of which 18 assessed validity of measurement tools of different aspects of transitional safety, and 191 assessed current transitional patient safety or effect of interventions. In the validated measurement tools, the overall quality of content and structural validity was acceptable; other COSMIN criteria, such as reliability, measurement error and responsiveness, were mostly poor or not reported. In our outcome inventory, the most frequently used validated outcome measure was the Care Transition Measure (n = 9). The most frequently used non-validated outcome measures were: medication discrepancies (n = 98), hospital readmissions (n = 55), adverse events (n = 34), emergency department visits (n = 33), (mental or physical) health status (n = 28), quality and timeliness of discharge summary, and patient satisfaction (n = 23).
CONCLUSIONS
Although no validated measures exist that assess all aspects of transitional patient safety, we found validated measurement tools on specific aspects. Reporting of validity of transitional measurement tools was incomplete. Numerous outcome measures with unknown measurement properties are used in current studies on safety of care transitions, which makes interpretation or comparison of their results uncertain.
Topics: Humans; Patient Safety
PubMed: 29864119
DOI: 10.1371/journal.pone.0197312 -
Neurogastroenterology and Motility Sep 2012In the absence of a clear biomarker for irritable bowel syndrome (IBS), clinical criteria are used. In this study, we conduct a systematic review to examine the... (Review)
Review
BACKGROUND
In the absence of a clear biomarker for irritable bowel syndrome (IBS), clinical criteria are used. In this study, we conduct a systematic review to examine the validation and utilization of IBS criteria.
METHODS
A systematic review was performed in two stages. The first was a review of literature from 1978 validating IBS diagnostic criteria. The second stage of review was to select studies published in IBS between 1992 and 2011. This time period was divided into three segments (Rome I era from 1992 to 1999, Rome II era from 2000 to 2006, and Rome III era from 2007 to 2011). The number and type of study (RCT or other) and criteria used were evaluated for each era.
KEY RESULTS
The first stage of the systematic review identified only 14 published studies validating diagnostic tests for IBS (with three studies evaluating more than one criterion). There were eight validations for Manning, three validations for Kruis, four validations for Rome I, three validations for Rome II, and no validation for Rome III. In the second review of utilization of Rome criteria, only 25.7% of published IBS papers used Rome III criteria during the Rome III era (Rome II was used most in 64.8% of studies).
CONCLUSIONS & INFERENCES
This review identified that comparator groups varied widely between studies making comparison of criteria impossible. Manning criteria are the most valid and accurate criteria. More importantly, Rome III is not validated and is poorly adopted in clinical research trial enrollment.
Topics: Guideline Adherence; Humans; Irritable Bowel Syndrome; Reproducibility of Results
PubMed: 22632582
DOI: 10.1111/j.1365-2982.2012.01943.x -
Radiotherapy and Oncology : Journal of... Jun 2023Multiple outcome prediction models have been developed for Head and Neck Squamous Cell Carcinoma (HNSCC). This systematic review aimed to identify HNSCC outcome... (Review)
Review
Multiple outcome prediction models have been developed for Head and Neck Squamous Cell Carcinoma (HNSCC). This systematic review aimed to identify HNSCC outcome prediction model studies, assess their methodological quality and identify those with potential utility for clinical practice. Inclusion criteria were mucosal HNSCC prognostic prediction model studies (development or validation) incorporating clinically available variables accessible at time of treatment decision making and predicting tumour-related outcomes. Eligible publications were identified from PubMed and Embase. Methodological quality and risk of bias were assessed using the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) and prediction model risk of bias assessment tool (PROBAST). Eligible publications were categorised by study type for reporting. 64 eligible publications were identified; 55 reported model development, 37 external validations, with 28 reporting both. CHARMS checklist items relating to participants, predictors, outcomes, handling of missing data, and some model development and evaluation procedures were generally well-reported. Less well-reported were measures accounting for model overfitting and model performance measures, especially model calibration. Full model information was poorly reported (3/55 model developments), specifically model intercept, baseline survival or full model code. Most publications (54/55 model developments, 28/37 external validations) were found to have high risk of bias, predominantly due to methodological issues in the PROBAST analysis domain. The identified methodological issues may affect prediction model accuracy in heterogeneous populations. Independent external validation studies in the local population and demonstration of clinical impact are essential for the clinical implementation of outcome prediction models.
Topics: Humans; Bias; Head and Neck Neoplasms; Outcome Assessment, Health Care; Prognosis; Squamous Cell Carcinoma of Head and Neck
PubMed: 36934895
DOI: 10.1016/j.radonc.2023.109629