-
Nursing Open Aug 2023This study aimed to develop a tool to measure paediatric specialist nurses' core competencies and examine the scale's validity and reliability. (Review)
Review
AIM
This study aimed to develop a tool to measure paediatric specialist nurses' core competencies and examine the scale's validity and reliability.
DESIGN
An exploratory quantitative study.
METHODS
This study was performed in April 2022 with 302 paediatric specialist nurses in mainland China. Items were created through a literature review, a qualitative interview, and the Delphi method. The data were evaluated using descriptive statistics, independent sample t-test, explanatory factor analysis, Pearson correlation coefficient, Cronbach's alpha coefficient and split-half reliability.
RESULTS
The final scale consisted of five factors and 32 items. The factors were communication, coordination and judgement abilities; professional technology mastery ability; specialist knowledge mastery ability; medical-related processes; and evidence-based nursing competencies. The explained total variance of the five factors was 62.216%. The scale-level and item-level CVI of this scale was 1.00, and the mean CVR of the total scale was 0.788. The Pearson correlation coefficients of each dimension and the total dimension of the scale was 0.709-0.892, and within each dimension, it was 0.435-0.651. The Cronbach's α of this scale was 0.944, and its split-half reliability was 0.883.
Topics: Child; Humans; Reproducibility of Results; Psychometrics; Surveys and Questionnaires; China; Nurses
PubMed: 37040438
DOI: 10.1002/nop2.1745 -
BMC Pediatrics Oct 2023Early childhood self-regulation (SR) is key for many health- and education-related outcomes across the life span. Kindergarten age is a crucial period for SR...
BACKGROUND
Early childhood self-regulation (SR) is key for many health- and education-related outcomes across the life span. Kindergarten age is a crucial period for SR development, and within this developmental window, potential SR difficulties can still be compensated for (e.g., through interventions). However, efficient measurement of SR through brief, comprehensive, and easy-to-use instruments that identify SR difficulties are scarce. To address this need, we used items of an internationally applied kindergarten teacher questionnaire-the Early Development Instrument (EDI) - to develop and validate a specific SR measurement scale.
METHODS
The psychometric evaluation and validation of the selected SR-items was performed in data collected with the German version of the EDI (GEDI), in two independent data sets - (a) the development dataset, with 191 children, and b) the validation dataset, with 184 children. Both included three- to six-year-old children and contained retest and interrater reliability data. First, three independent raters-based on theory-selected items eligible to form a SR scale from the two SR-relevant GEDI domains "social competence" and "emotional maturity". Second, exploratory and confirmatory factor analysis using structural equation modeling examined the item structure across both data sets. This resulted in a defined SR scale, of which internal consistency, test-retest and interrater reliability, cross-validation, and concurrent validity using correlation and descriptive agreements (Bland-Altman (BA) plots) with an existing validated SR-measuring instrument (the Kindergarten Behavioral Scales) were assessed.
RESULTS
Confirmatory factor analysis across both data sets yielded the best fit indices with 13 of the GEDI 20 items initially deemed eligible for SR measurement, and a three-factor structure: a) behavioral response inhibition, b) cognitive inhibition, c) selective or focused attention (RMSEA: 0.019, CFI: 0.998). Psychometric evaluation of the resulting 13-item-GEDI-SR scale revealed good internal consistency (0.92), test-retest and interrater reliability (0.85 and 0.71, respectively), validity testing yielded stability across populations and good concurrent validity with the Kindergarten Behavioral Scales (Pearson correlation coefficient: mean 0.72, range 0.61 to 0.84).
CONCLUSIONS
The GEDI contains 13 items suitable to assess SR, either as part of regular EDI developmental monitoring or as a valid stand-alone scale. This short 13-item (G)EDI-SR scale may allow early detection of children with SR difficulties in the kindergarten setting in future and could be the basis for public health intervention planning. To attain this goal, future research should establish appropriate reference values using a representative standardization sample.
Topics: Child; Humans; Child, Preschool; Reproducibility of Results; Surveys and Questionnaires; Psychometrics; Social Skills; Self-Control
PubMed: 37845613
DOI: 10.1186/s12887-023-04334-1 -
Nursing Open Oct 2023This study aimed to develop the nurses' Work Values Scale (WVS) to determine how important certain values are for nurses and to psychometrically test the scale.
AIM
This study aimed to develop the nurses' Work Values Scale (WVS) to determine how important certain values are for nurses and to psychometrically test the scale.
DESIGN
Instrument development and validation study.
METHOD
A two-phase scale development process comprising item generation, scale improvement and psychometric property evaluation was used. In the first phase, scale items were identified. In the second phase, item and exploratory factor analyses were performed in Study 1, and confirmatory factor analysis, validity verification and reliability verification of the nurses' WVS were performed in Study 2.
RESULTS
As a result of the analysis, a scale of 30 items with four subdomains was developed. In convergent validity and reliability verification, it was shown that the nurses' WVS has acceptable validity and reliability.
NO PATIENT OR PUBLIC CONTRIBUTION
Patients or members of the public were not involved in this study.
Topics: Humans; Psychometrics; Reproducibility of Results; Factor Analysis, Statistical; Nurses
PubMed: 37518936
DOI: 10.1002/nop2.1950 -
Behavior Research Methods Sep 2023Rasch analysis is a procedure to develop and validate instruments that aim to measure a person's traits. However, manual Rasch analysis is a complex and time-consuming...
Rasch analysis is a procedure to develop and validate instruments that aim to measure a person's traits. However, manual Rasch analysis is a complex and time-consuming task, even more so when the possibility of differential item functioning (DIF) is taken into consideration. Furthermore, manual Rasch analysis by construction relies on a modeler's subjective choices. As an alternative approach, we introduce a semi-automated procedure that is based on the optimization of a new criterion, called in-plus-out-of-questionnaire log likelihood with differential item functioning (IPOQ-LL-DIF), which extends our previous criterion. We illustrate our procedure on artificially generated data as well as on several real-world datasets containing potential DIF items. On these real-world datasets, our procedure found instruments with similar clinimetric properties as those suggested by experts through manual analyses.
Topics: Humans; Psychometrics; Surveys and Questionnaires; Probability; Reproducibility of Results
PubMed: 36070131
DOI: 10.3758/s13428-022-01947-9 -
BMC Medical Education Mar 2024The introduction of competency-based education models, student centers, and the increased use of formative assessments have led to demands for high-quality test items to...
BACKGROUND
The introduction of competency-based education models, student centers, and the increased use of formative assessments have led to demands for high-quality test items to be used in assessments. This study aimed to assess the use of an AI tool to generate MCQs type A and evaluate its quality.
METHODS
The study design was cross-sectional analytics conducted from June 2023 to August 2023. This study utilized formative TBL. The AI tool (ChatPdf.com) was selected to generate MCQs type A. The generated items were evaluated using a questionnaire for subject experts and an item (psychometric) analysis. The questionnaire to the subject experts about items was formed based on item quality and rating of item difficulty.
RESULTS
The total number of recurrent staff members as experts was 25, and the questionnaire response rate was 68%. The quality of the items ranged from good to excellent. None of the items had scenarios or vignettes and were direct. According to the expert's rating, easy items represented 80%, and only two had moderate difficulty (20%). Only one item out of the two moderate difficulties had the same difficulty index. The total number of students participating in TBL was 48. The mean mark was 4.8 ± 1.7 out of 10. The KR20 is 0.68. Most items were of moderately difficult (90%) and only one was difficult (10%). The discrimination index of the items ranged from 0.77 to 0.15. Items with excellent discrimination represented 50% (5), items with good discrimination were 3 (30%), and only one time was poor (10%), and one was none discriminating. The non-functional distractors were 26 (86.7%), and the number of non-functional distractors was four (13.3%). According to distractor analysis, 60% of the items were excellent, and 40% were good. A significant correlation (p = 0.4, r = 0.30) was found between the difficulty and discrimination indices.
CONCLUSION
Items constructed using AI had good psychometric properties and quality, measuring higher-order domains. AI allows the construction of many items within a short time. We hope this paper brings the use of AI in item generation and the associated challenges into a multi-layered discussion that will eventually lead to improvements in item generation and assessment in general.
Topics: Humans; Cross-Sectional Studies; Education, Medical; Competency-Based Education; Psychometrics; Research Design
PubMed: 38448870
DOI: 10.1186/s12909-024-05250-3 -
PloS One 2024While measures to detect psychophysical olfactory ability are a crucial part of clinicians' assessment of potential olfactory loss, it gives no indication of how...
While measures to detect psychophysical olfactory ability are a crucial part of clinicians' assessment of potential olfactory loss, it gives no indication of how olfaction is experienced by the patient and these different aspects often deviate substantially. To ensure quality and reproducibility of subjectively reported olfactory experience and significance, the Importance of Olfaction Questionnaire (IO-Q) was introduced around a decade ago, and while initial validations have produced promising results, important aspects remain nearly unexamined. For example, the test-retest reliability has rarely been examined and the difference of online versus pen-and-paper administration remains unexplored. Here, we translated IO-Q to Danish and examined its validity, test-retest reliability and mode of administration. A cohort of 179 younger, Danish participants with a high level of English proficiency took the test twice with varying time in-between. The first test was taken digitally and in English, while the second was taken using pen-and-paper and in Danish. The distribution of scores and the relationship between the IO-Q and subscale scores were nearly identical between tests, indicating little to no influence of language/test modality in the sampled population. The internal consistency was comparable to previously published results. Likewise, an acceptable test-retest reliability was observed for the full IO-Q and slightly lower for subscales. No significant effect of time was found across several weeks. In conclusion, the IO-Q performed satisfactorily in all examinations and could therefore serve as a valuable clinical measure of subjective olfactory experience, and its Danish translation shows highly similar characteristics to the original, English version.
Topics: Humans; Smell; Reproducibility of Results; Surveys and Questionnaires; Physical Examination; Denmark; Psychometrics; Quality of Life
PubMed: 38241356
DOI: 10.1371/journal.pone.0269211 -
Assessment Oct 2023Despite growing attention surrounding impostor phenomenon (also known as "imposter syndrome"), recent reviews have suggested that current measures may be inadequate in... (Review)
Review
Despite growing attention surrounding impostor phenomenon (also known as "imposter syndrome"), recent reviews have suggested that current measures may be inadequate in capturing the complex and multifaceted nature of this construct. The objective of the current studies was to clarify the theoretical conceptualization of impostor phenomenon based on experiences in an achievement-oriented setting. We conducted a review of the literature and developed an item pool for a novel impostor phenomenon assessment (IPA) (Study 1). Exploratory factor analyses (Study 1) and confirmatory factor analyses (Study 2) assessed this initial item pool to determine the factor structure and initial psychometric properties of the preliminary IPA (Studies 2 and 3). Our findings offer preliminary evidence for the reliability and validity of the IPA as a novel measure of impostor phenomenon.
Topics: Humans; Psychometrics; Reproducibility of Results; Self Concept; Anxiety Disorders
PubMed: 36591960
DOI: 10.1177/10731911221141870 -
Nursing Open Aug 2023To identify the measures assessing the nursing image, and to analyse, evaluate and synthesize the psychometric properties of these available instruments.
AIM
To identify the measures assessing the nursing image, and to analyse, evaluate and synthesize the psychometric properties of these available instruments.
DESIGN
A systematic review of the psychometric properties of instruments assessing the nursing image. The system review has been registered in the PROSPERO database with the registration number CRD42020221511.
METHODS
The systematic review was conducted according to COSMIN guidelines. Search strategies were run in CINAHL, MEDLINE, APA PsycINFO, Web of Science, Scopus, COSMIN systematic review database, Cochrane Library and China National Knowledge Infrastructure (CNKI). Two reviewers independently screened the studies, extracting the psychometric properties of the included study characteristics and instruments. The methodological quality of the studies, instrument measurement performance, risk of bias and grade of evidence were evaluated and disagreements were resolved via discussion.
RESULTS
The search strategy yielded a total of 24 studies and 11 instruments assessing nursing image. None of the measures fulfilled all criteria in the COSMIN Risk of Bias checklist. The Nurses Self-Concept Instrument (NSCI) and the Nurses Self-Concept Questionnaire (NSCQ) were recommended for measuring the professional image of nursing, and the Nursing Brand Image Scale (NBIS) have the potential to be recommended for measuring a more comprehensive image of nursing. Every instrument evaluated in this review had different characteristics in the nursing image (e.g. the traditional image, the professional image, and the brand image). Therefore, the selection of the most appropriate instrument depends on the psychometrics, the context and the aim of the assessment.
Topics: Humans; Psychometrics; Surveys and Questionnaires; Dissent and Disputes; Checklist; Databases, Factual
PubMed: 37086148
DOI: 10.1002/nop2.1742 -
Scandinavian Journal of Caring Sciences Sep 2023Interaction between caller and telenurse in telenursing is important for caller satisfaction and subsequent compliance. Despite this, satisfaction measures with focus on...
AIMS AND OBJECTIVES
Interaction between caller and telenurse in telenursing is important for caller satisfaction and subsequent compliance. Despite this, satisfaction measures with focus on interaction in telenursing are scarce and rarely anchored in nursing theory. The aim was to evaluate the psychometric properties of the Telenursing Interaction and Satisfaction Scale (TISS) with focus on data quality, factor structure, convergent validity, and reliability.
METHODOLOGICAL DESIGN AND JUSTIFICATION
This psychometric study was based on cross-sectional data.
RESEARCH METHODS, INSTRUMENTS, AND/OR INTERVENTIONS
Callers to the National Medical Advisory Service in Sweden (n = 616) completed the 60-item Telenursing Interaction and Satisfaction Questionnaire based on Cox's Interaction Model of Client Health Behavior. Twenty-five of these items were selected to form the TISS in four subscales according to the model. Data quality was evaluated in terms of missing data patterns and score distributions. The factor structure was evaluated with confirmatory factor analysis for ordinal data, convergent validity with Spearman correlations, internal consistency with ordinal alpha, scale reliability with composite reliability coefficients, and test-retest reliability with intraclass correlations.
RESULTS
The amount of missing data was acceptable and equally distributed. Data deviated significantly from a normal distribution. All response options were endorsed. The factor analysis confirmed the hypothesised four-factor structure; factor loadings ranged from 0.56 to 0.97 and factor correlations were high (0.88-0.96). Internal consistency (ordinal alpha = 0.82-0.97), scale reliability (0.88-0.99), and test-retest reliability (ICC = 0.77-0.86) were satisfactory for all scales.
STUDY LIMITATIONS
The study design did not allow drop-out analysis.
CONCLUSIONS
The TISS showed satisfactory psychometric properties in the study sample. It provides a measure that enables quantitative measurement of caller satisfaction with interaction in telenursing.
Topics: Humans; Reproducibility of Results; Psychometrics; Telenursing; Cross-Sectional Studies; Surveys and Questionnaires; Personal Satisfaction
PubMed: 36718016
DOI: 10.1111/scs.13149 -
Journal of Neuromuscular Diseases 2024Patient-reported outcome measures (PROMs) are valuable in comprehensively understanding patients' health experiences and informing healthcare decisions in research and... (Review)
Review
Patient-reported outcome measures (PROMs) are valuable in comprehensively understanding patients' health experiences and informing healthcare decisions in research and clinical care without clinicians' input. Until now, no central resource containing information on all PROMS in neuromuscular diseases (NMD) is available, hindering the comparison and choice of PROMs used to monitor NMDs and appropriately reflect the patient's voice. This scoping review aimed to present a comprehensive assessment of the existing literature on using PROMs in children and adults with NMD. A scoping methodology was followed using Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) and COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guidelines to assess the literature on PROMs in NMDs. Eligibility criteria encompassed articles describing psychometric development or evaluation of generic or disease-specific PROM-based instruments for adults and children with specific NMDs. The data charting process involved extracting measurement properties of included PROMs, comprising validity, reliability, responsiveness, and interpretability information. The review identified 190 PROMs evaluated across 247 studies in individuals with NMDs. The majority of PROMs were disease specific. The physical functioning domain was most assessed. Validity was the most frequently investigated measurement property, with a limited number of PROMs sufficiently evaluated for a range of psychometric characteristics. There is a strong need for further research on the responsiveness and interpretability of PROMs and the development of PROMs on social functioning in NMD.
Topics: Humans; Patient Reported Outcome Measures; Neuromuscular Diseases; Psychometrics; Reproducibility of Results; Child; Quality of Life; Adult
PubMed: 38517800
DOI: 10.3233/JND-240003