-
JMIR Medical Informatics Jun 2024Integrating machine learning (ML) models into clinical practice presents a challenge of maintaining their efficacy over time. While existing literature offers valuable...
Integrating machine learning (ML) models into clinical practice presents a challenge of maintaining their efficacy over time. While existing literature offers valuable strategies for detecting declining model performance, there is a need to document the broader challenges and solutions associated with the real-world development and integration of model monitoring solutions. This work details the development and use of a platform for monitoring the performance of a production-level ML model operating in Mayo Clinic. In this paper, we aimed to provide a series of considerations and guidelines necessary for integrating such a platform into a team's technical infrastructure and workflow. We have documented our experiences with this integration process, discussed the broader challenges encountered with real-world implementation and maintenance, and included the source code for the platform. Our monitoring platform was built as an R shiny application, developed and implemented over the course of 6 months. The platform has been used and maintained for 2 years and is still in use as of July 2023. The considerations necessary for the implementation of the monitoring platform center around 4 pillars: feasibility (what resources can be used for platform development?); design (through what statistics or models will the model be monitored, and how will these results be efficiently displayed to the end user?); implementation (how will this platform be built, and where will it exist within the IT ecosystem?); and policy (based on monitoring feedback, when and what actions will be taken to fix problems, and how will these problems be translated to clinical staff?). While much of the literature surrounding ML performance monitoring emphasizes methodological approaches for capturing changes in performance, there remains a battery of other challenges and considerations that must be addressed for successful real-world implementation.
PubMed: 38941140
DOI: 10.2196/50437 -
Age and Ageing Jun 2024Incontinence is a common, distressing condition, most prevalent in older people. There is an unmet need for effective interventions to support continence. This review...
Effectiveness of non-pharmacological interventions delivered at home for urinary and faecal incontinence with homebound older people: systematic review of randomised controlled trials.
INTRODUCTION
Incontinence is a common, distressing condition, most prevalent in older people. There is an unmet need for effective interventions to support continence. This review focuses on non-pharmacological interventions to reduce incontinence among homebound older people. Aim: to identify interventions with potential to be delivered by care workers, nurses or family members in a person's home.
METHODS
Multiple databases were searched until 15 September 2023 for randomised controlled trials reporting home-based interventions for incontinence for older people (≥65 years) living at home. Two reviewers independently screened titles, abstracts and papers against inclusion criteria, then assessed for the Risk of Bias (RoB2). A third reviewer resolved the discrepancies. Primary data were extracted and synthesised.
RESULTS
A full-text review of 81 papers identified seven eligible papers (1996-2022, all USA), including n = 636 participants (561 women and 75 men). Two studies focusing on multicomponent behavioural interventions showed benefit, as did one study of transcutaneous tibial nerve stimulation self-administered through electrode-embedded socks. Three, which included cognitively impaired people, reported improvement with toileting assistance programmes, but the effects were not all significant. Results were inconclusive from a study examining the effects of fluid intake adjustments. Interventions were delivered by nurses, three in collaboration with family caregivers. No faecal incontinence interventions met the criteria.
CONCLUSION
There is scant evidence for continence supporting interventions delivered in older people's own homes. With an ageing population often reliant on family or social care workers well-placed to support continence promotion and policy drives for services to support older people remaining at home, this evidence gap needs addressing.
Topics: Humans; Fecal Incontinence; Aged; Randomized Controlled Trials as Topic; Urinary Incontinence; Homebound Persons; Home Care Services; Female; Male; Treatment Outcome; Aged, 80 and over
PubMed: 38941119
DOI: 10.1093/ageing/afae126 -
JAMA Network Open Jun 2024
PubMed: 38941103
DOI: 10.1001/jamanetworkopen.2024.15407 -
JAMA Network Open Jun 2024
PubMed: 38941101
DOI: 10.1001/jamanetworkopen.2024.19094 -
JAMA Network Open Jun 2024Understanding the cost of drug development can help inform the development of policies to reduce costs, encourage innovation, and improve patient access to drugs.
IMPORTANCE
Understanding the cost of drug development can help inform the development of policies to reduce costs, encourage innovation, and improve patient access to drugs.
OBJECTIVE
To estimate the cost of drug development by therapeutic class and trends in pharmaceutical research and development (R&D) intensity over time.
DESIGN, SETTING, AND PARTICIPANTS
In this economic evaluation study, an analytical model of drug development constructed using public and proprietary sources that collectively cover data from 2000 to 2018 was used to estimate the cost of bringing a drug to market, overall and for specific therapeutic classes. The analysis for the study was completed in October 2020.
MAIN OUTCOMES AND MEASURES
Three measures of development cost from nonclinical through postmarketing stages were estimated: mean out-of-pocket cost or cash outlay, mean expected cost, and mean expected capitalized cost. Pharmaceutical R&D intensity, defined as the ratio of R&D spending to total sales, from 2008 to 2019, based on the time frame for available data, was also analyzed.
RESULTS
The estimated mean cost of developing a new drug was approximately $172.7 million (2018 dollars) (range, $72.5 million for genitourinary to $297.2 million for pain and anesthesia), inclusive of postmarketing studies. The cost increased to $515.8 million when cost of failures was included. When the costs of failures and capital were included, the mean expected capitalized cost of drug development increased to $879.3 million (range, $378.7 million for anti-infectives to $1756.2 million for pain and anesthesia); results varied widely by therapeutic class. The pharmaceutical industry as a whole experienced a decline of 15.6% in sales but increased R&D intensity from 11.9% to 17.7% from 2008 to 2019. By contrast, R&D intensity of large pharmaceutical companies increased from 16.6% to 19.3%, whereas sales increased by 10.0% (from $380.0 to $418.0 billion) over the same 2008 to 2019 period, even though the cost of drug development remained relatively stable or may have even decreased.
CONCLUSIONS AND RELEVANCE
In this economic evaluation of new drug development costs, even though the cost of drug development appears to have remained stable, R&D intensity of large pharmaceutical companies remained relatively unchanged, despite substantial growth in revenues during this period. These findings can inform the design of drug-related policies and their potential impacts on innovation and competition.
Topics: Drug Development; United States; Humans; Drug Costs; Drug Industry; Pharmaceutical Research
PubMed: 38941099
DOI: 10.1001/jamanetworkopen.2024.15445 -
JAMA Network Open Jun 2024Air pollution is a recognized risk factor associated with chronic diseases, including respiratory and cardiovascular conditions, which can lead to physical and cognitive...
IMPORTANCE
Air pollution is a recognized risk factor associated with chronic diseases, including respiratory and cardiovascular conditions, which can lead to physical and cognitive impairments in later life. Although these losses of function, individually or in combination, reduce individuals' likelihood of living independently, little is known about the association of air pollution with this critical outcome.
OBJECTIVE
To investigate associations between air pollution and loss of independence in later life.
DESIGN, SETTING, AND PARTICIPANTS
This cohort study was conducted as part of the Environmental Predictors Of Cognitive Health and Aging study and used 1998 to 2016 data from the Health and Retirement Study. Participants included respondents from this nationally representative, population-based cohort who were older than 50 years and had not previously reported a loss of independence. Analyses were performed from August 31 to October 15, 2023.
EXPOSURES
Mean 10-year pollutant concentrations (particulate matter less than 2.5 μm in diameter [PM2.5] or ranging from 2.5 μm to 10 μm in diameter [PM10-2.5], nitrogen dioxide [NO2], and ozone [O3]) were estimated at respondent addresses using spatiotemporal models along with PM2.5 levels from 9 emission sources.
MAIN OUTCOMES AND MEASURES
Loss of independence was defined as newly receiving care for at least 1 activity of daily living or instrumental activity of daily living due to health and memory problems or moving to a nursing home. Associations were estimated with generalized estimating equation regression adjusting for potential confounders.
RESULTS
Among 25 314 respondents older than 50 years (mean [SD] baseline age, 61.1 [9.4] years; 11 208 male [44.3%]), 9985 individuals (39.4%) experienced lost independence during a mean (SD) follow-up of 10.2 (5.5) years. Higher exposure levels of mean concentration were associated with increased risks of lost independence for total PM2.5 levels (risk ratio [RR] per 1-IQR of 10-year mean, 1.05; 95% CI, 1.01-1.10), PM2.5 levels from road traffic (RR per 1-IQR of 10-year mean, 1.09; 95% CI, 1.03-1.16) and nonroad traffic (RR per 1-IQR of 10-year mean, 1.13; 95% CI, 1.03-1.24), and NO2 levels (RR per 1-IQR of 10-year mean, 1.05; 95% CI, 1.01-1.08). Compared with other sources, traffic-generated pollutants were most consistently and robustly associated with loss of independence; only road traffic-related PM2.5 levels remained associated with increased risk after adjustment for PM2.5 from other sources (RR per 1-IQR increase in 10-year mean concentration, 1.10; 95% CI, 1.00-1.21). Other pollutant-outcome associations were null, except for O3 levels, which were associated with lower risks of lost independence (RR per 1-IQR increase in 10-year mean concentration, 0.94; 95% CI, 0.92-0.97).
CONCLUSIONS AND RELEVANCE
This study found that long-term exposure to air pollution was associated with the need for help for lost independence in later life, with especially large and consistent increases in risk for pollution generated by traffic-related sources. These findings suggest that controlling air pollution could be associated with diversion or delay of the need for care and prolonged ability to live independently.
PubMed: 38941096
DOI: 10.1001/jamanetworkopen.2024.18460 -
JAMA Network Open Jun 2024While adults aged 80 years and older account for 70% of hip fractures in the US, performance of fracture risk assessment tools in this population is uncertain.
IMPORTANCE
While adults aged 80 years and older account for 70% of hip fractures in the US, performance of fracture risk assessment tools in this population is uncertain.
OBJECTIVE
To compare performance of the Fracture Risk Assessment Tool (FRAX), Garvan Fracture Risk Calculator, and femoral neck bone mineral density (FNBMD) alone in 5-year hip fracture prediction.
DESIGN, SETTING AND PARTICIPANTS
Prognostic analysis of 3 prospective cohort studies including participants attending an index examination (1997 to 2016) at age 80 years or older. Data were analyzed from March 2023 to April 2024.
MAIN OUTCOMES AND MEASURES
Participants contacted every 4 or 6 months after index examination to ascertain incident hip fractures and vital status. Predicted 5-year hip fracture probabilities calculated using FRAX and Garvan models incorporating FNBMD and FNBMD alone. Model discrimination assessed by area under receiver operating characteristic curve (AUC). Model calibration assessed by comparing observed vs predicted hip fracture probabilities within predicted risk quintiles.
RESULTS
A total of 8890 participants were included, with a mean (SD) age at index examination of 82.6 (2.7) years; 4906 participants (55.2%) were women, 866 (9.7%) were Black, 7836 (88.1%) were White, and 188 (2.1%) were other races and ethnicities. During 5-year follow-up, 321 women (6.5%) and 123 men (3.1%) experienced a hip fracture; 818 women (16.7%) and 921 men (23.1%) died before hip fracture. Among women, AUC was 0.69 (95% CI, 0.67-0.72) for FRAX, 0.69 (95% CI, 0.66-0.72) for Garvan, and 0.72 (95% CI, 0.69-0.75) for FNBMD alone (FNBMD superior to FRAX, P = .01; and Garvan, P = .01). Among men, AUC was 0.71 (95% CI, 0.66-0.75) for FRAX, 0.76 (95% CI, 0.72-0.81) for Garvan, and 0.77 (95% CI, 0.72-0.81) for FNBMD alone (P < .001 Garvan and FNBMD alone superior to FRAX). Among both sexes, Garvan greatly overestimated hip fracture risk among individuals in upper quintiles of predicted risk, while FRAX modestly underestimated risk among those in intermediate quintiles of predicted risk.
CONCLUSIONS AND RELEVANCE
In this prognostic study of adults aged 80 years and older, FRAX and Garvan tools incorporating FNBMD compared with FNBMD alone did not improve 5-year hip fracture discrimination. FRAX modestly underpredicted observed hip fracture probability in intermediate-risk individuals. Garvan markedly overpredicted observed hip fracture probability in high-risk individuals. Until better prediction tools are available, clinicians should prioritize consideration of hip BMD, life expectancy, and patient preferences in decision-making regarding drug treatment initiation for hip fracture prevention in late-life adults.
PubMed: 38941095
DOI: 10.1001/jamanetworkopen.2024.18612 -
JAMA Network Open Jun 2024Studies on the familial effects of body mass index (BMI) status have yielded a wide range of data on its heritability.
IMPORTANCE
Studies on the familial effects of body mass index (BMI) status have yielded a wide range of data on its heritability.
OBJECTIVE
To assess the heritability of obesity by measuring the association between the BMIs of fathers, mothers, and their offspring at the same age.
DESIGN, SETTING, AND PARTICIPANTS
This cohort study used data from population-wide mandatory medical screening before compulsory military service in Israel. The study included participants examined between January 1, 1986, and December 31, 2018, whose both parents had their BMI measurement taken at their own prerecruitment evaluation in the past. Data analysis was performed from May to December 2023.
MAIN OUTCOMES AND MEASURES
Spearman correlation coefficients were calculated for offsprings' BMI and their mothers', fathers', and midparental BMI percentile (the mean of the mothers' and fathers' BMI cohort- and sex-specific BMI percentile) to estimate heritability. Logistic regression models were applied to estimate the odds ratios (ORs) and 95% CIs of obesity compared with healthy BMI, according to parental BMI status.
RESULTS
A total of 447 883 offspring (235 105 male [52.5%]; mean [SD] age, 17.09 [0.34] years) with both parents enrolled and measured for BMI at 17 years of age were enrolled in the study, yielding a total study population of 1 343 649 individuals. Overall, the correlation between midparental BMI percentile at 17 years of age and the offspring's BMI at 17 years of age was moderate (ρ = 0.386). Among female offspring, maternal-offspring BMI correlation (ρ = 0.329) was somewhat higher than the paternal-offspring BMI correlation (ρ = 0.266). Among trios in which both parents had a healthy BMI, the prevalence of overweight or obesity in offspring was 15.4%; this proportion increased to 76.6% when both parents had obesity and decreased to 3.3% when both parents had severe underweight. Compared with healthy weight, maternal (OR, 4.96; 95% CI, 4.63-5.32), paternal (OR, 4.48; 95% CI, 4.26-4.72), and parental (OR, 6.44; 95% CI, 6.22-6.67) obesity (midparent BMI in the ≥95th percentile) at 17 years of age were associated with increased odds of obesity among offspring.
CONCLUSIONS AND RELEVANCE
In this cohort study of military enrollees whose parents also underwent prerecruitment evaluations, the observed correlation between midparental and offspring BMI, coupled with a calculated narrow-sense heritability of 39%, suggested a substantive contribution of genetic factors to BMI variation at 17 years of age.
Topics: Humans; Body Mass Index; Male; Female; Israel; Adolescent; Obesity; Cohort Studies; Adult; Fathers
PubMed: 38941093
DOI: 10.1001/jamanetworkopen.2024.19029 -
JAMA Health Forum Jun 2024
Topics: Artificial Intelligence; Humans; Patient Safety; Hospitals
PubMed: 38941085
DOI: 10.1001/jamahealthforum.2024.1369 -
Journal of General Internal Medicine Jun 2024Electronic consultations (eConsults) enable asynchronous consultation between primary care providers (PCPs) and specialists. eConsults have been used successfully to...
BACKGROUND
Electronic consultations (eConsults) enable asynchronous consultation between primary care providers (PCPs) and specialists. eConsults have been used successfully to manage a variety of conditions and have the potential to help PCPs manage polypharmacy and promote deprescribing.
OBJECTIVE
To elicit clinician perspectives on barriers/facilitators of using eConsults for deprescribing among older adults within a university health network.
DESIGN
Semi-structured interviews.
PARTICIPANTS
PCPs, geriatricians, and pharmacists.
APPROACH
We used the COM-B (Capability, Opportunity, Motivation, and Behavior) model to structure the interview guide and qualitative analysis methods to identify barriers/facilitators of (1) deprescribing and (2) use of eConsults for deprescribing.
KEY RESULTS
Of 28 participants, 19 were PCPs (13 physicians, 4 residents, 2 nurse practitioners), 7 were geriatricians, and 2 were pharmacists. Barriers and facilitators to deprescribing: PCPs considered deprescribing important but identified myriad barriers (e.g., time constraints, fragmented clinical care, lack of pharmacist integration, and patient/family resistance). Use of eConsults for deprescribing: Both PCPs and geriatricians highlighted the limits of contextual information available through electronic health record (vs. face-to-face) to render specific and actionable eConsults (e.g., knowledge of prior deprescribing attempts). Participants from all groups expressed interest in a targeted process whereby eConsults could be offered for select patients based on key factors (e.g., polypharmacy or certain comorbidities) and accepted or declined by PCPs, with pithy recommendations delivered in a timely manner relative to patient appointments. This was encapsulated by one PCP: "they need to be crisp and to the point to be helpful, with specific suggestions of something that could be discontinued or switched…not, 'hey, did you know your patient is on over 12 medicines?'".
CONCLUSIONS
Clinicians identified multifaceted factors influencing the utility of eConsults for deprescribing among older adults in primary care. Deprescribing eConsult interventions should be timely, actionable, and mindful of limitations of electronic chart review.
PubMed: 38941059
DOI: 10.1007/s11606-024-08899-0