All enthusiasm aside: Are we even on the right path with medical AI?
Written by: Derya Dogan
The dichotomy between the perception and the practical implementation of Artificial Intelligence (AI) in medicine currently presents a fascinating paradox. While the theoretical potential of AI in healthcare seems endless, its real-world application often differs from these expectations. The expectations and aspirations sometimes even border on the fantastical–compared to the real-world conditions.
But now that we have enhanced computational capabilities at hand, this has made previously theoretical machine learning (ML) applications practically possible at last. Natural language processing, in particular, holds great promise for advancing neuroscientific research. However, this rapid progression raises questions about the right integration of AI into medical scientific practice.
Figure 1: The Dutch research group used natural language processing (NLP) to generate insights into overlooked patterns and the interplay between certain components. Their approach is a novel contribution to neuroscientific research. Having a closer look at their methods and other similar approaches, certain questions arise, mainly about medical data complexity and the (mathematical) description of the human body.
Are we, perhaps, allowing our enthusiasm to outpace medical considerations, and potentially overlooking the nuanced specialties of medical data and its complexity in reality?
The current medical approach to study neurodegenerative disorders
Traditional research methods for neurodegenerative disorders primarily rely on neuroimaging techniques, biomarker studies, etc., with the primary goal to develop drugs for treatment. These methods aim to understand basic disease mechanisms, identify diagnostic markers, primarily in the immediate physiological vicinity by focusing on specified molecular targets or genetic factors. However, this approach faces challenges such as limited efficacy in slowing disease progression due to lack of complete knowledge of the human body and the specific disease, and an almost rigid focus on drug development .
The biggest part of usable data comes from research in clinical trials and with the primary goal to develop drugs:
Figure 2: The current drug development process incorporates a comprehensive pipeline from identifying unmet clinical needs to market approval and sales. This process primarily generates patient-derived clinical or demographical data, (experimental) chemical compound data, and empirical statistics. This approach may overlook other potentially significant physiological factors that could influence a physiological disorder.
Based on these data, we cannot definitively conclude that the disease trajectories derived are fully reliable. This limitation arises because the analysis focuses exclusively on lead compounds of pharmaceuticals and their direct interactions, without incorporating a holistic and systemic approach or a comprehensive observation of physiologically healthy and dysregulated elementary processes.
While it may be statistically justifiable to focus on empirically demonstrating the efficacy of lead compounds, valuable insights can also be gained by examining why certain compounds fail to work universally and understanding the mechanisms underlying these failures. These variations contain a wealth of information that could enhance medical knowledge and contribute to uncovering quasi-causal properties of disease mechanisms.
Data quantity, quality, availability…
The generation of medical data, particularly in clinical studies and diagnostics, often takes 12-15 years due to time-consuming steps like patient recruitment, data collection, regulatory approvals, and analysis. Emerging medical technologies, including real-time data exchange with advanced sensor technologies, aim to accelerate this process but are still under development.
Data availability and quality in the medical domain are often limited, which has an impact on the performance of novel technologies. [1] While the generation of synthetic data is increasing [2], its overall efficacy is still investigated. Some studies indicate that models trained on synthetic data may experience 'forgetting' of learned information. [3]
Rethinking the approach to medical data management and breaking away from the entrenched practice of maintaining institution- or purpose-dependent data silos is crucial for developing a more integrated and robust medical scientific research practice. By enabling the exchange of data across institutions, practitioners, and researchers, the overall volume, availability and utility of medical data can naturally expand. The establishment of critical technical and infrastructural prerequisites, such as the widespread adoption of standardized data formats, like HL7 FHIR or SNOMED CT [4], is a future-proof and efficient way of making this possible.
Could NLP lead to more insights in neuroscientific research?
Figure 3: The research team used donor files from the Netherlands Brain Bank (NBB), and conducted temporal profiling of the diagnoses. These diagnoses were then compared to the neuropathological diagnoses. In the last step they used a model that is specified on handling missing values to predict the disease progression. The progression was then specified into subtypes.
A Dutch research team aimed to advance neuroscientific research by using natural language processing (NLP) to predict clinical disease progression. To achieve this, they utilized data from the Netherlands Brain Bank (n=3,042), which included comprehensive medical record summaries. These records provided clinical diagnoses—identifying diseases based on symptoms, history, physical exams, and non-invasive tests—and neuropathological diagnoses, determined postmortem through microscopic and molecular brain tissue analysis.
Their goal was to generate a framework to identify the specific disease trajectory and the clinical subtype for more accurate diagnosis and personalized, targeted treatment, with the help of NLP.
But is it really that easy?
To answer this question, the data complexity in the medical field must be understood first, at the most elementary level, meaning on the level of elementary cellular processes with all components and interactions involved.
Figure 2 highlights that most of the data is generated from clinical trials, which consists of mosttly clinical and directly related unstructured data lacking context or broader physiological dependencies. This is important to keep in mind for the following arguments below.
Do we understand medical data complexity, fully, yet?
Figure 4: Currently mostly clinical anamnesis, brain imaging, blood analyses and neurographic analyses are conducted when diagnosing neurodegenerative diseases. The subtypes of, e.g., dementia mostly have overlapping symptoms or unspecific signs, which makes accurate diagnosis even harder. The red structure represents beta-amyloid plaque, a prominent marker for Alzheimer's Disease besides tau protein. In Vascular Dementia, the blood flow to the brain is reduced which leads to cognitive decline characteristic for dementia.
Medical data is distinct and cannot be directly compared to financial data, as it possesses unique properties that demand specialized consideration [5] :
- Data complexity and structure: Medical data often comprises diverse formats such as clinical notes, medical images, and genetic information, mostly unstructured. This complexity in medical data requires ML algorithms capable of processing heterogeneous and unstructured inputs.
- Data volume and quality: Medical datasets can be limited due to privacy concerns and the high cost of data collection. Additionally, medical data may contain missing or inconsistent information.
- Regulatory and ethical considerations: The use of medical data is subject to strict regulations to protect patient privacy and safety. These regulations necessitate that ML models in medicine not only perform accurately but also maintain transparency and fairness.
- Interpretability and clinical relevance: In healthcare, ML models must provide interpretable results that clinicians can trust and act upon. Medical applications demand transparency to ensure patient safety and facilitate clinical decision-making.
- Unbalanced data: in medicine, such as the underrepresentation of rare diseases, sex disparities, and differences between sexes in clinical research, often leads to biased outcomes. These biases can hinder the development of accurate diagnostics, treatments, and policies.
By gathering data from elementary biological and physiological processes, researchers could identify and address variability at a more granular level. This approach could minimize the influence of such imbalances and highlights universal mechanisms.
Understanding data complexity and strengthening interdisciplinary cooperation is not enough, though - the approach for data collection and data processing needs a revolution in the medical field.
Current practices focus on isolating mechanisms and compounds, a justifiable approach given constraints in resources, time, and computational capacity. Projecting into future times, though, without embracing holistic strategies and rethinking diagnostic frameworks, advancements risk creating an illusion of progress rather than tangible improvements in the physical reality of medicine. These considerations reflect current challenges within global healthcare and medicine.
Real-world impact through real-world data
Figure 5: Although already an integral part of neuroscientific research, the gut-brain axis is not studied elaborately in neurodegenerative diseases. Intracellular mechanisms and individual neurochemistry is also often neglected, but can contribute a big part to both the disease and the wellbeing of the patient. More knowledge about these could lead to more specific subclustering or even renewed context of neurodegenerative diseases.
These arguments may undermine the scientific contribution of the researchers, but like the core argument of this post, everything must be put in broader context.
Given the arguments above, the neuroscientific context that holds most potential for the usage of NLP are domains that integrate real-world data. Real-world data (RWD) are data relating to patient health status or the delivery of healthcare that are routinely collected from various sources outside of traditional clinical trials, in the real world, under real-world conditions. [6]
The properties of real-world data make the outcomes of ML models more reliable:
- Observational nature: Unlike randomized controlled trials, RWD is observational and not collected in experimental settings.
- Heterogeneity: RWD reflects outcomes in diverse patient populations and real-world settings.
- Volume: RWD often involves large and complex datasets, covering extensive time frames and diverse patient populations.
- Variety: RWD provides insights into a wide range of information types:
- Behavioral and lifestyle data
- Demographic information
- Financial data (e.g., from hospital and insurance)
- Administrative data (e.g., insurance claims)
- Social data (e.g., employment, family, social status)
- Sources: RWD can be derived from multiple sources, including:
- Electronic health records (EHRs)
- Insurance claims and billing data
- Patient and disease registries
- Patient-reported outcomes
- Mobile devices and wearables
- Pharmacy databases
- Social media and web-harvested data
The paper further provides valuable key findings, particularly in addressing the challenges of diagnosing neurodegenerative disorders, which only supports the points made above. One significant finding is the substantial number of misdiagnoses when comparing clinical diagnoses with postmortem neuropathological diagnoses. Up to a third of cases with a specific dementia were clinically misdiagnosed. This means that approximately 33% of dementia cases had a discrepancy between their clinical diagnosis and the postmortem neuropathological diagnosis. This highlights the complexity and heterogeneity of neurodegenerative disorders, which not only leads to clinical confusion but also contributes to trial failures in research, and when left unnoticed, can lead to the illusion of medical advancements.
The researchers Lam et al. [7] provide additional evidence supporting this argument. These conditions can be changed using a holistic approach, or using “systems biology”. A holistic systems biology approach enables the integration of multi-omics (in which the data sets are multiple "omes", such as the genome, proteome, transcriptome, epigenome, metabolome, and microbiome) data and informs discovery of biomarkers, (personalized) drug targets, and novel treatment strategies.
The “math of the human body”
Many aspects of the physical world can be described and predicted using mathematical models, as seen in physics, where phenomena are governed by well-defined and established equations that provide quantitative reliability. However, the human body lacks an equivalent comprehensive quantitative framework. While some physiological systems, such as cardiovascular dynamics or neural signaling, have partial mathematical representations, there is no unified "mathematics of human physiology" to fully describe and predict the complexities of the human body. Developing such an approach would require integrating biology, physiology, systems theory, and more.
An important and often overlooked aspect in the implementation of ML models is the necessity for regular updates to integrate recent findings. This aspect is not explicitly addressed in the current body of scientific literature, yet. The process of model refinement is likely to require significant computational resources and scientific expertise. However, these considerations are essential to capture the complexity and dynamic nature of the subject matter. It is possible that this will lead to bottlenecks in the development of ML models in medicine in the future.
Science ≠ politics?
Healthcare systems influenced by economic and political motives may generate artifacts that prioritize financial or healthcare economical efficiency over medical (diagnosis) accuracy. Such diagnoses can skew data: the datasets become unreliable, and reflect external pressures rather than genuine conditions of health and healthcare status. This mostly neglected aspect undermines research, predictive modeling, and the development of evidence-based medicine.
But someone must have thought about it already, right?
“Digital twins”–quantitative and data-based representations–of the human body are currently in early stages of development. Most research is still focusing on specific organs or body parts rather than a complete human model. Significant progress has been made in simulating organ functions. [8]
A comprehensive digital twin of the entire human body still remains a distant goal.
Achieving this level of detail first requires advanced sensor and laboratory technology to accurately monitor and analyze the intricate information patterns of the human body. Medical technology advancements may still take some time. In the meantime, we could pave the way for real medical breakthroughs by at least advancing our thinking.
We might never reach the elementary molecular processes of the human body in its entirety, but we must at least aim to reach as far down as possible.
References:
[1] Syed R, Eden R, Makasi T, Chukwudi I, Mamudu A, Kamalpour M, Kapugama Geeganage D, Sadeghianasl S, Leemans SJJ, Goel K, Andrews R, Wynn MT, Ter Hofstede A, Myers T. Digital Health Data Quality Issues: Systematic Review. J Med Internet Res. 2023 Mar 31;25:e42615. doi: 10.2196/42615.
[2] Gonzales A, Guruswamy G, Smith SR. Synthetic data in health care: A narrative review. PLOS Digit Health. 2023 Jan 6;2(1):e0000082. doi: 10.1371/journal.pdig.0000082. PMID: 36812604; PMCID: PMC9931305.
[3] Shumailov, Ilia & Shumaylov, Zakhar & Zhao, Yiren & Papernot, Nicolas & Anderson, Ross & Gal, Yarin. (2024). AI models collapse when trained on recursively generated data. Nature. 631. 755-759. 10.1038/s41586-024-07566-y.
[4] Bender, D., & Sartipi, K. (2013). HL7 FHIR: An Agile and RESTful approach to healthcare information exchange. Proceedings of the IEEE 26th International Symposium on Computer-Based Medical Systems, 326–331. DOI: 10.1109/CBMS.2013.6627810
[5] Deo RC. Machine Learning in Medicine. Circulation. 2015 Nov 17;132(20):1920-30. doi: 10.1161/CIRCULATIONAHA.115.001593.
[6] Klimek P, Baltic D, Brunner M, Degelsegger-Marquez A, Garhöfer G, Gouya-Lechner G, Herzog A, Jilma B, Kähler S, Mikl V, Mraz B, Ostermann H, Röhl C, Scharinger R, Stamm T, Strassnig M, Wirthumer-Hoche C, Pleiner-Duxneuner J. Quality Criteria for Real-world Data in Pharmaceutical Research and Health Care Decision-making: Austrian Expert Consensus. JMIR Med Inform. 2022 Jun 17;10(6):e34204. doi: 10.2196/34204.
[7] Simon Lam, Abdulahad Bayraktar, Cheng Zhang, Hasan Turkez, Jens Nielsen, Jan Boren, Saeed Shoaie, Mathias Uhlen, Adil Mardinoglu, A systems biology approach for studying neurodegenerative diseases, Drug Discovery Today, 25, 7, 2020, https://doi.org/10.1016/j.drudis.2020.05.010.
[8] Tang, C., Yi, W., Occhipinti, E. et al. A roadmap for the development of human body digital twins. Nat Rev Electr Eng 1, 199–207 (2024). https://doi.org/10.1038/s44287-024-00025-w
AI assistance in the creation of this post:
- Following ChatGPT prompts were applied:
“Proofread following blog post and check for right grammar and punctuation, do not change anything in the text.”
- Following prompt for the AI generated picture in Canva:
"scientific picture of a human head from the side and the ear develops into the fractal mandelbrot quantity"
Surprisingly, the ear was almost in an anatomically acceptable position.