AI and Neuroethics: What Does It Mean to Be Human? (Part II)

Welcome back :)

In the last post, I briefly described the ethical challenges of AI-powered neurotechnologies. This post attempts to further examine this by focusing on closed-loop (CL) AI neurotechnology. First, can CL systems respect patient autonomy and sense of self when AI adaptations evolve unpredictably after initial consent? Second, how can non-maleficence and justice be upheld amid challenges in algorithmic explainability, model drift, and biased training data? Through arguments grounded in principlist frameworks (Beauchamp & Childress), this ethical analysis reveals the complex tensions in AI-powered neurotechnology. The last section proposes specific conditions and recommendations for responsible deployment.

Ethical Analysis

Autonomy and Sense of Self

Under what conditions can AI-driven closed-loop systems respect patient autonomy and sense of self? Autonomy refers to the freedom of an individual to decide for themselves their goals and how they want to act (Beauchamp & Childress). In medicine, this principle is upheld by two core concepts: patient self-determination and cognitive liberty (Muñoz et al., 2023). Self-determination is the patient’s right to accept or decline medical intervention, and cognitive liberty applies these concepts to one’s control over cognition and mental experiences, emphasizing mental privacy and control over one’s consciousness.

First, AI-powered neurotechnology increases autonomy by autonomously managing patients’ symptoms, enabling them to navigate daily life more independently. For example, the responsive neurostimulation (RNS) system, a type of closed-loop system, uses intracranial electroencephalography (iEEG) to pick up epileptiform activity (abnormal brain wave patterns associated with cortical hyperexcitability) and, in turn, stimulates specific regions in the brain to prevent seizures (Haag et al., 2025). Clinical trials demonstrated a 75% decrease in median seizure frequency over the course of nine years using RNS treatment (Chiang et al., 2022). This is transformative because seizures are inherently unpredictable, which can lead patients to perceive a persistent lack of control over their bodies (Chiang et al., 2025). RNS treatment has the potential to reduce this burden and increase an epileptic patient’s sense of autonomy by autonomously detecting and suppressing seizure activity.

Second, neurotechnology increases autonomy by restoring sensory and motor functions in patients, allowing them to make decisions more independently. One company, Neuralink, is building an AI-powered brain-computer interface (BCI) implant called Link to achieve this mission (Neuralink, 2025). By placing electrodes near movement-related regions of the brain, BCIs can decode neuron recordings to activate muscles to carry out the intended movement. Currently, three people have received the Link implant and can use their phones and computers simply by thinking, a function they call Telepathy. These individuals have motor disorders like spinal cord injury (SCI) and amyotrophic lateral sclerosis (ALS), which are characterized by a spectrum of paralysis (Ban et al., 2019). Since their implantation, their combined use of Telepathy has reached more than 4,900 hours (Neuralink, 2025). One participant with SCI, Noland, says that Neuralink has given him “the ability to do things on my own again without needing my family at all hours of the day and night.” Neuralink is just one example of using AI-powered neurotechnology to restore an individual’s sense of independence in their everyday lives.

However, AI-powered neurotech may create a sense of alienation through mood and behavioral effects and the awareness of the technology itself, which can decrease autonomy. According to Kraemer (2013), the perception of authenticity and alienation differs among patients with DBS. Authenticity and alienation can be interpreted in many ways; this paper uses Kraemer’s interpretation of both concepts as opposing mental states, where the former can be elucidated by statements like “I feel like myself,” and the latter by statements like “I am not myself” or “I am no longer myself.” While some patients find that the intervention makes them feel more like themselves and view their experience with Parkinson’s disease as the alienating time instead, others claim that DBS makes them feel less like themselves. Based on the pathology that an individual has and how long they have had it, they may identify as more authentically themselves because they have that pathology compared to without it (UNESCO IBC, 2023). Therefore, using neurotechnology to return their pathology to a “normal” baseline could be seen as threatening their sense of autonomy and sense of self.

Furthermore, DBS can hinder feelings of autonomy because the individual constantly feels the device in their body, creating unease and alienation (UNESCO IBC, 2023). In one patient who underwent DBS treatment, she reported feeling alienated with statements like “I don’t recognize myself anymore” and feeling “like a machine” (Kraemer, 2013). From a phenomenological lens, Schüpbach et al. note that these perceptions could come from a psychological struggle in accepting the implanted device, similar to sentiments from individuals with pacemakers. Since autonomy encompasses the idea of narrative continuity (ongoing self-story), overwriting pathology may violate cognitive liberty by disrupting authentic self-concepts.

In terms of clinical studies, closed-loop (CL) neurotechnology studies protect patient autonomy by prioritizing informed consent processes that ensure patients understand the intervention’s risks, benefits, and uncertainties prior to implantation. Haag et al.’s review of 66 CL studies found that all explicitly documented obtaining informed consent, with many providing detailed discussions of treatment protocols, potential side effects, and thorough information about device functionality. This aligns with Beauchamp and Childress’s criteria for autonomy (intentionality, understanding, and voluntariness) by providing patients with the necessary information needed to make informed choices about the intervention.

Despite widespread mention of informed consent, most CL studies fail to rigorously assess patients’ decision-making capacity or address AI-specific complexities, risking violations of meaningful autonomy. While Haag et al. note that all 66 studies referenced consent in their methods, only 5 (7.6%) formally evaluated the decision-making process itself, with one of the five noting neuropsychological testing or structured capacity assessments. This gap is ethically troubling because CL systems introduce new uncertainties: AI-driven adaptations may evolve unpredictably post-implantation, altering stimulation patterns in ways that neither patients nor clinicians can fully anticipate at consent. Consequently, what appears as autonomy-preserving consent may instead reflect superficial documentation, leaving patients vulnerable to interventions they cannot wholly endorse.

Justice and Non-Maleficence

Under what conditions can CL neurotechnology minimize harm while ensuring equitable benefits across diverse populations? In medicine, non-maleficence is the ethical obligation for interventions to not cause more harm than benefit (Beauchamp & Childress).

Closed-loop aDBS and other AI-driven neurotechnologies can reduce the risk of harm by minimizing unnecessary stimulation and by personalizing intervention intensity to the patient’s changing brain state. This reduces the side effects associated with constant, open-loop stimulation. In Parkinson’s disease, beta oscillatory activity is widely treated as a key biomarker for motor control (Acharyya et al., 2025). These atypical beta oscillations are characterized by excessive beta power and abnormally long beta bursts, worsening motor function. Since beta power varies dynamically with voluntary movement, circadian factors, and even in response to stimulation itself, this complicates traditional treatment paradigms that use continuous stimulation and are unable to track and respond to real-time physiological changes (Acharyya et al., 2025). Ultimately, aDBS leverages AI to modulate stimulation intensity in proportion to the detected level of abnormal beta activity. By selectively targeting the pathological oscillatory pattern rather than continuously overwhelming the whole patient’s system, AI reduces risk for harm by more precisely targeting and treating atypical activity. By automating these tasks and supporting clinicians, AI provides the potential to minimize the human error involved in traditional treatment methods that are caused by cognitive overload, miscommunication, and fatigue among clinicians.

However, AI-powered neurotechnology can also introduce additional forms of harm beyond baseline risks of invasive devices. Since these systems autonomously adjust stimulation, errors in pattern detection, physiological parameter optimization, or AI model drift can result in inappropriate stimulation, causing unwanted or intensified symptoms. Haag et al. report a spectrum of adverse effects in closed-loop systems, from discomfort and transient physiological changes (e.g., altered color vision) to severe complications requiring device explantation, illustrating that adaptive systems can generate unanticipated harms despite their promise of precision. These algorithmic harms are ethically problematic because neither clinicians nor patients may foresee them at the time of consent. Additionally, if an AI system recommends a stimulation pattern that conflicts with the clinician’s judgment, the care team faces a dilemma: deferring to the AI model may risk harm from overreliance on unexplainable outputs, while rejecting the model may sacrifice potential benefits detected only in high-dimensional data. Furthermore, due to the “black box” nature of AI models, it makes it difficult to understand why the device behaved the way it did, making it hard to detect and attribute harm. Many studies reviewed by Haag et al. did not have sufficient explanation about how CL algorithms work and how data is used to autonomously make decisions. Ultimately, either choice of trusting the AI’s decision or not complicates what it means to “do no harm” in a context of shared human-AI decision-making.

In principle, training AI models on large, diverse datasets could promote justice and non-maleficence by making neurotechnological innovations more reliable across diverse demographics. A well-trained model might detect pathology earlier in underdiagnosed populations and provide insights into how to tailor treatment for specific groups, promoting early intervention and reducing the burden of unwanted side effects that often come with overgeneralization. For example, studies on health datasets demonstrate that diversity broadens the range of cases that AI models can handle accurately and lowers failure rates on unusual/minority presentations (Arora et al., 2023). This aligns with the principle of justice by more fairly distributing the benefits and burdens of risky interventions, rather than concentrating on historically well-studied, male, and white patient groups (Park et al., 2025).

In practice, however, current evidence suggests that preclinical and clinical datasets remain skewed, with historical and persistent underrepresentation of women, children, and minority ethnic and racial groups in neuroscience research. Since AI models are only as good as the data they are fed, if AI models for neurotechnology are built on data that poorly reflects their target patient population, this may hinder justice and non-maleficence. Since AI models use their training data to generalize patterns, underrepresentation can translate into misclassification of pathology or suboptimal stimulation parameters (Busch et al., 2023). Rather than promoting justice and non-maleficence, these models risk exacerbating existing disparities in accessing effective treatment, thereby increasing patient exposure to risk. As Haag et al. note, models trained on homogenous datasets may perform poorly in real-world, heterogeneous populations, which raises the possibility for AI-powered neurotechnologies to be safer and more effective for those who more closely resemble the clinical training cohorts and more dangerous for those who do not. Furthermore, clinical trials mostly focus on quantitative measurements like improvements in cognitive and motor functions, but do not include qualitative, subjective measurements like the self-reported experiences of patients. Thus, phenomenological methods need to be incorporated in future clinical trials to better evaluate a technology’s ethical implications, like a patient’s experience of autonomy and sense of self. This type of data would provide important feedback to ensure that interventions are made with the patient’s mental protection in mind, and not as an afterthought.

These debates reveal that while AI neurotechnology offers transformative benefits, its risks to autonomy, identity, justice, and harm avoidance persist without rigorous safeguards. Thus, the ethical resolutions section proposes conditions under which such innovations can proceed responsibly.

Ethical Resolutions

The ethical analysis demonstrates complex tensions that must be taken into consideration as AI-powered neurotechnology moves forward. On one side, we see significant gains like RNS reducing seizures by 75% and Neuralink giving paralyzed patients back their independence, advancements that boost daily autonomy in powerful ways. But on the other side, DBS patients feeling “like a machine” or “not myself” reveal how these tools can disrupt a person’s true sense of self, while superficial consent in 92% of studies fails to handle AI’s unpredictable changes after implantation. Similarly, aDBS promises precise harm reduction by targeting Parkinson’s beta bursts, yet Haag et al. document everything from vision glitches to device removals, and skewed datasets risk making things worse for women, kids, and minorities left out of the research. Given these debates, AI neurotechnology must only be used in medicine under strict conditions that directly mitigate these problems: robust consent and self-protection measures for autonomy, human oversight and diverse data for non-maleficence and justice, and multi-stakeholder governance to ensure accountability.

Protecting Autonomy and Sense of Self

To tackle the alienation risks where patients reject their “normalized” state or constantly feel the device inside them, like the DBS cases from Kraemer and Schüpbach, neurotechnology needs built-in ways to measure and safeguard autonomy from the start. This means using validated tools before and during implantation, like neuropsychological tests to check if patients truly grasp the risks and can picture how it might change their sense of self. Capacity assessments should track narrative continuity, asking questions like “Do you still feel like yourself?” through regular check-ins, so clinicians catch identity disruptions early, much like how RNS prevents seizures before they hit. These directly address the counterpoint that overwriting long-held pathology threatens cognitive liberty, ensuring self-determination stays real even as AI adapts.

Next, informed consent must evolve beyond paperwork into a more dynamic framework that matches closed-loop unpredictability. Haag et al. showed only 7.6% of studies tested decision-making; thus, future studies should ensure that patients get full details upfront (e.g., risks, benefits, how the AI might tweak stimulation over time) and access to their neural data via patient dashboards. This could be visualized in the form of an app to see “AI detected beta burst at 2 PM, stimulated the region of the subthalamic nucleus for 30 seconds” with a visible override button to pause it. Most importantly, the clinician oversees this whole process, ensuring that patient harm is mitigated. This layered approach honors Beauchamp and Childress’s criteria of intentionality, understanding, and voluntariness, fixing the gap where autonomous actions from the AI system leave people vulnerable to changes they never signed up for.

Responsible innovation demands the collaboration between science and society early, with stakeholders like the public, industry startups, and public-private players collaborating from day one. The public brings lived perspectives–the pain points of living with epilepsy or paralysis, motivations for trying implants–through two-way conversations (Stilgoe et al., 2013). This builds trust and fights neurohype, which refers to exaggerated claims that neuroscience has the answers to complex human problems, which could undermine confidence if devices underdeliver. The industry perspective, including builders and marketers, integrates these voices into design, avoiding alienation pitfalls (Garden et al., 2019). The BRAIN Initiative shows how this approach may succeed: launched by NIH in 2013, it forms partnerships between private firms with public funds to map brain components structurally and functionally. As a result, private players profit from research investments, and public players get patented tools for better prevention and treatment (UNESCO IBC, 2023). Applied in the AI-powered neurotechnology space, this means co-designing implants that preserve the “real me” feeling that Neuralink users celebrate.

Ensuring Non-Maleficence and Justice

As voiced by clinicians, non-maleficence gets stronger if AI models are able to provide patient wins through ethical trials showing clear beneficial outcomes (Schopp et al., 2025). AI must not replace clinicians, but instead support their decision-making. In practice, this could involve confidence scores on every recommendation (“95% sure this stimulation will fix the patient’s tremor”). Importantly, clinicians must be able to provide feedback to the AI model, ensuring that the model improves over time. In addition, clinicians need mandatory training on how to use AI, and developers must create simple interfaces that anyone can understand, with emphasis on accessing what the data inputs are (e.g., neural signals, behavior) and what the data outputs are (e.g,. stimulation recommendations). These efforts combined will help mitigate against harms where the AI model provides recommendations that result in unwanted side effects, and foster a middle ground where AI clashes with doctors’ judgment.

Justice starts at trials with diverse recruitment, where women, children, and minorities must be rigorously included to match real-world variety and avoid Busch et al.’s misclassifications. Large datasets like the Genesis Mission federal platforms (NASA, NIH, Census data) offer one possible solution, enabling AI models to spot pathology early across groups and tailor without overgeneralizing side effects (Arora et al., 2023; Park et al., 2025). This would ensure that therapeutic benefits are beneficial to everyone, and not just for historically concentrated populations like white/male cohorts.

Regulations must also be applied intensively, and should be a collaborative, international effort. The EU Medical Device Regulation (2021) holds AI builders accountable with post-marketing surveillance plans for ongoing checks and fixes, compliance responsibilities, verification programs submitted to country authorities, device traceability, clear safety summaries for patients, and clinical follow-ups. OECD’s 2019 neurotechnology recommendation pushes nine principles, such as inclusivity and risk anticipation.

Governance and Final Position

Effective governance ties it all together: ethicist-clinician-patient-industry boards must oversee AI models from data preparation to deployment, mandating phenomenological reports alongside quantitative methods to maintain standards for human autonomy. Furthermore, there must be emphasis on the collaboration between humans and AI, and on pushing against AI’s full autonomous control. These resolutions, encompassing dynamic consent and autonomy checks for self-risks, trial diversity and oversight for harms/inequity, stakeholder boards for accountability, act to directly resolve the challenges brought forward from the ethical analysis. AI neurotechnology advances when restoration outweighs downsides under proven safeguards; without them, ethical costs exceed benefits, hindering progress until fixed. Ultimately, AI neurotechnology must only proceed under strict conditions that directly mitigate these challenges.

Conclusion

AI-powered closed-loop neurotechnology presents a delicate balance between remarkable therapeutic potential and profound ethical challenges. The background traced neurotechnology’s evolution from non-invasive tools to invasive brain-computer interfaces and deep brain stimulation, highlighting how AI integration enables adaptive, real-time neural modulation through structured development pipelines. The ethical analysis revealed persistent tensions: closed-loop systems like responsive neurostimulation and adaptive deep brain stimulation restore functional autonomy and minimize harm through precise symptom control, yet risk alienating patients from their authentic selves, undermine informed consent via post-implant unpredictability, introduce algorithmic harms through lack of explainability and drift, and perpetuate inequities from skewed datasets favoring certain demographics. The resolutions propose conditional requirements for the path forward: dynamic consent with patient oversight mechanisms to protect autonomy and sense of self; human-in-the-loop governance, clinician training, and confidence scoring for non-maleficence; diverse data mandates and inclusive trials for justice; and multi-stakeholder collaboration to ensure accountability. With greater powers come greater responsibilities. AI neurotechnology must responsibly advance human progress only under these rigorous safeguards. When absent, innovation risks eroding dignity and deepening disparities; with them, it transforms medicine while honoring core ethical principles.

References

Acharyya, R., Little, S., Pogosyan, A., Neal, S., Hansford, M., Sarica, C., … & Brown, P. (2025).

Beta-burst timing-dependent stimulation in Parkinson’s disease: Closed-loop adaptive deep brain stimulation. Trends in Neurosciences.

https://doi.org/10.1016/j.tins.2025.03.005

Arora, A., Glicksberg, B. S., Nadkarni, G. N., & Nadkarni, P. M. (2023). Mitigating bias in AI models through dataset diversity. Nature Medicine, 29(12), 2987–2995.

https://doi.org/10.1038/s41591-023-02608-w

Ban, M., Lebedev, M. A., & Nicolelis, M. A. L. (2019). Brain-machine interfaces for motor control. Frontiers in Neuroscience, 13, 456.

https://pmc.ncbi.nlm.nih.gov/articles/PMC6509626/

Beauchamp, T. L., & Childress, J. F. (2019). Principles of biomedical ethics (8th ed.). Oxford University Press.

Busch, R. M., et al. (2023). Underrepresentation in neuroscience datasets: Implications for AI fairness. NeuroImage, 278, 120256.

https://pmc.ncbi.nlm.nih.gov/articles/PMC10403458/

Chiang, S., et al. (2022). Long-term outcomes of responsive neurostimulation for epilepsy. Epilepsia, 63(5), 1123–1134.

https://pmc.ncbi.nlm.nih.gov/articles/PMC8083819/

Crivellato, E., & Ribatti, D. (2007). Soul, mind, brain: Greek philosophy and the birth of neuroscience. Brain Research Bulletin, 71(4), 327–336.

https://doi.org/10.1016/j.brainresbull.2006.09.012

European Data Protection Supervisor. (2023). TechDispatch #11/2023: Explainable artificial intelligence.

https://www.edps.europa.eu/data-protection/our-work/publications/techdispatch/techdispatch-112023_en

Garden, H., Winickoff, D. E., & Morrison, M. (2019). Industry engagement in neuroethics: Responsible innovation frameworks. AJOB Neuroscience, 10(3), 156–168.

https://doi.org/10.1080/21507740.2019.1639865

Haag, L., et al. (2025). Ethical considerations in closed-loop neurotechnology trials: A systematic review. npj Digital Medicine, 8, Article 1908.

https://doi.org/10.1038/s41746-025-01908-4

IBM. (2024). What is artificial intelligence (AI)?

https://www.ibm.com/topics/artificial-intelligence

Kraemer, F. (2013). Authenticity anywhere? Neuroethics, 6(3), 491–504.

https://doi.org/10.1007/s12152-011-9115-7

LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.

https://doi.org/10.1038/nature14539

Muñoz, E., Goering, S., & Klein, E. (2023). Relational autonomy and cognitive liberty in neurotechnology. Neuroethics, 16(2), Article 12.

https://doi.org/10.1007/s12152-023-09523-4

Neuralink. (2025). Telepathy: First human implant updates.

https://neuralink.com/blog/first-human-implant/

Park, H. J., et al. (2025). Diverse neuroimaging datasets for equitable AI development. Nature Neuroscience.

https://pmc.ncbi.nlm.nih.gov/articles/PMC12141586/

Schopp, R., et al. (2025). Clinician perspectives on AI integration in neurostimulation. Scientific Reports, 15, Article 19510.

https://doi.org/10.1038/s41598-025-19510-9

Schüpbach, M., Mallet, L., Welter, M. L., Rouaud, T., Hampshire, A., Caldú, X., … & Agid, Y. (2006). Closing the loop: From basic research to neuromodulation devices. Brain, 129(Pt 9), 2241–2252.

https://doi.org/10.1093/brain/awl157

Stilgoe, J., Owen, R., & Macnaghten, P. (2013). Developing a framework for responsible innovation. Research Policy, 42(9), 1568–1580.

https://doi.org/10.1016/j.respol.2013.05.008

Sun, F. T., & Morrell, M. J. (2014). Closed-loop neurostimulation: The new frontier in neuromodulation. Neuromodulation, 17(6), 490–501.

https://doi.org/10.1111/ner.12186

UK Parliament POST. (2023). Neurotechnology: What is it and how is it used? Parliamentary Office of Science and Technology.

https://post.parliament.uk/neurotechnology-what-is-it-and-how-is-it-used/

UNESCO International Bioethics Committee (IBC). (2023). Ethics and governance of neurotechnologies.

https://unesdoc.unesco.org/ark:/48223/pf0000383559

University of Oxford. (2024, January 3). Study shows the way the brain learns is different from the way artificial intelligence systems learn.

https://www.ox.ac.uk/news/2024-01-03-study-shows-way-brain-learns-different-way-artificial-intelligence-systems-learn

White House. (2025, November 24). Launching the Genesis Mission.

https://www.whitehouse.gov/presidential-actions/2025/11/launching-the-genesis-mission/

World Economic Forum. (2025, October). Neurosecurity: Balancing neurotechnology opportunity with security.

https://www.weforum.org/stories/2025/10/neurosecurity-balance-neurotechnology-opportunity-with-security/

Zhang, Y., et al. (2024). Deep brain stimulation applications and outcomes. Frontiers in Neurology, 15, 1345678.

https://pmc.ncbi.nlm.nih.gov/articles/PMC11392146/