ADHD (Hyperactivity) Detection in Children by Smartwatch

Co-authors: Karan Ahuja, Kristie Mak, Gwendolyn Thompson, Sam Shaaban, Oliver Lindhiem, Mayank Goel

At ACM IMWUT 2023, Riku Arakawa (Carnegie Mellon University) published a paper entitled “LemurDx: Using Unconstrained Passive Sensing for an Objective Measurement of Hyperactivity in Children with no Parent Input,” which was done at CMU, University of Pittsburgh, NuRelm, and University of Pittsburgh Medical Center.

We proposed a framework for detecting Hyperactivity, a core symptom of ADHD, using a smartwatch. In this article, we would like to introduce it briefly.

This work was partially funded by a National Institute of Mental Health grant (R41MH119644) and a Pitt Innovation Challenge Award (2020) from the University of Pittsburgh. Part of the results were presented in our pilot study [1].

1. Background

Attention-Deficit Hyperactivity Disorder (ADHD) is often seen in children and adults and causes difficulties in daily life. There are presentations of ADHD, including inattentiveness, hyperactivity, and impulsivity. Hyperactivity is especially common in young children with ADHD [2]. The issue addressed in this work is that there is no objective measure of hyperactivity. Currently, most adults around children, such as parents and teachers, answer questionnaires from which numerical values are calculated, but these are not objective indicators because they are time-consuming and subject to bias.

Therefore, this project aims to quantify hyperactivity using a ubiquitous wearable device — a smartwatch.

2. Related Work

Several studies attempted to detect hyperactivity using wristwatch-type devices. For example, Lin et al. [3] compared motion data from seated children in a classroom and found large differences between hyperactive and non-hyperactive children.

Many of these previous studies compared data in a specific context or the same environment. In this study, we aimed to extract useful information from the data of children who spent a day freely without contextual restrictions.

3. Proposed Approach

First, this study began with data collection: in collaboration with the University of Pittsburgh, N=75 children were asked to wear a smartwatch for a few days. After checking if the data was properly recorded, we got 61 participants’ data. Approximately half, 25, had a diagnosis of ADHD with hyperactivity (the rest were controls).

Also, in this data collection, parents were asked to recall and fill in at the end of the day a behavioral label of what their child was doing every 30 minutes to provide context. We knew from related research that context is important when comparing motion data. This study compares the model's behavior with and without this parent-assigned context label.

We compared the measured acceleration for each context for children diagnosed with ADHD and those without ADHD. It was observed that there are large differences in the time corresponding to the “sitting/quiet” behavior label, although there does not seem to be much difference in other contexts.

Comparing samples of measured acceleration. Large differences exist between children with and without hyperactivity in the “sitting/quiet” context.

Based on this observation, we proposed a method to train a machine learning model by first performing context filtering on data measured throughout the day and then generating features. In addition, we tested two patterns: one using labels assigned by the parent as the context, and the other a two-step pattern in which the context itself is estimated from the smartwatch data, and then context filtering is performed.

Pipeline with the proposed context filtering. We tried two approaches: one that uses as context the behavioral labels filled in by the child’s parents and one that estimates even the labels.

4. Experiments and Results

We compared three patterns: (1) without context filtering, (2) with context filtering applied using the behavior labels given by the parents, and (3) with context filtering applied using the inferred behavior labels. The results showed that the order of accuracy is (2) > (3) > (1). Please refer to the paper for additional analyses, etc., but the main message is that using estimated behavior labels suggests the possibility of calculating a measure of hyperactivity without the need for parents to assign behavior labels.

Results of the three conditions compared in this study.

The trained machine learning model can produce a Risk Score for hyperactivity. Using this Risk Score as an indicator, we considered how to support physicians in their diagnostic decision-making. In this study, through interviews with clinicians, we implemented an initial prototype with a design for transparency, which is considered important in Human-AI Interaction research. Please read the paper for details.

Initial prototype of a diagnostic support tool based on Risk Score.

5. Summary

In summary, we made the following contributions

・We proposed a machine learning model that calculates an objective measure of hyperactivity from smartwatch data, and validated it using actual patient data (N=61).

・We showed that the accuracy of the model improves when given a context (what kind of activity the patient is doing).

・Through interviews with clinicians, we designed a tool that can potentially assist with diagnosis from the perspective of human-AI interaction.

A future direction is Human-AI Interaction research on presenting information as a diagnostic tool to support clinicians’ decision making. Since machine learning models can make mistakes, we are considering the impact of such mistakes, how to present information that will help rather than replace diagnosis with AI, and whether other information that can support diagnosis can be extracted from smartwatch data. We are thinking about how to present information that will help, not replace, the diagnosis. This research is a long-term project that I will be working on throughout the Ph.D. period, so please watch for more information in the future!

6. FAQ

Q1. What problem does LemurDx solve?

A1: LemurDx provides an objective measure of hyperactivity, which is a core component of ADHD. Currently, providers rely largely on subjective questionnaires that are completed by parents and teachers.

Q2. How is LemurDx different from an actiwatch or FitBit?

A2. One key difference is context. LemurDx goes beyond just measuring physical activity. It also classifies the environment/context within which the physical activity is happening.

Q3. Is there any problem with a child wearing a smartwatch all day long?

A3: This was raised as a concern in our interviews with clinicians. In particular, wearing a smartwatch is likely to be disturbing and distracting. Previous studies had confirmed that signs of hyperactivity were strongly present in the data while sleeping, but in this study, we considered it particularly difficult to wear the watch while sleeping, so we limited the study to daytime wear. For future research, it is possible to reduce the wearing time while maintaining the indicator's accuracy by estimating the context of the time when it was worn successfully and inputting it into the model.

Q4. Is it a good idea to take advantage of the different nature of the data measured for hyperactive and non-hyperactive children, but use that data to estimate the context equally for both groups?

A4. The current context estimation model also uses motion data, but further improving the accuracy is the next step. For example, better context estimation can be achieved by combining other data that can be measured by smartwatches (heart rate, GPS, etc.). However, it is important to note that the behavior labels assigned by the current parent are not accurate either. (If you recall that act of recalling behavior in 30-minute increments at the end of the day, you would imagine that.) What is interesting about the results of this study is that filtering based on this kind of “not necessarily 100% accurate” contextual information contributed to the improvement in accuracy. We want to explore this further from a machine-learning perspective.

References

[1] Lindhiem, Oliver, et al. Objective measurement of hyperactivity using mobile sensing and machine learning: Pilot study. JMIR Formative Research 6.4 (2022): e35803. [2] Erik G Willcutt. The prevalence of DSM-IV attention-deficit/hyperactivity disorder: a meta-analytic review. Neurotherapeutics 9, 3 (2012), 490 -499 [3] Lung-Chang Lin et al. Quantitative Analysis of Movements in Children with Attention-Deficit Hyperactivity Disorder Using a Smart Watch at School. Applied Sciences 10, 12, 4116.