Traditional neuropsychological testing in the context of a clinical trial follows a relatively standard protocol. The lead investigators determine which tests to administer, then the tests are completed at various intervals throughout the trial, typically at pre- and post-intervention. These testing sessions occur in one sitting and provide a “snapshot” of cognitive abilities at that time. Test performance can vary due to a myriad of contextual factors during the testing session. Was the participant running late, couldn’t find parking, and arrived stressed and frazzled? Did they not sleep well the night before or get in their regular two cups of coffee prior to testing? Did a plane fly overhead during a memory test so they couldn’t hear the words they were supposed to learn and remember? Did the participant get fatigued after 1-2 hours of testing and quit giving their full effort? Were they having a depression-free day when normally depressive symptoms interfere with their daily cognitive abilities and functioning? The list goes on.
If a participant performs poorly on their pre-intervention cognitive testing and then performs well on their post-intervention cognitive testing, the investigative team would conclude the intervention was effective at improving cognitive abilities. However, what if contextual factors were at play during these testing sessions, and performance was not a true representation of cognitive ability?
Mobile cognitive testing using a burst design can help overcome these challenges. In a burst design, participants are administered brief cognitive tests on their smartphones multiple times over a short period of time. Scores can be aggregated to represent mean cognitive performance, and variability across time and contexts can be examined. Coupling mobile cognitive testing with ecological momentary assessment (EMA) surveys can be particularly advantageous in examining how context is influencing cognitive abilities. For example, in one study of older adults with and without cognitive impairment (N=103), we administered EMA surveys 4x/day and mobile cognitive tests of memory and executive function 2x/day over a 14-day period (Campbell et al., 2020). Surveys and cognitive tests were administered at random times (and at least 2.5 hours apart) within the participants’ preferred sleep/wake cycle. We found that when participants reported they were doing a cognitively stimulating activity (e.g., reading, work, puzzles), they had better performance on the mobile executive function task, but when they were doing a passive leisure activity (e.g., watching TV, resting) they performed worse on the same task. Further, the more time participants spent in cognitive stimulating activities over the 2-week period, the greater their averaged performance on the mobile memory task, whereas a greater percentage of time spent in passive leisure activities was associated with worse averaged memory performance.
When designing a burst design protocol for a clinical trial, there are several factors to consider:
Selection of tests and other data streams
- What tests to administer, and if you want to incorporate other data streams such as EMA surveys, passive data from smartphones/wearables, etc.
Timing of mobile cognitive tests
- Do you want to do a one-week burst pre- and post-intervention?
- Testing during the intervention? At what timepoints (e.g., 2-weeks after new dosing? 6-weeks after dose change?) and how frequently
Frequency of test administration
- Do you want to administer tests daily (and once/day or multiple times per day), every other day, once per week, other?
Participant burden
- The more intensive the burst the more burdensome it will be to the participant
What are your clinical endpoints?
- Change in cognition? Daily functioning? Interrelationships of factors?
Are there strong psychometric properties for the mobile cognitive tests you’re administering?
- An absence of validity data in mobile cognitive testing platforms could lead to unreliable information about possible cognitive impairment or change in cognitive outcomes.
All these decision points should be weighed in the protocol design. On average, collecting ~28 data points (assuming 20-30% of the tests will be missed) in a burst seems to be fairly consistent in the literature. Our work has frequently used a 14-day burst protocol, with mobile cognitive tests administered 2-3x/day (each test administered 1x/day), provide strong data to aggregate scores across time and context, provide variability in test scores, and have minimal fatigue effects (i.e., adherence rates generally ~80% for our 14-day protocols, and participants generally complete the tests for the full 14-days). Other groups have used a 7-day burst protocol, with mobile cognitive tests 4x/day, also yielding strong data.
In addition to providing a mobile cognitive testing and EMA platform, the NeuroUX team has helped numerous research groups with protocol design and obtaining NIH and VA funding. Currently, several studies are underway utilizing the NeuroUX mobile cognitive tests in clinical trials as well as observational studies.
.png)
Smartphones present unique opportunities for assessing numerous aspects of cognition, including subtle changes in cognition over time, cognitive performance in the context of everyday life (including relationships between cognition and mood, daily functioning, sleep, physical activity, nutrition, social relationships, pain, substance use, etc.), real-time response to treatment or medication change (and reducing the Number Needed to Treat or NNT; see Moore et al., 2016), and other socio-cultural and environmental factors impacting the complexity of cognitive abilities. Overall, smartphones can provide a digital solution in clinical trials with cognition as an endpoint, thus minimizing cost and increasing sensitivity and specificity of study findings.
