As part of our World Mental Health Day Blog Series, Dr Aja Murray introduces DigiCAT, a digital tool developed as part of her Wellcome Trust Mental Health Data Prize winning project, which aims to assist researchers in conducting counterfactual analysis.
To develop effective prevention and treatment strategies for mental health issues it’s important to know what the ‘active ingredients’ in mental health are. Many different active ingredients in mental health have been proposed (see this report from Wellcome Trust), including things like thinking styles, emotion regulation, arts engagement, social connection, sleep, and physical activity.
However, as we discussed in an earlier blogpost it can be difficult to tell which of the myriad factors that show an apparent relationship with mental health are truly playing an active role. A major difficulty comes from the challenge of ‘confounding’ whereby some candidate active ingredient and mental outcome appear to be linked, but this is only because they are both influenced by a set of other factors. For example, if social media use is associated with an outcome like depression, is that only because people who are more depressed have traits, experiences, or circumstances (confounders) that also mean they are more likely to be high social media users?
Data analysis techniques under the umbrella of ‘counterfactual analysis’ have been developed to deal with this problem. It works, in essence, by comparing people who are highly similar in their profiles of confounding variables but who differ on the candidate active ingredient. If these groups differ on a mental health outcome, we can be more confident that this is truly due to the candidate active ingredient rather than the confounding variables.
Our team has applied counterfactual analysis to explore candidate active ingredients suggested by the previous literature and our young person advisory panel. Using the Millennium Cohort Study dataset, for example, we found no evidence that reading for pleasure was an active ingredient in adolescents’ mental health. In the same dataset, we found that during certain phases of adolescence, social media use may be an active ingredient in depression, anxiety, self-harm, and suicidality.
Our young person advisory group helped us make sense of these findings and highlighted some important limitations of work in this field. They particularly emphasised that existing measures of social media use don’t really capture how people engage with social media (which for them was the critical factor), pointing to a need to develop new and better measures in the future.
More broadly, counterfactual analysis is not very well-used in the mental health field and this is likely attributable to the fact that the technique is not very accessible. In particular we noted a lack of accessible tools and resources to support understanding and implementing it. With funding from the Wellcome Trust we, therefore, developed a digital tool: DigiCAT (Digital Counterfactual Analysis Tool) to make counterfactual analysis easier to use for exploring active ingredients in mental health.
DigiCAT is available to download and a web app demo version is also available. The code for the tool is available here. It was developed using a user-centred approach and we involved lived experience experts in the process too. Our user consultations thus far suggest that it can help do the job of empowering mental health researchers to use counterfactual analysis but we’re continuing to solicit feedback to make further improvements.
The next step for us will be to build more features into DigiCAT, introduce it to more audiences, and to continue to use it ourselves to conduct research that we hope can inform interventions to improve the mental health of the population.
As we continue to develop DigiCAT, we welcome very much welcome any feedback to: uoe_digicat-group@uoe.onmicrosoft.com.
Dr Aja Murray, Reader in Psychology, University of Edinburgh.
Comments