Researchers have recently found that some studies on mental illness are not wide-scale enough to retrieve trustworthy data. In order to gather more reliable and consistent data on how mental illness behaves in the brain, studies need to include a sample size of thousands of brains, according to a study published in Nature.
About the study: In this study, the researchers did an analysis of several other mental health studies — including around 50,000 brains — and found some disparities between data collected in smaller studies and larger studies on the same subject.
MRI scans: The main study noted that data from MRI scans is far more accurate than other methods of assessing mental illness, such as questionnaires. The best way to get accurate data is through an MRI, which isn’t always accessible.
- One problem with performing MRI scans is the cost. MRIs can add up to a total of around $1,000 an hour, which can stand in the way of using larger sample sizes, the study stated.
The issue with small sample sizes: Many small mental health studies have pulled data from sample sizes as small as 25 people. It is becoming more clear in the neuroscience community that thousands of brains are needed for accurate data.
- The study reads, “Small studies are most vulnerable to sampling variability, the random variation of an association across population subsamples. Sampling variability decreases and associations stabilize with increasing sample sizes.”
- If a smaller study has confirmed a finding, scientists recommend double-checking this data with a larger study.
- Small studies can only highlight brain features that are usually associated with mood, behavior or mental abilities, not a mental illness, according to NPR’s reporting on the study.
Why larger studies are needed: Differences in the brain that signify mental illness are less obvious and more controversial, according to NPR. Larger studies are needed to ensure the data tracks across a wide range of different brains.
- When small studies were performed, an area of the brain or connection that seemed important in one set of scans sometimes seemed insignificant in another set.
- When the sample size was increased to thousands of brains, the results became more reliable, per NPR.
Publication bias: Several small studies are likely to come up with similar, seemingly significant findings. This can sometimes lead the general public to think that these findings are “conventional wisdom,” when really the data isn’t complete and often misleading. NPR states that this is known as publication bias.
- When larger studies are done, it is much more difficult to come to the same conclusions, which can lead to contradictions in the data based on study size, according to NPR.