Researchers have successfully reconstructed a piece of music from neural recordings using computer modeling, according to a recent article published in PLOS Biology. The study aimed to investigate the spatial neural dynamics underlying music perception using encoding models and ablation analysis.
Music activates many of the same brain regions as speech, and researchers have long been interested in understanding the neural basis of music perception. While distinct neural correlates of musical elements have been identified, the interaction between these neural networks in processing the complexity of music remains unclear.
The study’s lead author, Dr. Robert Knight from the University of California, Berkeley, noted that the research could potentially add musicality to future brain implants for individuals with neurological disorders that affect speech.
In the study, researchers implanted 2,668 electrocorticography (ECoG) electrodes on the cortical surfaces of 29 neurosurgical patients. The patients were asked to passively listen to a three-minute snippet of the Pink Floyd song “Another Brick in the Wall, Part 1.” This method of stimulus presentation prevented any confounding effects from motor activity and decision-making.
Using data from 347 electrodes, the researchers successfully reconstructed the song, closely resembling the original but with less detail. This marks the first time that music has been reconstructed using this approach, although similar methods have been used to reconstruct speech from brain activity in the past.
The study combined the use of intracranial electroencephalography (iEEG) data, which provides excellent temporal resolution and signal-to-noise ratio, and nonlinear decoding models to uncover the neural dynamics underlying music perception. The team also examined the effect of dataset duration and electrode density on reconstruction accuracy.
The results showed that both brain hemispheres were involved in music processing, with the superior temporal gyrus (STG) in the right hemisphere playing a more crucial role in music perception. In addition, a new STG subregion tuned to musical rhythm was identified. Most electrodes responsive to music were implanted over the STG, indicating its importance in music perception.
Furthermore, the study found that nonlinear models provided the highest decoding accuracy, with an r-squared value of 42.9%. However, adding too many electrodes beyond a certain amount diminished decoding accuracy. The study also highlighted the impact of dataset duration, with the model achieving 80% of the maximum decoding accuracy in just 37 seconds.
The findings of this study have potential implications for brain-computer interface (BCI) applications, particularly in communication tools for individuals with speech disabilities. Incorporating musical elements could improve the quality of speech generated by current BCI-based interfaces, which often sound unnatural and robotic. The study’s findings could also be relevant for patients with auditory processing disorders.
In conclusion, the study provides new insights into the neural dynamics underlying music perception and confirms previous findings on the bilateral network involved in music processing. Future research may look into extending electrode coverage to other cortical regions and exploring different decoding models and behavioral dimensions.