A conversation with Japanese neuroscientist Yu Takagi on the emerging boundaries between the human brain and generative artificial intelligence by Andrea Monti – Initially published in Italian by MIT Technology Review Italia
A study published in 2023 on IEEEXplore and authored by professors Yu Takagi of the Nagoya Institute of Technology and Shinji Nishimoto of the Graduate School of Frontier Biosciences at Osaka University has demonstrated how it is possible to integrate functional magnetic resonance imaging (fMRI) results with Stable Diffusion, anormal” generative AI model with no intellectual property restrictions, to translate brain activity into images.
MIT Technology Review Italia had the opportunity to interview Professor Takagi to learn more about the impact this result may have on the interaction between the brain and artificial intelligence.
In this exclusive interview, Takagi discusses the scientific, technological, and ethical implications of his work: from the challenges of non-invasive models to the transhumanist potential of neural interfaces, to the crucial role of dataset transparency.
A lucid and rigorous vision of the horizon where mind and machine meet.
Professor Takagi, could you describe your field of research?
I am deeply engaged in research focusing on two fundamental aspects: the artificial brain, represented by artificial intelligence, and the human brain. My primary interest lies in the intersection of these two domains—how they can be connected and understood in relation to one another.
For instance, I am particularly fascinated by the potential connections between generative AI and human cognitive processes. Currently, my focus is on exploring how large language models can be integrated with and compared to human brain functionality. My specialization sits squarely in the nexus of these two areas, bridging artificial systems and biological intelligence.
Do you believe that having freely available models, such as Stable Diffusion, has been beneficial to your research? Did it help in saving research funds and allow you to concentrate on core research activities rather than on building datasets and algorithms from scratch?
Yes, absolutely. The availability of such models has been transformative. Historically, there have been many AI models, but in my view, Stable Diffusion marked a turning point in the field, offering a quality and versatility that surpassed previous iterations.
Between 2010 and 2015, various AI models for image processing existed, but they often fell short when applied to our specific datasets and experimental requirements. Developing such sophisticated models internally would have demanded enormous financial resources—something often out of reach for us. Thanks to openly available tools like Stable Diffusion, we were able to achieve far superior results with limited resources, avoiding the need to develop everything independently. This was a significant advantage.
Q: Did you fine-tune the model you used to adapt it to reading fMRI scans, or did you employ the Stable Diffusion model without further modifications?
That is an excellent question. In our case, we did not perform any fine-tuning. Our approach was straightforward: we established a simple mapping between the stable diffusion model’s output and human brain activity. We avoided complex adjustments, primarily because we lacked the computational resources, such as powerful GPUs, and the financial means to support more elaborate modifications. Our strategy was to keep the process simple yet effective, which, fortunately, produced meaningful results.
Do you think that having freely available tools, including datasets and algorithms, is crucial for advancing scientific and academic research?
Yes, indeed. Two key components are essential: the pre-trained model itself, such as Stable Diffusion, and a robust dataset. For example, in 2021, a critical dataset—the Natural Scene Dataset—was released by the University of Minnesota (US), which played a pivotal role in our research. Today, while many research groups are making advancements, not all are publishing openly, which creates barriers. Open access to both models and data is, in my view, indispensable for the continued progress of the field.
Regarding MRI scans, how did the resolution of the scans impact your final results?
Unfortunately, the resolution of our MRI scans was not as high as we would have liked, especially compared to what is achievable in other advanced fields. The low resolution limited the granularity of our results. Some aspects of our findings generated excitement, especially among the general public and media, however, realistically, we are still far from achieving highly detailed, image-like reconstructions purely from brain scans.
Do you think using a brain-computer interface (BCI), or implants such as Neuralink, could improve the results?
Very likely. Such technologies hold significant promise. However, they are not yet feasible for use in healthy individuals. Currently, invasive BCIs are primarily reserved for patients with serious medical conditions. If we could use these technologies in healthy subjects, the quality and resolution of brain data would improve dramatically, leading to far more accurate and insightful results.
So, in an ideal scenario, you would need volunteers willing to receive brain implants to collect high-quality data directly from the brain?
Precisely. In an ideal setting, having participants with brain implants would allow us to gather much richer data. Additionally, we would need more sophisticated models. While AI technology is progressing rapidly, neuroscience is developing at a slower pace, which presents a challenge for closing this gap.
Speaking about BCI implants: are there some concerns regarding data bandwidth between the brain and the processor, as well as the heat generated by the processor? To collect more information, you need higher voltage, which, in turn, generates more heat. This seems to be a fundamental limitation for BCIs. Do you think this is a reasonable concern?
Yes, I believe this is indeed one of the biggest challenges in BCI technology—the physical limitations of the device itself. While many hope for breakthroughs, these physical constraints—such as heat generation and bandwidth—remain significant hurdles. We need innovative solutions to overcome these barriers if we want to develop BCIs that are practical, safe, and effective for human use over long periods.
Do you think it is necessary to rely on invasive implants to collect reliable data, or could non-invasive interfaces be equally effective in the future?
That is an excellent and very timely question. In my opinion, within the next five to ten years, non-invasive methods are unlikely to match the precision and reliability of invasive implants. While there may be significant advancements in the longer term—say, twenty or thirty years—current non-invasive techniques still fall short when compared to what invasive methods can achieve. We have seen limited progress in the last decade, and I expect that trend to continue for the foreseeable future.
Do you foresee your research leading to a practical product, perhaps in healthcare or another sector, designed to assist disabled or disadvantaged individuals?
Yes, that is certainly one of our ambitions. If our technology can be applied effectively, it could lead to innovative tools to assist people with various disabilities. For instance, we could develop AI systems that support vision, hearing, or even speech for individuals who have lost these abilities. We believe that our methods could contribute to multimodal AI systems that enhance sensory functions, offering significant improvements in quality of life for such individuals.
Are you moving towards developing multimodal AI in your future work?
Yes, indeed. We are expanding our focus to apply our techniques across a broader range of modalities, combining neuroscience with advanced AI. We are exploring ways to translate these developments not just into scientific research but also into tangible tools that could serve both patients and, in the long term, healthy individuals.
Do you believe that your research might ultimately lead to creating a new breed of human being—one that is enhanced by prosthetics and artificial implants, not just for medical needs but also as part of a broader evolution towards transhumanism?
That is a fascinating question, and yes, I do think so. There are many neuroscientists in Japan who are also intrigued by these ideas. Ultimately, the goal of brain-machine interfaces could be to link one brain directly to another, facilitating a form of communication or perception that transcends current limitations. This could lead to profound changes in how we perceive and interact with the world—possibly even allowing individuals to share perceptions or experiences directly, creating a fundamentally different type of human experience.
Do you think that using external datasets or AI-generated interpretations might distort human understanding of reality? For instance, if a dataset mistakenly labels a tree as a car, wouldn’t this corrupt the way we perceive the world?
Yes, this is a very insightful observation. The reliance on datasets and AI models introduces a layer of abstraction between raw perception and interpretation. If the underlying data or the model’s training is flawed, it could indeed distort reality as we understand it. This is why rigorous quality control over datasets and models is crucial—without it, the risk of misinterpretation or misinformation increases significantly.
Do you think it’s important to have independent bodies that oversee and certify the quality and neutrality of these datasets to prevent manipulation, especially in politically sensitive contexts?
Absolutely. The potential for data poisoning or intentional bias in datasets is a serious concern. To ensure the integrity of research and applications—especially in critical areas like neuroscience and AI—we need independent, transparent bodies to regulate and verify the quality and neutrality of datasets. This is essential to prevent misuse and to maintain trust in the technology.
In the context of neuroscience, do you think models should remain entirely neutral, even if they process content that might be considered ethically or morally questionable? Or should there be ethical oversight that restricts certain types of data?
I strongly resonate with your point. While there are certainly ethical considerations, particularly when working with sensitive or potentially harmful content, I believe that scientific tools themselves should remain neutral. The purpose of these models is to reflect and interpret brain activity as accurately as possible. Ethical oversight is vital, but it should focus on how the tools are applied, rather than limiting the scope of what the models can process.
Do you think your technology could also work in reverse? That is, could you start with an image, process it through a model, and then stimulate corresponding activity in the brain?
This is a highly challenging but fascinating prospect. Currently, stimulating specific brain activity based on external images requires advanced technologies and invasive methods, which we do not yet fully understand or possess. Although we can already simulate certain types of brain activity, truly inducing precise perceptual experiences is a significant technical hurdle. I don’t expect major breakthroughs in this area within the next five to ten years, but in the longer term, with advances in both neuroscience and hardware, it may become possible.
So, essentially, this technology could create a kind of virtual world—a mediated reality where experiences are filtered through a system before being perceived by the brain?
Exactly. This could open the door to new forms of entertainment, communication, and even therapeutic applications. However, it also raises serious ethical and philosophical questions, particularly concerning autonomy, freedom of thought, and government control. It’s a powerful tool that must be approached with caution.