| アイテムタイプ |
学術雑誌論文 / Journal Article(1) |
| 公開日 |
2024-08-08 |
| タイトル |
|
|
タイトル |
Inter-individual deep image reconstruction via hierarchical neural code conversion. |
|
言語 |
en |
| 言語 |
|
|
言語 |
eng |
| 資源タイプ |
|
|
資源タイプ識別子 |
http://purl.org/coar/resource_type/c_6501 |
|
資源タイプ |
journal article |
| 著者 |
Jun Kai Ho
Tomoyasu Horikawa
Kei Majima
Fan Cheng
Yukiyasu Kamitani
|
| 抄録 |
|
|
内容記述タイプ |
Abstract |
|
内容記述 |
The sensory cortex is characterized by general organizational principles such as topography and hierarchy. However, measured brain activity given identical input exhibits substantially different patterns across individuals. Although anatomical and functional alignment methods have been proposed in functional magnetic resonance imaging (fMRI) studies, it remains unclear whether and how hierarchical and fine-grained representations can be converted between individuals while preserving the encoded perceptual content. In this study, we trained a method of functional alignment called neural code converter that predicts a target subject's brain activity pattern from a source subject given the same stimulus, and analyzed the converted patterns by decoding hierarchical visual features and reconstructing perceived images. The converters were trained on fMRI responses to identical sets of natural images presented to pairs of individuals, using the voxels on the visual cortex that covers from V1 through the ventral object areas without explicit labels of the visual areas. We decoded the converted brain activity patterns into the hierarchical visual features of a deep neural network using decoders pre-trained on the target subject and then reconstructed images via the decoded features. Without explicit information about the visual cortical hierarchy, the converters automatically learned the correspondence between visual areas of the same levels. Deep neural network feature decoding at each layer showed higher decoding accuracies from corresponding levels of visual areas, indicating that hierarchical representations were preserved after conversion. The visual images were reconstructed with recognizable silhouettes of objects even with relatively small numbers of data for converter training. The decoders trained on pooled data from multiple individuals through conversions led to a slight improvement over those trained on a single individual. These results demonstrate that the hierarchical and fine-grained representation can be converted by functional alignment, while preserving sufficient visual information to enable inter-individual visual image reconstruction. |
| 書誌情報 |
NeuroImage
巻 271,
p. 120007,
発行日 2023-03
|
| 出版者 |
|
|
出版者 |
Elsevier |
| ISSN |
|
|
収録物識別子タイプ |
ISSN |
|
収録物識別子 |
1095-9572 |
| PubMed番号 |
|
|
|
識別子タイプ |
PMID |
|
|
関連識別子 |
36914105 |
| DOI |
|
|
|
識別子タイプ |
DOI |
|
|
関連識別子 |
10.1016/j.neuroimage.2023.120007 |
Inter-individual deep image reconstruction via hierarchical neural code conversion.pdf (4.2 MB)
