Recent functional MRI studies identified sensory-biased regions across much of the association cortices and cerebellum. However, their anatomical relationship to multiple-demand (MD) regions, characterized as domain-general due to their coactivation during multiple cognitive demands, remains unclear. For a better anatomical delineation, we used multimodal MRI techniques of the Human Connectome Project to scan subjects performing visual and auditory versions of a working memory (WM) task. The contrast between hard and easy WM showed strong domain generality, with essentially identical patterns of cortical, subcortical, and cerebellar MD activity for visual and auditory materials. In contrast, modality preferences were shown by contrasting easy WM with baseline; most MD regions showed visual preference while immediately adjacent to cortical MD regions, there were interleaved regions of both visual and auditory preference. The results may exemplify a general motif whereby domain-specific regions feed information into and out of an adjacent, integrative MD core.