Borges’s “The Library of Babel” describes imaginative visions of a total library with an infinite spatial layout. In full recognition of “Relativity” by Escher as setting, the design method is “optical illusion”. By adding Chinese and Hong Kong culture into it, the recomposed library is called “Illusio Bibliotheca” with six domains, including Heavens/Time/Calendrics, In Situ, Man/Society, Nature, Philosophy and Economy. Through associating symbols to setting, spaces are no longer only drawing on flat surfaces, but also in imaginary 3D spaces, making hyperbole and paradox work into setting.