Building Tools for Quantum Cities: The Journey So Far
In the spring of 2024, I embarked on an ambitious project: developing a graduate-level course titled Superposition: Quantum Cities for the Integrated Media and Design program at NYU Tandon School of Engineering. This course aimed to do something bold—create layered realities for a neighborhood, specifically the Brooklyn Navy Yard, that could explore what it means to exist in multiple, simultaneous versions. The concept drew from quantum physics, specifically superposition, where multiple states coexist until observed. In these alternate, simultaneous realities, each version of the Navy Yard starts with the same layout but diverges over time while retaining coherence with the “ground truth.”
The goal is to use generative AI to create alternate versions of the Brooklyn Navy Yard—versions with different histories, characters, and day-to-day activities, but all fundamentally grounded in a consistent reality. The class intends to encourage students to creatively explore what might have happened, might be happening, or might happen in the future. To bring this idea to life, I quickly realized we would need tools to maintain rigorous continuity across these branching narratives. This need led to the development of two key supporting projects, each crucial for ensuring the consistency and coherence of the Superposition project.
Supporting Projects: Laying the Foundations
Patterns of Consistent Environment Evolution
The first supporting project, Patterns of Consistent Environment Evolution, draws inspiration from Christopher Alexander’s A Pattern Language. Alexander’s work is all about understanding and articulating the organic, interconnected nature of built environments—whether they are neighborhoods, buildings, or streets. This project translates Alexander’s patterns into a computational form, using knowledge graphs to represent the relationships between architectural elements in a way that is adaptable, dynamic, and richly detailed.
The goal here is to develop a digital knowledge graph that not only captures these architectural patterns but also provides a foundation for understanding how changes in the physical environment impact each other. This knowledge graph ensures that as new elements are introduced into our simulated version of the Brooklyn Navy Yard, they respect the underlying principles of coherent and meaningful design. Additionally, both of these experiments will involve the creation of multi-modal embeddings (text, image, audio, video) that will be properties of the nodes and relationships in the knowledge graphs. The technique is called GraphRAG, and this is a multimodal extension of that technique. In short, Patterns of Consistent Environment Evolution helps to keep our evolving urban environment consistent and believable.
Coherent Story Growth
The second supporting project is Coherent Story Growth, which focuses on the narrative aspects of building expansive, evolving story worlds. Using Rudyard Kipling’s novella The Man Who Would Be King as a starting point, chosen because it is a public domain story of an appropriate length with characters that evolve over time and intertwined events, this project aims to extend the original narrative into a larger, interconnected story universe. In the class, we will fashion a similar short seminal story that aligns with the real-world creation of the Navy Yard but allows for divergence in the past, present, and future. The challenge here is to create new characters, events, and perspectives while ensuring everything remains consistent with the core storyline.
A knowledge graph is again central to this effort, capturing relationships among characters, places, and events, and serving as the foundational structure from which new, coherent stories can be generated. By leveraging generative AI and agents that create new story elements, Coherent Story Growth ensures that all the branching paths in the narrative still logically connect. This approach allows for the creation of new characters and events that are consistent with the past and become facts for future story additions. Both of these experiments will use multi-modal embeddings (text, image, audio, video) as properties of the nodes and relationships in the knowledge graphs, extending the GraphRAG technique into a multimodal domain. The aim is for students to use this framework to extend the Navy Yard’s story with rich characters and complex interactions that evolve over time while maintaining continuity.
Learning and Looking Ahead
Working on these projects has been a transformative experience. Initially, the Superposition project was driven by an interest in using AI to create layered realities, but it soon became clear that maintaining continuity and coherence across these different realities was the real challenge—and perhaps the real opportunity. Patterns of Consistent Environment Evolution has provided the architectural coherence needed to keep physical spaces meaningful, while Coherent Story Growth has addressed the narrative complexity needed to grow believable story worlds.
These supporting projects are foundational because they allow the alternate versions of the Brooklyn Navy Yard to maintain an internal logic that feels authentic. The knowledge graphs act as the connective tissue that binds these different aspects together, ensuring that each new addition to the environment or the narrative is not just consistent but enriches the overall story world.
The ultimate goal remains the same: to provide students with the tools and frameworks to create and explore speculative versions of cities and stories that are as complex and interconnected as the real world. Whether it’s an alternate version of the Brooklyn Navy Yard shaped by Alexander’s timeless design principles or a reimagined story that extends Kipling’s original narrative, these projects lay the groundwork for what I hope will be a deeply creative, rigorous, and exciting exploration of what might be.
Glossary of Terms and Concepts
- Large Language Models (LLMs): AI systems that use deep learning to understand and generate human-like text based on large-scale datasets. Examples include GPT-4 and other transformer-based architectures.
- Generative AI: A type of AI capable of generating new content, such as text, images, or music, that is similar to the data it was trained on. This research uses generative AI to expand story worlds and create visual representations.
- Simulacra: A simulation or representation of a world, whether factual or fictional, that enables the exploration and testing of concepts in a controlled environment.
- Knowledge Graph: A data structure that represents entities and their interrelationships, often used to capture complex networks of information such as people, places, events, and concepts. These relationships explicitly include temporal (when events happen) and spatial (where events happen) aspects to ensure coherence.
- Coherence: The logical and consistent relationship between different elements of a narrative or system, ensuring that all aspects are interconnected in a meaningful way, with explicit consideration of temporal (when events occur) and spatial (where events occur) aspects to maintain continuity and contextual accuracy.
- Multimodal Integration: The combination of different types of data, such as text, images, and even sound, to create a richer and more comprehensive representation of knowledge or a narrative.
- Timeless Way of Building: A concept by Christopher Alexander describing a way of designing environments that are inherently meaningful and enduring, often characterized by organic and contextually responsive design principles.
- Agents: Autonomous computational units that act to expand the storyline by generating new content or retellings. These agents ensure that new information remains coherent with existing narrative structures.
- Hypertextual Relationships: Non-linear connections between elements that allow for a complex and interconnected web of references, much like hyperlinks in digital text.
- Embedding: A representation of data, such as text or images, as vectors in a high-dimensional space, often used to capture semantic meaning for use in AI and machine learning.
- Acceptable Incoherence: A categorization for new elements added to a knowledge graph that may have minor inconsistencies but are still valuable for expanding the narrative in an understandable way.
- Fractal Growth: A process of iterative expansion where each new layer adds more detail, maintaining the structural integrity of the whole. This concept is applied to narrative and architectural growth in this research.