Or How Might We Interact With Agents

<aside> ℹ️

This document is a working draft and is subject to ongoing development. Certain sections may be incomplete or updated as our research and perspectives evolve. It reflects Symbiotic’s current position and thinking, which may change as we refine our approach.

</aside>

<aside>

Reader’s Note

We often think of products we use in terms of the features they provide — this one has “Slack integration”, which allows building “dashboards”. This is like thinking about a book in terms of its words — this book has a "serendipitous", that one has a "generous". What matters is not individual words, but how the words together convey a message. Likewise, a well-designed system is not simply a bag of features. A good system is designed to encourage particular ways of thinking, with all features carefully and cohesively designed around that purpose.

This document presents a lot of features. The trick is to see through them—to see the underlying design principles they represent and understand how they enable the creator to think.

What’s an ‘Intermedium’?

The term was coined by J.C.R. Licklider in his conception of human interaction with ‘Procognitive Systems’– which were his idea of intelligent knowledge systems of the future. Intermedium, according to him, extends beyond the Interface. Interface is thought of as a mere surface, a plane of separation between man and machine, while Intermedium subsumes the tools and modes of perception, reasoning, and action by both participants.

</aside>

1. Introduction

Since deep learning’s breakthrough decade, the focus of computing has moved toward model-centric software: generative models sit at the core while surrounding systems orchestrates context, tools, and guardrails. After a period spent mapping their capabilities and limits, the central question now is interaction: how should humans create, collaborate, and manage these agents? This is less a break with history than a return to Licklider, Engelbart and Kay’s vision of interactive augmentation, updated for systems that can reason, act, and learn in the loop.

Until now, computer users are considered "end users", meaning that they are at the end of the process of computer programming, far removed from the programmer. The applications you use were not written with your particular needs in mind. Your calendar app doesn't know that you prefer taking meetings before 6 PM, or that your favorite pizza place closes early on Wednesdays. Your word processor doesn't know that your English teacher requires book titles to be underlined in your bibliographies. As a result, 'end users' must map their activities into the capabilities of any application.

What’s needed is a composable platform that empowers you to create and use tools for thought and action. Today, agent‑oriented and multi‑agent systems present a promising new way in this direction. They offer a fresh set of metaphors and abstractions for building and interacting with software in ways that align with how you work. This document introduces Noa, a general-purpose AI that can create and coordinate a network of agents. It briefly introduces the agent metaphor in Noa, defines ways to create, collaborate with and manage agents, and presents some archetypal applications that illustrate the power of this new paradigm.

2. The Agent Metaphor

The word 'agent' means different things to different people. The idea of authoring and working with agents is barely new. Leading AI textbooks define "Artificial Intelligence" as "the study of intelligent agents". Every few decades, the word 'agent' escapes the province of esoteric technical enthusiasts and seeps into public debate. This summer of AI, the concept of agents and multi-agent systems is driven by a Generative AI model with access to external tools and information. Instead of getting into a debate about what defines an agent, here I'll try to present a few principal ideas on how we might model intelligent agents that help us to augment and amplify collective human cognitive abilities.

We believe that a truly agentic paradigm needs a more sophisticated building block, one that is malleable enough to let you define concepts and composable enough to combine seamlessly for performing everyday cognitive tasks. While generative models play a key role in equipping these agents with messy pattern-recognition abilities, such as understanding natural language and creating images, agents also need certain core characteristics that enable them to reason and act contextually, whether they’re drafting a report, helping you plan your week, or uncovering meaningful insights.

One of the things that's worked best the last three or four hundred years, is that you get simplicity by finding a slightly more sophisticated building block, and build your concepts out of it. – Alan Kay

What are Noa Agents?

Agents in Noa can be understood as simulations of real-world concepts or entities that can sense and act in a digital environment.

General agent architecture in Noa (Agent A1, top right) and an illustration of the multi-agent framework.

More concretely, an agent can possess the following core characteristics that allow it to perceive, reason, and act contextually and reliably:

• Agents are defined by their roles. Noa agents can take on different roles depending on the kind of concept they represent or the task they perform, like Meeting Agent or Call Transcriber Agent. All agents defined by a role share a common blueprint.
• Agents can know things. Noa Agents can maintain a tailored knowledge base made up of individual records. For example, your Meeting Agent holds entries for every meeting, complete with structured fields such as “Meeting Name” and “Attendees,” as well as any related images, screenshots, or recordings. Likewise, your Task Agent tracks every to-do item, capturing both the key details and any attached files. This rich, multimodal knowledge empowers agents to reason more effectively and deliver contextually relevant responses.
• Agents can reason and act. Noa Agents can reason, enabling them to take action. Agents create and execute plans, made of a sequence of reasoning steps. These plans represent your way of getting something done, like transcribing a meeting, routing leads to the right sales team member, or forecasting revenue.
• Agents can interact. Noa Agents can interact with humans and other agents in a collaborative environment, enabling them to reference knowledge and execute plans together.
• Agents can learn.

3. Creating agents