Knowledge Representations+DAS (Distributed AtomSpace)+DAS Deep Dive

DAS (Distributed AtomSpace) is Hyperon's distributed knowledge-management system — a high-speed, dynamic memory fabric for massive, mutable hypergraphs. It decouples durable persistence from fast-changing attention dynamics, letting an AtomSpace span multiple machines while keeping the cognitive properties needed for AGI. This card provides technical depth beyond the concise DAS index card.

On this page

• How DAS Relates to MORK and ECAN — what is genuinely implemented vs. easy to overstate
• Core Mechanisms — STI/LTI separation, the Attention Broker, and the query-tree execution model
• Storage Backends — Redis+MongoDB, in-memory, MORK/MorkDB, RocksDB
• Mathematical / Formal Foundations — the proposed state-management algebra (monoid / coalgebra / CRDT / lenses)
• System Interfaces and Dependencies — MORK, ECAN, ASI Chain, MeTTa
• Implementation Anchors — the actual repositories
• Current Status — operational / under development / proposed
• Open Problems — the live research and reconciliation questions
• Primary Sources

Papers: Goertzel (2025), Hyperon Whitepaper §2.5–2.6; Goertzel (2025), TrueAGI AI State Management Platform: Initial Design

Status: Current. The DAS core (query engine, AtomDB, attention broker) is operational with Redis+MongoDB and in-memory backends. The MeTTa parser and CLI toolbox are functional. MORK integration is code-real with caveats (see Storage Backends and the MORK/ECAN section immediately below). The broader State Management Platform with CRDT geo-replication, Rholang lens contracts, and Kubernetes orchestration is proposed.

How DAS Relates to MORK and ECAN

DAS sits at Layer 3 of the four-layer AtomSpace family — the abstract AtomSpace concept, the in-memory Hyperon Space, the distributed DAS, then MORK as the high-performance backend (see AtomSpace Full → Implementations). Two implementation realities are worth stating plainly, because both are easy to overstate:

1. The MORK backend is real, but not a drop-in mutable store. DAS contains a code-real MorkDB AtomDB backend, but it is a split implementation: the DAS-side AtomDB code is real (singnet/das: MorkDB.h, MorkDB.cc, AtomDBSingleton.cc), while the MORK server itself is pinned through a Docker / server-branch path. Critically, link / S-expression delete is not supported in the MorkDB backend — MorkDB.cc:268-270 hard-fails with “MORKDB does not support deleting links.” Treat MORK-as-DAS-backend as integration-ready for loads and queries, not as a Decko-compatible mutable store.

2. The Attention Broker approximates ECAN; it is not the literal 2009 formulas. The AttentionBroker stores per-context STI outside AtomDB, updates exact-count Hebbian networks, and uses a token rent/wages/spreading model. It is an ECAN-compatible engineering surrogate, not the literal 2009 ECAN formula set. Implementation: ExactCountHebbianUpdater (HebbianNetworkUpdater.cc:57-96, weights = count(A→B)/count(A)) plus TokenSpreader, a fixed-token rent/wage/spreading model (StimulusSpreader.cc:54-60 rent = rent_rate × importance; :140-157 wages proportional to request counts; :75-82 deterministic arity-weighted spreading; fixed defaults RENT_RATE = 0.75, lower/upper spreading = 0.10). This places the Attention Broker in the same lineage as the classical ECAN C++ implementation and the metta-attention MeTTa port — all three score 0/4 on strict-literal compliance with the 2009 ECAN paper. DAS is the most production-oriented of the three, but it is the same engineering-surrogate family.

Hyperon Experimental bridge. The new-das! MeTTa op constructs a DistributedAtomSpace and returns a DynSpace (hyperon-experimental/lib/src/metta/runner/builtin_mods/das.rs:156-199). The das feature is default-enabled in lib/Cargo.toml:45-55, depending on metta-bus-client from singnet/das tag 1.0.2.

Server-branch drift (as of 2026-04-29). The DAS Dockerfile pins MORK commit 578a759 (2025-07-21); local origin/server is at 5b04a1d (2026-04-18) with 49 subsequent commits including server-shutdown deadlock, lock-held-too-long deadlock, UTF-8 validation, and edge-case malformed-symbol fixes. das-toolbox CLI defaults to image tags trueagi/das:mork-server-1.0.5 and mork-loader-1.0.5 — three potentially-different references (image tag, Dockerfile pin, server-branch HEAD) that a production deployment must reconcile.

Core Mechanisms

Separation of Concerns. DAS's key architectural insight is decoupling two timescales of knowledge management:

• Long-Term Importance (LTI): Vast persistence stored in distributed backends (Redis+MongoDB clusters, RocksDB, or MORK). This is the system's durable memory — everything it has learned and experienced.
• Short-Term Importance (STI): Immediate dynamics managed in high-speed RAM by the Attention Broker. This is the system's working memory — the subset of knowledge currently relevant to active cognitive processes.

Attention Broker. The Attention Broker mitigates combinatorial explosions inherent in graph traversal. It maintains separate Hebbian networks for different query contexts, each with independent STI values per atom — the same atom can have different importance in different contexts. Hebbian link weights encode co-occurrence probabilities: the probability of atom B appearing in a query answer given that atom A is present. STI updates can be implicit (automatically triggered during query execution as answers flow through the tree) or explicit (user-requested stimulation and activation spreading). This enables the Attention Broker to learn query-relevant importance patterns from the actual queries being processed. (See “How DAS Relates to MORK and ECAN” above for the engineering-surrogate vs. literal-2009-ECAN distinction.)

Query Architecture. DAS supports multiple simultaneous query agents, each specialized for different cognitive operations: Query Engine (pattern matching), Evolution Agent (fitness-driven search), Inference Agent (PLN-style reasoning), Link Creation Agent (dynamic graph modification), Context Broker (context-sensitive routing), and AtomDB Broker (backend-agnostic storage). A Service Bus mediates inter-component communication in a service-oriented architecture, supporting pluggable transport (GRPC, MQTT, in-RAM queues).

Query Tree Execution. The Query Engine processes queries in three steps: (1) parsing (MeTTa expressions or token vectors), (2) execution planning (constructing an optimized query tree), and (3) query execution (streaming results). Query trees are composed of three element types, each an asynchronous independent processing unit:

• Sources (leaf nodes) — Fetch candidate answers from AtomDB matching a LinkTemplate pattern, query the Attention Broker for STI values, and emit candidates sorted by decreasing STI
• Operators (internal nodes) — Apply Boolean logic (AND, OR, NOT) to combine candidate streams from multiple sources, preserving STI ordering
• Sinks (root) — Deliver final results to the caller, who can iterate or interrupt at any time

Complex queries compose LinkTemplates with Boolean operators in arbitrarily nested expressions (e.g., (or (contains $s (word "aaa")) (contains $s (word "bbb")))). Elements can be deployed as threads, processes on the same server, or across different machines — the communication channel (RAM queues, GRPC, MQTT) adapts to the deployment topology.

Query Evolution. For queries with billions of potential results where only a few optimal answers are needed, DAS provides an evolutionary algorithm inspired by BOA (Bayesian Optimization Algorithm). Rather than mixing individuals with genetic operators, it samples each generation using the Attention Broker's probabilistic criteria: (1) sample a population by executing the query, (2) evaluate a user-provided fitness function on each result, (3) select the fittest individuals to stimulate importance in the Attention Broker's Hebbian network, (4) repeat — each generation biases STI allocation toward atoms that compose high-fitness answers. This leverages DAS's existing attention infrastructure to perform fitness-directed optimization without a separate evolutionary framework.

Deployment Architecture. DAS supports a Lambda Architecture using OpenFaaS (preferred) or AWS Lambda for distributed query execution. A Pattern Inverted Index maps patterns to their knowledge base occurrences, enabling efficient pattern matching without exhaustive graph traversal — analogous to inverted indexes in full-text search but operating over hypergraph patterns rather than text tokens. A Cache Layer sorts and partitions results by relevance, returning iterators that "fetch additional chunks on demand." A Traverse Engine handles hypergraph navigation for remote DAS connections, pre-fetching surrounding atoms to reduce round-trip latency. (Provenance: blog/post, SingularityNET Medium — "The Distributed Atomspace (DAS): A New-age Knowledge Repository", Jun 2024)

(Provenance: repo-doc, DAS conceptual documentation)

Storage Backends

DAS abstracts over multiple storage backends via the AtomDB interface:

• Redis + MongoDB (current, primary) — Redis for fast key-value lookups and caching; MongoDB for persistent document storage. The standard production deployment.
• In-memory (current) — For development and testing without external services.
• MORK / MorkDB (code-real with caveats) — DAS-side MorkDB AtomDB class (subclass of RedisMongoDB at MorkDB.h:43) talks to a MORK HTTP server (deployed from MORK's server branch). Add-side: MorkDB.cc:197-281 generates MeTTa expressions and posts them to MORK while upserting Mongo metadata. Read-side: MorkDB.cc:150-186 converts link schema → MeTTa, calls MORK export, decodes results. Delete-incomplete: MorkDB.cc:268-270 hard-fails on link/S-expression delete; flush_pattern + re_index_patterns provide batch-rebuild only, NOT live mutable-store CRUD. Server pin: das/src/docker/mork/Dockerfile.server pins MORK 578a759 (2025-07-21). See the “How DAS Relates to MORK and ECAN” section above for the full server-branch drift caveat.
• RocksDB (via adapter) — Single-host persistent storage using embedded key-value store.

Expression hashing and HandleTrie data structures provide efficient content-addressed storage and retrieval across all backends.

Mathematical / Formal Foundations

The "TrueAGI State Management Platform" design document describes the mathematical foundations for DAS's next-generation distributed architecture:

• State-Transition Monoid: Each user interaction or AI update is treated as an endomorphism \(\delta_i : S \to S\), where sequences form a free monoid \(I^*\). A snapshot after inputs \(i_1 \ldots i_n\) is \(s_n = \delta_{i_n} \circ \cdots \circ \delta_{i_1}(s_0)\). This gives a precise way to record, snapshot, and replay state.
• Coalgebraic Streams: Each AI instance defines streams of outputs and next states as an \(F\)-coalgebra. Final coalgebra uniqueness guarantees consistent replay semantics.
• CRDT Join-Semilattices: For geo-distributed deployment, the state space \(S\) is equipped with a join-semilattice \((S, \sqcup)\). Concurrent branches merge via \(s_A \sqcup s_B\), ensuring associative, commutative, and idempotent convergence without coordination overhead.
• Lenses and Permissions: Category-theoretic lenses (get, put) project and update subviews of state, obeying round-trip laws. These map directly to Rholang contracts on MeTTaCycle for permissioned access control.

These foundations are described in the design document as a proposed architecture; the current DAS implementation uses simpler replication and access control mechanisms.

System Interfaces and Dependencies

• MORK: DAS provides distributed persistence via MorkDB; MORK handles in-memory triemap processing on the server side. See the “How DAS Relates to MORK and ECAN” section above for the split-implementation caveats and server-branch drift.
• ECAN: The Attention Broker is an ECAN-compatible engineering surrogate (not literal 2009 ECAN); see Attention and Motivation for the broader attention lineage.
• ASI Chain: In decentralized mode, DAS event streams replicate across ASI Chain nodes with CRDT merging. Rholang contracts enforce permissioned access.
• MeTTa: DAS is queryable from MeTTa via hyperon-experimental's new-das! op (das.rs:156-199) and the hyperon_das Python package, both routing through the Space API.

Historical: legacy AtomSpace storage ecosystem →

Implementation Anchors

• das (primary) — C++ with Python client (hyperon_das). Bazel build via Docker. Query agents, Attention Broker (ExactCountHebbianUpdater + TokenSpreader), HandleTrie, MorkDB AtomDB backend.
• das-metta-parser — C Flex/Bison parser loading MeTTa expressions into MongoDB/Redis backends.
• das-toolbox — Python CLI tools (das-cli) for DAS infrastructure management; defaults to image tags trueagi/das:mork-server-1.0.5 and mork-loader-1.0.5.
• MORK server branch — Pinned via das/src/docker/mork/Dockerfile.server at MORK commit 578a759; see MORK Full Status for drift detail.
• Hyperon Experimental DAS feature — hyperon-experimental/lib/src/metta/runner/builtin_mods/das.rs; das feature default-enabled with metta-bus-client from singnet/das tag 1.0.2.
• Legacy: atomspace-storage, atomspace-rocks, atomspace-cog

Current Status

Last verified: 2026-06-01

• Operational: DAS query engine; AtomDB with Redis+MongoDB AND code-real MorkDB AtomDB backend (with delete-incomplete caveat); Attention Broker (ECAN-compatible engineering surrogate; ExactCountHebbianUpdater + TokenSpreader) with Hebbian activation spreading; MeTTa parser; CLI toolbox; Python client; hyperon-experimental new-das! bridge.
• Under development: Production reconciliation of MORK server-branch pin / image tags / origin/server HEAD; Decko-compatible mutable-store semantics (link delete blocks live CRUD); improved query agent specialization.
• Proposed: State Management Platform with CRDT geo-replication, coalgebraic replay semantics, Rholang lens contracts for permissioned access, Kubernetes-orchestrated multi-region deployment.

Open Problems / Research Directions

• MORK + DAS integration — defining the boundary between hot in-memory processing and cold distributed persistence; reconciling the three server-branch references for production.
• Live mutable-store CRUD over MorkDB — link/S-expression delete is currently blocking; flush_pattern batch-rebuild is not Decko-compatible.
• CRDT convergence at scale — ensuring efficient join-semilattice merging for large, rapidly evolving AtomSpaces.
• Attention Broker scaling — maintaining Hebbian activation efficiency as graph size grows.
• ASI Chain integration — event stream replication with cryptographic provenance.

Primary Sources

• Goertzel, B. (2025). Hyperon for AGI⇒ASI Whitepaper, §2.5–2.6: Distributed Execution, State Management.
• Goertzel, B. (2025). TrueAGI AI State Management Platform: Initial Crude Design and Development Plan.
• Source-code review archive (2026-04-29): the AtomSpace backend-integration extraction + reconciliation files at scripts/archive/atomspace_pilot/ in the wiki repository are the provenance for the MORK-coupling and AttentionBroker corrections on this card.

---

Related cards: AtomSpace · MORK · ASI Chain