Sensory

help edit space_dashboard
Draft — This content has not been approved for publication.

Author: Linas Vepstas (sole author)

Repo: opencog/sensory @ 64849f3 (master, 2026-02-24)

Status: Operational on the AtomSpace-Scheme stack. ~733 commits. 7 wired sensory atom types with per-type C++ factory functions and Scheme runtime examples. Identified as the previously-unmapped Hyperon-era perception bridge by the Perception/Neural-Symbolic cluster pilot Source 5 (2026-05-01).

What Sensory Is

Sensory is a perception/action substrate for AtomSpace: it provides AtomSpace-native input nodes for files, sockets, terminals, IRC chat, and LLM prompts/embeddings, plus the C++ factory layer and Scheme bindings to instantiate and consume them at runtime. The 705-line README at 64849f3 explicitly frames the project as a perception/action revival on AtomSpace, positioned against failed prior embodiment efforts (Hanson Sophia ROS+Blender; Minecraft MineRL+Malmo+ROCCA; Unity pet-dog avatars; Ethiopian soccer-robothon).

Sensory does not cite DeSTIN, FISHGRAM, or the 2013 Goertzel/Sanders/O'Neill/Yu paper architecture by name -- pickaxe across the repo at HEAD AND in git history returns zero hits for fishgram, sushigram, DeSTIN tokens. It is a BROAD perception/action revival, not a specific 2013-paper-architecture revival. See the 2013 paper card for the editorial-terminology note on this distinction.

Wired Sensory Atom Types

Seven sensory atom types are declared at opencog/sensory/types/sensory_types.script:29-51 and built via opencog/atoms/CMakeLists.txt:7-14:

Atom typeFactoryUse
TEXT_FILE_NODETextFileNode.cc:296Text file open/read/write
FILE_SYS_NODEFileSysNode.cc:515Filesystem traversal/listing
UNIX_SOCKET_NODEUnixSocketNode.cc:310UNIX domain socket I/O
TCP_SOCKET_NODETcpSocketNode.cc:320TCP socket I/O
TERMINAL_NODETerminalNode.cc:216Terminal/console I/O
I_R_CHAT_NODEIRChatNode.cc:528IRC chat client
OLLAMA_NODEOllamaNode.cc:676Ollama LLM generate / chat / embed (optional, gated via CMake FIND_PROGRAM at CMakeLists.txt:45-55, 71-72)

All 7 are [WIRED] per the cluster-pilot 3-state classification (no [STUB-NOT-WIRED] and no [UNIMPLEMENTED] in this set).

OllamaNode Dual Characterization

OllamaNode supports two distinct uses, both runtime-wired and both with non-trivial implementation. The Perception cluster pilot V5-2 split-verdict resolution treats them as separate characterizations rather than collapsing them into a single label:

  • [LLM-MEDIATED-PERCEPTION] -- IRC text-stream wiring at examples/ollama-bot.scm:104-141 (:20-35 setup). The LLM is the intermediate language-understanding layer between IRC sensory I/O and the AtomSpace. This is sensory I/O where an LLM happens to be the parsing layer.
  • [LLM-AS-KNOWLEDGE-SOURCE] / [LLM-AS-MEMORY-SUBSTRATE] -- embedding-vector extraction at examples/ollama-embed.scm:20-31, 42-46; storage via *-embedding-* key at OllamaNode.cc:629-664. Embeddings are used as semantic-memory primitives. This is not perception -- it's external-memory access via LLM-vended representations.

Both characterizations co-exist in the same OllamaNode implementation but address different use-cases. The duality is a concrete instance of how LLM nodes resist single-label classification in cognitive-architecture terms. Definition at OllamaNode.h:34-45; generate/chat/embed paths at OllamaNode.cc:362-430. C++ HTTP via cpp-httplib. Tests at tests/sensory/ollama/ollama-embed-test.scm (Scheme).

Sister Portfolio (Vepstas Hyperon-era research portfolio)

Sensory is part of a 5-repo sole-authored Vepstas portfolio plus 2 co-authored repos. Cross-references in READMEs confirm portfolio cohesion: the sensory README cites evidence, motor, agents, atomese-simd; the motor README cites sensory and agents.

Sister repoHEADCommitsRole
opencog/agents0718ba089Pure-Atomese learning agents. experiments/02-memory/CMakeLists.txt:14-15; count-agent.scm:37-40, 73.
opencog/motord5ba07e76Sensory-backed crawler project. demos/crawler.scm:15-20, 69-81, 200-213, 268.
opencog/evidenceba0314c30Theory/docs portfolio (CLAUDE.md:20-23; README.md:141-155, 210-216); not executable integration.
opencog/atomese-simdb8be282372OpenCL/CUDA Atomese/Sensory acceleration. 4 .cl + 1 .cu files. CMakeLists.txt:84-101, 115-177.
opencog/miner(co-authored with Nil Geisweiller)--The C++/Scheme symbolic miner that the hyperon-miner trio ports to MeTTa.
opencog/vision(co-authored with Adrian Borucki)--Minimal AtomSpace-OpenCV scaffolding (5 wired ImageLink operation types + 1 stub-not-wired); not a perception pipeline. See Vision.

Stack Identity: AtomSpace-Scheme, not MeTTa-runtime

Sensory and the entire Vepstas portfolio are [ATOMSPACE-SCHEME-STACK]:

  • C++ implementation with Guile Scheme examples and tests.
  • No .metta files anywhere in the portfolio.
  • No hyperon-experimental bindings.
  • Uses cpp-httplib for Ollama HTTP, AtomSpace C++ API + Scheme bindings throughout.

This is a cluster-narrative-deciding finding: the Hyperon ecosystem operates on TWO parallel stacks, both receiving concurrent active development as of 2026-04-30:

  • AtomSpace-Scheme stack: Vepstas perception portfolio + opencog/miner + opencog/vision.
  • MeTTa-runtime stack: hyperon-experimental + hyperon-miner trio + neural-subgraph-matcher-miner + metta-attention.

Bidirectional cross-grep between Sensory + sister repos and the MeTTa-runtime mining stack returned ZERO references in either direction (Codex token list: sensory, opencog/sensory, OllamaNode, neurospace, torchme, ImageLink, Fishgram, sushigram, DeSTIN, hyperon-miner, neural-subgraph). The two stacks operate in code-isolation. [PARALLEL-NON-INTEGRATED] holds.

Architecture Status: [PARTIAL-FRAGMENTED-REVIVAL]

The 2013 DeSTIN-FISHGRAM-PLN architecture has fragments scattered across parallel research tracks but no end-to-end runtime pipeline:

  • Perception layer: opencog/sensory (this card).
  • Symbolic mining layer: opencog/miner + hyperon-miner trio + neural-subgraph-matcher-miner.
  • Inference layer: hyperon-pln + chaining.

The layers are not interconnected at runtime. Sensory → Miner: 0 hits. Sensory → Hyperon-miner: 0 hits. The LLM-mediated path via OllamaNode exists but is NOT wired to any downstream reasoner (PLN or Miner). The cluster-pilot verdict is [PARTIAL-FRAGMENTED-REVIVAL] -- generic architectural concerns recur in fragmented form across the three traditions but no end-to-end pipeline ties them together.

Quick Start

# Build sensory (requires AtomSpace + cogutil)
cd sensory && mkdir build && cd build && cmake .. && make && sudo make install

# Ollama embedding example (requires Ollama installed and running)
guile -L /usr/local/share/opencog/scm \
      examples/ollama-embed.scm

# IRC chat bot (requires IRC connection details)
guile -L /usr/local/share/opencog/scm \
      examples/ollama-bot.scm

# Run sensory tests
cd build && make test

Open Questions

  • Cross-stack bridge. If Hyperon production work consolidates on the MeTTa-runtime stack, will the Vepstas perception portfolio be ported to MeTTa, or will an AtomSpace-Scheme ↔ MeTTa-runtime bridge be built? No work on either path is documented at this snapshot.
  • Downstream reasoner wiring. Sensory provides runtime-wired perception input. No current wiring routes that input to PLN or to any pattern miner. Bridging the perception layer to a reasoner is the structural gap that [PARTIAL-FRAGMENTED-REVIVAL] describes.
  • OllamaNode classification. The dual [LLM-MEDIATED-PERCEPTION] / [LLM-AS-KNOWLEDGE-SOURCE] characterization is a cluster-pilot resolution but suggests OllamaNode wants a richer cognitive-architecture vocabulary than perception alone provides.
  • Higher-layer sensory goal. README.md:636-648 sketches a higher-layer sensory goal that is not yet realized in code.

References



Discussion