Self-Programming= Learning about Intelligence-Critical System Features
← Back to Publications
Self-Programming = Learning about Intelligence-Critical System Features
Authors:
Year: 2011
Venue: Self-Programming in AGI Systems Workshop (AGI-11)
Links: paper (PDF)
Summary
Offers a precise characterization of “self-programming”: a system self-programs when it learns about elements of its own “cognitive infrastructure” — the fuzzy set of “intelligence-critical” features whose quality most affects the system's overall intelligence (framed as an optimization over a multi-feature system). The paper argues that the distinction between self-modification and ordinary learning is fundamentally fuzzy and formalizes it through this feature-criticality lens.
Relevance to Hyperon
A conceptual-foundations source for the Self-Modification and Safety Deep Dive — it defines what self-modification (self-programming) means and how it relates to ordinary learning, underpinning the card's treatment of self-modification as a first-class, typed, bounded operation rather than ad-hoc patching.