This document describes the core principles and philosophy behind graphden. For technical architecture, see ARCHITECTURE.md. For packages system, see PACKAGES.md. For implementation status, see ROADMAP.md.
The one-sentence why (canonical — keep every outward surface in
sync): Graphden makes software something your whole team can see —
and your AI can safely change. The living, typed graph is the
mechanism, not the headline: because the program is a graph and not
text in files, engineers review structural diffs instead of prose,
non-programmer teammates compose ready-made typed blocks without being
able to break them, and AI edits land as inspectable, effect-gated
changes. Lead outward-facing copy with the benefit; introduce the graph
one beat later as the how. Surfaces that carry this line and must stay
in sync: the graphden.dev hero (graphden-cloud
resources/packages/landing/landing/fns.edn), this repo’s README lead,
and the social profiles/banners (docs/brand/).
Graphden is built first as the author’s own tool — to make day-to-day software work less annoying — and only secondarily as a product. This ordering is load-bearing for design decisions:
When evaluating a proposed change, the question is not “does someone want this?” but “does this remove a pain the author has, or does it unlock a deployment shape the author needs?”. If neither — defer.
AI-assisted editing is a first-class consumer of the model. The author uses AI in their daily work and hits the text-diff review pain (whitespace / line-break / formatting noise drowning out the behaviour change) every time the AI proposes a multi-file edit. Graphden’s entity model makes graph-diff review possible — an AI proposal lands as “added N fn-defs, changed M bindings, deleted K refs” against the current graph, reusing the branch-diff UI that already exists. Two consequences for design:
See ROADMAP Block 9.
The graph substrate is not one audience — it stratifies naturally into three roles that compose over the same entities, so a person (or an AI) can work at exactly the altitude they’re comfortable with and hand the result up:
impls.clj
base-fns and the fns.edn that wrap them into an integration — a
Telegram binding, a Google-Sheets writer, a Postgres query surface.
Their craft is type design and interface design: picking narrow
types, naming slots so a stranger understands them, and exposing a
small, honest public surface for the package. In LEGO terms they
mould new brick shapes (and, rarely, new ways bricks connect —
discouraged, but possible under real need). This tier is where deep
programming skill lives.The point of the stratification is to insert intermediaries into the
usual “programmer → end user” chain: to build a complex discount rule
you should not also have to solve concurrency, secret storage, and
hosting. The expert tier solves those once, correctly, inside a package;
the composer tier reuses the result by shape. This is also the
monetisation seam — building is free, keeping a composition running
(as a service) is the billed layer.
Why AI fits the graph — and fits tier 2. Two properties compound:
Taken together, an AI can plausibly be the composer tier — installing packages the expert tier published and wiring them by type — while the human stays in the reviewer seat over a legible graph diff. Stated as a bet the product now has to test in the market, not a settled fact.
In any popular system the trade-offs are chosen by someone else — language designer, framework author, vendor. Those choices often optimise for the chooser’s pain, not the user’s, and stay frozen long after the original constraint has passed (because too many downstream users now depend on them).
Graphden is built so that the source of trade-offs is the author. Decisions about what to optimise for — readability vs. performance, expressiveness vs. minimal model, explicitness vs. brevity — are made internally and can be re-evaluated at any time, because no external party is locked into the previous choice.
This is both a reason to build the tool from scratch (existing tools bake in someone else’s trade-offs) and a guard rail for weighing external pressure: the contributions that land easiest are the ones that add capability without putting anyone else in vendor-position for the author — a new base-fn, a package, a bug fix, a doc, a feature the author would also reach for. A request that would lock the author into supporting a use case they don’t share isn’t a bad idea; it’s just one the author can’t take stewardship of, and it fits better as a downstream package or fork than in the core.
Practical consequences:
This is the load-bearing reason for “tool for myself first, product second” — only the author-as-sole-source-of-trade-offs framing guarantees the author retains the ability to make this tool ever work the way it should.
A programming language is a bridge between:
Humans rarely understand exactly what they want or how to achieve it. Computers can help, but they need precise instructions. The journey from human intent to machine execution requires multiple steps: understanding, translation, verification.
Programming languages help traverse this path. The question is: how optimally?
Modern programming languages use text as the primary representation:
Machine side: Text → AST → bytecode/machine code. This works well — it’s fast, precise, and reliable. No complaints here.
Human side: Humans write and read text to express their thoughts. This is where we see room for improvement.
Problems with text-based code:
let for single-use values — Clojure (and most other languages) name intermediate values even when they are used exactly once, purely to shorten the surrounding form. The name has no semantic value; it is a layout device.Every file in a classical project starts with a block of imports declaring which namespaces / libraries / modules it depends on. The import block is visual noise that the system could derive from which names the file actually references — and indeed many IDEs auto-manage it. But the noise stays on screen.
Worse, opening a library to understand or extend it is a separate step that requires navigating to that library’s source, switching projects, possibly checking out a different repository. In practice this step is skipped: a library is treated as a black box documented by its README. Extending or improving a library while using it is rare even for one-line changes.
Graphden’s “code = graph in DB” removes both: there are no files, so no imports; libraries (packages) are graphs of fns in the same DB, visible and editable in the same UI as user code.
Production software has a chain of roles with very different technical depth (developer, integrator, domain expert, end user — see § Role Chain for the full breakdown). Classical languages give us one knob to expose code: source-code visibility. This forces a binary:
What we want is a graded surface where each role sees only the operations and primitives it needs, and adding a new operation does not require building a new UI from scratch. This is a top-level motivation for graphden, not just a feature — see § Role Chain and Extensibility for the design that addresses it.
Almost all code, regardless of language, is first converted to an Abstract Syntax Tree (AST) before execution. An AST is a graph. This suggests that graphs are a natural representation for computation.
| Aspect | Graph Representation |
|---|---|
| Visual | Nodes connected by edges — intuitive to read |
| Editing | Connect nodes, break connections — direct manipulation |
| Storage | Structured data (entities, relationships) — queryable, versionable |
| Execution | Traverse graph, evaluate nodes — natural for lazy evaluation |
Goal: Match or exceed classical languages in execution speed, safety, and correctness.
Strategies:
Non-negotiable: We will not sacrifice correctness for any other goal.
Goal: Make development simpler than text-based programming.
This breaks down into:
Following SICP, a language has three aspects:
In graphden:
value / ref-fn-id); binding-list-item.position for ordered chainsparent-ids inheritance + bindings)We resist adding new entity types or edge types. Every addition increases cognitive load and implementation complexity.
Abstractions must minimize the need to define anything twice:
ref-fn-id within execution) enables sharing computed resultsparent-ids + bindings) enable reuse without duplicationMust have: Everything expressible in classical languages should be expressible in graphden. No “sorry, you can’t do that here.”
Must avoid: Unnecessary expressiveness. Examples:
let bindings in Clojure are essential for Clojure, but graphs name things differentlyThe system should support (or enable) development tools:
| Tool Category | Classical Languages | Graphden Approach |
|---|---|---|
| Version control | Git (text diffs) | Graph versioning via VersionedStorage decorator — branches, fork-point conflict detection, merge |
| Testing | Unit test frameworks | Isolated node execution with mocked dependencies |
| Linting | Static analysis | Graph structure constraints (already enforced) |
| IDE features | Indexing, navigation | Database queries (inherent) |
| Debugging | Stack traces | Execution path traces (see Open Questions) |
Five entity types, with deliberate roles:
| Entity | Purpose |
|---|---|
fn |
Function entity OR type-row. Inheritance via parent-ids (M:N). |
slot |
Atomic (name, type-fn-id) pair. Immutable post-create; shared across fns. |
fn-slot |
Junction: which slots a fn exposes, with position. |
binding |
Per-(fn, slot) overlay (value, ref, type-override, terminal, list flags). Renames are rename-view slots via slot.source-slot-id, not a binding column. |
binding-list-item |
Sequence content under a list-typed binding. |
Slots are first-class because two fns can carry the same slot identity, which is how multiple-inheritance recognises that “two parents expose the same parameter” rather than colliding on names.
References point at fns via binding.ref-fn-id (or
binding-list-item.ref-fn-id for sequence items). The executor
dispatches based on the slot’s effective type:
| Resolved slot type | Behavior |
|---|---|
:fn |
Pass fn-id directly — for HOF callables |
| anything else | Execute the ref, use its result (cached per ref-fn-id) |
The legacy :is-fn boolean flag on args was retired — slot.type-fn-id
(overlaid by binding.type-override-fn-id) IS the HOF marker. One
affordance, one concept.
Base functions are minimal primitives that wrap Clojure/Java capabilities.
They should be:
Examples of GOOD base functions:
| Name | Purpose | Why it’s a base-fn |
|---|---|---|
add |
(apply + nums) |
Wraps Clojure + |
http-server |
Start http-kit server | Wraps http-kit library |
render-hiccup |
Convert hiccup to HTML | Wraps hiccup library |
router |
Create Ring router | Wraps reitit library |
query-entities |
Query storage | Access to storage protocol |
env |
Get environment variable | Access to system |
Examples of BAD base functions (anti-patterns):
| Name | Problem | What it should be |
|---|---|---|
graph-editor-server |
250+ lines, hardcoded routes, HTML, CSS, JS | Multiple fn-defs composing router, http-server, render-hiccup |
user-login-handler |
Hardcoded auth logic | fn-def composing query-entities, if, json-response |
dashboard-page |
Hardcoded page structure | fn-def composing html-page, with-htmx, etc. |
Rule of thumb: If a base function contains:
…it should be a fn-def (graph composition), not a base-fn.
The graph editor UI is built as fn-defs across the core/web/app packages:
;; In web/response/fns.edn — reusable response block (MI over a
;; content-type axis + a status axis); free arg: :body
{:name :json-ok-response
:parents [:json-content-type :ok-response]}
;; In app-base/routes-method/fns.edn — a GET route template;
;; free args: :path :handler
{:name :get-route
:parent :route
:args {:method "get"}}
;; In app/editor/fns.edn — editor UI composition
{:name :page
:parent :html-page
:args {:title "Graphden - Graph Editor"
:head :head
:body :body
:scripts :_editor-scripts}}
;; In app/routes/fns.edn — a concrete route leaf
{:name :health
:parent :get-route
:args {:path "/health" :handler :_health-handler}}
;; In app/server/fns.edn — server composition
{:name :web-server
:parent :http-server
:args {:handler :_app-error-bounded
:port 8080}}
This approach:
:json-ok-response are reused)fn report
binding {slot s-sales, ref-fn-id calculate-sales} ← executes calculate-sales
binding {slot s-snapshot, ref-fn-id calculate-sales} ← same ref-fn-id, result cached
This enables:
ref-fn-id = computed once)We wanted a single reference behavior, but HOFs need to receive
functions as values. Rather than a side-channel flag, a slot
typed :fn IS the HOF marker — the editor shows it through the
type chip, the executor dispatches on it. One affordance for one
concept.
The minimum needed for the model we want:
fn carries inheritance + type-row metadataslot and fn-slot separate “what is this parameter” from
“which fns expose it” — sharing slot identity is what makes MI
inheritance picky about identity vs. namebinding overlays per-fn customisation without forking the slotbinding-list-item keeps sequence content indexable so
reverse-queries (“which fns ref X via list?”) stay cheapEach entity earns its row. Removing any would either lose
expressiveness (no slot sharing → no MI), or push semantics into
ad-hoc fields on fn (binding overlays inside the fn entity →
versioning per-fn instead of per-binding, lost dedup).
Concern: “Storing every function in a database will be slow.”
Response:
Evidence: Benchmarks needed, but architecture supports this claim.
Context-dependent semantics — “if child has special type X and another child has field Y, then compute parent differently.” This is complex and error-prone.
Implicit behavior — Everything should be explicit in the graph structure.
Magic — No hidden transformations or special cases.
The executor’s own filesystem — no base-fn writes a file or reads
one by path. The executor is shared (multi-tenant on the cloud, one
process per instance on self-host), and a path is the one input that
escapes both the effect gate and the type checker. Text comes in as
streams (:slurp of a request body), classpath resources (:read-resource*)
and HTTP; state that must outlive a request goes to SQL (:pg-execute,
web/sql) or the queue. The same boundary keeps HTTP bodies TEXT: a
binary payload is base64 (:base64-encode) or an object store behind
an HTTP API, not bytes in the graph.
These are recurring design temptations that look local-good but lose against principle 2 (Minimal entities). Recording them so future-us can re-derive the answer fast.
A test in graphden is a fn-def with all free-args bound —
already expressible in the existing model. The temptation is to add
fn.is_test (or a test entity), so the UI can filter tests out of
the main list and tools can run them in batch. We reject that:
tests namespace SEGMENT. Any named,
non-_-prefixed fn whose namespace path contains a tests segment
(tests.parser, myproj.tests.api) is a test. Zero schema change.
(Shipped as any-segment rather than the root-only prefix first
sketched here: a project’s tests then live inside the project’s own
root namespace, where a namespace filter on that root still sees them.
Matched by segment, never substring — testsuite is not a test
namespace. _-prefixed fns are the tests namespace’s private
scaffolding, not tests.)tests segment” — same cost as a flag.:assert / :assert-eq, the
runner + statuses, the editor surfaces, auto-run on writes.“The part of the graph I look at” — a set of namespaces, everything
built on a fn, everything with an effect — is a filter set, and a
named one is a view. The temptation is a workspace / view
entity with a fn↔view M:N junction, or a text rule language the
server parses. We reject both:
graphden.explorer.filters /
.views), exactly like the branch choice.:explorer-view
base-fn, its axes bound as slots. uses and also preserve the original
scalar identity slots; uses-all and also-all hold additional identities
through ordinary const compositions with fn-ref value bindings.
References survive renames and do not execute their targets. The problems
slot stores selected predicates, evaluated against the current branch.
Versioned, branch-scoped, runnable
(▶ = the member list), listed back by GET /api/views. Zero schema
change, no DSL: the graph IS the rule.crud.entities.views/view-members), never
maintained by hand.exclude.Logging / intermediate-result capture is per-execution context, not
per-fn state. The temptation is fn.trace_enabled. We reject that:
trace is a runtime decoration applied by the executor when asked,
not a property of the fn entity. See
§ Open Questions → Debugging for the
runtime constraints.
This section maps each system component to the principles it serves. Use this to evaluate changes: modifying a component should improve its target principles without harming others.
| Entity | Principles Served | How |
|---|---|---|
fn |
Minimal entities, Expressiveness, DRY | Function or type-row; M:N inheritance via parent-ids |
slot |
Minimal entities, Explicit | Atomic identity for a parameter; sharing enables MI |
fn-slot |
Locality, Explicit | Which fn exposes which slot in what order |
binding |
Locality, Explicit | Per-(fn, slot) overlay; closer-fn-wins makes inheritance lookups O(chain length) |
binding-list-item |
Indexability | Sequence content as rows so reverse-queries stay cheap |
| Component | Principles Served | How |
|---|---|---|
storage-protocol |
Correctness, Minimal entities | Generic CRUD interface for all backends; schema-agnostic |
postgres-storage |
Performance, Correctness | Production-grade ACID transactions; recursive CTE for cycle walks |
| Component | Principles Served | How |
|---|---|---|
storage/protocol/graph |
Correctness, Modularity | ExecutionGraphResult record + accessor functions |
graph-data-schema |
Minimal entities | The five core entities (fn, slot, fn-slot, binding, binding-list-item) |
versioning/storage |
Dev simplicity | Versioned-decorator wrapping any base storage |
| Component | Principles Served | How |
|---|---|---|
executor |
Correctness, Performance | Lazy evaluation; depth/timeout protection |
base-functions |
Expressiveness, Minimal entities | Rich library; no need for user to implement basics |
fn-registry |
DRY, Correctness | Single registration point; deterministic UUIDs |
fn-composition |
DRY, Explicit | Data-driven composition; no hidden behavior |
| Constraint | Principles Served | How |
|---|---|---|
| No dependency cycles | Correctness | Prevents infinite loops at write time (binding.ref-fn-id graph) |
| Uniqueness keys | Correctness | fn-slot/binding schema UNIQUE; fn (namespace, name) + binding-list-item per-branch resolved-view |
| Decision | Principles Served | Trade-off |
|---|---|---|
| Slot type IS the HOF marker | Expressiveness, Explicit | One concept (type=:fn) instead of two (type + flag) |
| Union value/ref-fn-id on binding | Minimal entities | Mutual-exclusion checked at write |
| Result caching by ref-fn-id | DRY, Performance | Cache lives in executor, not schema |
| Slots shared across fns by id | DRY, MI correctness | Sharing is opt-in via fn-slot pointing at same slot-id |
| Bindings overlay per (fn, slot) | Locality | One row per customisation; no fork-on-rename |
| Component | Principles Served | How |
|---|---|---|
system/* |
Dev simplicity | Integrant lifecycle wires storage + executor + base-fn registry + http server |
packages/* |
Dev simplicity | One directory per package; loader auto-syncs at startup |
| Component | Status | Target Principles | Expected Trade-offs |
|---|---|---|---|
| Distributed execution | Planned | Performance | Complexity in coordination |
| Type system | Done (refinements / records / lists / unions / variants) | Correctness, Dev tools | Save-time check + rich-type registry |
| Versioning | Done (VersionedStorage decorator) |
Dev tools, Correctness | Storage overhead; migration complexity |
| Permissions | Planned | Correctness | Query overhead; complexity |
| Visual UI | Done (HTML overlays + SVG edges, no graph library) | Dev simplicity | Large implementation effort |
When proposing a change:
Example evaluation:
Proposal: Add
priorityfield tofnentity for execution ordering.
- Affects: graph-data-schema, executor
- Improves: Performance? (maybe, if we can optimize hot paths)
- Violates: Minimal entities (+1 field), Explicit (implicit ordering)
- Verdict: Reject — ordering should be explicit in graph structure, not metadata
In traditional software, the chain from code to end user is short: a developer writes code, users use the product. But in practice, there are always intermediate layers with varying levels of technical expertise:
| Role | Technical Depth | Domain Knowledge | Examples |
|---|---|---|---|
| Platform developer | Deep (writes Clojure, base-fn) | Low | Core team |
| System integrator | Medium (composes graphs, configures) | Medium | DevOps, architects |
| Domain builder | Low (uses pre-built blocks) | High | Forum admin creating categories, CMS editor |
| End user | None | Full (their own domain) | Forum user, app consumer |
Even a classic forum has this: developers write the engine, admins create sections and configure rules, users post content. Modern no-code/low-code platforms extend this chain further.
Graphden should support this entire chain. Through access levels to functions and graph operations, each role sees only what they need:
fn entity with a return-type-fn-idThe boundaries between roles are enforced by the permission system, not by technical barriers. A domain builder doesn’t need to know Clojure — they work with the same graph primitives, just at a higher level.
The core system is simple: base-fn implementations + graph composition (fn, slot, fn-slot, binding, binding-list-item) + executor. But production features — versioning, caching, logging, permissions, environment management, secret storage — all require modifications to either storage (new fields, tables, query logic) or executor (new resolution steps, middleware).
Hardwiring these features into the core has problems:
| Approach | Description | When to Use |
|---|---|---|
| 1. Hardwired | Extension is part of core storage/executor code | Security-critical, performance-critical, or fundamentally inseparable from core |
| 2. Component | Separate module wrapping core with additional behavior | Most production features (versioning, caching, permissions) |
| 3. Graph-native | Extension described in the same graph language as user programs | Logging, monitoring, domain-specific middleware |
Current assessment of planned features:
| Feature | Recommended Approach | Reasoning |
|---|---|---|
| Core executor loop | 1 (hardwired) | Bootstrapping problem — cannot execute graph without executor |
| Core storage CRUD | 1 (hardwired) | Same — cannot store graph without storage |
| Versioning | 2 (component) | Storage wrapper: VersionedStorage(BaseStorage) |
| Caching | 2 (component) | Already implemented as CachedStorage wrapper |
| Permissions (enforcement) | 2 (component) | Security-critical — must not be bypassable by user graph |
| Permission policies | 2-3 (data/graph) | Policy rules can be declarative data or graph predicates |
| Secret storage | 2 (component) | Infrastructure concern, not expressible in pure graph |
| Environment management | 2 (component) | Branch-level metadata, storage concern |
| Logging | 3 (graph-native) | with-logging base-fn wrapper, user controls placement |
| Domain middleware | 3 (graph-native) | User-defined request/response processing |
Current storage implementations must satisfy a long protocol interface. The goal is to reduce this to a minimal set of generic operations:
Minimal Storage:
put(entity-type, id, data)
get(entity-type, id)
query(entity-type, predicates)
delete(entity-type, id)
apply-migration(migration-spec)
Extensions declare their needs through migrations (new fields, indexes, tables) rather than requiring protocol changes. Each storage backend translates migration specs into its own DDL. This allows adding versioning, caching, or permissions without modifying the base storage interface.
Independent composability requirement: Each feature module (caching, versioning, permissions, etc.) must be an independent component implementing the storage decorator pattern. Any combination of decorators must work correctly:
BaseStorage (minimal CRUD)
Valid compositions (any combination):
BaseStorage — no extras
CachedStorage(BaseStorage) — cache only
VersionedStorage(BaseStorage) — versioning only
CachedVersionedStorage(VersionedStorage(BaseStorage)) — both
PermissionStorage(CachedStorage(BaseStorage)) — permissions + cache
... any other combination
The executor works with any storage through the unified ExecutionGraph protocol and does not need to know which decorators are active. Each decorator is transparent to layers above it.
When two features interact (e.g., cache invalidation on branch merge), a dedicated combining module handles the interaction rather than coupling the features directly.
The system separates concerns into three distinct layers:
┌─────────────────────────────────────────────────────────────────┐
│ EXECUTOR │
│ - Receives fn-id, executes function graph │
│ - Knows about laziness (Clojure delays) │
│ - Resolves base-fn implementations │
│ - Does NOT know about storage details │
└─────────────────────────────────────────────────────────────────┘
│
│ uses ExecutionGraph + StorageCRUD protocols
▼
┌─────────────────────────────────────────────────────────────────┐
│ GRAPH LAYER │
│ - Provides execution graph for a fn-id │
│ - Knows about graph entities (fn, slot, fn-slot, binding, │
│ binding-list-item) │
│ - Middleware pattern: composable storage decorators │
│ • Base storage — direct queries │
│ • CachedStorage(base) — DB-level caching │
│ • VersionedStorage(base) — branch version resolution │
│ - Does NOT know about execution semantics │
└─────────────────────────────────────────────────────────────────┘
│
│ uses StorageCRUD protocol
▼
┌─────────────────────────────────────────────────────────────────┐
│ STORAGE LAYER │
│ - Generic CRUD: create/read/update/delete/query │
│ - Schema-agnostic (works with any entity types) │
│ - Storage decorators modify CRUD behavior: │
│ • VersionedStorage — intercepts CRUD for version resolution │
│ • PermissionStorage — enforces access control │
│ - Does NOT know about graph semantics │
└─────────────────────────────────────────────────────────────────┘
│
│ uses DataSchema protocol
▼
┌─────────────────────────────────────────────────────────────────┐
│ DATA SCHEMA LAYER │
│ - Defines entity types, fields, constraints │
│ - Schema extensions are composable: │
│ • graph-data-schema — fn / slot / fn-slot / binding / │
│ binding-list-item │
│ • versioned-data-schema — branch + version table per │
│ versioned entity │
│ - Pure data definitions, no behavior │
└─────────────────────────────────────────────────────────────────┘
Key principle: Each layer depends only on the layer below it. The executor doesn’t know if storage is versioned or cached — it only sees ExecutionGraph + StorageCRUD. Storage doesn’t know about execution — it only sees CRUD operations.
Graph-specific protocols in the right place: The GraphConstraints and ExecutionGraph protocols belong in the Graph Layer, not Storage Layer:
| Protocol | Layer | Reason |
|---|---|---|
StorageCRUD |
Storage | Generic CRUD, schema-agnostic |
GraphConstraints |
Graph | Knows about fn → binding.ref-fn-id → fn relationships |
ExecutionGraph |
Graph | Resolves graph structure for execution |
Storage decorator pattern enables composition:
;; Direct storage (simplest)
(def storage (pg/create-storage cfg))
;; With versioning (branch resolution)
(def storage (vs/wrap-with-versioning (pg/create-storage cfg)))
;; Caching layer (planned)
(def storage (cs/wrap-with-cache (vs/wrap-with-versioning base) cache))
The executor takes any storage through the unified protocols:
(executor/create-context {:storage storage
:base-fns (registry/get-base-fns)})
This design follows Django’s “apps” pattern where each feature is an independent module that can:
ExecutionGraph implementations)The ultimate goal: the system’s own infrastructure is described in the same graph language it provides to users. Initialization graphs define storage setup, migrations, executor configuration. Extension graphs add fields, modify behavior. UI is a graph of components (e.g., HTMX + templating).
This is analogous to Lisp’s self-hosting capability: the language is expressive enough to describe its own extensions. The practical limit is the bootstrapping boundary — a minimal Clojure core (executor + base CRUD) that cannot itself be a graph, because it’s needed to execute graphs.
The editor frontend today is partially graph-described: its JS and
CSS are stored as :const fn-defs in resources/packages/app/editor/
(big strings of source) and the route composition is a graph. But
the structure of the UI — DOM tree, components, data bindings —
is text inside those :const strings, not graph nodes.
The author’s eventual goal is a single fully graph-described UI. That is a large project, so we commit to it as a two-step roadmap, where each step delivers value on its own:
Step 1 — Make the editor’s own JS/CSS editable from inside the running editor. (SHIPPED 2026-08-18.)
:resource-override entity (path → content, per
branch) shadows the shipped classpath asset; every bundle reads
through :read-resource-overridable. No rebuild and no blob store —
the override lives in Postgres like any other versioned row, and
:frontend-hash-effective rolls the ?v= asset hash on save so the
browser re-fetches. (S3 was considered and rejected: versioning and
atomicity are exactly what the existing storage already gives.)GRAPHDEN_DISABLE_ASSET_OVERRIDES
rescue hatch keep a bad override from bricking the editor.This removed the immediate pain: editing the editor’s own JS/CSS no longer means going to disk and rebuilding the jar.
Step 2 — Express UI structure as fn-defs.
element, attrs, text, plus a renderer that takes a fn-id and
produces the rendered HTML/HTMX/web-component.Step 2 is the deeper ambition (think Clojurescript / Hyperfiddle /
Electric Clojure but graph-native). It is not scheduled as a
near-term feature; it is logged here so that step 1 is built without
accidentally precluding step 2 — e.g. by hard-coupling the editor’s
DOM building to text-string :const blobs.
:fnClojure’s defprotocol packs three concerns into one declaration:
the shape of the contract, the registration of an
implementation, and the dispatch at the call site. In graphden
each of these falls out of existing mechanisms — no protocol
entity is added to the schema.
Contract shape — a type-row (a fn entity with no
return-type-fn-id and no parent-ids) whose slots are typed :fn declares
the contract. Each :fn-typed slot is one abstract method; the
slot’s effective signature is the method signature.
Implementation — a fn-def with :parent <type-row> and bindings
from each abstract slot to a concrete callable IS an implementation.
The existing sync-time type-check guarantees all required slots are
bound and the bound callables match the slot’s signature; no
separate “extend-protocol” mechanism needed.
Dispatch — free-arg injection. The canonical pattern is that a
consumer graph carries a free-arg whose type is the type-row, and
the concrete implementation is supplied at execution time or by a
parent graph one level up. This keeps dispatch explicit: which
implementation was used is visible in the graph structure, not in
ambient runtime state. An ambient-singleton alternative (“the fn-def
named :current-storage, referenced from anywhere”) is technically
possible but violates Principle 3 (Explicit) and is not the
recommended pattern.
Contract verification — protocol contract tests live in the
tests/ namespace (§ Patterns We Don’t Introduce)
under a sub-namespace named after the protocol. A test is a fn-def
with one free-arg of the protocol’s type and all other args bound;
running the contract suite against a concrete implementation =
passing that implementation as the free-arg’s value. This mirrors
how graphden.storage.protocol.contract-tests works at the Clojure
level, lifted into the graph.
Two shapes — the realisable one today, the aspirational one for later.
The runtime POC validated end-to-end in
test/graphden/integration/storage_protocol_poc_test.clj. Each
abstract operation is its own :fn-typed free-arg on the consumer;
the caller binds each to a concrete base-fn at the top of the
chain.
;; Consumer — declares one :fn-typed free-arg per abstract op.
{:name :user-profile-handler
:parent :call
:args {:func {:as :storage-query} ; :fn-typed by inheritance from :call
:arg {:select [:id :name] :from :users :where [:= :id 42]}}}
;; Top-level wire-up — bind each abstract op to a concrete base-fn.
;; Swap to a future backend = sibling top-level binding to its
;; equivalents (`:storage-query`, `:storage-execute`, `:storage-tx`).
{:name :web-server
:parent :http-server
:args {:handler :api-router
:storage-query :pg-query
:storage-execute :pg-execute
:storage-tx :pg-tx}}
Swappability works as designed: nothing in :user-profile-handler
or below names “postgres”. Only the top-level binding does. A
second backend ships its own equivalent :pg-* peers; the top-
level switch is the only edit point.
{:name :Storage
:type {:query [:fn {:hsql :jsonb} [:list :jsonb]]
:execute [:fn {:hsql :jsonb} :int]
:tx [:fn {:body [:fn {} :any]} :any]}}
{:name :postgres-storage-impl
:parent :Storage
:args {:query :pg-query :execute :pg-execute :tx :pg-tx}}
;; Consumer would receive ONE free-arg `:storage` of type :Storage
;; and navigate via :get on the record.
This shape is not runnable today. Two gaps surfaced during the POC attempt:
:postgres-storage-impl :parent
:Storage) does NOT auto-construct a record-of-fn-ids value at
run time. Records are type-level constructs; instance
construction is the body fn’s job.:assoc chain to build the map) won’t
keep :fn-typed values as raw fn-ids — :assoc :value evaluates
the ref and stores the result.Bridging the gap needs a new primitive — something like an
:as-callable base-fn that promotes a stored fn-id into a Clojure
callable, plus a record-of-callables construction path that
captures fn-ids without executing them. We did not ship this in
Block 1; it’s documented here so a future implementation has a
clear target.
For now, use Shape 1 for any new code. Both shapes deliver the same swappability guarantee — Shape 2 only adds compact “one free-arg” ergonomics, which Shape 1 trades for working-today.
Problem: Classical languages have:
Question: What’s the graphden equivalent?
Possible approaches:
Implemented (P1) — execution-path capture: the first approach
shipped as the trace? flag on the /api/execute submit body plus a
runtime-only per-fn traced set
(graphden.executor.compile-eager/set-traced-fn-ids! — deliberately
NOT a stored fn field, per § “Per-fn debug/trace toggles are not a
stored field”). Each traversed :ref invocation of a traced fn
records {fn-id, cache-hit?, duration-ms} (secret-touching fns record
a :hidden entry instead — constraint 4 below, applied at capture
time), snapshotted onto the :fn-execution row’s :path-trace field
with a 256 KB byte cap and oldest-first truncation (constraint 5’s
size half).
Implemented (P2) — the second approach (node highlighting) plus
the UI half of the first: the execute popover’s “Trace path” checkbox
(off by default) submits trace?, which binds the execution-scoped
trace-all sentinel — running a specific fn with the box checked IS
the “small subtree they explicitly select” of constraint 1 (capture
admits only frames actually reached inside that one execution, and
stays bounded by the P1 caps; there is still no global switch). The
per-fn traced set remains the selective gate for programmatic
captures and the P3 ambient-sampling hook. editor-path-view.js
renders a stored trace read-only: traversed fn cards highlight with
per-fn aggregate badges (invocation count, total/max duration, cache
hits, “[hidden — secret]”), off-canvas fns are listed without being
force-loaded, and the view clears on any canvas rebuild.
Implemented (P3) — ambient sampling + full intermediate-VALUE capture; every constraint below now has an enforcement site:
trace? flag (execution-scoped trace-all sentinel)
or membership in the runtime-only selective set
(compile-eager/set-traced-fn-ids!). Still no global switch
reachable from the UI.compile-eager/set-trace-sampling! (session-scoped runtime atom
*trace-sample-rate*, default 0.01, resets on restart — NEVER
persisted: no config key, env var, or stored field exists for it).
An fn in the selective set submitted WITHOUT trace? gets its path
captured at the rate; the draw happens ONCE per top-level execution
at run-future binding time (ambient-sample?), never per-node,
and sampled captures are path-only — never values. A full (100 %)
rate refuses without an explicit {:confirm-full true} second
argument — the programmatic mirror of the UI confirm. Backend +
REPL surface only, deliberately: no ambient-sampling UI ships until
a real need (the constraint protects the user from cost, not UI
completeness — see ROADMAP).capture-values? submit flag (implies trace?). The editor’s
“+ capture values” checkbox is a second-step control: disabled
until “Trace path” is checked, and checking it opens an explicit
confirm dialog with an estimated cost line (“up to ~N KB”, from the
fn’s client-known ref-closure size × the 4 KB per-value cap);
declining reverts it. Captured values render through the same
streaming JSON byte counter as :result persistence
(graphden.util.json-size, via
compile-eager/render-captured-value).:secret auto-skip) — unchanged mechanism,
extended guarantee: a secret-touching fn’s frame records
{:hidden :secret} and its value is never read into the capture
buffer — the value renderer is not invoked on that branch (unit
test asserts the renderer probe stays at zero). Hardened
2026-08-19 (registry/trace-capture-class): the classification
FAILS CLOSED — a frame with no registry entry hides as
{:hidden :unknown-type} instead of capturing; a
secret-returning (or unknown) frame POISONS its open ancestors
(consumers record :value-hidden :secret-derived, the dynamic
complement to the static taint rules); and every READ of a stored
trace re-redacts through the current registry
(persist/re-redact-path-trace), so a fn that became secret
after the run stops serving its historical values. See
SECRETS.md § Path-trace capture (T6).:value-truncated? true marker), 16 MB total value budget
enforced AT CAPTURE TIME with oldest-first drop
(:values-dropped? true tells the user), on top of the P1
10k-entry capture cap and 256 KB persist-side byte cap. The TTL
half, read honestly: the “1 hour after last view” example is about
capture BUFFERS — here the in-memory buffer is a per-execution atom
whose lifetime IS the execution (snapshotted and released when the
completion reaper fires), so there is no long-lived buffer to
expire; persisted :path-trace rows ride :fn-execution’s
existing 7/30-day TTL sweeper.Implemented (P4) — the time-travel half, as REPLAY rather than
live stepping: every trace entry now carries :seq/:parent-seq
(entry-order frame number + forcing-frame link), so a stored trace
reassembles into the call TREE. GET /partials/execute-trace renders
it server-side; editor-trace-view.js steps through it (row click /
◀ ▶ / arrow keys) with canvas highlight — “пошагово проследить” over
the record, deterministic and pause-free. The companion entry point
is the catch-next-request trap (crud.debug-capture, the trap
block on a service fn’s Runs tab): a one-shot, TTL-bounded,
org+branch-scoped arm that runs the
next matching HTTP request through the same trace machinery and
persists it as a standard :fn-execution row — no new entity, no new
viewer (docs/EXECUTION.md § Debug). A LIVE pausing stepper was
considered and deliberately deferred: laziness makes “the next step”
follow thunk-forcing order (cache hits skip bodies, :cell bakes at
compile, HOF lambdas run outside the seam), so a paused walk is
frequently LESS legible than the completed tree — and pausing holds
executor-pool threads against the watchdog. Revisit only if the tree
proves insufficient.
Non-negotiable constraints (kept verbatim from before the implementation — they remain the contract any future change must honour):
:secret-typed values. We already have the
:secret information-flow type-marker
(SECRETS.md). The capture pipeline reuses it:
any value whose static type is :secret-marked is never read
into the capture buffer. The fn that produced it shows as
“[hidden — secret]” in the trace.These constraints together define the answer to “what stops a user from accidentally turning debug capture into a DoS on themselves” — which is the single failure mode that matters for this feature.
Problem: In classical languages, let bindings name intermediate results for readability:
(let [users (fetch-users)
active (filter :active users)
emails (map :email active)]
(send-newsletter emails))
Question: How does graphden handle this?
Current answer: Result caching by ref-id provides sharing, but primarily for efficiency, not readability. Visual UI may need to support annotations or labels.
Problem: Text-based errors reference line numbers.
Question: What do graphden errors reference?
Current answer: fn-id, arg-id, execution path. Visual UI can highlight the relevant node. But textual representation needs thought.
Problem: Changing base-fn args affects all composed fns that inherit from it.
Question: How to handle breaking changes?
Possible approaches:
VersionedStorage decorator + branch fork-point conflict detection)| # | Principle | Why |
|---|---|---|
| 1 | Correctness first | No feature justifies bugs |
| 2 | Minimal entities | Each addition increases complexity everywhere |
| 3 | Explicit over implicit | Behavior should be visible in graph structure |
| 4 | DRY | Never define the same thing twice |
| 5 | Expressiveness parity | Can do everything classical languages can |
| 6 | No unnecessary expressiveness | Don’t add features just because we can |
| 7 | Locality of changes | Changing one node shouldn’t require changes elsewhere |
| 8 | Incrementality | Adding features shouldn’t require rewriting existing ones |
Every change to the project must improve at least one principle without violating others.
If a proposed change:
This document serves as the philosophical foundation for design decisions.
A base-fn impl should ideally be 1-2 lines: call the library, return the result. Everything else — composition, defaults, multi-step processing — belongs in fn-defs.
The fundamental rule: a base-fn MUST NOT call another base-fn. If it does, the composition is hidden in code instead of being visible in the graph. Fix: compose via fn-def, or make shared logic a private helper (not a registered base-fn).
Acceptable exceptions for longer impls: input validation/size limits (safely wrapping the library) and library adapter boilerplate (e.g., Ring handler format).
If two base-fns do the same thing with different type signatures, keep one with :any types.
Examples:
assoc + assoc-any → single :assoc with :any argsconj + conj-any → single :conjidentity → fn-def of :const (same implementation)wrap-style + wrap-script → fn-defs of :wrap-element (parameterized by :tag)with-htmx → :with-cdn-script (parameterized by :url); HTMX is the only CDN script leftWhen the same logic appears in multiple base-fns, extract it as a standalone base-fn.
Examples:
parse-query-string — duplicated in 3 places in crudparse-json — inline in layout and crud handlersformat-display-value — nested cond in graph arg->nodeBase-fns must not contain hardcoded configuration values. Defaults belong in fn-defs (via arg bindings), not in implementation code. The impl should use the arg directly without (or arg default).
Examples:
router → fn-defs via MI (text-error-router)(or port 8080) in the http-server impl — the :web-server fn-def binds :port 8080 explicitly:range’s :start 0 / :step 1), not (or arg val) in implHTML, SVG, CSS, hardcoded strings belong in fn-defs (:parent :const), not in implementation code. Impl stays a pure library wrapper.
Examples:
:body → :const fn-def with hiccup structure_favicon-svg-body → :const fn-def:with-cdn-script :url, not constants in impl):const fn-defs (_editor-script-paths, _graphden-runtime-js-paths)_ PrefixIntermediate fn-defs that have no standalone semantic meaning use _ prefix. This signals they exist only as wiring, not as reusable abstractions.
Examples:
health-handler-fn → _health-handlereditor-response → _editor-responsefavicon-svg-body → _favicon-svg-bodyGeneric arg names (key, value) should be renamed via :as to reflect domain semantics at the point of use.
Examples:
method-map.key → :method (HTTP method)method-map.value → :handler (request handler)assoc-status.value → :status (status string)assoc-timestamp.value → :timestamp (timestamp value)