From Data-Oriented Architecture to Intelligent Systems🔗

Listen to the Podcast🔗

Prefer to listen? Here's an audio deep-dive exploring the evolution from data-oriented architecture to intelligent systems:

Watch the Video Explainer🔗

Get a visual overview of the 2013 Future Organisation diagram and its modern relevance:

---

In 2013, before the AI gold rush and the rise of vector databases, a hand-drawn diagram called The Future Organisation mapped an idea that most businesses hadn't yet realised:

Organisations don't run on applications — they run on data, relationships, and the intellectual property built between them.

That diagram became the foundation for the Data Management System and Method patent (WIPO WO2013106883) — an early attempt to unify integration, data federation, and service orchestration into a single intelligent platform that later evolved into an iPaaS pattern (commercialised as Universal Platform) and, today, into AI orchestration for agents, prompts, and retrieval pipelines — with contemporaries and successors across the space (e.g., MuleSoft, Boomi, SnapLogic, Workato; Azure Logic Apps/Functions, AWS Step Functions; and modern AI orchestration/agent frameworks such as LangChain, LlamaIndex, Semantic Kernel, Temporal, and Prefect).

---

The 2013 Vision🔗

At the time, cloud adoption was accelerating, but few were thinking holistically about data gravity or knowledge reuse. Most digital strategies revolved around:

• Building more applications
• Migrating legacy systems
• “Modernising” infrastructure

The Future Organisation diagram inverted that hierarchy. It placed data at the core, IP as the transformation layer, and relationships — both human and system — as the boundary through which value flows.

---

What is Data‑Oriented Architecture (DOA)?🔗

A data‑oriented architecture designs the system around data first — its meaning, structure, ownership, and movement — rather than around applications or services.

Core principles🔗

• Canonical models over app‑specific schemas.
• Data as a product: documented contracts, owners, SLAs.
• Lineage & identity: persistent IDs, audit trails, reproducibility.
• Immutability & events where possible; changes are new facts.
• Late binding: transform on read, not on write, to preserve optionality.
• Idempotent processing: safe to replay.
• Separation of concerns: Control / Definition / Execution layers.
• Governed access: policy, privacy, and entitlements built‑in.

How it differs (at a glance)🔗

Related but not the same🔗

• Data mesh: federated ownership + data‑as‑a‑product. DOA is the architectural stance that enables it.
• Data fabric: connective tech & metadata; DOA provides the modelling and governance backbone.
• CQRS/Event sourcing: patterns that fit DOA’s immutable/eventful mindset.
• MDM: a DOA capability for truth, not the whole architecture.

Why DOA helps in the AI era🔗

• Grounding & provenance for LLMs and agents (traceable inputs).
• Reusability: one governed corpus → many use‑cases (ops, analytics, AI).
• Optionality: new models and vector stores can adopt shared contracts without re‑plumbing.

A client profile exists once (governed model, single ID). Portals, care apps, analytics, and AI agents all read from the same contract via APIs/events. Transformations happen at consumption, preserving source truth and lineage.

---

The Patent: Building an Information Platform🔗

In 2011, the Data Management System and Method patent (WO2013106883) described a platform with three tiers — Control, Definition, and Execution — that could translate data between any source and any target without rewriting code.

It proposed:

• Configuration-only integration — adapters replacing code.
• Data DNA — unique GUIDs for every data element, preserving full lineage.
• Multi-node swarm processing — distributed asynchronous execution.
• Master Data Management baked in — real-time, rule-based reconciliation.
• Visualised data landscapes — a “Galaxy” map of every data source and flow.

These ideas anticipated today’s AI pipelines — embedding lineage, explainability, and real-time data flow into the fabric of enterprise systems.

---

The Architecture Lives On🔗

Many of the concepts outlined in that early work now exist across the AI stack:

---

From Data Architecture to Intelligent Systems🔗

The shift from data integration to intelligent systems is evolutionary, not revolutionary.

In practice, this means the system itself becomes an agent:

• Reading, reasoning, and reacting to data.
• Tracking lineage.
• Producing reusable insights across the organisation.

The architecture of the 2010s becomes the substrate of today’s intelligent systems.

Wise, M. (2016). Journey from Data Integration to Data Science. InfoQ Presentation.

---

Organisational Design Implications🔗

The 2013 Future Organisation diagram implied that value creation happens at the intersection of:

1. Data integrity
2. IP reuse
3. Relationship capital

Those same intersections now define AI-ready enterprises:

• Trustworthy AI → governed data with traceable provenance.
• Reusable knowledge bases → fine-tuned, versioned models.
• Human-in-the-loop systems → co-adaptive feedback between people and AI.

---

ISO as the Future-Shaping Guardrails🔗

ISO/IEC TR 24368:2022 — AI: Overview of ethical and societal concerns (JTC 1/SC 42) adds the why behind governance: it frames the societal risks and ethical considerations that should inform your design choices. In practice, it asks: who benefits, who could be harmed, and what safeguards are in place?

To make it actionable, align TR 24368 with three companion standards:

• ISO/IEC 42001:2023 — AI Management System (AIMS): establishes a management system for policy, objectives, roles, and continual improvement across the AI lifecycle.
• ISO/IEC 23894:2023 — AI Risk Management: integrates AI‑specific risk identification, analysis, and treatment with ISO 31000 thinking.
• ISO/IEC 38507:2022 — Governance implications of AI: clarifies the board and executive accountabilities for acceptable, effective AI.

What this means for the Data–IP–Relationships model🔗

• Data → TR 24368 pushes you to design for transparency, fairness, and context; 23894 operationalises risk controls (e.g., data quality, drift, misuse).
• IP → 42001 ensures your models, prompts, and policies are governed as reusable institutional IP with auditability and change control.
• Relationships → 38507 anchors decision rights and accountability for customers, clinicians, partners, and regulators.

A pragmatic checklist (map to your architecture)🔗

• Purpose & harm assessment (TR 24368): document intended use, affected parties, potential harms, and mitigations.
• Controls catalogue (23894): bias, robustness, security, privacy, and model‑ops controls linked to risks.
• Management system (42001): policy, roles, competence, supplier management, monitoring, incident handling, continual improvement.
• Governance (38507): board‑level principles, KPIs, audit trails, and escalation paths.

---

Integrating Oblique Strategies🔗

The 2013 Future Organisation diagram implied that value creation happens at the intersection of:

1. Data integrity
2. IP reuse
3. Relationship capital

Those same intersections now define AI-ready enterprises:

• Trustworthy AI → governed data with traceable provenance.
• Reusable knowledge bases → fine-tuned, versioned models.
• Human-in-the-loop systems → co-adaptive feedback between people and AI.

At Trilogy Care, these ideas underpin the Trilogy Graph Architecture and AI Assistant ecosystem, connecting data, cognition, and care operations across cloud, voice, and workflow.

---

Integrating Oblique Strategies🔗

The principles behind Oblique Strategies and the Push Manifesto — introduced in Moving the Needles: Oblique Strategies for System Transformation — act as a creative framework for evolving these architectures.

The seven needles — Viscosity, Friction, Automation, Immediacy, Niche, Governance, and Learning — mirror the dynamics of data and organisational systems:

• Viscosity represents resistance to change in data flows. Intelligent architectures lower viscosity through composability and API-first design.
• Friction mirrors resistance from people and process; AI systems reduce this through automation and intuitive interaction.
• Automation and Immediacy define system responsiveness — how fast insights turn into actions.
• Niche captures context — focusing intelligence where it delivers the most value.
• Governance ensures control and compliance within self-learning systems.
• Learning represents adaptation — systems that not only perform but evolve.

When these needles are consciously moved, organisations transition from managing systems to orchestrating ecosystems — intelligent, adaptive, and resilient.

---

Reducing Friction and Lowering Barriers🔗

Modern architectures must do more than manage complexity — they must remove it from the user’s path. Intelligent systems reduce friction by automating handoffs, simplifying access, and making data discoverable at the point of need.

Lowering barriers means enabling customers, partners, and internal teams to access the data and IP they already rely on without gatekeeping. Systems designed with openness and clarity allow innovation to flow — empowering people to create value without asking permission.

In practice, this means:

• Removing blockers — replace approval chains with audit trails.
• Transparent APIs — customers and staff can interact directly with trusted data sources.
• Self-service intelligence — insights and actions should be available where work happens.
• Reducing cognitive load — design systems that anticipate user intent rather than force configuration.

Research on cognitive friction shows that interface complexity increases mental load and slows decision-making; reducing cognitive friction improves task performance and comprehension. See Cognitive Friction Measurement (CMS OpenAccess, 2023) and foundational HCI work on Fitts’s Law for minimizing interaction effort.

In digital journeys, studies find that small friction points inflate abandonment and reduce conversion; removing micro‑barriers leads to measurable gains. See Improving Digital Sales Through Reducing Friction Points in the Customer Digital Journey (IJIRMPS, 2022) and industry analyses on friction mapping and reduction.

Not all friction is bad: positive friction — deliberate pauses, checks, or reflective prompts — can improve reliability and decision quality in human‑AI systems. See Better Slow than Sorry: Introducing Positive Friction for Reliable Dialogue Systems (arXiv, 2025), Exploring a Behavioral Model of “Positive Friction” in Human‑AI Interaction (arXiv, 2024), and MIT Sloan Management Review on when friction can enhance customer experience.

The goal is to get out of the customer’s way — to build an environment where access to information, automation, and IP is seamless, secure, and intuitive.

---

Why It Matters Now🔗

Today’s “AI race” looks like the web boom of 2010 — but with higher stakes. Every organisation is re-learning an old truth:

Applications fade. Data persists. Relationships compound.

The next generation of AI-ready enterprises will succeed by balancing data discipline with creative adaptability — reducing friction, maintaining governance, and fostering continuous learning.

Designing intelligent systems isn’t about chasing models. It’s about building architecture that learns — the same principle drawn in a notebook twelve years ago.

---

Appendix: DOA in Game Development🔗

Game engines have adopted data‑oriented design for years to maximise cache efficiency, parallelism, and determinism:

• Mike Acton’s Data‑Oriented Design (Insomniac Games): foundational talks and materials emphasising designing around data and work, not objects or types.
• Unity DOTS (ECS, Jobs, Burst): Unity’s data‑oriented stack enabling high‑performance, multithreaded gameplay with explicit data layouts.
• EA/DICE Frostbite: public engineering posts and GDC sessions show DOD in production (e.g., “Culling the Battlefield”) and schema‑first content pipelines.
• GDC Vault case studies: multiple sessions detail why engines/platforms chose DOD patterns for large‑scale, cross‑platform performance.

This heritage matters for enterprise AI: the same data‑first, layout‑aware, pipeline‑driven mindset translates directly to low‑friction, high‑throughput intelligent systems.

---

Appendix: Glossary (DOA → AI)🔗

• Canonical model — A shared, system-agnostic schema for a domain.
• Lineage — Trace of where data came from and how it changed.
• Late binding — Transforming on read to preserve optionality.
• Idempotent processing — Safe to replay the same input/events.
• Data as a product — Documented contracts, owners, SLAs.
• RAG — Retrieval-augmented generation (ground LLM outputs with enterprise data).
• Embedding — Vector representation for semantic search/similarity.
• Vector store — Index for embeddings (ANN search).
• Agent — Orchestrated tool-using process guided by policy and context.

---

Appendix: DOA vs Data Mesh vs Data Fabric🔗

---

Appendix: DOA Implementation Patterns🔗

• Canonical Events: Domain events carry canonical payloads; consumers transform locally.
• API + Event Pairing: Read via query APIs; react via events (CQRS-friendly).
• Schema Registry: Versioned JSON/Avro/Protobuf with compat checks in CI.
• Data Contracts: Contract tests in CI; break builds on incompatible changes.
• Lineage Hooks: Emit lineage metadata (source, hash, transform) to a catalog.

---

Appendix: RAG/Agent Pipeline mapped to DOA🔗

1. Ingest → canonicalise → store with lineage
2. Chunk & embed → attach policy/entitlements to chunks
3. Index to vector store with metadata (owner, PII flags, TTL)
4. Retrieve by policy-aware filters (tenant, role, consent)
5. Generate with citations and model card
6. Post-process: PII redaction, safety, logging (AIMS controls)
7. Feedback loop: capture corrections → update contracts/quality rules

---

Appendix: ISO Mapping Matrix (Controls → Pipeline)🔗

---

Appendix: Metrics & KPIs🔗

• Friction: time-to-first-data (TTFD), steps to access, auth prompts per task
• Quality: contract breaks, lineage coverage %, data freshness SLA adherence
• AI: citation coverage %, hallucination rate, harmful output rate, drift alerts
• Ops: MTTR incidents, failed deployments %, cost per 1k requests
• Adoption: API call growth, self-service queries, time saved per workflow

---

Appendix: Reference Architecture (Text Sketch)🔗

Sources → Ingest (schema registry, PII scan) → Canonical store (immutable log + gold) → Lineage catalog → Services (APIs) + Events (bus) → AI Layer (RAG, embeddings, vector index) → Consumers (apps, analytics, agents) with policy enforcement.

---

Appendix: Platforms & Frameworks Mentioned (Links)🔗

• Universal Platform (iPaaS) — commercialisation referenced; recognition: Gartner Cool Vendor 2015
• MuleSoft — mulesoft.com
• Boomi — boomi.com
• SnapLogic — snaplogic.com
• Workato — workato.com
• Azure Logic Apps — Microsoft Learn
• Azure Functions — Microsoft Learn
• AWS Step Functions — aws.amazon.com/step-functions
• LangChain — langchain.com
• LlamaIndex — llamaindex.ai
• Semantic Kernel — Microsoft Learn
• Temporal — temporal.io
• Prefect — prefect.io
• Apache Airflow — airflow.apache.org
• Dagster — dagster.io
• Zapier — zapier.com
• n8n — n8n.io

---

References🔗

• Wise, M. (2011). Data Management System and Method. WO2013106883.
• Wise, M. (2013). Future Organisation Diagram. Original Drawing
• Wise, M. (2016). Journey from Data Integration to Data Science. InfoQ Presentation
• Wise, M. (2025). Trilogy Graph Architecture. Internal Trilogy Care R&D
• Wise, M. (2025). Moving the Needles: Oblique Strategies for System Transformation. WiseJNRS Blog
• Gartner (2015). Cool Vendors in Integration. Recognition of Universal Platform (iPaaS)
• ISO/IEC TR 24368:2022. Information technology — Artificial intelligence — Overview of ethical and societal concerns. iso.org/standard/78507.html
• ISO/IEC 42001:2023. Information technology — Artificial intelligence — Management system. iso.org/standard/42001
• ISO/IEC 23894:2023. Information technology — Artificial intelligence — Guidance on risk management. iso.org/standard/77304.html
• ISO/IEC 38507:2022. Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations. iso.org/standard/56641.html
• CMS OpenAccess (2023). Cognitive Friction Measurement: Interaction Assessment of Interface Information in Complex Information Systems.
• IJIRMPS (2022). Improving Digital Sales Through Reducing Friction Points in the Customer Digital Journey.
• MIT Sloan Management Review (2018). Can Friction Improve Your Customers’ Experiences?
• ArXiv (2025). Better Slow than Sorry: Introducing Positive Friction for Reliable Dialogue Systems.
• ArXiv (2024). Exploring a Behavioral Model of “Positive Friction” in Human‑AI Interaction.
• HCI Foundation. Fitts’s Law.

---