Skip to main content

Bassirah Technology

Perspectives

Governance Has to Travel With the Data

Most governance programs protect the store where data first arrives. Copies, models, logs and partners need the rules to move with the data.

Most data governance programs are built around a store: a warehouse, a lake, a catalog. The rules say who may read it, what it means and how long it is kept. Then the data leaves. It is copied into a feature store, summarized into a dashboard, sent to a partner, pasted into a prompt or used to train a model. At each step, the rules tend to stay behind.

Our six-paper Data Governance series, published this month, kept arriving at the same conclusion from different directions. Governance works when it moves with the data, in a form machines can read, through every copy and every use. It fails at the boundaries.

Where the rules get left behind

A few boundaries come up again and again:

  • From production into evaluation. Prompts and responses captured for debugging become test sets. The retention rule that applied to the logs does not follow them.
  • From one team to another. A table changes shape, and a downstream report breaks, because the expectations were never written down.
  • From one region to another. A replica or backup lands in a country the data was never meant to reach.
  • From one organization to another. Data is shared under a contract, and nothing technical records what the recipient may do with it.

None of these is exotic. Each is a normal engineering step that the governance program did not see.

Why documents are not enough

Policies written for people are necessary, but they do not travel. A spreadsheet of processing activities, a wiki page of data owners or an annual access review describes the state of things on the day it was written. Data platforms change every day.

Regulation has moved in the same direction. The GDPR asks controllers to keep data no longer than necessary and to be able to demonstrate compliance, which is hard to do for copies nobody has recorded. Newer rules for AI and for digital resilience describe risk management as continuous rather than periodic. The expectation is evidence, not intentions.

What traveling governance looks like

In practice, governance that moves with the data has four parts.

  1. Labels that stay attached. Classification, purpose and location tags are carried in metadata, so that a copy inherits the rules of its source.
  2. Lineage that records every move. Open standards such as OpenLineage record which job read and wrote which data set, so you can find every copy before you change or delete anything.
  3. Policies written as code. Access, masking and location rules are evaluated by engines at the point of use, not checked by hand once a year.
  4. Commitments written as contracts. Producers state the schema, quality and service they promise, in a format tools can test.

Each part is available today in mature tooling. What is usually missing is the decision to connect them, so that a tag set in the catalog changes what a query engine returns, and a lineage record drives a deletion.

The limits are worth stating

Some things do not travel well yet. Once data has left your systems and reached a partner, technical controls depend on trusting the partner’s platform. Automated classification proposes labels with errors that people must review. Risk scores for access decisions are still a research topic. A realistic program uses automation where it is reliable and keeps people responsible for thresholds, exceptions and judgment.

Where to start

Pick one boundary that causes real trouble, often production logs or a partner feed, and trace a single data set across it. Record where every copy lives, which rules should apply to each and which of those rules are enforced by a system rather than a document. The gaps you find are the program.

How Bassirah can help

We help teams map where their data goes, connect catalogs, lineage and policy engines, and decide which controls to automate first. The six papers in the series set out the research and regulation behind each step.

Sources

  1. Governing the AI Data Lifecycle (Bassirah white paper)
  2. Continuous, Risk-Adaptive Data Governance (Bassirah white paper)
  3. Regulation (EU) 2016/679 (GDPR), EUR-Lex
  4. OpenLineage documentation

Bassirah Technology — Innovation & R&D

The Innovation & R&D team at Bassirah Technology researches data, analytics and AI platforms and writes the white papers, explainers and perspectives published here.

Related reading

More in Perspectives

Discuss This With Our Team

Bring your questions, constraints and current platforms. We will help you decide a practical next step.

Talk to an Expert

Product and project names are trademarks of their respective owners. Their mention describes publicly documented technology and does not imply a business relationship, except where Bassirah names a Strategic Technology Partner.