BASSIRAH DATA INTELLIGENCE SUITE · DATA QUALITY & MDM
One Governed, Quality-Checked View of Every Data Asset.
Data Quality & MDM for DataHub turns the open-source DataHub metadata platform into a ready-to-run catalogue for a bank’s data estate. It arrives wired to your lakehouse, with a governance model, automated data quality from Great Expectations and explainable master data management from Splink.
Built by Bassirah on open-source DataHub, Great Expectations and Splink · Runs in your environment
OVERVIEW
Find It, Trace It, Trust It, Own It
Most catalogues answer one question: does this dataset exist? Data Quality & MDM for DataHub answers the four that come next, on a single page, without a ticket.
Find it
Search every table, pipeline, job, chart and dashboard across the platform in one place.
Trace it
See where a dataset came from and everything downstream that depends on it.
Trust it
Check today’s data quality results, freshness and volume before you use it.
Own it
Know the business domain, the owner and the steward accountable for it.
DataHub is an open-source metadata platform. Data Quality & MDM for DataHub is a Bassirah product built on it, with Great Expectations and Splink. It is not affiliated with or endorsed by the DataHub project or its maintainers. For independent support of DataHub itself, see DataHub Core.
WHAT YOU GET
A Catalogue That Is Ready on Day One
Instead of an empty platform to configure and populate, you start with a catalogue connected to your lakehouse and a governance model already defined as code.
Unified catalogue
Operational PostgreSQL sources, Apache Iceberg medallion tables (bronze, silver, gold, serving and master data), Airflow and Flink pipelines, and Superset dashboards in one searchable view.
End-to-end lineage
Follow a record from the source database through change data capture, streaming or batch ingestion and each medallion layer to the dashboard that shows it. Pipelines appear as nodes with inputs and outputs.
Impact analysis
Before you change or delay a table, see every downstream table, chart and dashboard it feeds.
Business glossary
A banking glossary linked to the datasets and columns that implement each term, so “settled” or “HOLD” means the same thing to everyone.
Domains, owners and tags
Assets organised by business domain, with named owners, stewardship groups and tags for sensitivity and tier.
Data products
Group the assets behind a business capability, name its owner and mark the output ports consumers should build on.
Structured properties
Typed metadata such as criticality, retention and payment scheme, so governance rules are searchable and reportable rather than buried in free text.
Access policies as code
Role-based policies defined as code and loaded into DataHub, ready to enforce when authentication is enabled.
Always current
An orchestrated, ordered pipeline refreshes metadata on a schedule (every six hours by default), so the catalogue does not drift from the platform.
Reference deployment at a glance
| Catalogued assets | Pipelines and BI | Governance |
|---|---|---|
| 75 datasets across the medallion layers and master data | 8 pipelines (data flows) and 19 jobs | 5 business domains |
| Bronze, silver, gold, serving and master data layers | 29 charts and 2 dashboards | 48 glossary terms and 3 data products |
HOW IT WORKS
Quality Gates and Master Data, Published to One Catalogue
Every layer is validated before the next one runs, master data is resolved in its own service, and everything lands in DataHub with lineage, owners and results.
DATA QUALITY · GREAT EXPECTATIONS
Trust, Answered on the Dataset Page
Trust is the first question every data consumer asks. The product answers it where people already look: on the dataset page, next to the lineage and the owner.
- Quality at every layer. Expectation suites cover bronze, silver, gold, serving and master data. Raw data is validated at ingestion, and each later layer after its transformation.
- Results where people look. Every check is published to DataHub as an assertion, so the dataset’s Validations tab shows what passed, what failed and when.
- Freshness and volume watchers. Native DataHub assertions track whether each table updated within its agreed window and whether its row count is plausible.
- Failures raise the alarm. A failed suite opens an incident on the affected dataset and notifies the operations channel, so the owner hears first.
- A hard stop, not a warning. Validation is a gate in the pipeline: a failing layer fails the run instead of being reported as a success.
- One quality record. Results are written to a queryable table that feeds a Data Quality dashboard, and Great Expectations Data Docs are kept for audit.
What is checked at each layer
| Layer | Examples of checks |
|---|---|
| Bronze | Row count above zero; primary key present and unique; event time present; amounts not negative; status values within the allowed set. |
| Silver | Stricter business rules: amounts greater than zero, a closed set of statuses, valid currencies, load timestamps present. |
| Gold | KPI sanity bounds: rates between 0 and 100, counts not negative, scheme values limited to known payment rails. |
| Serving | Everything in gold, plus cross-layer reconciliation: a downstream layer may not count more than the layer that feeds it. |
| Master data | Golden record keys unique; every golden record backed by at least one source record; match probabilities between 0 and 1; known source systems only. |
MASTER DATA · SPLINK
One Golden Party, With the Evidence for Every Decision
The same customer or counterparty often appears under several identifiers across systems. A master data service built on Splink, an open-source probabilistic record-linkage engine, resolves those records into one golden party and keeps the evidence behind each match.
Probabilistic matching
Records are compared field by field with exact and fuzzy rules, such as Jaro-Winkler similarity on names and addresses. Each pair gets a match probability, not a yes-or-no guess.
Explainable by design
Every link above the threshold is stored with its probability and weight, so a steward can see why two records were merged and why others were not.
Tunable caution
A configurable threshold sets how much evidence a merge needs. Shared names or addresses alone do not merge two people whose identifiers differ.
Golden record with survivorship
Each resolved party gets a stable master identifier and a golden record. Survivorship rules pick the canonical values and report breadth signals, such as how many handles or banks sit behind one party.
Cross-reference for every record
A cross-reference table maps each source record to its master, so any payment can be traced to the party behind it.
Run-by-run telemetry
Each run records its volumes, cluster sizes and settings, giving a durable history of match quality.
Isolated and scalable
Matching runs in its own service, triggered by the orchestrator over HTTP. It uses DuckDB for current volumes and can move to Splink’s Spark backend for very large candidate sets, without changing the model or its outputs.
Governed like everything else
Master data tables appear in the catalogue with lineage from source payments through candidates to the golden party, and pass the same Great Expectations checks.
WHAT BASSIRAH ADDS
Open-Source DataHub, Delivered Ready to Run
DataHub is a capable metadata platform that each team configures and populates for its own estate. This product delivers that work done for a banking lakehouse, and adds master data management, which DataHub does not include.
| Open-source DataHub | Data Quality & MDM for DataHub | |
|---|---|---|
| Catalogue and lineage | A platform your team connects and populates | Pre-wired to the lakehouse, streaming and batch pipelines and BI layer, refreshed on a schedule |
| Governance model | Defined by your team | Domains, owners, glossary, tags, structured properties, data products and access policies, defined as code |
| Data quality | Displays assertions, including results from Great Expectations | Great Expectations suites for every layer, results on the dataset, incidents and alerts on failure, and a pipeline hard stop |
| Master data | Not included | Splink entity resolution with explainable links, golden records and a cross-reference |
| Conversational access | Search and browse | Feeds GenBI, so people can ask questions in plain language and get lineage-aware answers |
WHO BENEFITS
One Catalogue, Four Teams Better Served
Data engineers
See the blast radius of a change before making it, and get alerted when a layer breaks.
Data stewards and owners
Manage definitions, ownership and quality in one place instead of spreadsheets.
Risk and compliance
Show where data came from, who owns it, which checks it passed and which party a record belongs to.
Analysts and business users
Find trusted data quickly and know whether it is fresh before using it.
Frequently Asked Questions
Is this the same as open-source DataHub?
It is built on open-source DataHub. Bassirah delivers DataHub pre-wired to your lakehouse with a governance model, adds Great Expectations data quality suites and pipeline gates, and adds Splink master data management. It is a Bassirah product and is not affiliated with or endorsed by the DataHub project.
Does it run in our own environment?
Yes. It is deployed on your infrastructure, alongside your lakehouse, pipelines and BI tools.
What happens when a data quality check fails?
The pipeline run fails at that layer instead of passing bad data downstream. An incident is opened on the affected dataset in DataHub and the operations channel is notified.
Can stewards see why two records were merged?
Yes. Every link above the matching threshold is stored with its probability and weight, and each source record maps to its golden party through a cross-reference table.
Does it work with GenBI?
Yes. GenBI reads lineage, sensitivity, freshness and ownership from the catalogue, and can answer questions such as “what breaks if this table is late?” directly from DataHub.
GenBI (Chat With Your Data)
Ask governed questions in Arabic or English, on top of this catalogue.
See Your Data Estate in One Governed Catalogue
Book a demo of Data Quality & MDM for DataHub, shown together with GenBI on the same reference platform.