Data Reconciliation
Your CRM has "Falcon Tech Incorporated." Your ERP has "Falcon Tech Inc." Same customer, no shared key, and a VLOOKUP that returns nothing.
DataFuseAI reconciles records across systems with Fuzzy Filter and Fuzzy Join — individual or weighted multi-column scoring against a threshold you set — alongside exact Dedupe, on the same pipeline canvas as every other transform. Schedule the match as a job that keeps run history, instead of re-running a spreadsheet formula by hand.
Where Exact Joins Stop Being Enough
Two systems that describe the same real-world entity rarely agree on how to spell its name, and almost never share a primary key. Exact joins and VLOOKUP handle the matches that agree by coincidence and drop everything else, which is usually most of what you actually need to reconcile.
The same entity, spelled two different ways
"Falcon Tech Incorporated" in the CRM. "Falcon Tech Inc." in the ERP. An exact join treats them as two different customers, because to a database they are. Multiply that across every abbreviation, typo, and formatting quirk two systems accumulate independently, and the gap between what the join returns and what is actually true keeps growing.
No shared key across the two systems
The CRM assigns its own customer ID. The ERP assigns its own. Nobody wrote down a mapping between them, because on the day both systems were adopted, nobody expected to need one. Without a shared key, matching has to run on the fields that describe the record — name, address, email, phone — not an ID lookup.
Reconciliation that lives in a spreadsheet
VLOOKUP finds the exact matches and silently skips the rest. Someone eyeballs what is left over, decides which near-misses count, and pastes the result into a report. The decision of what counts as a match is made by hand, once, and is not repeatable the next time the source data changes.
No record of which match was accepted, or why
A reconciliation run happened last quarter. The output shipped. The match logic — which threshold, which columns, which near-misses were accepted — lived in one person's head or a since-overwritten spreadsheet tab. When the same reconciliation is asked for again, or questioned, there is nothing to point to. This is the same gap covered in building audit-ready operations rather than bolting them on.
Fuzzy Matching, Weighted Across the Columns You Trust
Fuzzy Filter and Fuzzy Join sit next to Dedupe, Join, and every other transform on the same pipeline canvas — reconciliation is a pipeline, not a separate product. You set which columns matter, how heavily each one counts, and what threshold counts as a match.
Fuzzy Filter: score a column against a value, or against another row
Match Scoring Mode is Individual Score or Combined Score. Individual scores one column against a threshold on its own — full_name against "Anika Banarjee" at a score of 85 catches "Anika Banerjee" without catching an unrelated name. Combined weighs several columns into one overall score before the threshold applies, for when no single field is reliable enough on its own.
Fuzzy Join: match two sources on approximate agreement, not identical keys
Weighted, multi-column fuzzy joins — full_name → client_name at weight 60 and company_name → organization at weight 50, against an overall threshold — join two sources that were never given a shared key. It runs as a join node on the same canvas as every other transform, between a source and a sink, not as a separate matching tool.
Dedupe before or after the match
The Dedupe transform defines the columns that determine uniqueness — for a CRM–ERP reconciliation, typically the ID, name, email, and address fields — and removes duplicate rows against that definition. Run it before a fuzzy join to clean each source first, or after to collapse duplicate matches the join produced.
Reconciliation as a scheduled job, not a one-time script
Once the pipeline is built, schedule it as a job that keeps run history. Each run leaves a record — when it ran, how many rows matched, and against which threshold — instead of a spreadsheet tab nobody dated. The next reconciliation request is a re-run, not a rebuild.
"Anika Banarjee" and "Anika Banerjee", caught as one
These are the real screens from a CRM–ERP customer reconciliation pipeline built on DataFuseAI — the Fuzzy Filter transform, configured and run against real rows in the same source table.
Set the column and the threshold
Read the rows it caught
Two sources, joined without a shared key
Fuzzy Join runs as a join node between two sources, weighing several columns into one score instead of requiring an exact key match on any of them.
Weight the columns that matter
Get back matched pairs, not a miss
A Threshold You Set, Not a Black Box
Every one of these is a setting you can open and change during a trial — the scoring mode, the column weights, the threshold — not an outcome you have to take on faith. Bring the two sources that never agreed on a key and see where the threshold needs to sit.
Individual or combined match scoring — score one column against a threshold on its own, or weight several columns into one overall score, per reconciliation.
Weighted, multi-column fuzzy joins — match on name and company together, not one field in isolation, with a weight and a threshold you set per column pair.
Fuzzy Filter, Fuzzy Join, and Dedupe live on the same pipeline canvas as every other transform — no separate matching tool to license or maintain.
Every reconciliation run keeps history — when it ran, how many rows matched, at what threshold — visible from inside the product on the next run.
The same RBAC and audit-log model as the rest of the platform — who can edit a matching rule versus who can only run it, with before-and-after change detail on every edit.
Or run the reconciliation yourself against your own data. Run a POC →
Frequently Asked Questions About Data Reconciliation
An exact join matches rows only when the join columns are byte-identical, so 'Falcon Tech Incorporated' and 'Falcon Tech Inc.' are treated as two different customers. A fuzzy join scores how similar the columns are and matches rows that clear a threshold you set, so near-identical spellings, abbreviations, and formatting differences can still match without a shared key.
There is no universal number — it is a trade-off you set and can change. A lower threshold catches more true matches but risks pulling in unrelated rows; a higher threshold is stricter but can miss real matches with more spelling drift. The practical approach is to start around 80–85 on a sample you can eyeball, check the false positives and false negatives it produces, and adjust from there.
Individual Score checks one column against its own threshold independently — useful when a single field, like a full name, is reliable enough on its own to decide the match. Combined Score weighs several columns into one overall score before the threshold applies, which fits cases where no single column is trustworthy alone but agreement across several of them together is convincing.
Fuzzy Filter and Fuzzy Join run as nodes in a pipeline that can execute on the managed DataFuseAI engine, a Databricks workspace, or your own Spark cluster through Apache Livy, so the matching workload scales with the engine you assign it to rather than being fixed to the platform itself. As with any comparison-heavy join, narrowing the candidate set first — for example by an exact match on a coarser field like postal code — keeps the fuzzy comparison itself smaller and faster.
Both. The Dedupe transform defines the columns that determine uniqueness and removes duplicate rows against that definition — run it before a fuzzy join to clean each source first, or after to collapse duplicate matches the join itself produced. Which order makes sense depends on whether the duplicates already exist within a single source or only appear once the two sources are joined.
The same role-based access control that covers the rest of the platform: action-level permissions separate who can edit a Fuzzy Filter or Fuzzy Join node from who can only run the pipeline, and audit logs record changes with before-and-after detail. That is the same audit-ready operations model used across the platform, not a separate control layer for reconciliation.
As often as the two source systems drift out of agreement, which is usually daily or weekly rather than continuously. Once the pipeline is built, schedule it as a job that keeps run history, so each run leaves a record of how many rows matched and at what threshold instead of a spreadsheet nobody dated.
They stay visible rather than being silently accepted. The output of a Fuzzy Filter or Fuzzy Join carries the matched rows and, depending on how the pipeline is built, the score itself as a column — so a near-miss that clears the threshold for the wrong reason can be caught in the output and the threshold or weights adjusted, rather than being merged into a downstream report unnoticed.
Continuous, real-time deduplication at the point of entry — for example, blocking a duplicate signup as it happens. Fuzzy Filter and Fuzzy Join run inside a pipeline built for repeatable, scheduled, monitored batch workflows, which fits periodic reconciliation between systems well but is not built to sit in front of a live transaction and approve or reject it in milliseconds.