Semarchy XDM Ebook
Semarchy XDM Ebook
Master Data
Management &
Data Governance
Module
Comprehensive Product
Overview (v 5.3.16)
Contents
What is Semarchy xDM? 3
Overview3
Semarchy’s Three Pillars for Accelerating MDM Success 4
xDM: Packaged Enterprise Software Platform 5
Product Solution Architecture 7
Multiple Implementation Style Support/Multiple Domain Support 9
Multiple Usage Scenarios (Operational/Analytics) Support 10
Data Application Suite - Internal Integration 11
System Requirements 40
Application Server 40
RDBMS 40
Data Storage & Legibility 41
Data Modeling, Mapping, and Integration Patterns 41
4
What is Semarchy xDM?
Our People
MDM Success
Our Platform
5
What is Semarchy xDM?
: xDM is a packaged enterprise software platform.
Customers are not provided with a predefined
xDM comes out-of-the-box with Design-Time UIs enabling app designers and data architects with:
6
What is Semarchy xDM?
UIs for Business users, Data Stewards, and Data Champions are automatically
generated from the design and definitions from Application/Dashboard Builder.
The following integration/programmatic interfaces are also available out of the box.
7
What is Semarchy xDM?
Product Solution xDM is an enterprise-scale integrated data
hub unifying Master Data Management (MDM),
Architecture Reference Data Management (RDM), Data
Governance, Data Quality, and Data Integration
(xDI). The platform design is consistent across
on-premises and cloud-based deployments.
Web Browser
Data applications & Design-time
Same architecture/product
HTTPs
REST
SSO Core Engine Workflows Dashboards
Operations
SAP, Salesforce, Oracle, Infor,
NetSuite, Marketo, HubSpot,
Streaming Workday, SugarCRM, Dynamics,
Plugins REST API SemQL Data Access ELT, Batch, etc.
Apache Tomcat real-time,
streaming,
integration
8
What is Semarchy xDM?
xDM’s architecture is a classical yet highly scalable three-tier architecture, using a
Java EE application server with a relational database backend, as follows.
User Interfaces for design-time and business These interfaces and the REST APIs are served
users run in a web browser. They use the most by the Java EE application, which provides the
recent front-end frameworks (Angular, React) interaction layer with the core MDM services
for optimal user experience. while enforcing data and metadata access
security.
The backend database stores the xDM Cloud Databases and Multi-Database
metadata (models, policies, etc.) and certification for repositories and data hubs.
the hub data. This database is used to xDM supports Oracle database, PostgreSQL,
query the hub data and as the processing and Microsoft SQL on-premises or on the
engine for the xDM Data Certification most common cloud platforms (Amazon
Process, storing the data along its lifecycle Web Services, Microsoft Azure, and
for full data lineage and traceability. Google Cloud Platform). Please contact us
if you have questions on how xDM needs
to be deployed in your environment.
9
What is Semarchy xDM?
xDM is designed from the ground up to be multi-
vector (multi-style, multi-domain, multi-usage
Multiple scenario, multi-organizations, cross-industries).
Registry Consolidation
systems.
Co-Existence Centralized
Matches and physically stores a Matches and physically stores the up-to-date
consolidated view of master data. consolidated view of master data. Authoring
of master and reference data in MDM.
xDM supports the four most common implementation styles of MDM concurrently
10
What is Semarchy xDM?
Multiple Usage xDM is primarily used by our clients to support operational
usage scenarios. Very few clients use xDM only for purely
Scenarios analytical usage (less than 10%). However, most clients
(Operational/
using xDM in operational scenarios also leverage it for
analytical use cases. Meaning, data from xDM is used as
Your new data hub built with xDM becomes the central point for information lifecycle management
and orchestrates business-critical processes to improve overall data quality and semantics. Most of
our clients integrate the data hub with their:
xDM tightly interacts with the enterprise processes (Order to Cash, Health and Environmental
Compliance, Consent Management, Privacy and GDPR, IDMP, CFR 11, etc.) from within the calling
application by invoking xDM services through REST APIs.
For example, a shipping company using SAP determined risky by xDM, the SAP transaction
decides to enhance the booking process to is suspended, and the Party data is submitted
Sales Party
Rep Data
Semarchy xDM:
11
What is Semarchy xDM?
Our clients usually have more than one model
Data Application for maintainability reasons and faster design
cycles. They usually decide where to add
Suite - Internal the “intersection” data sets by selecting one
For example, a client might decide to design into the hub. xDM would then naturally match
a centralized hub for Customer/Organization and merge the two data sets and provide
data where the Customer entity is defined as all data stewardship workflows. After this
a “Basic Entity.” During an M&A operation, and integration is performed, and when the client
to streamline the integration of the Customer is ready to de-commission the Customer data
data owned by the second party, the client applications of the acquired party, the client
can change the entity type to “fuzzy-matched” can switch back to the centralized mode again.
and load all the third-party Customer data
12
What is Semarchy xDM?
What Domains Can Clients
Master with Semarchy?
Data Governance
Business glosary, policies, processes, roles, accountability, lists of values, data issues tracking, collaboration
Supply
CRM Finance Call Center POS Marketing ERPs Analytics
Chain
13
What Domains Can Clients Master with Semarchy?
With the xDM, operational analytics, predictive Similarly, operational business applications
analytics, and traditional analytics share leverage xDM in their core processes to
the same master data, reference data, guarantee semantic consistency across the
business glossary, and data catalogs, to enterprise, reducing costs and risks while
transform the interactions and observations increasing revenue-generating opportunities.
into measurable business value.
Interactions
Things &
Semarchy xDM:
14
What Domains Can Clients Master with Semarchy?
xDM for Parties
15
What Domains Can Clients Master with Semarchy?
xDM for Things
16
What Domains Can Clients Master with Semarchy?
This data model centers on Places. It is augmented with information
elements such as geography, points of interest, administrative areas,
public facilities, terrain, elevation, schedules, maintenance, rich
content (pictures, videos, PDFs), contacts, facilities costs, business
continuity, insurance, sales performance, etc. Hub management
xDM for values propagation and lifecycle. Processes and touch points in
the applications ecosystem include taxonomy governance, code-
Reference to-code translation, global versus local versus industry definitions,
industry-standard onboarding, outbound reporting, auditing,
Data compliance enforcement, sales impact analysis, etc.
xDM includes pre-built applications for data governance, discovery, Award-Winning Master Data Management & Data Governance Module
and data cataloging. The Data Governance application is
customizable with a starter pack that includes:
Data
structural metadata and sample data for discovery
• Users, roles, authorizations, access contexts, risks, business
17
What Domains Can Clients Master with Semarchy?
Value-Driven
Design Semarchy develops a single platform to help
18
What Domains Can Clients Master with Semarchy?
Core Capabilities
Business users and data champions analyze With data discovery metrics, data champions
Semarchy xDM:
the profiles using the fully automated, built-in have a clear assessment of the data quality
xDM Discovery dashboards with drill-down to at any time and measure the evolution of
the source data to identify root causes and/or such metrics against KPIs over time.
potential remediation processes.
19
xDM Core Capabilities
Integrate
Loading/Synchronization/Business Services/Integration
xDM supports both batch integration using process, which ensures consistency
ETL/Data Integration products and real- and accuracy of the received data.
time integration patterns. Both patterns can The data certification process is highly
use either the built-in SQL or REST APIs. optimized for:
Designers of the data hub models can define • Real-time access/low-volume per
their subscription or publishing endpoints call/high-velocity through the xDM
called named queries. These then generate Continuous Loads asynchronous
the appropriate APIs (REST or SQL), supporting mechanism.
complex structures as query parameters.
• Batch loading/high-volume/low-
Batch loading or real-time data submission frequency type of integrations using
always flows through the data certification in-database set-based processing.
Chaos Clarity
ETL/ELT/ ETL/ELT/
CDC CDC
SQL
SQL Data Certification
Connectors BATCH Connectors
Enrich > Validate > Match > Merge > Publish
JMS
Landing Golded
ESB/ REAL-TIME ESB/
Data Masters Rejects Matching & XRef, Data
Messaging REST Continous Messaging
Loads
(enriched) User Lineage and REST Operational
Decisions History
Apps
Direct API Direct API
Calls Calls
20
xDM Core Capabilities
Most of our clients use xDI: Data Integration • Portal/process integration for
which can reduce TCO by up to 80%. Others any data managed within the hub.
choose to use their own integration middleware
to integrate with xDM. • Data virtualization for building an enterprise
virtual layer.
Point-to-point real-time integration for various
use cases is usually done through our REST Querying REST APIs for reading or writing
APIs. These common use cases include: data respects the logical model semantics.
• Search in the data hub before simultaneously xDM has built-in auto-documented APIs
creating a record in both the operational dynamically generated using swagger
application and the hub. standard. This has dramatically increased the
adoption of the APIs by our customer base.
• Embedding the data hub records within the
operational application UI using configurable
queries.
21
xDM Core Capabilities
Enrich
Data Quality result is certified golden data. The automated Data Certification
Process is entirely generated from the rules defined in the data
Rules model for both batch and real-time certification. Data lineage
Enforcement and audit trails are submitted for compliance requirements.
Enrich &
Standardize SE/AE Enrich
1 Source/Authoring
5
Errors
SD 2 Validate MI
Master
3 Match &
4 Consolidate
GI
Golden
Source Data
Integration Integration
SA
Source
Override
6
Publish Golden
Authoring Data
MD GH GE
7
Validate
Master Data Golden Data Golden Errors
MH
Master History
GH
Golden History
Enrichment/ xDM Data Certification Process uses the data enrichment rules to
Standardization augment, cleanse, or standardize the data using trusted external
data providers or automated data transformations. Enrichment
Rules rules can use Semarchy data manipulation language (SemQL) that
Semarchy xDM:
22
xDM Core Capabilities
Enrichment Plugins List
Enrichment plugins fit in the following categories. language translation using Google Translate,
etc.
• Content-lookup enrichment plugins: B2B
party lookup (Experian, D&B), address lookup • REST Clients invoke a REST service API
(Google, Bing, OpenStreetMap), person that is presented for communication by
name (with gender/nickname detection), any system or service provider such as
phone standardization (locations, carrier, Google Geocoding API, or Melissa Global
time zones phone types, etc.), etc. Email Web API. Rest clients are designed
in Semarchy xDM with no code and
• AI/ML-based enrichments: Amazon used as an enricher for B2B party lookup
Rekognition for images parsing, sentiment (Experian, D&B), address lookup (Google,
analysis, text extraction with domain- Bing, OpenStreetMap), person name
specific ontologies, OCR leveraging (with gender/nickname detection), etc.
USCDataScience, dynamic classification of
data based on textual input and taxonomies,
Validation Rules
23
xDM Core Capabilities
Example of a data validation rule in xDM
24
xDM Core Capabilities
Match Rules
xDM provides a built-in Fuzzy Matching engine Rules in a Matcher define several conditions
leveraging the power of SemQL to detect for considering two records of a match
arbitrary matches between pairs of records. with scores representing the percentage of
Those pairs of records are automatically confidence. The match rule outputs are tracked
aggregated into match groups that will later and submitted to data stewards as an “explain
form unique golden records. Multiple Match data” graph view like the one shown below.
25
xDM Core Capabilities
Survivorship The quality of golden records depends on the ability of the data hub
Deletion Semarchy xDM allows deletion of golden and master data with propagation/
veto rules to comply with GDPR/CCPA and other privacy regulations. A
delete action may be:
• A Hard Delete that physically deletes records from the hub database
and data cannot be recovered.
26
xDM Core Capabilities
Traceability & Historization
Semarchy xDM has an automated mechanism change, creation, update, or deletion. Semarchy
to trace all data changes and historize golden xDM automatically creates the structures
and master data for all entities. Model designers, and processes to store and maintain the
with a key click, can configure historization for record history. Record history is also available
new or existing entities, to trace every record for integration via the built-in REST API.
Historization allows business users to easily browse the hub at a given point in time
27
xDM Core Capabilities
Manage
Data Modeling
xDM provides a fast and iterative modeling users, data stewards, data champions, and
framework that simplifies the design of data architects for defining high-level concepts
semantically complete enterprise data models. expressed using business terminology.
It enables collaboration between business
These data models act as the single point business requirements. The data model,
of logical definition of entities, attributes, rules, and policies are exposed to users
relationships, and the rules managed in within the data management application as
the data hub. Data modeling flexibility is documentation endpoints, supporting full
the key driver that enables the agility to navigation as well as contextual help, putting
support rapid implementation of changing data literacy at the core of the data hub.
Semarchy xDM:
28
xDM Core Capabilities
A model diagram exposed in the application documentation
xDM supports bi-temporal modeling and • Data rules rely on a powerful graph
automatically generates the appropriate language (SemQL) and/or an extensible
historization and lineage data infrastructure plugin framework for defining:
and processes for all entities. Entities also ◦ In-line data enrichment and
support record deletion in all hub styles data standardization rules
(including consolidation), handling cascade and ◦ ML-based rules for dynamic
veto through the model’s relationships. Key data data classification
modeling capabilities include: ◦ External AI-driven microservices call-
outs (Google, Amazon, Microsoft, etc.)
• Full multi-domain modeling framework ◦ Data validation and data quality rules
supporting all end-user requirements ◦ Match rules with scoring
29
xDM Core Capabilities
Hierarchy Management
The xDM logical model defines all relationships • Collaboration for defining new hierarchies
between entities and supports an unlimited and/or mapping to existing hierarchies
number of customizable balanced, unbalanced,
and recursive hierarchies for all data domains. • Managing time-varying relationships and
Hierarchies defined in xDM organize entities browsing hierarchies as-of-date
into single or multiple related schemes for data
management, classification, and reporting. The SemQL language provides intuitive hierarchy
Hierarchies defined in the model are used for: navigation and search, allowing users to
navigate both hierarchical and non-hierarchical
• Searching/navigating, both in the generated relations using the same language patterns.
user interfaces and via APIs
30
xDM Core Capabilities
Workflows & Business
duration to complete. Non-compliance with
Processes Management SLAs can trigger special actions such as
escalation, re-assignment, or user-defined
hooks.
xDM supports stewardship using human
workflows for data authoring, collaboration, • Workflows are designed graphically using
validation, and duplicate records management, BPMN.
to achieve the highest quality content for golden
data records. Key workflows and business • Advanced notifications using email, JMS,
process management capabilities include: or other notification mechanisms allow for
maximum interaction with users. In-app
• Policies/rules defined in the model are notifications and badges help guide business
automatically enforced in the workflow. users to act.
• Privileges for data access (at row and • Isolated data manipulation transactions can
attribute level) are automatically enforced to be shared across roles without impacting the
guarantee security compliance. golden data of the hub.
• Tasks can be assigned or re-assigned to • Built-in workflow metrics and KPIs are
roles (groups of users) or individual users. automatically exposed in xDM Dashboards.
• Rules can be defined to escalate or re- • REST APIs are exposed natively by the
assign tasks to ensure compliance with SLAs platform for potential integration with
dynamically. These can be applied to tasks external business process engines
to define a maximum pending duration or a (data, rules, UI Forms, workflows).
Workflow Runtime
Semarchy xDM:
31
xDM Core Capabilities
Information Stewardship, Governance & Policy Enforcement
xDM provides a flexible and complete managing the data of the hub. Applications
framework for generating a Material Design generated by xDM are contextually adapted
user experience for collaborating and and optimized for various user personas:
Wants to enforce data quality rules defined by the business to ensure data
consistency, completeness, and compliance.
Spots and resolves data discrepancies with the help of the business stakeholders.
Understands the impact of data changes across multiple business lines and
applications.
Data Performs data analysis using graph queries to reveal new relationships and
Champion opportunities.
Drives the data management roadmap while keeping a strong handle on the data.
32
xDM Core Capabilities
Security policies, data validation rules, data • Work queues for fixing rejected data
standardization rules, matching rules, and
survivorship rules defined in the logical model • Dynamic data classification rules with a
(and/or in the Data Governance Application) are user feedback loop
enforced automatically for any data managed by
xDM. Data published via direct authoring or from • Review of automated match/merge and
source applications are automatically enriched, survivorship results using graph view and
standardized, validated, matched, and merged side-by-side record comparison.
into golden records. Key information stewardship
capabilities include: • Manual match/merge with decisions with
possible data overrides
• Guided data authoring, import & export
using a step-by-step user experience • Filtering, searching and taking actions on
the data
• Data quality feedback according to policies
and rules • Graph analysis for discovering new
relationships and opportunities
• Mass updates in separate transactions to
execute what-if scenarios • Inbox for receiving workflow notifications
and data subscriptions
• Collaborative workflows for complex data
authoring with parallel teams collaborating • Comprehensive linking to the
on the same objects enterprise business glossary
33
xDM Core Capabilities
Report
Data Quality Monitoring & Data quality metadata (definition) and metrics
in visualizations contextually adapted for the REST API endpoints or table exports to
various personas (data stewards, business visualize the mastered hub data using your
users, etc.). The dashboards are presented in existing BI/Analytics or reporting tool.
34
xDM Core Capabilities
A sales performance dashboard blending data from the hub and the data lake
35
xDM Core Capabilities
Govern
Enterprise Glossary
The application provides an enterprise glossary • Processes, Tasks, Key Performance
to manage: Metrics, Lists of Values, and their Mappings
to existing Taxonomies
• Business Terms with their relationships,
ownership, and RACI matrices • Domains and Contexts to refine the glossary
Data Dictionary Award-Winning Master Data Management & Data Governance Module
The data dictionary allows capturing technical • Attributes, columns, referential integrity rules
metadata from various endpoints to govern the
physical deployment of applications. It manages • Relationships to the data hub models and
information about: glossary
36
xDM Core Capabilities
Users & Roles
The cornerstone of the Data Governance • Roles and authorizations
application resides in the definition of roles
and users with their expected authorizations • Access contexts, risks, and compliance
in the application’s ecosystem. The policies
governance application helps to document:
• Business stakeholders and their roles
in the governance processes (RACI)
Sample term definition and associated knowledge graph of “Invoice” in xDM Data Governance
37
xDM Core Capabilities
Deployment
Environment Details
Cloud-Based Your Own License) model. xDM is the most
popular MDM and RDM solution on the AWS
Deployments Marketplace for master data management.
Web Browser
Data applications & Design-time
Amazon AWS
Cloud Arhitecture Overview
Discovery Data Apps Dashboards Governance App Builder
Operational Applications
SAP, Salesforce, Oracle, Infor,
NetSuite, Marketo, HubSpot,
Plugins REST API SemQL Data Access
Real-time Workday, SugarCRM, Dynamics,
(REST, JMS) etc.
Apache Tomcat
Remote Data
Enrichment (JDBC)
JDBC
Plugin API Analytics
Data Lake, Data Warehouse,
Batch Operational Data Stores, etc.
Amazon RDS
Content Providers Single or Multi-AZ database instance
(SQL/JDBC)
Real-time or batch
running on RDS
data blending
Semarchy xDM:
38
xDM Deployment Environment Details
xDM provides an architecture for enterprise-
Performance/ scale deployments, based on Java EE. xDM
Scalability, Availability,
uses PostgreSQL, Microsoft SQL Server, or
Oracle relational databases, for storing and
Typical Application
server configuration Apache Tomcat Apache Tomcat Apache Tomcat
(per node) whether on- 8GB RAM, 2CPU 8GB RAM, 2CPU 8-16GB RAM, 2CPU
prem or cloud
Recommended Number
of app server nodes (in 2 2-6 6+
the cluster)
Semarchy xDM:
39
xDM Deployment Environment Details
1,800 20M ~ 48ms
requests per golden response time
second parties
Our team has benchmarked a highly loaded environment for a Government related portal with
a peak test of 1,800 requests per second hitting a B2B database of over 20 million golden
parties for one hour.
xDM REST APIs have performed as expected with a constant response time of ~48ms. The
architecture used a PostgreSQL database load balanced on three mid-sized application
servers (2 CPU Cores, 8GB RAM).
Semarchy xDM:
40
xDM Deployment Environment Details
System Requirements
41
xDM System Requirements
Using xDI: Data Integration’s ELT capabilities,
Data Storage & or your own middleware/data integration layer,
An xDM entity definition and the corresponding SQL query for Golden Data (GD)
42
xDM System Requirements
Support & Services
Proof of Value
The Proof of Value
Approach
80% process has enabled
80% of our clients to
have a fully functional
Semarchy enables clients to rapidly generate
MDM solution up
value with a start-small/scale-fast approach.
and running in 10
When considering a master data project,
weeks or less.
organizations quickly face the challenge of
evaluating ROI and value to gain executive
sponsorship. Sponsors frequently perceive such
projects as risky and time-consuming with low Sixty percent of
chances of success. The Two-Week Proof of
Value (PoV) service addresses this challenge. 60% our clients have
implemented more
than one domain.
43
Support & Services
Partners Implementation Partners
Distribution/Selling Partners
Software Partners
44
Support & Services
About Semarchy
Business Operations
Semarchy is a leader in Master Data Management and Data Integration. Its unified data platform
(xDM + xDI) provides enterprise-grade Master Data Management (MDM), Application Data
Management (ADM), Collaborative Data Governance, and Data Integration solutions for some of
the most well-known brands across the US and Europe. Our agile platform leverages intelligent
algorithms and business-driven design to simplify data discovery, stewardship, quality, enrichment,
workflows, and measurements. xDM is implemented via an agile and iterative approach that
delivers business value in under 10 weeks, and scales to fulfill any enterprise data challenges.
Gartner MQ Leader
Leader
Fall
2022
Semarchy xDM:
45
Free your data.
Start fast. Scale infinitely.
Intelligently secure & govern.
www.semarchy.com
Resources Contact us