Data Governance

Data Governance Portal

The Data Governance Portal provides a user interface to define data governance scopes based on repository filters and tracked fields.
These scopes are used to define which fields will be audited regarding its completeness and currentness.
E.g. ‘Business Applications that are productive and critical need to have these fields maintained and reviewed in this interval'.

To open the portal navigate to Data Governance -> Data Governance Portal and open the portal to start creating data governance scopes.

eb6784b-UPMX_Data_Governance_Portal_-_Overview

A – Indicates the amount of scopes ( 1 ), the amount of fields tracked [ 1 ] and table 'Symbol + Table Name'

B – These ‘cards’ represent a Data Governance Scope

C – Click here to create a new Scope for the selected/expanded table

D – Click here to create a new Scope for a new table not being tracked yet

 
Creating a new Scope: step-by-step guide
37b811a-UPMX_Data_Governance_Portal_-_Create_Scope

A – Click the plus sign in the center of a card within a table definition to create a new scope for that table or the plus sign in the bottom right to create a new scope for a new table

B – Provide a meaningful name, e.g. ‘Bus. Applications in Production’

C – These buttons are used for saving or discarding the scope

D – Specify the target table

E – Specify the steward type – single means the steward role is responsible for data maintenance, when selecting ‘Steward Group’ a ServiceNow group can be selected that will be responsible

F – Review cycle determines how often the data needs to be verified (as current)

G – Reminder cycle determines the interval people are reminded to verify

H – The audit field can be deactivated if the field is not in focus for data completeness and currentness reviews. Activate the field, to send out notifications to the steward or steward group members

I – Select the checkbox to include all records in the table or click on the ‘Filters’ button to open a window to specify filter criteria for the scope records to satisfy

J – Specify your filter criteria here and confirm with ‘Save’

K – After the scope card has been created, you can add fields relevant to the scope/to monitor 

🚧

Caution when working with huge amounts of data

When creating scopes for Data Governance, UPM-X creates evaluation entries in the data quality table (x_inpgh_upmx_data_quality) for each tuple of table and field.
For tables with large amounts of data (>10.000 elements), we recommend to create the scope with 2-3 fields, wait for the evaluation to complete and add the next set of fields.

 
Data Quality Widget

The Data Quality Widget provides an overview of tracked fields and allows to easily verify fields that are due for verification.
Control its visibility with ACLs on the Data Quality tab of an element/table where data quality is tracked.

99fbe78-UPMX-DQ-Widget-Admin

A – Field tracked by Data Governance

B – Current value of the field or ‘(Empty)’

C – Completeness Indicator – either ‘Complete’ or ‘Missing’

D – Currentness Indicator – either ‘Current’, ‘Overdue’ or ‘Missing’/’No Value’

E – ‘Next verification in’ indicates when the next review is due according to the review cycle definition, or how many days/months/years it is overdue

F – Verification button to instantly verify the field content

G – ‘Verify All’ – instantly verifies all fields where verification is due

H – Example of countdown to next due verification

 
Data Governance Reporting

UPM-X stores Data Governance data in these tables

  • x_inpgh_upmx_data_quality - main data quality table
  • x_inpgh_upmx_portfolio_element - all UPM-X tables are extended from this and have a calculated ‘Data Completeness’ and ‘Data Currentness’ field
  • Since v4.2 calculated ‘Data Completeness’ and ‘Data Currentness’ values are available on all tables inherited from ‘cmdb_ci’

To create Data Governance Reports and Dashboards, start with the Data Quality table and if you need to filter on further criteria on the tables themselves, switch to the attributes directly on the tables

d936f1c-UPMX-DQ-Dashboard
 
 
Scheduled Jobs in Data Governance

The Data Governance feature in UPM-X leverages Scheduled Jobs, especially to conduct administrative work

d488017-UPMX-DQ-Scheduled_Jobs

A – Sets data currentness to zero on Data Quality table if it exceeds the review cycle

B – Sends the notification containing information about Data Completeness and Currentness to steward/steward group if reminder cycle is over

C – Sets the ‘Value’ field on Data Quality table with the actual field value

D – Sets the average data completeness and currentness for an element in ‘Data Completeness’ and ‘Data Currentness’ fields of the element

E – Removes the entire Data Quality configuration – e.g. in case you want to get rid of demo data/configuration and start with a clean slate

F – Creates Data Quality records for the scopes defined

G – Sets the Steward Type, Reminder Cycle and Last notified Date fields on Data Definition Scope  ‘Legacy/pre-v4.1’ for migration purposes

🚧

Caution when clearing data quality data

Use ‘Remove complete Data Quality configuration’ with caution – it removes everything related to DG. It might be an alternative to clear the x_inpgh_upmx_data_quality table to achieve the desired outcome

 
 
Notifications in Data Quality

UPM-X leverages the ServiceNow platform functionality for sending notifications to stakeholders. Notifications are sent out for scopes defined in the data governance portal that are flagged with 'audit=yes'.

  • To edit Notifications navigate to ‘System Notification -> Notifications’
  • To edit Email scripts, navigate to ‘System Notification -> Notification Email Scripts’
6bed354-UPMX-DQ-Notifications

A – List of notifications in the context of UPM-X Data Governance. The notification invokes a corresponding Notification Email Script

B – Name of the Notification

C – Name of the Notification Email Script invoked

D – List of Notification Email Scripts in the context of UPM-X Data Governance.

 

Set the notification schedule via the data governance portal

The schedule when the notifications are sent out is controlled by the Data Governance Portal settings in the individual scopes