1. Data Management
  2. Data Quality Studio
  3. Apply data quality policy rules

Monitor and clean up data quality execution history

You can monitor and clean up data quality execution history for:

  • Validation rules that are applied when you manually add or change records. Validation history is logged only if logging is enabled. 
  • Fuzzy duplicate checks.


Data quality officer Data quality officer The data quality officer (DQSDataQualityOfficer) can view and process data quality assessment results. Start Start Which history to  monitor and clean up? Which history to  monitor and clean up? Monitor validation history Monitor validation history In the Data quality studio parameters, you can enable logging of data quality policy validation rules execution. Validation rule execution results are Logged only when you manually add or change records. Each time a validation rule is executed, an entry is logged. Not logged if the data quality policy is run by a: Data quality assessment. Connectivity Studio data import. You can review and process the logged validation rule execution results. For each logged validation rule execution result, you can: Set the review status. Open the source record to make changes, if desired. The validation status of a logged validation rule execution result can be: Success: The validation rule is met. Failed: The validation rule is not met and results in an error or warning. Skipped: The validation rule is not executed because its conditions are not met. Procedure 1. Go to Data quality studio > Inquiries > Policy execution logs. 2. In the list, find and select the desired record. 3. Sub-task: Change review status to 'In progress'. 4. Click Change status to open the drop dialog. 5. In the Review status field, select '0'. 6. Click OK. 7. Click Yes. 8. Sub-task: Review and edit source record. 9. In the list, find and select the desired record. 10. Click Open source record. 11. Click Edit. 12. Close the page. 13. Sub-task: Change review status to 'Accept deviation' or 'Completed'. 14. In the list, find and select the desired record. 15. Click Change status to open the drop dialog. 16. In the Review status field, select 'Accept deviation' or 'Completed'. 17. In the Reason code field, enter or select the desired reason code. 18. In the Reason notes field, type a value. 19. Click OK. 20. Click Yes. 21. Close the page. Clean up validation history Clean up validation history In the Data quality studio parameters, you can enable logging of data quality policy validation rules execution. Validation rule execution results are logged only when you manually add or change records. Each time a validation rule is executed, an entry is logged. You can clean up the data quality policy execution log once or in recurring mode. For example, you want to keep validation history records for one month. Each week, you can do a cleanup, deleting history records older than one month. The steps of this topic explain how to clean up the validation history. Procedure 1. Go to Data quality studio > Periodic tasks > Clean-up logs. 2. In the Validation logs retention (Days) field, enter a number. 3. Sub-task: Set up batch processing. 4. Expand the Run in the background section and fill in the fields as desired. Note: The validation history cleanup always runs in batch. 5. Click Recurrence and fill in the fields as desired. 6. Click OK. 7. Click OK. Monitor fuzzy duplicate check history Monitor fuzzy duplicate check history If you have checked a record for fuzzy duplicates, you can view the fuzzy duplicate check history for that record. The history shows all fuzzy duplicate checks that are done for the record. If values are merged from duplicate records to the selected record, you can view the merged values. Procedure 1. To view the fuzzy duplicate check history, go to the applicable page. 2. Select the desired record. 3. Click History. Note: On the action pane, depending on the setup, the History button can be shown:On the 'Data quality' tab, in the 'Duplicate check' button group.As a button.On an existing action pane tab, in the 'Duplicate check' button group. 4. To view merged values, on the Duplicate check history page, click Show merge values. Clean up fuzzy duplicate check history Clean up fuzzy duplicate check history If you have checked a record for fuzzy duplicates, you can view the fuzzy duplicate check history for that record. The history shows all fuzzy duplicate checks that are done for the record. You can clean up the logged fuzzy duplicate check history manually or in recurring mode. For example, you want to keep fuzzy duplicate check history records for one month. Each week, you can do a cleanup, deleting history records older than one month. The steps of this topic explain how to clean up the fuzzy duplicate check history. Procedure 1. Go to Data quality studio > Periodic tasks > Clean-up duplicate check history. 2. In the Retention days field, enter a number. 3. Sub-task: Set up batch processing. 4. Expand the Run in the background section and fill in the fields as desired. Note: The fuzzy duplicate check history cleanup always runs in batch. 5. Click Recurrence and fill in the fields as desired. 6. Click OK. 7. Click OK. End End Validation  history Fuzzy duplicate  check history

Activities

Name Responsible Description

Monitor validation history

Data quality officer

In the Data quality studio parameters, you can enable logging of data quality policy validation rules execution.

Validation rule execution results are

  • Logged only when you manually add or change records. Each time a validation rule is executed, an entry is logged.
  • Not logged if the data quality policy is run by a:
    • Data quality assessment.
    • Connectivity Studio data import.

You can review and process the logged validation rule execution results.

For each logged validation rule execution result, you can:

  • Set the review status.
  • Open the source record to make changes, if desired.

The validation status of a logged validation rule execution result can be:

  • Success: The validation rule is met.
  • Failed: The validation rule is not met and results in an error or warning.
  • Skipped: The validation rule is not executed because its conditions are not met.

Clean up validation history

Data quality officer

In the Data quality studio parameters, you can enable logging of data quality policy validation rules execution.

Validation rule execution results are logged only when you manually add or change records. Each time a validation rule is executed, an entry is logged.

You can clean up the data quality policy execution log once or in recurring mode.

For example, you want to keep validation history records for one month. Each week, you can do a cleanup, deleting history records older than one month.

The steps of this topic explain how to clean up the validation history.

Monitor fuzzy duplicate check history

Data quality officer

If you have checked a record for fuzzy duplicates, you can view the fuzzy duplicate check history for that record. The history shows all fuzzy duplicate checks that are done for the record.

If values are merged from duplicate records to the selected record, you can view the merged values.

Clean up fuzzy duplicate check history

Data quality officer

If you have checked a record for fuzzy duplicates, you can view the fuzzy duplicate check history for that record. The history shows all fuzzy duplicate checks that are done for the record.

You can clean up the logged fuzzy duplicate check history manually or in recurring mode.

For example, you want to keep fuzzy duplicate check history records for one month. Each week, you can do a cleanup, deleting history records older than one month.

The steps of this topic explain how to clean up the fuzzy duplicate check history.

Provide feedback