ServiceNow CMDB Identification and Reconciliation Engine (IRE)

A configuration management database (CMDB) is a database used for the purpose of populating, monitoring, and maintaining information about Configuration Items (CIs). CIs are the various components, like services, devices, applications, etc., that are present in an organization’s infrastructure. The ServiceNow CMDB can be used to populate data for important attributes of various types of CIs and the relationships between different CIs. The information available in the CMDB is typically used to support various business objectives of an organization, like IT service management, IT operations management, IT asset management, security operations, risk management, etc. Maintaining the data quality and accuracy of the CMDB is very important to support the business objectives more effectively and efficiently. ServiceNow’s Identification and reconciliation engine (IRE) plays a key role in this. The basic building blocks of Identification and reconciliation engine (IRE) are CI Identification rules, CI reconciliation rules, and IRE data source rules.

 

1. CI Identification rules

Identification rules are a set of criteria that ServiceNow utilizes to identify already existing Configuration Items (CIs) within the CMDB. These rules specify how a CI is recognized using some unique attribute (e.g., Name, Serial number, Device ID, Object ID, etc.) or a combination of one or more unique attributes. Properly defined identification rules are essential for maintaining and improving CMDB data quality by reducing the possibility of creating duplicate CIs.

 

1.1 How to access identification rules?

It can be accessed using any of the 2 ways mentioned below:

  1. Go to ‘CI class manager’. Search for any CI class that you want to check (e.g., Network Gear) and open it. Click on the ‘Identification Rule’ tab.

'Identification rule' tab in CI Class Manager

This method is more suitable to check all the identifier entries defined for any specific CI class.

  1. Go to ‘Identification/Reconciliation>CI identifiers from the application navigator.

'Identifier entries' module in application navigator

This method is useful for checking identifier entries defined across all CI classes.

 

1.2 Understanding CI identification rules in action

Let’s take the ‘Network Gear’ class as an example. Open ‘CI class manager’ from the application navigator. Search for Network Gear. Click on ‘Identification Rule’ tab. The ‘Identifier entries’ section shows the list of attributes which are specified as the identifier entries for this class.

'Identifier entries' example for 'Network Gear' CI Class

We can see ‘Priority’ values assigned to each identifier entry. The entry with the lowest ‘Priority’ value (i.e., 100 here) would be evaluated first, followed by the remaining 3 entries in the increasing order of ‘Priority’. In this example, ServiceNow IRE will initially check for the presence of existing CIs with the combination of ‘Serial Number’ and ‘Serial Number Type’. If a match is found, the data will be updated for the identified CI, and the remaining identifier entries won’t be evaluated. If a match is not found, it will continue by checking the next identifier entry (only Serial Number) and so on. If a match is not found even after evaluating all the 4 identifier entries, a new CI will be created.

 

1.3 Configuring new Identification Rules

The ‘Replace’ button can be used to define a new identification rule for this class. Then an ‘Add’ button will appear under ‘Identifier Entries’.

Configuring a new identification rule

Click the ‘Add’ button. A window will appear where we can choose whether we want to use the attributes from the main table or from another table (lookup table) or from a combination of tables. Click ‘Next’ after choosing.

Choosing the table which contains the identifier entry attributes

Specify the ‘Priority’, select the ‘Criterion Attributes’ and click ‘Save’ to create the identifier entry. Repeat the process for adding more attributes with a different priority.

Selecting the attributes to be added as identifier entries

 

1.4 Best practices to consider for CI identification rules

Following are some best practices to consider regarding identification rules:

  • Review the out-of-the-box (OOB) identification rules for all the CI classes that you are planning to manage in your CMDB. Modify the rules wherever required based on the unique attributes present in the data sources that will be used to populate each class.
  • The attributes considered as identifier entries should be unique and must remain unchanged throughout the CI lifecycle. It should also be populated for most of the records. Leverage a combination of multiple attributes wherever possible.
  • Use Service Graph Connectors (if available) for third-party data integrations that populate the CMDB. Integrations that are NOT done using this method will NOT go through the IRE. i.e., they do NOT consider the identifier entries defined for the CI classes that are populated by the integration.
  • Use integration hub ETL for manual data imports. Any data imports that are NOT done using this method will NOT go through the IRE.
  • Review the identification rules periodically to check if any modifications are required due to any changes in IT infrastructure. E.g., introduction of a new data source

1.5 Identification inclusion rules

This is an additional feature that may be used to limit the list of CIs that are considered for the identification procedure. If a CI class has an Identification inclusion rule specified, ServiceNow’s IRE only processes CIs that satisfy the requirements specified by the identification inclusion rules for that CI class. You may, for instance, establish a rule that only considers CIs with the operational status=Operational. There are no predefined inclusion rules in an out-of-the-box (OOB) ServiceNow instance.

Identification inclusion rules can be accessed and configured by going to the ‘Inclusion Rule (Advanced)’ section present under the Identifier entries defined in the ‘Identification Rule’ tab of a CI class. Click on ‘Add’ and then define the ‘Active record condition’. For example, the following screenshot shows how to set an inclusion rule for the ‘Computer’ CI class with the condition Operation status=Operational.

Adding 'Identification inclusion rules'

 

2. Reconciliation rules

CI reconciliation rules play a vital role in maintaining the integrity and accuracy of the Configuration Management Database (CMDB). These rules are important to resolve conflicts and discrepancies that arise when multiple data sources update the data for the same CI class in the CMDB. For example, if both ServiceNow discovery and an integration from a third-party data source (e.g., SCCM) are updating the same CI, reconciliation rules determine which data source should have higher priority.

2.1 How to access Reconciliation rules?

It can be accessed using any of the following 2 methods:

  1. Navigate to ‘CI class manager’. Search for any CI class that you want to check (e.g., Server) and open it. Click on the ‘Reconciliation Rules’ tab

Accessing 'Reconciliation Rules' tab in 'CI Class Manager'

This method is more suitable for checking all the reconciliation rules defined for any specific CI class.

  1. Go to ‘Identification/Reconciliation>Reconciliation Definitions’ from the application navigator.

'Reconciliation Definitions' module in application navigator

This method is useful for checking and analyzing reconciliation rules defined across all CI classes.

 

2.2 How to configure Reconciliation rules?

Go to the ‘Reconciliation Rules’ tab of the required CI class (E.g., Server) from the CI class manager as mentioned in section 3.1. Click on the Add button under ‘Reconciliation Rules’ section.

Configuring a new 'Reconciliation Rule'

Step 1: A window will appear with the tab named ‘Add Discovery Sources & Prioritize’. Specify each ‘Discovery source’ and its ‘Priority’ using the + button. Then click on the ‘Next’ button to move to the next tab. The following screenshot indicates that ServiceNow discovery will be given a higher priority over manual entry as per this reconciliation rule.

Adding Discovery sources and their priorities for the reconciliation rule

Step 2: From the ‘Select Attributes’ tab, choose the attributes for which this rule will be applicable. The ‘Apply to all attributes’ option can be enabled if we need to use the rule for all attributes of that class. As per the attributes selected in the following screenshot, manual entry will not be able to overwrite the data from ServiceNow Discovery for Manufacturer, Model ID, Name, and Serial Number.

Choosing the attributes for which the rule will be applicable

Step 3: From the ‘Set Filter Condition’ tab, define a filter condition only if you need to restrict this reconciliation rule to CIs that meet some criteria. Click on ‘Save’ to finish creating the rule.

Setting filter condition to define the criteria for which the reconciliation rule will be applicable

Note: Multiple reconciliation rules can be created for the same CI class to define discovery sources and their priorities for various attributes that are populated in the CI class.

 

2.3 Dynamic reconciliation rules

What we saw in Section 3.2 are static reconciliation rules. Dynamic reconciliation rules were introduced in the San Diego release. We need to enable Multisource CMDB to use this feature. Dynamic reconciliation rules can be used to choose one of the following types of values from the Multisource CMDB data:

  • First Reported
  • Most Reported
  • Last Reported
  • Largest Value
  • Smallest Value

If the same CI attribute has both static and dynamic reconciliation rules, the dynamic reconciliation rule will take precedence.

 

2.4 Data refresh rules

This feature can be used to specify when to determine if a CI is stale for a specific discovery source and override the static reconciliation rule for stale CIs so that they can be updated by a lower-priority authorized discovery source. They are present on the ‘Reconciliation Rules’ tab itself. Click Add and then specify the ‘Discovery Source’ and its ‘Effective Duration’ (i.e., the number of days beyond which the CI will be considered as stale for that data source). Click ‘Save’ to create the rule.

Adding a new 'Data Refresh Rule' from the 'CI Class Manager'

For example, let’s consider that the following data refresh rule is defined for the ‘Server’ class. As per this, a lower priority data source would be able to update the data for all Server CIs that are NOT updated in 14 days by ServiceNow discovery.

Example of a 'Data Refresh Rule'

 

2.5 Best practices to consider for CI reconciliation rules

Identify all the CI classes that are populated by more than one method (even if it is just discovery + manual entry only). List down the key attributes that are being populated in these classes from various data sources or methods. Discuss with various stakeholders like the configuration manager, various CI class owners, teams that manage each data source (e.g., SCCM team), etc. to understand which data source or method should be given higher priority for different attributes. Also, discuss and understand whether we need to define data refresh rules and dynamic reconciliation rules for any of these CI classes.

 

3. IRE data source rules

Consider a scenario in which a certain data source is no longer trusted to create new CIs in a particular class, but it can be trusted to update existing CIs only. IRE data source rules can be created to prevent a specific data source from inserting new CIs for a specific class. For example, the following rule indicates that insert is not allowed from the LANDesk data source to the Computer CI class.

Example for 'IRE data source rule'

Only a single IRE data source rule can be active for any pair of CI class and data source.

 

Conclusion

The CMDB is a very powerful database that can support the business objectives of an organization across a multitude of IT and non-IT functions across the enterprise. An inaccurate, incomplete, or outdated CMDB can lead to erosion of user trust. This will impact all the business functions that leverage the CMDB. Properly reviewing and defining CI identification rules as well as reconciliation rules for each CI class will help to avoid a lot of CMDB data quality issues.

Date Posted:

May 29, 2024

Share This:

Leave A Comment

Categories

Tags

Loading

Related Posts

Fresh Content
Direct to Your Inbox

Just add your email and hit subscribe to stay informed.