Metadata Merger Tool

MAST Methodology
1

The Metadata Merger Tool is a new feature added to the Aristotle Registry to help with metadata harmonisation. This tool is useful for merging metadata at the “Audit and Harmonisation” phase of MAST methodology. Admin permissions are needed to use the metadata merger rule.

Sources required to carry out metadata merging:

Source Metadata: The metadata that will be merged with the target metadata. All the links associated with the source metadata will be linked to the target metadata after the merge.

Target Metadata: This is the metadata into which other metadata gets merged. The merger tool will remove all the linkages from the source metadata and create the same with the target metadata, in addition to the linkages it already has.

Merger tool functionality

Merging Metadata: Using the source and target metadata items, the same type of metadata, such as object class, data element, data element concept, and so on, can be merged. It will remove all the metadata linkages from the source metadata and add all those to the target metadata. The change will be reflected at all levels of the registry.

Undo feature: This merger tool has an awesome feature to undo the merge at any time. Once the rule is created and executed, you can undo that change by clicking the undo button.

Rules creation: You can use the merger tool to create rules that can be saved and used later.

In the example below, before merging, there are three object classes, ‘Person’ with ids 273 and 24 and ‘Citizen’ with id 547.
After merging, the ‘Person’ with id 24 and ‘Citizen’ with id 547 is merged to ‘Person’ with id 273’, all the metadata is now linked with the ‘Person’ with id 273.

Merger Tool (12)
Merger Tool (12)1650×1340 218 KB

For more information on the Metadata Merger tool, please see the attached pdf.

Merger tool.pdf (1.4 MB)

2 Likes
2

Thanks very much this is great!

If the Source Data Elements are linked into Data Element Paths in Distributions, will the Target Data Element flow through into those Data Element Paths?

3

Hey Andrew,

Thanks for asking this question.
If the source data elements are linked into data element paths in distributions, the target data element flow nicely through those Data Element Paths. Just to let you know, at the moment, this feature is in beta version, it’s working fine on executing the merge, but on undo, the target data element will not go back to the source data element, it’s still in progress.

Thanks,
Shikha

4

Thanks Shikha. Good to know it’s beta.

Another question, what happens to the registration status of any items that are merged? To me it seems that all source items should get a status of retired or superseded (or similar).

5

Hey Andrew,

That’s a really good idea; thanks for that. Currently, the registration status remains the same and gives a message at the top that the item has been merged.

About the registration status, I think we will probably go with ‘retired’ status’ as ‘retired’ means the item is not recommended for use, which we want to achieve in this context, whereas “superseded” status means that the item has been replaced or superseded by the latest version or another item.

1 Like
6

Thanks Shikha,

Are there plans to have a governance process around the merge. At a minimum, if a registration authority has endorsed the source metadata items as standard, they should need to approve the merge.

Beyond the simple case above, it is potentially very complex. For example, if data elements are merged, should the stewards of distributions that used the source data elements be asked to approve the merge? Does it make a difference if the distributions have been endorsed as standard vs not having been endorsed?

Andrew

7

Thanks Andrew, that’s a really good feedback.

We wanted to give admins the ease of merging metadata as required; at the moment, only the admins have access to do the merge, and a few tasks need to be done manually by the admin as per the requirement, but moving forward, for the phase 2 merger tool updates, we would love to implement the governance around the merge.

Thanks again, your suggestions are really helpful.

Shikha

1 Like
8

We have started to work through our first set of merger rules, and have a couple of initial observations:

  1. Can the definition of the updated item be displayed? For example with Object Class merging, one of the things we will need to assess and decide is whether the updated concept will need a definition change to be consistent with its new name. Or, if we are linking a concept to a more generic OC, do we need to make the property more specific (move some of the meaning from the OC to the Property level). If, based on the definition of the concept, we need to make a change to the property, we will need to decide whether to link to a different existing property, or change the property. And we also need to check that we are not creating any duplicates by executing the merge.

  2. Would it be possible to also apply adjustments to the new concept and element definitions within the tool. At the moment we will have to execute the merge, then edit all of the related concepts/elements definitions afterwards.

  3. Can the highest registration status of the items be displayed to assist with determining the impact of changes.

  4. Can we have the ability to edit a rule after it has been saved and/or ability to deselect/remove a particular item from being merged. Sometimes, we we may come across a concept that gets picked up for merging based on the OCs, but on closer inspection it actually needs to be changed to a completely different object class.

Thanks, looking good so far, will keep working through our first set of rules and post any other ideas/scenarios we come across.

Cheers,

9

hey @skew, Thanks for the feedback. I just wanted to let you know that the merger tool is still in beta and that we are working on it. To answer your questions:

  1. Did you mean something like showing the definition and the registration status in the preview and giving the option to exclude changes from the merge based on your choices via the check box? See below for an example.
    image
    image1567×876 231 KB

In this way, you will be able to see the definition while creating a rule, and if it needs a change, you can either edit the definition by going back to the element or exclude the change from the merging rule by checking the tick box before saving the merge rule.

Using the merger tool, you can only merge the same type of metadata item. Once the object classes have been merged, you can decide which properties to merge based on your analysis and then perform the merge for properties.

You can check for duplicates using the metadata report builder in the toolbox.

  1. Changes to the definitions will be hard to do while merging, and thus require manual assistance. The easy way to change the names and definitions is to generate a report of all the object classes with their UUIDs, and using the UUIDs in the SDDF excel, you can update the names and definitions in just one upload.

  2. At the moment, if the target metadata item has a registration status, it only shows that. The registration status is unaffected by the merger. If it makes it easier for you to decide whether to merge or not, we can add the registration status in the preview as shown in the above screenshot.

  3. Yes, we can scope this out in the next-year roadmap. As shown in the above screenshot, we can provide the ability to select the ones that you don’t want to include in the rule.

Cheers,
Shikha

10

Thanks Shikha, that’s great, exactly what I had in mind. The extra info and check options will give us greater control over the merge and ensure we don’t merge things that shouldn’t be, and help identify what other updates to the items might be required.

We can then make manual changes to definitions etc. if there are only a few impacted items. If there are many impacted items, we can use the report / import process.

Something that will help greatly with this might be if the generation of the report can be an option or a part of the merge execution process…? One thing that we were a little worried about was losing visibility of the changed items after merging, in order to be able to make the definition updates etc.

11

Hey @skew, thanks for the feedback!

by report, do you mean the record of changes that were performed as part of the merge, something like, logs of the merger changes?

As for the visibility of the changed items after merging, you can undo the changes to check the previous definitions. When the merge is executed, it saves a log of the before and after (which is how it knows what to undo). We just don’t have the download option for the log yet, which we would like to include in the next development phase of the merger tool.

Cheers,
Shikha

12

Yeah, similar, I was thinking if you can produce a download version of the log that is in the same format as the Bulk Import template, containing at least the ID, Name and Definition of updated items, then it will be easy for us to apply minor definition changes to all impacted items in that spreadsheet, and then apply them via the bulk import.

13

It sounds like a good idea. I have created a task for this one for the new year and will keep you updated on its progress.

Merry Christmas and happy holidays!