Delta processing - how to know what's changes to integrate

Is there a call that could be made to the API to retrieve a list of changes made from a given date?

When changes are made to Aristotle, are these deltas sent to a queue of some sort that can be accessed?

I really would like a simple way of integrating Aristotle with other tools making use of metadata definitions. When something of interest is updated, I’d like a way of finding out without needing to interrogate the entire content of Aristotle and sorting out the wheat from the chaff.

Whoops… I wrote the title of that post in a rush. Apologies for the grammar crimes :roll_eyes:

There’s no edit capability to fix poorly worded titles once they’re posted :crazy_face:

Hey @Michael this is a great suggestion.

At the moment we don’t have an API endpoint for only getting recently changed metadata items. We are constantly improving the API and this is an easy suggestion for us to add to Aristotle.

We’re currently on track with the Quarter 3 roadmap features and are already ahead on Quarter 4, so I’ll see if we can squeeze his feature in an upcoming release.

2 Likes

That would be awesome. It would help a lot with integration efforts. Much thanks @sam and Team.

2 Likes

Hi @Michael, just letting you know that a delta endpoint has been released today. You’ll be able to find it at /metadata_info/latest_action.

Let me know how it goes!

@silke, is that ?

  • https://server_name/metadata_info/latest_action
  • https://server_name/api/metadata_info/latest_action
  • https://server_name/home/metadata_info/latest_action

None of these work for me. Although maybe I don’t have sufficient credentials…

@Michael you can access it at:

  • https://server_name/api/v4/metadata_info/latest_action/

And if you go to https://server_name/api/v4 you’ll also see all of the necessary documentation.

Looks nice.

So I run an API call like “https://dss.aristotlecloud.io/api/v4/metadata_info/latest_action/” and everything is sorted in order of date created. The results (just the first two results out of 50K+ here) look like this:

So I know the UUID, date and action type.

I don’t know what entity type the UUID is… but I can run another graphQL to work that out. Something like:

So it takes an extra query to know that it was a distribution that was modified. Now that I know the UUID and the entity type (distribution), I can follow up with an additional query to find out what it looks like now and refresh any mirrors I might have set up. Actually, I can probably just widen the graphQL query above and hopefully get the information I need anyway in one go.

I’ll run with this and see what I can get out of it, thanks again.

We can certainly look at adding in an extra field into this API to return the item type as well.

Looking at the name of the queue, I guess it’s obvious, but I hadn’t realised that this queue only displays the latest change to a record.

So what if I have a case when a record changed radically a couple days ago by User A (let’s say someone wiped out a few hundred permissible values, when they should not have). Today, someone different, User B, makes a minor innocent change. Now I notice all those missing permissible values… I consult the log at metadata_info/latest_action/… and think the problem occurred when User B made a change, since the older log of User A’s change has been removed.

Is this how it currently works?

If so, then I suppose what I can gather from this list is that from a given date of interest, the metadata UUID I see listed has changed at least once, and an unknown number of times before that.

Yup - exactly! In essence, this endpoint is broadly designed to answer the question “What metadata has changed since I last checked the registry?”

The reason it doesn’t return a list of changes is that there are different permissions that can be applied to each individual change to an item, which a user may not be able to see. So a user may be able to see an item, but may not have permission to see who changed it and how - in fact, this is the default behaviour, and when items are published users by default cannot see any version histories that show how the item was changed.

This endpoint is designed to assist with federation between systems, and lets a system interrogate an Aristotle Metadata Registry to determine that changes have occurred to metadata they need to be aware of, but won’t return exact what changes have been made, so that it can update its own content.

In an upcoming release, we are also going to be including deletion policies, to allow administrators to set permissions around the deletion of content.

The approach described above means that the /latest_action/ endpoint can also be used to inform users that content has been deleted within the specified time frame as well, so that the other system can remove the content as required.

1 Like