TL;DR? Here’s a quick summary

Sitecore doesn’t remove xDB data automatically. You will need to execute the Right to be Forgotten either manually from the dashboard or from custom code. Setting your cookie lifetime to match your privacy policy only removes the link between the visitor’s browser and his data in xDB. If you do not act, or clean data manually, all data will be stored indefinitely.

Together with a few colleagues at We are you we have built a custom activity type that allows you to implement sliding expiration in the most elegant way: via a very simple marketing automation plan, each individual xDB contact will be anonymized automatically when the retention period has expired. You can download the installable Sitecore module over here: github.com/robhabraken/data-privacy/Sitecore modules.

Another ‘issue’ with the Right to be Forgotten in Sitecore, is that it doesn’t cover disposing data stored by Sitecore Forms. First of all, there is no link between a Contact in xDB and its form entries stored in the database of Sitecore Forms, and secondly, Sitecore leaves cleaning out those tables entirely to the implementation party. This would require you to start juggling with SQL queries, which is something you’d want to stay away from.

So I have created another installable Sitecore module that establishes a link between xDB Contacts and Form Entries of Sitecore Forms. It inserts a hidden field with the Tracker ID of the current contact into each form submission. I have also built another custom activity type to erase those form entries using a marketing automation plan. Additionally, using an xConnect service plugin, this module automatically includes cleaning up those entries whenever the right to be forgotten is executed (thanks to Anton Tishchenko for the great contribution to my project).

This means you now have automatic disposal of form submissions. And you can also delete form submissions as a part of specific marketing plans, whenever you have a shorter retention period than your general privacy policy states (or when a visitor requests removal of this data).

Data Privacy & Sitecore

A deep dive into data retention

You don’t want sites to store your personal data any longer than necessary, right? And when implementing Sitecore sites you of course want to respect the privacy policy. Now, we’ve got the ‘execute the Right to be Forgotten’ feature in Sitecore, but this is either a manual process (via the Experience Profile dashboard) or requiring custom code.

If you don’t act, all data will be stored indefinitely!

There’s simply no automatic disposal that removes (personal) data once the retention period from the privacy policy expired. Nor is there a way to configure such. I’ve seen the misconception that the cookie lifetime would cover this, which is a setting you can change in your Sitecore implementation. But the cookie lifetime simply ensures that the cookie isn’t stored any longer than necessary. If a visitor’s cookie is gone the connection between his browser (client) and the data in his Experience Profile (server) may be terminated, but his data still lives inside your Sitecore databases.

I came across a few solutions to this issue, but non of them are fully automatic, nor do they take advantage of the capabilities Sitecore offers. Apart from executing the ‘Anonymize’ feature from the dashboard as a marketer, or calling ExecuteRightToBeForgotten() from code, I have seen examples of using the scheduler to call the latter function once in a while. The difficulty here is that your scheduler runs on a Content Management role and your xDB data is managed from your xConnect deployment. Also, it may not be ideal performance-wise when the size of your xDB databases increase, and it isn’t easy to configure as a marketer. Secondly, there’s the Analytics Database Manager (ADM) from the marketplace, and although it does its job just fine, you have to run it manually every once in a while and pick a cut off date. I personally would like to see disposal happening continuously and also per xDB profile, not in batches. Even more since you might have different types of consent and different retention periods.

And then there’s another issue with executing the Right to be Forgotten: it simply doesn’t cover all the data of your Sitecore implementation. We’ll get to that later.

So, this confusion (and some complex or even incorrect documentation) could lead to shying away from storing personal data in xDB for some companies. And I think it is a pity that this lack of clarity would inhibit the use of this wonderful technology!

From Tracker to Database

First of all, it’s important to understand what happens when a visitor enters your website. Mostly because Sitecore uses different models (in code) for the Tracker data and for the xConnect model. And also because it clearly shows why deleting the cookie only breaks the connection, but doesn’t remove the data:

When a visitor hits your website, a Tracker object is created, which contains a so called Contact ID. Upon sessionEnd though, this data is sent to the databases and a Contact will be created in xConnect. It gets a new Contact ID and the ‘old’ ID of the tracker is now moved to the xDB.Tracker identifier. But as you know, Sitecore’s xDB doesn’t consist out of only one database, it has a whole collection of databases that stores your data. The Experience Profile is only saved in the xDB Collection database though, which we will cover first. The second part of this blog post will zoom in on the other databases as well. If you want to learn more about how the tracker works, checkout https://doc.sitecore.com/developers/92/sitecore-experience-platform/en/web-tracking.html.

Implementing Sliding Expiration

So now we know that we want to handle disposal of the data right after the consent of a visitor expires, which I’d like to call ‘sliding expiration’. We want to implement this with minimal code, staying as close to Sitecore’s capabilities out of the box, without putting any more strain on the system than necessary (performance-wise). And this functionality should be highly reusable by marketers, with the flexibility to use it in different scenarios as well. There’s only one way to do this right: a marketing automation plan!

Using a marketing automation plan you can easily configure when to execute the right to be forgotten, and it automatically kicks off when the time is right for each individual Contact in your database. And that’s exactly what we want. Also, it is very easy to change the logic later on. No hard coded logic but handing over the control over the data to the marketer. Additionally, we can also add some more conditions or steps in the process of anonymizing a certain contact.

But there’s no marketing action that lets you execute the right to be forgotten on a certain contact. So let’s make one!

Oh, and please make sure that any custom facet or facet property you create gets the PIISensitive attribute:

As I’ve mentioned above, you can download this custom activity type and marketing automation template as a Sitecore module, which is tested on Sitecore 9.1 update 1 and above. You can also use my example implementation on the Habitat Corporate project as a starting point for your own custom implementation, checkout my GitHub repository at https://github.com/robhabraken/data-privacy.

Conclusion

I think this is a very elegant solution that ensures the compliance with the privacy policy of your website at all times, while not having to worry about performing any manual action at all. On the other hand, it enables your marketers to perform the ‘Anonymize’ feature as a part of a specific marketing automation plan, like upon the end of a specific campaign or when a visitor clicks an unsubscribe link within a campaign email, routing him to a marketing automation plan that handles this fully automatically.

Interesting links

If you want to learn all about what to clean up in Sitecore to be complaint, checkout https://doc.sitecore.com/developers/93/platform-administration-and-architecture/en/right-to-be-forgotten.html.

For a good blog on how to build custom activities for a marketing automation plan, read this https://doc.sitecore.com/developers/93/platform-administration-and-architecture/en/right-to-be-forgotten.html.

For a detailed explanation of the process of identifying contacts and why there are different IDs I’d like to refer to https://doc.sitecore.com/developers/91/sitecore-experience-platform/en/tracking-contacts.html.

Up next…

In the next blog post I will dive into data storage within the other xDB databases and how to clean up personal data within your Forms database automatically as well.