Skip to main content

GDPR and your Exchange / Office365 Contacts from a data perspective

GDPR which stands for General Data Protection Regulation is one of those things that surfaces in the IT world (y2k would be another) that seems like a godsend to the lawyers and anybody doing project management.  Endless paperwork, meetings, manual writing and training that seems to achieve little but cost a ton. One of the problems with GDPR is the very broad brush it paints with about what is considered private data and what constitutes compliance. What I'm going to cover in this post is what you can do if your on the receiving end of a GDPR data request around Contacts that are stored in an Exchange Mailbox and how scripting can help you out. I'm not going to talk about any of the legalities around GDPR because its a bit of minefield but look more at it from the data prospective.

Mailbox Contacts

From a strictly data view the majority of properties that make up an Outlook Contact record is private data. Eg a Persons Name, PhoneNumber, Email Address, Address information etc is the data that essentially makes a contact functional. The volume of personal information people will store in contacts is a very personal thing to the Mailbox owner. Eg some people may store the birthday information of a client, spouse or children which Outlook can cater for to provide a more personal touch while for others just the EmailAddress and Name is enough. Contacts folders can then be shared within or outside an organization (policy dependant)which is where you can really start to getting into trouble and is where  IT professionals should have a part to play to ensure that when things are shared that it is done with security and privacy in mind. Data Stewardship is probably a new buzzword you might starting to here more often.

Should GDPR spell the end of using the default standard permission on Contacts folders ?

I'm posing this as a question more then a statement but my thoughts are probably. The default standard permission on a Mailbox Folder basically allows you to specify everyone (has to be an authenticated user with a Mailbox) can access a particular folder in your Mailbox. eg


By default on an Exchange folder this is set to None (so privacy by design would be good) but if a user (or and administrator using a PowerShell script) has set this to anything other then None on Contacts folders then they are basically sharing any private data they have stored in there own mailbox's contacts folder but can't account for who has access to it and in turn how it maybe used. EG a new Intern in the company decides to email everyone in the contacts folder to complete a task they didn't understand etc. Data Stewardship can mean at lot of different things but as a basic one you should know who has access to private data and be able to explain why people have that access and understand the privacy implications of accessing the underlying data. Eg your switching away from doing things for purely convenience.

Reporting on the state of the private data being stored

Reporting on private data can pose some special logical problems in that the report your producing shouldn't expose any of the underlying data your reporting one. One approach is a simple yes/no on which data is being stored or score each data point and provide a report on that. I've written a script that does both and added it to my contacts Module which is available on the PowerShell gallery https://www.powershellgallery.com/packages/ExchangeContacts and GitHub  https://github.com/gscales/Powershell-Scripts/tree/master/EWSContacts/Module basically what this script does is checks a number of the default contacts properties (not all of them) and counts each one that has data and what data type are available so you can then produce a report of that eg.


This script works in turn with other cmdlets in the Module eg to produce the above report on all contacts in the default Contacts Folder of a Mailbox

$Contacts = Get-EXCContacts -MailboxName gscales@datarumble.com -Folder \contacts $Contacts | ForEach-Object{     Get-EXCPrivacyReport -Contact $_ }
You could also do it as a one-liner if you want to limit the report for example to those contacts with Notes eg

 Get-EXCContacts -MailboxName gscales@datarumble.com -Folder \contacts  | ForEach-Object{ Get-EXCPrivacyReport -Contact $_ } | where-object {$_.HasNotes -eq $true}
I'm only reporting on certain contact properties so this if you have a request to do this type or reporting I would suggest taking a look at the code which is available on GitHub and customising it to suit whatever your reporting needs are https://github.com/gscales/Powershell-Scripts/blob/master/EWSContacts/Module/functions/contacts/Get-EXCPrivacyReport.ps1 (or hire me and I'll do it as I could use the work at the moment).

Searching for Contacts

Article 17 of GDPR covers the rights to erasure (right to be forgotten), while its probably unlikely that this should ever effect Outlook Contacts it maybe that one day you find yourself being asked to show that you  could comply with such a request.

Generally this means just going in and deleting one contact in the Outlook Contacts folders but if your asked to do this across a number of Mailboxes or the Mailbox has a number of Contacts folders where the contact maybe located (or maybe the user has copied the contacts) this is where some type of automated search can be useful. Usually the first port of call for searching would be either the Search-Mailbox cmdlet or eDiscovery/Compliance tools in Office365 portal or OnPerm server. These currently don't seem to offer the ability to do a simple search via Email Address or DisplayName for an Outlook contact so I've written a script and added this to my EWS Contacts Module that can do this.

This script first gets all the Contacts Folder in a Mailbox in the visible Mailbox Root and then does a KQL (or AQS if you still on 2010) search of each of these Contacts Folder. With this script I do a parameter-less Keyword search of the Contacts which given that the emailaddresses and displayname should be indexed should then return the contacts as need. I then do some validation at the client side to remove any false positives by checking each of the 3 EmailAddress properties if you searched via email address or the many displayName properties that are available. If you are searching by displayName you do need to be careful of the format and any punctuation that maybe have been used that could affect the search results.(if you where requested to search via Telephone number that's possible but each phone number is a different so would require extra code to support that)

The following is an example of using the Search cmdlet to search via email

Search-EXCAllContactFolders -MailboxName mailbox@domain.com -EmailAddress gscales@datarumble.com  -Credentials $cred
or to search via displayName


Search-EXCAllContactFolders -MailboxName mailbox@domain.com -DisplayName "user im lookingfor"  -Credentials $cred
This script returns the typed EWS Managed API objects which is useful if you want to do further manipulation (Copy,Move) or if you just want to delete the object just call the Delete Method with the Enumeration for the type of delete you want to do eg for a soft deleted

$contact.Delete([Microsoft.Exchange.WebServices.Data.DeleteMode]::SoftDelete) 

The other thing the script returns is what property it matched on so for instance if it is the EmailAddress you are searching for and the TargetContact has this email address as its EmailAddress3 property when you look at the results of the search if you check the matched property it will show you this eg


As you can see above I haven't filtered any of the hidden Contact folders in the Mailbox so it will search through some of the System Folders like the Recipient Cache etc.

One last thing to remember here is that if your just talking about the emailaddress of a person its going to be stored in other places in the Mailbox eg AutoComplete Cache and in any email correspondents. So completely forgetting an email address across all dataset in Exchange is extremely impractical. (eg consider that its also held in Tracking logs, Antispam gateways etc).

Contact Groups

Contract Groups are personal distribution lists that are stored in a Mailbox's Contact Folders. From a Search perspective if you where looking to find if an Email Address was a member of any Contact Groups in a Mailbox it not an easy task to fulfil (because there is no mechanism to search contact Group members).So for this I've written a script that will enumerate all the Contact Groups across all Contacts Folders, then enumerate all the members in theses groups and then check each of the EmailAddresses or DisplayName (Email displayname) properties to check if a users is member. This Script will then return the Group if found with a new property call MatchedMember with the Member that was found. This make it easier if you want then remove the member you can call the Member.Remove Method using this property. Eg to find and remove a user from a Contact Group using this cmdlet


$Groups = Search-EXCAllContactGroups -MailboxName user@youdomain.com -Credentials $cred -EmailAddress test@you.com Foreach($Group in $Groups){        write-host ("Removing : " + $Group.MatchedMember.AddressInformation.Address + " From " + $Group.DisplayName)        $Group.Members.Remove($Group.MatchedMember)        $Group.Update([Microsoft.Exchange.WebServices.Data.ConflictResolutionMode]::AlwaysOverwrite) }
All the source code for the scripts I've talked about in the post from GitHub here https://github.com/gscales/Powershell-Scripts/tree/master/EWSContacts/Module or you can get the Module from the PowerShell Gallery here https://www.powershellgallery.com/packages/ExchangeContacts/

Popular posts from this blog

Testing and Sending email via SMTP using Opportunistic TLS and oAuth in Office365 with PowerShell

As well as EWS and Remote PowerShell (RPS) other mail protocols POP3, IMAP and SMTP have had OAuth authentication enabled in Exchange Online (Official announcement here ). A while ago I created  this script that used Opportunistic TLS to perform a Telnet style test against a SMTP server using SMTP AUTH. Now that oAuth authentication has been enabled in office365 I've updated this script to be able to use oAuth instead of SMTP Auth to test against Office365. I've also included a function to actually send a Message. Token Acquisition  To Send a Mail using oAuth you first need to get an Access token from Azure AD there are plenty of ways of doing this in PowerShell. You could use a library like MSAL or ADAL (just google your favoured method) or use a library less approach which I've included with this script . Whatever way you do this you need to make sure that your application registration  https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-

How to test SMTP using Opportunistic TLS with Powershell and grab the public certificate a SMTP server is using

Most email services these day employ Opportunistic TLS when trying to send Messages which means that wherever possible the Messages will be encrypted rather then the plain text legacy of SMTP.  This method was defined in RFC 3207 "SMTP Service Extension for Secure SMTP over Transport Layer Security" and  there's a quite a good explanation of Opportunistic TLS on Wikipedia  https://en.wikipedia.org/wiki/Opportunistic_TLS .  This is used for both Server to Server (eg MTA to MTA) and Client to server (Eg a Message client like Outlook which acts as a MSA) the later being generally Authenticated. Basically it allows you to have a normal plain text SMTP conversation that is then upgraded to TLS using the STARTTLS verb. Not all servers will support this verb so if its not supported then a message is just sent as Plain text. TLS relies on PKI certificates and the administrative issue s that come around certificate management like expired certificates which is why I wrote th

The MailboxConcurrency limit and using Batching in the Microsoft Graph API

If your getting an error such as Application is over its MailboxConcurrency limit while using the Microsoft Graph API this post may help you understand why. Background   The Mailbox  concurrency limit when your using the Graph API is 4 as per https://docs.microsoft.com/en-us/graph/throttling#outlook-service-limits . This is evaluated for each app ID and mailbox combination so this means you can have different apps running under the same credentials and the poor behavior of one won't cause the other to be throttled. If you compared that to EWS you could have up to 27 concurrent connections but they are shared across all apps on a first come first served basis. Batching Batching in the Graph API is a way of combining multiple requests into a single HTTP request. Batching in the Exchange Mail API's EWS and MAPI has been around for a long time and its common, for email Apps to process large numbers of smaller items for a variety of reasons.  Batching in the Graph is limited to a m
All sample scripts and source code is provided by for illustrative purposes only. All examples are untested in different environments and therefore, I cannot guarantee or imply reliability, serviceability, or function of these programs.

All code contained herein is provided to you "AS IS" without any warranties of any kind. The implied warranties of non-infringement, merchantability and fitness for a particular purpose are expressly disclaimed.