Restricted SharePoint Search is a new Microsoft feature to mitigate sites oversharing issue when you are implementing Copilot. The feature is documented here, but still I have some questions, e.g.:
How about external data? Copilot can use external data to learn from via agents and connectors. But would Restricted SharePoint Search if implemented allow data from external connectors to be used in copilot?
“Users’ OneDrive files, chats, emails, calendars they have access to” – means own data for every single user or all shared OD data?
What exactly is “Files from their frequently visited SharePoint sites”? I mean, how frequently user needs to visit site for this?
What exactly means “Files that the users viewed, edited, or created.”
What about teams chat messages, e-mails, viva engage messages?
“Files that were shared directly with the users” – does that mean “individual files shared” or can include folders, libraries, sites?
If user is a member of a teams – would all team content included?
It says “Files…” but would site pages be included? Or list items? Or list items attachments? Pages is something that people use to create wiki to share knowledge.
How long it takes for Microsoft 365 to start restricting results after Restricted SharePoint Search is enabled
How to deal with “You do not have the required license to perform this operation”
Here I’m going to answer the questions above.
So far I build a test scenario using my dev tenant that includes multiple collaborated users and content in the form of files, pages, list items and messages spreaded across multiple sites falling into different categories of Restricted SharePoint Search allowed content.
You do not have the required license…
If you are getting “You do not have the required license to perform this operation” when you are trying Get-SPOTenantRestrictedSearchMode or Get-PnPTenantRestrictedSearchMode – that means there is no Copilot for Microsoft 365 licenses assigned to tenant yet. This feature – Restricted SharePoint Search – works only when at least one Copilot license is assigned to tenant.
Governance in IT is establishing rules, policies, tools and practices that helps you manage and protect your enterprise resources. SharePoint governance (or wider – Collaboration governance) covers
Microsoft 365 groups is a key concept in today’s collaboration landscape that includes Microsoft Teams, Viva Engage, SharePoint etc. Access to resources is organized via groups. It is essential that every Microsoft 365 group has an owner (owners) so we have somebody to enforce Collaboration governance through.
Scenario
Let say you administer a large Microsoft 365 environment (e.g. ~100k+ users and/or ~50K+ sites) and after some years you have a lot of ownerless groups and sites (around 5k probably), and a lot of inactive groups and sites (maybe 15k). You are getting more and more ownerless groups – dozens each week. You are thinking of stopping bleeding and cleaning this up…
Out-of-the-box we have Microsoft 365 groups expiration policy and Microsoft 365 ownerless groups policy. You might also have some 3-rd party tools implemented – e.g. ShareGate, SysKit Point.
If you do not care – you might just activate both OotB Microsoft policies – via GUI – they are simple to activate. But once you activated policies – they will trigger thousands of emails. Now imagine a person is getting dozens of emails asking him/her to be an owner or to renew the group that probably he/she has no idea about… What will happen next? People will probably ignore these alerts. Then? Groups and sites will be automatically deleted. And then? Right, there will be a huge noise and many angry users and high-priority tickets and you will have to restore sites/teams and finally you’ll have to deal with all that mess manually.
So, what is the right way to clean-up a large Microsoft 365 environment from ownerless and inactive teams, groups sites? Not a trivial question, hah?
Solution
Disclaimer: I’m sharing here my personal opinion with no obligations or warranty etc., so you’d dig into all the technologies used and based on your particular situation build your own plan. But my personal opinion is based on my 15+ years experience with SharePoint, including really large environments.
Note: It is always a good idea to discuss your plans with you org’s communication team and helpdesk/service-desk to adjust clean-up activities with other initiatives and let other people be prepared.
High-level steps for group-based Sites:
consider implementing Minimum 2 owners per group policy to stop bleeding. Currently Microsoft 365 does not have such functionality, so consider 3-rd party tool like SysKit Point or custom PowerShell script that sends notifications
apply this policy to groups where you already have 2+ owners – it’ll be safe
apply this policy to all other groups by chanks
consider custom PowerShell clean-up, e.g. you can simply delete groups with no owners and no members and/or inactive groups with no content and/or groups that are inactive for a long time (this must be aligned with business and legal)
avoid scoping down this policy via people (security groups)
implement it for all groups all users with 6-7 weeks and custom e-mail template
implement Microsoft groups expiration policy in “Clean-Up” configuration… again, there are a few different strategies – see this article
change Microsoft Ownerless groups policy configuration to a “Permanent” mode configuration set
(or) change Microsoft 365 groups expiration policy with a “Permanent” mode configuration
(or) develop and implement custom staged decommissioning process – kind of “last chance” set of scripts to discontinue groups that are still ownerless after all efforts above. Staged means we do not just delete these groups, but e.g. we can – rename ownerless groups – convert groups from public to private – set teams to archived mode – exclude sites from copilot search with “Restricted SharePoint Search” etc. – set site to no-access mode – remove members from the group – and finally delete the group with connected team team and site I have a separate article on custom staged decommissioning process
Note: There will always be ownerless groups in large environment. We have to live with it. So all steps above – think of it as a processes – we’d need to do it on regular basis.
All above was mostly about group-based sites (as we have OotB Microsoft policies for groups), but we probably have the same problem (or even worth) with standalone sites (that would be a separate topic).
There is a new feature published at Microsoft roadmap site:
Microsoft 365 admin center: Manage ownerless Microsoft 365 groups and teams
Teams, Outlook groups, Team Sites etc. powered by Microsoft 365 Groups supports two roles: members and owners. Members can collaborate with others in the group through files, emails, messages etc. Owners manage the group membership and monitor content and conversations. When employees leave an organization or switch projects internally, it results in their existing user accounts getting deleted. If such employees were group owners, keeping track of their groups becomes critical to ensure accountability within the organization. We have introduced a new ownership governance policy to help automate the management of ownerless groups by requesting active members to become owners of the group. Admins can define who is eligible for these notifications and configure what notifications and how often these notifications are sent to active group members. Users, who are members of the ownerless groups can simply accept or decline request via the actionable email message.
Feature ID: 180749
Added to roadmap: 10/10/2023
Last modified: 10/10/2023
Product(s): Microsoft 365 Admin Center
Cloud instance(s): GCC
Platform(s): Web
Release phase(s): General Availability
But based on the feature description – all looks exactly as what we already have for years as “Microsoft 365 ownerless groups policy” which you can configure under Microsoft 365 Admin Center -> Settings -> Org settings -> Microsoft 365 groups
As SharePoint or Teams admin you manage Microsoft 365 groups (create, update, delete, manage membership etc.) having your admin role activated. I prefer PowerShell 7 and Microsoft.Graph PowerShell module, and I need an Azure registered app with “Group.ReadWrite.All” Microsoft Graph API delegated permission.
Some findings:
If a user was not a group member or group owner – and the user is added to the group members – this user will get notification “You’ve joined the <Group Name> group” via e-mail that comes from a group e-mail address.
When a user is added to the group owners (or elevated to group owner if user was a group member) – user does not get notification.
When a user was a group owner and now you are adding this user to the group members – user does not get notification.
All the actions are logged into Microsoft 365 audit log under your personal Id.
Script samples:
# This script is just a sample to demonstrate basic technique on getting, updating groups membership and deletion m365 groups with PowerShell and MS Graph
#
# please do not run this script as is, but update it based on your needs
# authentication with personal Id
# app must have as minimum "Group.ReadWrite.All" Microsoft Graph API delegated permission
# user must have SharePoint admin (or Teams admin) roles activated
Connect-MgGraph -ClientId $clientid -TenantId $tenantId
Get-MgContext | Select-Object Scopes -ExpandProperty Scopes
# sample data
$groups = @()
$groups += [PSCustomObject]@{GroupId = '443d22ae-683a-4fe4-8875-7bd78227a026' }
$groups += [PSCustomObject]@{GroupId = 'e5805388-c18c-48c0-b42d-6223cf8f3d82' }
# Get Groups
foreach ($group in $groups) {
Get-MgGroup -GroupId $group.GroupId
}
# add members to the group
$groupId = '443d22ae-683a-4fe4-8875-7bd78227a026'
$userId = 'df74e0d3-d78c-495b-b47a-549437d93cf7' # Adele
New-MgGroupMember -GroupId $groupId -DirectoryObjectId $userId
# add Owner to the group
$groupId = '443d22ae-683a-4fe4-8875-7bd78227a026'
$userId = 'eacd52fb-5ae0-45ec-9d17-5ded9a0b9756' # Megan
New-MgGroupOwner -GroupId $groupId -DirectoryObjectId $userId
# Delete group
$groupId = '443d22ae-683a-4fe4-8875-7bd78227a026'
Remove-MgGroup -GroupId $groupId
It is known that a single Microsoft Exchange account is not sending more than 10k emails per day.
It is also know that once activated – Microsoft 365 groups ownerless policy will be sending notifications for all groups in scope to specified number of group members within 24 hours.
The question is: what if there are more than 10,000 notifications to send (e.g. 4,000 ownerless groups and the policy is configured to send notification to 3 members per group – that gives us 12,000 notifications to send)? Would the policy send 10k notifications and the rest 2k notifications the next day?
I’m conducting an experiment. I created 10k groups in my lab tenant with one owner and 3 random members. Then I configured a policy that is sending notification to a 3 most active members (in this case – random members). And then I made all these groups ownerless by deleting the single owner Id from Azure AD (Microsoft Entra).
Here is what I got from users perspective:
user
user groups number
got messages day 1
got messages day 2
got messages total
1 Roger
5012
1374
2 Dick
5039
1349
3 Bob
5108
3412
4 Bapu
4908
1376
5 Stas
4996
1437
6 David
4959
1377
total
10325
Here is what audit log says:
Events “OwnerlessGroupNotified” day 1: 4949 Events “OwnerlessGroupNotified” day 2: 95 Events “OwnerlessGroupNotified” total: 5044 Each event details says 3 members were notified.
It seems like groups are selected by policy in random order.
Massive E-mails sending was started 43 minutes after midnight UTC
“OwnerlessGroupNotified” were logged at the rate of 1925 events during 1-st hour, 2029 events during 2-nd hour, 785 events during 3-rd hour, 176 events during 4-th hour, 26 events during 5-24 th hour, 95 events during next 25-48 hours so max rate was one event every 3 seconds in the beginning (or 1 e-mail per second) …
Adaptive scopes are good, but what if both policies are implemented? Which one wins? The scenario for two policies might be: static retention policy is implemented as default retention policy for all sites, and if site require different retention or deletion – it should fall under one of the adaptive scopes and an adaptive retention policy will be applied.
Nobody likes garbage, including Microsoft 365 administrators. If any user can create a team or yammer community – they create, but then they leave company and we are getting more and more abandoned groups, teams and SharePoint sites. So we need a way to clean up environment. There is a Microsoft 365 groups expiration policy that can help remove unused groups from the system, but since all Teams and Yammer sites are group-based – it also helps SharePoint admins make things cleaner. In a nutshell what this policy does is it sends notifications to group owners so a group owner can renew the group, otherwise the group will expire and be deleted.
Who can configure the policy and how
The policy lives under Azure Portal, Azure Active Directory, Groups, Expiration:
Microsoft 365 groups expiration policy can be configured by Groups Admin or Global Admin (tenant admin) only. Microsoft 365 Teams or SharePoint admin cannot configure it. Microsoft says that User administrator can do it – so I need to verify it.
Here is the policy config screen:
Microsoft documented it well in the “Microsoft 365 group expiration policy“, but I completed some tests in my lab environment and here is what I found and what is not covered by Microsoft. Let me share it with Questions and Answers format:
Questions and Answers
General questions
Q: How long it takes for policy to start generating notification emails after activation? A: Immediately, i.e. minutes, maybe up to one hour (in case there groups that are subject for the policy).
Q: Can I customize email that is send to group owners? A: No, there is no such option at the moment.
Q: What is the email address notifications come from? A: It’s “msgroupsteam@microsoft.com” with the display name “Microsoft Groups Team”
Q: What does a notification email look like? A: Please find some examples below, in the end of this article.
Q: Are there other ways to get notifications? Teams? A: I have not seen any official Microsoft’s documentation on this, but yes – notifications are coming via Teams too: “TeamName is expiring soon. Renew now”:
though it is not clear what exactly should used do to renew the group, as after clicking on that alert a regular teams settings page is opened:
and I got just a few notification in teams, though e-mails notifications I got many.
Q: What happens when a user clicks “Renew group” button in the email notification? A: User will be sent to a Microsoft’s page and the following “Do you want to renew the group?” window will be shown:
On Yes, it says”<groupName> was successfully renewed. You can close this window now”:
And the group expiration date will be set up as current date. On “No” it says “Group was not renewed. You can close this window now.”:
And an expiration day will not be changed. No more notifications will be generated. The group will be active until expiration date. Then the group will be deleted.
Q: What if two owners choose opposite? A: The last action will take effect.
Q: what if one user choose “delete group” but the other one later decided “Renew group”? A: The one who click “Renew group” will see “<Group Name> successfully renewed. Because the group was deleted, it might take up to 24 hours to be fully restored. You can close this window now.”
Q: What if the group does not have owners? A: If the group is orphan (ownerless), the expiration emails will go to the email specified in policy configuration. Usually it is a distribution list with admins or other responsible team.
Q: What if the group does have a non-mail-enabled owner? A: I have tested 2 types of entities with no email: – just a contact in Outlook – user with no Exchange license assigned Results are: Outlook contact cannot be added to team, so there should be no contacts as teams/groups owners; a user with no Exchange licens can be added to team/group and Microsoft does not consider this group ownerless, so notification should be sent to group owners, but since there is no email associated to a group owner – e-mail are not sent, so we are having an issue here.
Q: What if I deactivate the policy – will email notifications sent earlier still be actionable? In other words, would users still be able to renew the group clicking on the “Renew group” button? A: Yes. Actually “Renew group” button is just a link to the Url: https://account.activedirectory.windowsazure.com/Group/RenewGroup?tenantId=<tenantId>&id=<groupId> where a group owner can renew group.
Q: If one of the owners renewed the group – what will happen with notifications sent to other owner? What if other owner click “Renew group” or “delete group”? A: Notifications sent will stay. Since buttons in the email are just links (not actionable buttons) – user will be redirected to a web-page where he/she will be able to renew or delete the group.
Q: As per MS: “Groups that are actively in use are renewed automatically around 35 days before the group expires. In this case, the owner does not get any renewal notifications. Any of the following actions will automatically renew a group…<list of actions>”. So, what exactly does “Groups that are actively in use” mean? A: This is not disclosed by Microsoft. They only say “Azure Active Directory (Azure AD), part of Microsoft Entra, uses intelligence to automatically renew groups based on whether they have been in recent use. This renewal decision is based on user activity in groups across Microsoft 365 services like Outlook, SharePoint, Teams, Yammer, and others.” Btw, <list of actions> includes almost all user actions – so basically any action – even just visit site/team is considered as activity.
Q: Can I track the policy in action via audit log? A: There is no “activity type” for this policy’s specific actions… You also cannot specify user “msgroupsteam@microsoft.com” to get all activities. So no tracks on the policy “before action” – i.e. at the detection and e-mailing stage. If a user clicks “renew” button or “delete group” link – this should be logged as this user action with Category “GroupManagement” and activity: “Update group” and “RenewedDateTime” as property modified. If it happens that the group is deleted by policy – this should be logged under policy’s account – see below.
Automatically renewed group appears as audit log event with – Workload: AzureActiveDirectory – RecordType: 8 “AzureActiveDirectory” – Activity: “Update group” – Properties modified would be “RenewedDateTime”
Automatically deleted group appears as audit log event with – Workload: AzureActiveDirectory – RecordType: 8 “AzureActiveDirectory” – Activity: “Delete group.”
Microsoft groups lifetime policy operates on behalf of Actor (first-party Microsoft service principal):
Q: After the group is deleted, who can restore it? A: MS says: “A deleted Microsoft 365 group can be restored within 30 days by a group owner or by an Azure AD administrator”. In fact, SharePoint admin (and maybe some other roles like Teams admin or Exchange admin) can restore group. SharePoint admin can restore site from recycle bin – and the group will be restored as well.
Q: My org is using retention policies. Will the lifecycle policy delete site if it contradicts with retention policy? A: Lifecycle policy respects retention policy, so if the site should not be deleted according to retention policy or legal hold – the site will not be deleted (TBC – need to be validated).
Q: What if a user forward this e-mail notification to other user? Can this other user renew or delete the group? A: When a user receive a notification email forwarded, and he/she click “Renew group” button – his/her experience will be the same if he/she is also a group owner. If a user is not a group owner – he/she will get “You don’t have permission to renew this group because you’re not an owner. To renew , contact a group owner. You can close this window now.”:
Note: if a user with active groups administration permissions receives email and try to renew or delete the group – he/she will also be able to do that.
Q: Can user get information on groups he/her owns, groups expiration data? Can user renew the group before the policy trigger email notification? A: yes, all that can be done from the page: https://myaccount.microsoft.com/groups/groups-i-own
Q: What if I activate m365 groups lifecycle policy for the selected groups only? Any insight on policy behavior? A: The policy will work as usual, but for the selected groups only. Separate from the policy – under “my groups” users will be able to see “Expiration date” and “Renew” option for groups in policy’s scope only:
Scenario with many existing inactive groups
Let say we have a large Microsoft 365 environment with many inactive groups, some of them are inactive for a long time – e.g. 1 or 2 years. We want to implement groups expiration policy, but we want to understand better the policy behavior.
Microsoft says: “The expiration period begins when the group is created, or on the date it was last renewed” and “When you change the expiration policy, the service recalculates the expiration date for each group. It always starts counting from the date when the group was created, and then applies the new expiration policy.” So in case we implement the policy first time, we know that Renewal Date for all groups is just a Group Creation Date.
Q: What will happen if I activate the policy – will the policy start generating emails immediately for all groups? A: Yes. Once activated – policy starts detecting expired groups and sending notifications to groups owners. So if you have 3k expired groups with 6k owners in it – expect policy will generate 6k e-mail notifications.
Q: Which groups the policy will be triggered against? All or Inactive only? A: As per Microsoft, if at around 35 days before expiration it will be determined that group is actually active, the policy can renew the group automatically. But definition of this activity is not disclosed and might be not the same as group activity status 90 days based on MS Graph data you can see at CA. (I got notifications for groups that were not active recently but with Active status).
Q: In the case above – what would be the deadline? When the policy will delete the group? A: If the group expiration period is passed, but the policy was just activated – it does not delete the group immediately. Policy allows ~30-35 days for owners to renew the group. E.g. My test policy was activated May 3 and I got message for old group immediately, but it said that the group will be deleted on June 7.
Q: What if there are more than 10K emails – will it trigger Exchange throttling? A: Most likely emails not sent will be sent next day.
Q: Can I specify a distribution list in the policy as an “Email contact for groups with no owners”? A: Yes
Q: Can I specify an external e-mail address as an “Email contact for groups with no owners”? A: TBC
Q: Can admin ask user to renew or delete the group by some other custom solution (skipping the policy)? A: yes. Actually, “Renew group” button is just a link to the following Url: https://account.activedirectory.windowsazure.com/Group/RenewGroup?tenantId=<tenantId>&id=<groupId> where <tenantId> is tenant id and <groupId> is group Id. So basically anyone
Microsoft 365 Groups object model
Let me explain the policy behavior in m365 group object model terms.
There is a group property “RenewedDateTime”. When group is created – this property is set up to group created date/time (same as group CreatedDateTime property value). For the notification purposes the policy calculates “Expected Expiration DateTime” as RenewedDateTime plus “Group LifeTime” (number of days specified in policy, e.g. 180). First notification is triggered about 30 days before “Expected Expiration DateTime”, so the policy simply selects groups with RenewedDateTime property value less then current DateTime minus “Group LifeTime days” minus 30 days and sends notification starting from oldest group:
RenewedDateTime < Today - GroupLifeTime -30
When owner confirms group is still needed – RenewedDateTime is setup to current DateTime.
Q: When a user chose to “Renew group” – will it impact group activity? A: No. If a user did not visit group – but just clicked “Renew group” – it will not trigger group last activity date. E.g. inactive group will still be inactive.
Q: Is there an API to configure Microsoft 365 groups expiration policy programmatically? A: Yes, in MS Graph API it is called Group Lifecycle Policy: groupLifecyclePolicy.
Q: Can I programmatically renew the group (all groups) as an admin? A: Yes, consider using Microsoft Graph API or PowerShell 7 with PnP.PowerShell module. PnP Doc says Reset-PnPMicrosoft365GroupExpiration command “Renews the Microsoft 365 Group by extending its expiration with the number of days defined in the group expiration policy set on the Azure Active Directory” – but that does not seem accurate. This command sets up “RenewedDateTime” group property to the current datetime, not related to current policy settings (the policy might even not have been activated). Microsoft Graph API entry point: “POST /groups/{id}/renew“ Group.ReadWrite.All permissions required.
Q: Is it possible to setup “RenewedDateTime” property to another date/time of my choice (not the current date)? A: Apparently that is not possible. I could not find a way so far… It says Property 'renewedDateTime' is read-only and cannot be set.
Q: What permissions are required to renew the group with Reset-PnPMicrosoft365GroupExpiration? A: Group.ReadWrite.All – delegated or application
Q: What exactly is behind the automatic groups renewal? A: Actually, the is a separate process in parallel with groups expiration policy – and this process starts monitoring groups activity ~35 days before expiration and once activity is detected – the process resets group RenewedDateTime property. And the moment this date is reset – the group is excluded from policy.
Q: If I activate the policy not for all but for a selected groups only, will I still be able to renew other groups programmatically? A: Yes, as an admin – you can resets group RenewedDateTime property programmatically all alone. It does not matter – whether this policy is activated or not.
Screenshots
Notification e-mail that comes to group owners “as is” – web outlook view:
Notification e-mail that comes to group owners when content is unblocked (web outlook):
Notification e-mail that comes to group owners when pictures are loaded (desktop Outlook):
Notification e-mail that comes to group owners some key areas:
And I’d add that e-mail says how many members in this group (number of members, not including owners… i.e. if you are the only owner – it’ll be zero members). Correction: “Renew group” is not an actionable button – it is just a html button with a link.
Screenshot of the notification that comes to email specified in policy for the groups that does not have owners:
Outlook icon link sends user to group mailbox
SharePoint icon is the link to the associated SharePoint site
Clicking on Teams icon will transfer user to a default team channel chat page
the last one – group icon – is the link to a Microsoft’s groups management page where user can edit group, manage membership, renew group or delete group (see screenshot below):
Renew group button is visible if the expiration policy is activated:
Deleted group
When the not renewed group reaches expiration date – the policy deletes the group and group owners get an e-mail notification like this:
Email subject would be “Attention: <group name> was deleted. Restore it by Thursday, August 10, 2023” and in the body “
<group name> expired on Monday, July 10, 2023. It was deleted, along with all associated communications, files, calendar events, and tasks. You have 30 days from the expiration date to restore Test Priv team – ownerless groups policy and its content. You received this email because you’re an owner of the group”.
Owner can restore group within 30 days by simply clicking “Restore group” button. Then owner would be redirected to the “https://myaccount.microsoft.com/groups/action?groupId=<groupId>&action=Restore” and get a message “The group was successfully restored. It might take up to 24 hours before you can access all associated content. You can close this window now.”
