What is archiving SharePoint sites and why we’d need it?
Disclaimer: Archival that was announced at Microsoft Inspire 2023 (Introducing Microsoft 365 Backup and Microsoft 365 Archive) is not what we are discussing here. Though it might be considered as an option (as archived sites are still visible for admins but not visible for users), MS SharePoint Archive require additional licensing.
Scenario
You are in the process of cleaning-up large Microsoft 365 environment. You need to delete SharePoint sites (e.g. due to inactivity) but you cannot get confirmation from site owners (e.g. sites or groups are ownerless).
Deleted sites could be restored within 93 days of deletion if somebody rise a hand, but there is still a risk of possible loosing of important information, e.g. in case site is needed one a year. So you need to do clean-up but at the same time you want to decrease risks of loosing information.
So, you might want to do something with sites to engage users to volunteer to be site owner if they want to keep this site – e.g. prevents using the site the regular way and let users know that the site will be deleted etc., but do not actually delete site until it will be fully clear that site is not needed for anyone and can be safely deleted.
Let us call it “Staging” period. Depending on your org culture/rules/licensing etc. it might be 6 months, or 1 year or 5 years or more.
Approach options
generally, the options are (random order):
Set site to Read-Only mode
Set site to No-Access mode
Convert group from Public to Private
Remove access to the site (remove users from group)
Rename the site
Put a banner on a top bar with a message
Message to Teams or Yammer chat
Send e-mail to site members
Implement a Microsoft 365 ownerless groups policy
You might choose to set sites to read-only mode or even no-access mode. If so – users that are still need this site are loosing ability to work with site, but site is not deleted. Consider archiving as kind of scream-test phase before actual sites deletion.
If a user who needs this site would scream (rise a ticket to restore site) – you can trigger processes of a) finding new owner for the site b) excluding the site from clean-up process c) actual restoring site to normal mode
There are some options to setup a site to Read-Only or NoAccess mode. Here is the PowerShell command:
$siteurl = "https://contoso.sharepoint.com/teams/Team-SO-B"
Get-PnPTenantSite -Identity $siteurl | ft -a Url, LockState
Set-PnPTenantSite -Identity $siteurl -LockState ReadOnly
Get-PnPTenantSite -Identity $siteurl | ft -a Url, LockState
Set-PnPTenantSite -Identity $siteurl -LockState NoAccess
Get-PnPTenantSite -Identity $siteurl | ft -a Url, LockState
Set-PnPTenantSite -Identity $siteurl -LockState Unlock
The problem is what if the site is teams-connected or yammer-connected or just group-based. Here are some test results:
Services SharePoint site is connected to/Site State
Read-Only
NoAccess
Outlook only
N/A
N/A
SharePoint and Outlook
Outlook emails: OK Outlook files: read-only experience; No options to upload or create document; Documents are open in read-only mode. “The file couldn`t be saved to group” error message when trying to save file to a group library.
Outlook emails: OK Outlook files: empty screen; No error messages; Documents are not visible; “The file couldn`t be saved to group” error message when trying to save file to a group library.
SharePoint and Yammer
SharePoint, Teams and Outlook
Teams chats: OK Teams files: documents are open as read-only; No options to upload or create a new document SharePoint: “This site is read-only at the administrator’s request.”
Teams chats: OK Teams files: “403 FORBIDDEN” error message SharePoint: “ This site can’t be reached The webpage at https://contoso.sharepoint.com/teams/Team-STO-B might be temporarily down or it may have moved permanently to a new web address. ERR_INVALID_RESPONSE”
So you can see – behavior is inconsistent – users can still chat in Teams and Yammer and consume SharePoint content (in case the site in read-only) or get error messages or not very meaningful results (in case the site is in NoAccess mode) – so it would be not clear for users that the site is gong to be decommissioned.
If you are seeing “Ownerless group policy configuration failed” and “Please try again.” error message:
there might be some different reasons:
Microsoft said it is (was) a know problem – it happens sometimes (timeout?), if you configured the policy properly and have enough permissions. So just go back one step and try again – all should be good.
Sometimes “Ownerless group policy configuration failed. Failure in configuring ownerless groups policy” is a permissions issue SharePoint admin, Teams admin: cannot configure Ownerless Groups Policy Global admin: yes, can configure Ownerless Microsoft 365 Groups Policy. What is the minimum role required? According to a recent update of the Microsoft’s article – “A Global administrator can create a policy…”. In my experience – groups admin can also configure the policy
Note: Groups admin when configuring the policy can see warning message “You don’t have permissions to save changes”. No worries 🙂 => You will be able to save changes 🙂
Video tutorial on the policy configuration (at around 5:00 you can see this error message):
getOffice365GroupsActivityDetail – details about Microsoft 365 groups and activity
getSharePointSiteUsageDetail – details about SharePoint sites and usage
getTeamsTeamActivityDetail – details about Microsoft Teams and activity by teams
Also we know, that Teams sites are group-based, and you can have private and shared channels under Teams – but these sites are not actually group-based and there are group-based SharePoint sites with no Teams behind.
And activities might be different – update document or just visit home page, provide permissions and update channel properties etc.
So the question is what kind of activity at what level is recorded at which report?
Sometimes, mostly during PoC or testing policies like retention policy or lifecycle policy you would need some documents created and updated weeks, months or even years ago.
But if you create or upload a document in SharePoint library – it will be just a regular new document. So, how to get old documents in the new environment?
I see two options:
Sync with OneDrive If you sync a library with your local folder (done Microsoft by OneDrive desktop app) and put some old document in your synced folder – the doc will be synchronized back to SharePoint library with Created and Modified properties preserved.
Make the document older with PowerShell With “Set-PnPListItem” PowerShell command you can update not only such properties like Title, but also “Created By”, “Modified By” and even date and time document was created and modified via “Created” and “Modified”. Optionally you can play with document history with “-UpdateType” parameter. UpdateType possible values are:
Update: Sets field values and creates a new version if versioning is enabled for the list
SystemUpdate: Sets field values and does not create a new version. Any events on the list will trigger.
UpdateOverwriteVersion: Sets field values and does not create a new version. No events on the list will trigger
As SharePoint or Teams admin you manage Microsoft 365 groups (create, update, delete, manage membership etc.) having your admin role activated. I prefer PowerShell 7 and Microsoft.Graph PowerShell module, and I need an Azure registered app with “Group.ReadWrite.All” Microsoft Graph API delegated permission.
Some findings:
If a user was not a group member or group owner – and the user is added to the group members – this user will get notification “You’ve joined the <Group Name> group” via e-mail that comes from a group e-mail address.
When a user is added to the group owners (or elevated to group owner if user was a group member) – user does not get notification.
When a user was a group owner and now you are adding this user to the group members – user does not get notification.
All the actions are logged into Microsoft 365 audit log under your personal Id.
Script samples:
# This script is just a sample to demonstrate basic technique on getting, updating groups membership and deletion m365 groups with PowerShell and MS Graph
#
# please do not run this script as is, but update it based on your needs
# authentication with personal Id
# app must have as minimum "Group.ReadWrite.All" Microsoft Graph API delegated permission
# user must have SharePoint admin (or Teams admin) roles activated
Connect-MgGraph -ClientId $clientid -TenantId $tenantId
Get-MgContext | Select-Object Scopes -ExpandProperty Scopes
# sample data
$groups = @()
$groups += [PSCustomObject]@{GroupId = '443d22ae-683a-4fe4-8875-7bd78227a026' }
$groups += [PSCustomObject]@{GroupId = 'e5805388-c18c-48c0-b42d-6223cf8f3d82' }
# Get Groups
foreach ($group in $groups) {
Get-MgGroup -GroupId $group.GroupId
}
# add members to the group
$groupId = '443d22ae-683a-4fe4-8875-7bd78227a026'
$userId = 'df74e0d3-d78c-495b-b47a-549437d93cf7' # Adele
New-MgGroupMember -GroupId $groupId -DirectoryObjectId $userId
# add Owner to the group
$groupId = '443d22ae-683a-4fe4-8875-7bd78227a026'
$userId = 'eacd52fb-5ae0-45ec-9d17-5ded9a0b9756' # Megan
New-MgGroupOwner -GroupId $groupId -DirectoryObjectId $userId
# Delete group
$groupId = '443d22ae-683a-4fe4-8875-7bd78227a026'
Remove-MgGroup -GroupId $groupId
It is known that a single Microsoft Exchange account is not sending more than 10k emails per day.
It is also know that once activated – Microsoft 365 groups ownerless policy will be sending notifications for all groups in scope to specified number of group members within 24 hours.
The question is: what if there are more than 10,000 notifications to send (e.g. 4,000 ownerless groups and the policy is configured to send notification to 3 members per group – that gives us 12,000 notifications to send)? Would the policy send 10k notifications and the rest 2k notifications the next day?
I’m conducting an experiment. I created 10k groups in my lab tenant with one owner and 3 random members. Then I configured a policy that is sending notification to a 3 most active members (in this case – random members). And then I made all these groups ownerless by deleting the single owner Id from Azure AD (Microsoft Entra).
Here is what I got from users perspective:
user
user groups number
got messages day 1
got messages day 2
got messages total
1 Roger
5012
1374
2 Dick
5039
1349
3 Bob
5108
3412
4 Bapu
4908
1376
5 Stas
4996
1437
6 David
4959
1377
total
10325
Here is what audit log says:
Events “OwnerlessGroupNotified” day 1: 4949 Events “OwnerlessGroupNotified” day 2: 95 Events “OwnerlessGroupNotified” total: 5044 Each event details says 3 members were notified.
It seems like groups are selected by policy in random order.
Massive E-mails sending was started 43 minutes after midnight UTC
“OwnerlessGroupNotified” were logged at the rate of 1925 events during 1-st hour, 2029 events during 2-nd hour, 785 events during 3-rd hour, 176 events during 4-th hour, 26 events during 5-24 th hour, 95 events during next 25-48 hours so max rate was one event every 3 seconds in the beginning (or 1 e-mail per second) …
You are trying to register an application at SharePoint site with appregnew.aspx page and you are getting an error or notification message “Your SharePoint tenant admin doesn’t allow site collection admins to create an Azure Access Control (ACS) principal“.
Or you are trying to provide ACS-based permissions for an application to SharePoint site with appinv.aspx page and you are getting “Your SharePoint tenant admin doesn’t allow site collection admins to update app permissions. Please contact your SharePoint administrator.”
You can still view and even delete your apps permissions from /_layouts/15/appprincipals.aspx page:
Reason
This is due to a recent update to Microsoft 365 (tenant governance security measures enhancement MC660075) implemented by Microsoft in Aug/Sep 2023. According to the update, only tenant administrators can create or update ACS service principal by default.
The root cause for this is that the Microsoft is pushing developers out of that legacy ACS-based SharePoint Apps-only service principals towards Azure-registered applications with Sites.Selected API permissions as they are more secure etc. In Nov 2023 Microsoft announcement retirement of ACS principals.
Key differences between ASC and Sites.Selected are:
ACS-based SharePoint app/permissions
Apps registered in Azure with Sites.Selected API permissions
Support authentication with client secret only, Secrets are valid for 1 year exactly.
Support authentication with client secret and/or certificate, custom expiration time.
Support granular access to SharePoint site, e.g. to site collection or web (subsite) or a specific list or library.
Support only access to entire site collection (but Microsoft says granular access is coming) Granular permissions are available – ‘Lists.SelectedOperations.Selected’, ‘ListItems.SelectedOperations.Selected’ and ‘Files.SelectedOperations.Selected’ permissions allows to provide application access to list, library or list item or particular documents
Support only classic SharePoint REST API and CSOM
Support both – classic SharePoint REST API and CSOM and Microsoft Graph API
App id (client id) is created via appregnew.aspx at a specific SharePoint site by site collection administrator (disabled in Sep 2023).
App id (client id) is created in Azure portal (Entra Id), API Sites.Selected permissions are configured via Azure portal (Entra Id) and require tenant admin consent.
Permissions for the app to a site are provided at the site by site collection administrator via appinv.aspx page (disabled in Sep 2023).
Permissions for the App to to a specific SharePoint site are provided via Graph API by SharePoint admin with PowerShell script.
Solution #1 – switch to Sites.Selected
Register an application in Azure (via Entra Id – Azure portal GUI, PowerShell script or your company’s specific helpdesk/servicedesk request)
Update the app so both – MS Graph API Sites.Selected and SharePoint Sites.Selected permissions are configured, then
API permissions must be consented – so you’d seek/request your tenant admin consent
Obtain and upload client certificate (recommended) or generate client secret (at this moment you should be able to authenticate to tenant)
Request access for the app to a specific SharePoint site – your SharePoint service admin should be able to do that (at this moment you should be able to authorize to your SharePoint site).
Validate your app has access to the target SharePoint site with PowerShell (check validation scripts below under References).
Use recommended by Microsoft technique, code samples are available for the most popular languages/platforms – Python, C#, Java etc. (check below under References).
Secure your certificate and/or secret. It is not a good idea to use hard-coded secrets, so consider using special services/storages for secrets (aka Vaults)
If you are hosting your application in Azure – consider using managed identity.
There are 3-rd party (and Microsoft) apps developed using classic approach (examples – Azure data Factory, Alteryx). So in some cases Sites.Selected permissions are not enough to get access to SharePoint.
Solution #2 – admin to register/update an ACS app
This option is acceptable if you have existing application that require ACS-based access. This option is not recommended for new development, as ACS is deprecated and scheduled for retirement.
Microsoft (MC660075 in Message Center): “site collection admin will be unable to register app or update app permissions through above pages unless authorized explicitly by the SharePoint tenant admin” and “With this update site owners will not be able to register/update apps unless the tenant admin explicitly allows it.”
That is incorrect. Site collection admin cannot register app (appregnew) or provide permissions to the app (appinv) anymore. Tenant admin does not authorize site collection admins. Instead tenant (or SharePoint) admin can register an app or provide permissions to the app at a specific site (not changing the entire default behavior back…). But there was no such option (!) in the middle of October 2023, when this feature was enabled at all tenants. Even having a SharePoint admin or tenant admin permissions – if you tried to register an app with AppRegNew.aspx – you got the same error message “Your SharePoint tenant admin doesn’t allow site collection admins to…”.
Later (Checked today – Nov 6, 2023) it seems like Microsoft has implemented it! E.g. now SharePoint or tenant admin is able to register an app with AppRegNew.aspx or update it with AppInv.aspx at any specific site collection. SharePoint or tenant admin must also be among this site collection admins.
It is ok (and I’d say the preferred way) to provide ACS permissions to the app registered in Azure, so do not register apps in SharePoint anymore (do not use AppRegNew.aspx).
Bottom line: if ACS-based permissions are required for app here you go:
register application in Azure (Entra id)
activate your SharePoint service/tenant admin role
ensure you are also target site collection administrator
navigate to the site appinv.aspx page – e.g. “https://yourtenant.sharepoint.com/sites/yoursite/_layouts/15/appinv.aspx” and us Azure registered app (client) Id. E.g. for lookup provide
Azure registered app (client) Id for – click lookup
localhost as app domain
https://localhost as redirect url
Permission Request XML – depending on permissions you need, e.g. for full app access to entire site collection:
It is possible to switch back this new default behavior that prevents site collection admin to register/update apps at SharePoint. This is done with PowerShell command
To run this command – you’d need to be a SharePoint service or tenant admin.
But this will be a step back on your journey in improving m365 tenant safety, as after that you’ll have a self-registered service principals out of control again. So devs will be using it not being aware of ACS retirement and when Microsoft switch off ACS – it will be a disaster, as all app will stop working. That is why Microsoft implemented this feature to soft-disable ACS and allowed us 2 years to redesign or apps and migrate from ACS to Entra Id apps with Sites.Selected. So this solution is not recommended.
In case you really need an exception to provide an ACS-based service principal – there is Solution number 2.
Full text of Microsoft’s MC660075 message
(Updated) SharePoint admin control for App registration / update
Tag MAJOR UPDATE ADMIN IMPACT FEATURE UPDATE
Message Summary Updated August 30, 2023: We have updated the content below for clarity. Thank you for your patience.
This is an enhancement to the security measures for administrative governance that modifies the default procedures for SharePoint app registration via AppRegNew.aspx page and permission updates via AppInv.aspx page. Following the implementation of this change, site collection admin will be unable to register app or update app permissions through above pages unless authorized explicitly by the SharePoint tenant admin.
Upon attempting to register an application on AppRegnew.aspx page, a notification will be displayed stating “Your SharePoint tenant admin doesn’t allow site collection admins to create an Azure Access Control (ACS) principal. Please contact your SharePoint tenant administrator.”
Similarly, upon attempting to update app permissions on AppInv.aspx page, a notification will be displayed stating “Your SharePoint tenant admin doesn’t allow site collection admins to update app permissions. Please contact your SharePoint tenant administrator.”
Kindly note that app registration and permission update via Microsoft Azure portal are not impacted by this change.
When this will happen:
The rollout process is scheduled to commence in late August and is expected to conclude in mid-September.
How this will affect your organization:
With this update site owners will not be able to register/update apps unless the tenant admin explicitly allows it.
To modify the default behavior, the tenant administrator must execute the following shell command to explicitly establish the flag as TRUE, thereby superseding the default value of FALSE. The service principal can only be created or updated by the tenant administrator by default. However, when the flag is set to TRUE, both the SharePoint tenant admin and site collection admin will be able to create or update the service principal through SharePoint.
The shell command is: Set-SPOTenant -SiteOwnerManageLegacyServicePrincipalEnabled $true
Note: The property ‘SiteOwnerManageLegacyServicePrincipalEnabled’ becomes visible in tenant settings after SharePoint Online Management shell is updated to 16.0.23710.12000 or a later version. But before this rollout, the value will always be TRUE even explicitly set to FALSE. It will only automatically be switched to FALSE as the default value after the rollout is launched.
What you need to do to prepare:
No proactive measures are required to prepare for this change. Nevertheless, it is advisable to inform your users of this modification and update any relevant documentation as necessary.
