Microsoft announced retirement for SharePoint Add-Ins and ASC-based app permissions (SharePoint app-only principals). Let me summarize here:
SharePoint Add-Ins, ASC-based app permissions and SharePoint app-only principals
Timeline of retirement and future plans
How to detect if there are legacy service principals used in the environment
Migration guidance
Timeline
Known key retirement milestone dates:
Mar 2024 – Microsoft will stop accepting new Add-In submission to app store
Jul 2024 – Microsoft will stop ability for customers to acquire add-ins from App Store (*)
using ACS and Add-ins will be blocked for new tenants since Nov 2024 (**)
using ACS and Add-ins will be blocked for all existing tenants since Apr 2, 2026
(*) SPFx based solutions will continue to be available, Installation from a private app catalog stays possible (**) regardless of their origin (public marketplace, private tenant catalog)
So timeline is generous, and we have plenty of time to
What you need to do to prepare
Detect if there are any legacy service principals used in the environment: Use the Microsoft 365 Assessment tool (by PnP) to scan your tenants for SharePoint Add-In usage.
After SharePoint Add-Ins are disabled, users will not be able to add SharePoint Add-Ins to their sites, and admins cannot add new SharePoint Add-Ins to the tenant and site collection app catalogs. SharePoint Add-Ins already added to sites will stay available and can still be used by the site’s users.
SharePoint Add-Ins will not be available from the public marketplace
After July 1, 2024, users browsing the public marketplace (AppSource) will see SharePoint Add-In, but if they select Get it now, a message will explain that SharePoint Add-Ins are retired and cannot be added. If you still require a specific SharePoint Add-In, contact the Add-In creator to understand the possible options.
Below is the my guide on how to connect Azure Data Factory to SharePoint and how to deal with connection error code 23201 “Failed to get metadata of odata service, please check if service url and credential is correct and your application has permission to the resource“
Scenario
You are configuring Azure Data Factory pipeline. You want to connect to SharePoint List as a data source.
Update (Nov 2024): Azure Data Factory V2 supports connection to SharePoint using certificate. So we do not need to provide legacy ACS permissions to the app registration. Instead we’d provide just Sites.Selected API permissions and use certificate to authenticate ADF to to Microsoft 365.
Below is legacy (classic) approach connecting ADF to SharePoint
To establish connection to SharePoint site you need to provide Site Url, tenant Id, service principal Id and service principal key:
Service principal
Service principal here could be
SharePoint app-only service principal registered at SharePoint site via appregnew.aspx
Azure registered app (app registered in EntraId)
In both cases you get “service principal” – which is App Id or Client Id and “service principal key” which is app secret (client secret).
Note: in Sep 2023 Microsoft implemented update to all Microsoft 365 tenants. According to the update, by default only tenant administrators can create or update ACS service principal by default. If site collection admin starting from Oct 2023 can register SharePoint app-only spn via appregnew.aspx or provide ACS-based permissions via appinv.aspx – that means tenant admins switched this back.
So, if registering a new SharePoint app-only service principal is still available for your tenant – you can get service principal Id and key from SharePoint via appregnew and/or provide ACS-based permissions via appinv – and there should be no problem connecting to SPO list from ADF (June 2024), but please review special note below.
If a site collection administrator or owner tries to register app in SharePoint with appregnew.aspx or provide permissions to the app with appinv.aspx – and he/she gets: Your SharePoint tenant admin doesn’t allow site collection admins to create (update) app permissions. Please contact your SharePoint administrator:
that means registering service principal in SharePoint is disabled. In this case Microsoft recommend using Azure application registration – with MS Graph API Sites.Selected and SharePoint Sites.Selected API permissions configured, consented by tenant admin and with access to specific SharePoint site provided by SharePoint admins (refer to this article for more details).
Issue
If you obtained service principal Id and key as Azure Registered App – connection to SharePoint site from Azure Data Factory does not always works. When you configure connection and click test – you might get an error:
Failed to get metadata of odata service, please check if service url and credential is correct and your application has permission to the resource. Expected status code: 200, actual status code: BadRequest, response is :
...
<title>Request Error</title>
<body>...
<p class="heading1">Request Error</p>
<p>The server encountered an error processing the request. See server logs for more details.</p>
...</body>
Reason
So the issue above is a combination of two controversial circumstances:
Microsoft discourages using SharePoint app-only service principals and disabled ability for site owners to register SharePoint app-only service principals and provide ACS-based permissions in favor of Azure Registerd Apps with Sites.Selected based permissions.
Azure Data Factory still require ACS-based permissions (Upd: June 2024 – still true)
Solution
If Microsoft disabled ability for site owners to to provide ACS-based permissions for the app – that does not mean it is fully disabled. It turns out – SharePoint admins are still able to register SharePoint app-only principals and provide ACS-based permissions.
The recommended steps are:
register Application in Azure (in EntraId, not in SharePoint) to get App (client) Id this could be done by user from Azure App Registrations (or, if this ability is disabled by tenant admin – there must be a way for users to request an application registered in Azure)
provide to this App Id ACS-based permissions at the target SharePoint site via appinv.aspx – this is done by a person who got at the same time two roles
SharePoint admin role enabled and
The specific site collection administrator permissions so if your role is a regular user or developer (not an admin) – you’d request this service from your admins
provide Sites.Selected permissions for the App to the target Site again, this is something you’d need to request from your admins – tenant admin should be able to provide admin consent to SharePoint and Graph API Sites.Selected permissions for your app and SharePoint admin should be able to provide actual permissions for the app to the site.
Technical Details
(as per June 2024 – still true) Actually, the only you need is to provide any ACS-based access for the application. Even to another site, web or list. You can also remove this just provided ACS-based access. It seems like the moment you click “Trust” when you provide access via AppInv.aspx – something is triggered in Microsoft Identity Management token issuing mechanics so Azure Data factory connection starts working (assuming Sites.Selected access was provides).
Surely connection will work if you provide only ACS-based permissions (with no Sites.Selected permissions), but this is what we all want to avoid by any means.
More fun! Connection to entire site will start working even if you provide SharePoint app-only (ACS-based) permissions to some specific list. Though later, when you try to ingest data – you will be able to ingest only this list data.
Environment this is tested:
Powershell module used to enable/disable 16.0.24120.12000 Microsoft.Online.SharePoint.PowerShell
Microsoft implemented Sites.Selected API permissions for Azure registered apps in 2021-2022 as a preferred way to access specific SharePoint site with application credentials. Microsoft recommend using Azure registered apps instead of SharePoint App-Only service principals and “softly” push developers toward Azure registered apps. Microsoft recently (Aug-Sep 2023) implemented an update and pushed it to all existing Microsoft 365 tenants – so that ability for site admins to register service principals at sites is turned off by default.
So starting Aug-Sep 2023 site owners/admins cannot register and provide ACS-based permissions for apps to their SharePoint sites.
In 2024 Microsoft announced EOL for SharePoint app-only spns and ACS-based permissions.
Special note
This article is written in 2023 with the sole purpose to help you resolve the issue. And it all is still true (validated in June 2024). But! I assume that sooner or later (before April 2026) Microsoft will address it’s own issue and update Azure Data Factory so ADF will be accepting permissions provided via Sites.Selected only. That is why – at the moment – I strictly recommend:
Use Azure Registered App (not a SharePoint app-only spn)
Get both types of permissions for this app:
modern – Sites.Selected SharePoint and Graph API permissions
old/classic – ACS-based permissions
if so – your data pipeline should continue working smoothly when MS implement modern authentication in ADF.
What if you need to bulk update Microsoft 365 groups membership e.g. to add a group owner or member for tens of thousands m365 groups? Iterating through groups one-by-one is unproductive and could take days. Can we do it faster? Here is what I found.
In my case, it was Microsoft 365 ownerless groups policy implementation for large tenant… Skipping details – I needed to update ownership for 10,000 Microsoft 365 groups and I was looking for a best/fastest possible option maybe some kind of bulk update or with multiple threads. And I figured out that the fastest way is to use PnP.PowerShell that calls Microsoft Graph API but run it against list of groups with PowerShell parallel trick. Here is the sample PowerShell code:
You administer Microsoft 365 SharePoint Online. Part of your daily activities is providing Microsoft Graph and SharePoint Sites.Selected API permissions to other users (developers).
In Aug/Sep 2023 Microsoft pushed an update that prevents site collection admins to create or update an Azure Access Control (ACS) principal (that was the way most of developers used to get Client Id and Client secret to access SharePoint site). So your users are probably getting something like Your SharePoint tenant admin doesn’t allow site collection admins to create or update an Azure Access Control (ACS) principal message attempting to create or update SharePoint App-only principal at AppRegNew.aspx or AppInv.aspx pages. Here are more details on the issue.
Microsoft and MVPs shared some technique how to provide Sites.Selected API permissions, but dealing with scripts manually, elevating individual permissions every time you need to run the script – it all takes time and not very efficient. More and more devs are reaching you on the app. So you want to automate this process.
SharePoint list as a frontend here you can accept intake requests, organize approval workflow and display automation results
Azure Function App as a backend here will be your PowerShell script hosted that runs on scheduled basis and takes care of actual permissions provisioning
Solution details
High-level, getting application permissions to some specific SharePoint site is a two-step process:
get application registration in Azure and properly configure it
get permissions for this application to a specific SharePoint site
For the first step – check this and this articles. I’ll focus on the second step below.
You can provide Sites.Selected permissions for the app to a site with
I will be using second one one. Also PnP.PowerShell will be used to get access to SharePoint intake site and read/update requests from SharePoint list and so on.
Azure App Registration
I registered an admin Application in Azure – “SharePoint Automation App”, added Graph Sites.FullControl.All and SharePoint Sites.FullControl.All permissions, then added Microsoft Graph Directory.Read.All permissions and got tenant admin consent:
I generated a self-signed certificate and added it to the app:
This app will be used to call provide permissions, and to connect to the SharePoint front-end.
Users will register their applications in Azure, add Graph Sites.Selected and SharePoint Sites.Selected permissions, got tenant admin consent, then request permissions to the specific site by creating an intake request – new list item.
Front-End SharePoint Site
I created a SharePoint site for automation. This site will play a front-end role for users. I created a list “Sites.Selected” and updated list columns so I have the following fields:
Target Site Url
Application Id
Permissions (read/write)
Automation Output
In real-world (Prod) – You can (should) also implement approval workflow as you’d provide permissions for the application to the site only with this site owner approval. The PowerShell code behind should also validate site owner’s consent with app access to site. But for the sake of simplicity I’ll skip this in my demo.
Azure Function App
I created an Azure Function App with the following parameters: – Runtime stack: PowerShell Core – Version: 7.2. – OS: Windows – Hosting plan: Consumption
And then PowerShell timer-triggered function in Visual Studio Code.
Function requirements.psd1 (it takes a few hours for Azure to install modules; while modules are installing – you might see “[Warning] The first managed dependency download is in progress, function execution will continue when it’s done. Depending on the content of requirements.psd1, this can take a few minutes. Subsequent function executions will not block and updates will be performed in the background.”):
@{
'Az' = '10.*'
'PnP.PowerShell' = '2.*'
}
Azure Az module to access other Azure resources. PnP.PowerShell module will be used to access SharePoint.
I will keep my admin Azure registered app in a key vault, so need somehow to let the key vault know that this specific app can access this specific credentials. So I enabled system assigned managed Identity for the Function App:
MS: “This resource is registered with Azure Active Directory. The managed identity can be configured to allow access to other resources…”. I’m going to use an object (principal) Id of this function to grant access to keyvault.
Azure key vault
Surely we do not hard-code app secrets. So we need a key vault o store app credentials.
I created a key vault under the same resource group in Azure and named it “SharePointAutomationDemo”. Then I added a roles assignment – “Key Vault Secret User” and “Key vault Reader” to the Function App via it’s managed identity:
I also assigned “Key Vault Administrator” role to the user (developer) who will add certificates/secrets to this key vault and develop Azure function code.
With the new ‘Lists.SelectedOperations.Selected’, ‘ListItems.SelectedOperations.Selected’ and ‘Files.SelectedOperations.Selected’ permissions it is new possible to provide application permissions at a specific list, library or list item levels or at a particular document level, so automation solution would be a little more complicated.
So far some findings I came up with after several Microsoft forms troubleshooting sessions… I’ll keep all the gotchas here as “how to” guide for myself. I’d be glad if this also helps you troubleshoot your Microsoft forms.
Microsoft forms links
You know, a user can create a Microsoft form. Then user can share it. There are two kind of links –
to respond
to edit/view/export results
Link to respond is kind of : https://forms.office.com/Pages/ResponsePage.aspx?id=FHPcfQGf1UWwEnFmW7HFRMgvShgV5J1Phpi7J1M_UoVUOUI1TzNQUEdWOTAzVVdRUVYzVVg4MlhZNC4u or short one: https://forms.office.com/r/kDKaHWauj7
Link “to collaborate” -e.g. with the link a person can edit and view results – is created under … “Create or duplicate”, and could be for anyone, for all people in org, and for specific people in org
if the link looks like "https://forms.office.com/Pages/DesignPageV2.aspx?subpage=design&FormId=<FormId>" then it’s for specific people in org
if the link looks the same but also contains "&Token=e3cd16ccf8034a3e868c68747e1f9584" then it’s for anyone with work or school account or for anyone in the organization
The one with the “edit” link can edit the form (including questions, answers options, and form visibility , view responses, delete responses, create a “summary link”, create a duplicate link, and export responses to excel (“Open in Excel” button). But cannot change collaboration options.
When user complete the form (after submit button), there is an option “Save my response” – if so – user will see this for with only one (his/her) response under forms app.
Collaborator is not seeing the form he/she has access to until follow the link.
Move the form to a group
Form owner can move the form to a group (and this is strictly recommended for all production forms). If so:
people who are group members (not only owners) will see this form under forms app – under specific group
form id will be changed, so the long “respond” link will be different. Though the short link will be the same. All links should continue to work: Old and New long and short respond links. Group-owned form id seemed to me be little longer – 88 characters vs 80 chars for individual-owned forms.
The trick Tomasz Szypula @toszypul shared here (I’m also citing the trick below) on how to find form owner having just a link works like a charm! Even for deleted owner`s IDs.
But let me share some more here. If the form is owned by group – the link will be similar, but with “/group/<groupId>” instead of “/user/<UserId>” . E.g. here: https://forms.office.com/formapi/api/7ddc7314-9f01-45d5-b012-71665bb1c544/groups/65714e55-87f4-49c3-b790-fc75d7349c8a/light/...
you can see “65714e55-87f4-49c3-b790-fc75d7349c8a” which is group Id. So you can use the same trick to figure out what group owns a form.
Deleting user who owns forms
What if the original owner of a form is no longer with a company? How can I transfer ownership of the form?
If the employee account was deleted or disabled, the global administrator or office application administrator of the organization who have a valid Forms license can transfer for ownership within 30 days of when an account was disabled/deleted. See details. Note that all forms user owned will be transferred to an admin, then an admin can transfer forms to a group so new owners can have access to answers etc.
Deleting a group that owns forms
When a form is owned by group and the group is getting deleted… tbp…
Audit log events
You can get some ideas on the form from an audit log, including
is the form owned by group or by user
to whom the form was shared with collaborator link
Below are kinds of events related to Microsoft 365 forms:
ListForms – Listed forms – viewed forms home page with list of forms
ViewForm – Viewed Form –
ViewRuntimeForm – Viewed response page
ViewResponses- Viewed responses
CreateResponse – Created response
ExportForm – Exported form – “export to excel” – file saved to the local machine (form owner=user)
ConnectToExcelWorkbook – Connected To Excel Workbook – “export to excel” – file saved to the teams SharePoint site under Documents (form owner = group)
toszypul replied to Jason_B1025
Jan 03 2022 03:17 AM - edited Jan 03 2022 03:18 AM
@Jason_B1025 I was able to get the ID of the user with a bit of a hack. Here are sample steps:
-Access the form using this designer direct URL https://forms.office.com/Pages/DesignPage.aspx?origin=shell#FormId=<YourFormID>
-Inspect the network traces. You will find a request similar to this
https://forms.office.com/formapi/api/72f988bf-86f1-41af-91ab-2d7cd011db47/users/e5351c57-d147-418e-89ab-3a3d50c235b6/light/forms('v4j5cvGGr0GRqy180BHbR1ccNeVH0Y5Bias6PVDCNbZUOUg4TkZJUEswSVQ1ODhNNkpHVVlMMldPTi4u')?$select=id,...
-The ID in bold is the AAD ID of the user
-Use Graph Explorer - Microsoft Graph to run this request to retrieve the username and email address of the owner https://graph.microsoft.com/v1.0/users/<UserID>
My 3×5 cents to this clever trick:
Not only “collaborator” link helps, but also “respond to”
If the form is owned by the group – the link would be similar but with “…/group/group_id/…” instead of “…/user/user_id/…”
If you have a SSO in your org and cannot find this call under network – try different browser or incognito mode or logging out before the call – as what you need appears at early stages – even before authentication
How do I know – is it a person-owned or group-owned form
Let say you got a claim that “we were able to work with the form, and now it is gone”, and the only you have is the “collaborators” link to the form – so you can edit form, view responses etc. but nobody knows who created that form… So how to determine who owns the form – person or group and what person/group.
The form is owned by a person if
form id is 80 characters length
on “Export to Excel” button – it saves/downloads excel file to the file system
audit log contains ExportForm (Exported form) event – as clicking “Export to Excel” button generates ExportForm (Exported form) event
As SharePoint or Teams admin you manage Microsoft 365 groups (create, update, delete, manage membership etc.) having your admin role activated. I prefer PowerShell 7 and Microsoft.Graph PowerShell module, and I need an Azure registered app with “Group.ReadWrite.All” Microsoft Graph API delegated permission.
Some findings:
If a user was not a group member or group owner – and the user is added to the group members – this user will get notification “You’ve joined the <Group Name> group” via e-mail that comes from a group e-mail address.
When a user is added to the group owners (or elevated to group owner if user was a group member) – user does not get notification.
When a user was a group owner and now you are adding this user to the group members – user does not get notification.
All the actions are logged into Microsoft 365 audit log under your personal Id.
Script samples:
# This script is just a sample to demonstrate basic technique on getting, updating groups membership and deletion m365 groups with PowerShell and MS Graph
#
# please do not run this script as is, but update it based on your needs
# authentication with personal Id
# app must have as minimum "Group.ReadWrite.All" Microsoft Graph API delegated permission
# user must have SharePoint admin (or Teams admin) roles activated
Connect-MgGraph -ClientId $clientid -TenantId $tenantId
Get-MgContext | Select-Object Scopes -ExpandProperty Scopes
# sample data
$groups = @()
$groups += [PSCustomObject]@{GroupId = '443d22ae-683a-4fe4-8875-7bd78227a026' }
$groups += [PSCustomObject]@{GroupId = 'e5805388-c18c-48c0-b42d-6223cf8f3d82' }
# Get Groups
foreach ($group in $groups) {
Get-MgGroup -GroupId $group.GroupId
}
# add members to the group
$groupId = '443d22ae-683a-4fe4-8875-7bd78227a026'
$userId = 'df74e0d3-d78c-495b-b47a-549437d93cf7' # Adele
New-MgGroupMember -GroupId $groupId -DirectoryObjectId $userId
# add Owner to the group
$groupId = '443d22ae-683a-4fe4-8875-7bd78227a026'
$userId = 'eacd52fb-5ae0-45ec-9d17-5ded9a0b9756' # Megan
New-MgGroupOwner -GroupId $groupId -DirectoryObjectId $userId
# Delete group
$groupId = '443d22ae-683a-4fe4-8875-7bd78227a026'
Remove-MgGroup -GroupId $groupId
It is known that a single Microsoft Exchange account is not sending more than 10k emails per day.
It is also know that once activated – Microsoft 365 groups ownerless policy will be sending notifications for all groups in scope to specified number of group members within 24 hours.
The question is: what if there are more than 10,000 notifications to send (e.g. 4,000 ownerless groups and the policy is configured to send notification to 3 members per group – that gives us 12,000 notifications to send)? Would the policy send 10k notifications and the rest 2k notifications the next day?
I’m conducting an experiment. I created 10k groups in my lab tenant with one owner and 3 random members. Then I configured a policy that is sending notification to a 3 most active members (in this case – random members). And then I made all these groups ownerless by deleting the single owner Id from Azure AD (Microsoft Entra).
Here is what I got from users perspective:
user
user groups number
got messages day 1
got messages day 2
got messages total
1 Roger
5012
1374
2 Dick
5039
1349
3 Bob
5108
3412
4 Bapu
4908
1376
5 Stas
4996
1437
6 David
4959
1377
total
10325
Here is what audit log says:
Events “OwnerlessGroupNotified” day 1: 4949 Events “OwnerlessGroupNotified” day 2: 95 Events “OwnerlessGroupNotified” total: 5044 Each event details says 3 members were notified.
It seems like groups are selected by policy in random order.
Massive E-mails sending was started 43 minutes after midnight UTC
“OwnerlessGroupNotified” were logged at the rate of 1925 events during 1-st hour, 2029 events during 2-nd hour, 785 events during 3-rd hour, 176 events during 4-th hour, 26 events during 5-24 th hour, 95 events during next 25-48 hours so max rate was one event every 3 seconds in the beginning (or 1 e-mail per second) …
You are trying to register an application at SharePoint site with appregnew.aspx page and you are getting an error or notification message “Your SharePoint tenant admin doesn’t allow site collection admins to create an Azure Access Control (ACS) principal“.
Or you are trying to provide ACS-based permissions for an application to SharePoint site with appinv.aspx page and you are getting “Your SharePoint tenant admin doesn’t allow site collection admins to update app permissions. Please contact your SharePoint administrator.”
You can still view and even delete your apps permissions from /_layouts/15/appprincipals.aspx page:
Reason
This is due to a recent update to Microsoft 365 (tenant governance security measures enhancement MC660075) implemented by Microsoft in Aug/Sep 2023. According to the update, only tenant administrators can create or update ACS service principal by default.
The root cause for this is that the Microsoft is pushing developers out of that legacy ACS-based SharePoint Apps-only service principals towards Azure-registered applications with Sites.Selected API permissions as they are more secure etc. In Nov 2023 Microsoft announcement retirement of ACS principals.
Key differences between ASC and Sites.Selected are:
ACS-based SharePoint app/permissions
Apps registered in Azure with Sites.Selected API permissions
Support authentication with client secret only, Secrets are valid for 1 year exactly.
Support authentication with client secret and/or certificate, custom expiration time.
Support granular access to SharePoint site, e.g. to site collection or web (subsite) or a specific list or library.
Support only access to entire site collection (but Microsoft says granular access is coming) Granular permissions are available – ‘Lists.SelectedOperations.Selected’, ‘ListItems.SelectedOperations.Selected’ and ‘Files.SelectedOperations.Selected’ permissions allows to provide application access to list, library or list item or particular documents
Support only classic SharePoint REST API and CSOM
Support both – classic SharePoint REST API and CSOM and Microsoft Graph API
App id (client id) is created via appregnew.aspx at a specific SharePoint site by site collection administrator (disabled in Sep 2023).
App id (client id) is created in Azure portal (Entra Id), API Sites.Selected permissions are configured via Azure portal (Entra Id) and require tenant admin consent.
Permissions for the app to a site are provided at the site by site collection administrator via appinv.aspx page (disabled in Sep 2023).
Permissions for the App to to a specific SharePoint site are provided via Graph API by SharePoint admin with PowerShell script.
Solution #1 – switch to Sites.Selected
Register an application in Azure (via Entra Id – Azure portal GUI, PowerShell script or your company’s specific helpdesk/servicedesk request)
Update the app so both – MS Graph API Sites.Selected and SharePoint Sites.Selected permissions are configured, then
API permissions must be consented – so you’d seek/request your tenant admin consent
Obtain and upload client certificate (recommended) or generate client secret (at this moment you should be able to authenticate to tenant)
Request access for the app to a specific SharePoint site – your SharePoint service admin should be able to do that (at this moment you should be able to authorize to your SharePoint site).
Validate your app has access to the target SharePoint site with PowerShell (check validation scripts below under References).
Use recommended by Microsoft technique, code samples are available for the most popular languages/platforms – Python, C#, Java etc. (check below under References).
Secure your certificate and/or secret. It is not a good idea to use hard-coded secrets, so consider using special services/storages for secrets (aka Vaults)
If you are hosting your application in Azure – consider using managed identity.
There are 3-rd party (and Microsoft) apps developed using classic approach (examples – Azure data Factory, Alteryx). So in some cases Sites.Selected permissions are not enough to get access to SharePoint.
Solution #2 – admin to register/update an ACS app
This option is acceptable if you have existing application that require ACS-based access. This option is not recommended for new development, as ACS is deprecated and scheduled for retirement.
Microsoft (MC660075 in Message Center): “site collection admin will be unable to register app or update app permissions through above pages unless authorized explicitly by the SharePoint tenant admin” and “With this update site owners will not be able to register/update apps unless the tenant admin explicitly allows it.”
That is incorrect. Site collection admin cannot register app (appregnew) or provide permissions to the app (appinv) anymore. Tenant admin does not authorize site collection admins. Instead tenant (or SharePoint) admin can register an app or provide permissions to the app at a specific site (not changing the entire default behavior back…). But there was no such option (!) in the middle of October 2023, when this feature was enabled at all tenants. Even having a SharePoint admin or tenant admin permissions – if you tried to register an app with AppRegNew.aspx – you got the same error message “Your SharePoint tenant admin doesn’t allow site collection admins to…”.
Later (Checked today – Nov 6, 2023) it seems like Microsoft has implemented it! E.g. now SharePoint or tenant admin is able to register an app with AppRegNew.aspx or update it with AppInv.aspx at any specific site collection. SharePoint or tenant admin must also be among this site collection admins.
It is ok (and I’d say the preferred way) to provide ACS permissions to the app registered in Azure, so do not register apps in SharePoint anymore (do not use AppRegNew.aspx).
Bottom line: if ACS-based permissions are required for app here you go:
register application in Azure (Entra id)
activate your SharePoint service/tenant admin role
ensure you are also target site collection administrator
navigate to the site appinv.aspx page – e.g. “https://yourtenant.sharepoint.com/sites/yoursite/_layouts/15/appinv.aspx” and us Azure registered app (client) Id. E.g. for lookup provide
Azure registered app (client) Id for – click lookup
localhost as app domain
https://localhost as redirect url
Permission Request XML – depending on permissions you need, e.g. for full app access to entire site collection:
It is possible to switch back this new default behavior that prevents site collection admin to register/update apps at SharePoint. This is done with PowerShell command
To run this command – you’d need to be a SharePoint service or tenant admin.
But this will be a step back on your journey in improving m365 tenant safety, as after that you’ll have a self-registered service principals out of control again. So devs will be using it not being aware of ACS retirement and when Microsoft switch off ACS – it will be a disaster, as all app will stop working. That is why Microsoft implemented this feature to soft-disable ACS and allowed us 2 years to redesign or apps and migrate from ACS to Entra Id apps with Sites.Selected. So this solution is not recommended.
In case you really need an exception to provide an ACS-based service principal – there is Solution number 2.
Full text of Microsoft’s MC660075 message
(Updated) SharePoint admin control for App registration / update
Tag MAJOR UPDATE ADMIN IMPACT FEATURE UPDATE
Message Summary Updated August 30, 2023: We have updated the content below for clarity. Thank you for your patience.
This is an enhancement to the security measures for administrative governance that modifies the default procedures for SharePoint app registration via AppRegNew.aspx page and permission updates via AppInv.aspx page. Following the implementation of this change, site collection admin will be unable to register app or update app permissions through above pages unless authorized explicitly by the SharePoint tenant admin.
Upon attempting to register an application on AppRegnew.aspx page, a notification will be displayed stating “Your SharePoint tenant admin doesn’t allow site collection admins to create an Azure Access Control (ACS) principal. Please contact your SharePoint tenant administrator.”
Similarly, upon attempting to update app permissions on AppInv.aspx page, a notification will be displayed stating “Your SharePoint tenant admin doesn’t allow site collection admins to update app permissions. Please contact your SharePoint tenant administrator.”
Kindly note that app registration and permission update via Microsoft Azure portal are not impacted by this change.
When this will happen:
The rollout process is scheduled to commence in late August and is expected to conclude in mid-September.
How this will affect your organization:
With this update site owners will not be able to register/update apps unless the tenant admin explicitly allows it.
To modify the default behavior, the tenant administrator must execute the following shell command to explicitly establish the flag as TRUE, thereby superseding the default value of FALSE. The service principal can only be created or updated by the tenant administrator by default. However, when the flag is set to TRUE, both the SharePoint tenant admin and site collection admin will be able to create or update the service principal through SharePoint.
The shell command is: Set-SPOTenant -SiteOwnerManageLegacyServicePrincipalEnabled $true
Note: The property ‘SiteOwnerManageLegacyServicePrincipalEnabled’ becomes visible in tenant settings after SharePoint Online Management shell is updated to 16.0.23710.12000 or a later version. But before this rollout, the value will always be TRUE even explicitly set to FALSE. It will only automatically be switched to FALSE as the default value after the rollout is launched.
What you need to do to prepare:
No proactive measures are required to prepare for this change. Nevertheless, it is advisable to inform your users of this modification and update any relevant documentation as necessary.
Let say you need to run some code against Microsoft 365 on a scheduled basis. For this, you can use Azure Function App and timer-triggered Functions. From the code you’d call Microsoft Graph API and/or SharePoint API, so you’d need your Function somehow to get credentials on the fly from the KV and use it to call APIs. For this – you’d have a key vault where credentials will be be stored and retrieved by functions when needed. How does KV know if it’s ok to share secrets with the function app? Below I’ll share step-by-step how to create, configure and secure an Azure Function with system-assigned managed identity, and Azure Key Vault.
Below is a “classic” approach with client id, client secret and a key vault.
First, we’ll create a resource group, e.g. “Azure-Func-Secured”.
Next, we’ll create a function app:
We’ll avoid the default “Flex Consumption” service plan (as it is Linux-only and does not support dependencies) and select “Consumption” a hosting option for now:
Runtime stack we’ll be using is PowerShell Core 7.4, but you can choose your own (options are – Python, .Net (C#), Node.js, Java). Let us leave other configuration settings by default (e.g. Enable Public access: On) for now, we’ll fix (secure) it later.
Ok, the function app has been created. Notice that app service plan, application insights and storage account, as well as some other services were created automatically under the same resource group:
Now we can create one or more functions under this function app, also we’d need to create a key vault and a registered app. Let us start with function and VS Code would be our environment of choice.
Let us start Visual Studio Code in a separate folder. Ensure you have “Azure Functions”, “Azure Resources” and PowerShell extensions by Microsoft. You’d sign-in to Azure:
and create a function project:
You’d choose “Timer triggered” function (as we need the function running on schedule), the function name would be MyScheduledFunct01 and we’d leave the default schedule as “0 */5 * * * *” – so our function will be triggered every 5 minutes.
Let us deploy the function right away and see if it works. For this, we can use “deploy” icon under “WORKSPACE” panel:
or “Deploy to function app” option from the function app context pop-up menu under the Azure “RESOURCES” panel:
After successful deployment give it some time, then check function invocations and ensure function is triggered an running:
Next, we’d update “requirements.psd1” to include Azure Key Vault and PnP PowerShell modules as it takes some time for the function app to pull in and install dependencies. Requirements.psd1:
# This file enables modules to be automatically managed by the Functions service.
# See https://aka.ms/functionsmanageddependency for additional information.
#
@{
# For latest supported version, go to 'https://www.powershellgallery.com/packages/Az'. Uncomment the next line and replace the MAJOR_VERSION, e.g., 'Az' = '5.*'
# 'Az' = 'MAJOR_VERSION.*'
'Az.KeyVault' = '6.*'
'PnP.PowerShell' = '2.*'
}
And we’d update function itself to monitor if dependencies are installed, than we’d deploy the function again so time would work for us. MyScheduledFunction/run.ps1:
# Input bindings are passed in via param block.
param($Timer)
$currentUTCtime = (Get-Date).ToUniversalTime()
if ($Timer.IsPastDue) {
Write-Host "PowerShell timer is running late!"
}
Write-Host "PowerShell timer trigger function ran! TIME: $currentUTCtime"
#############################################################################################
Write-Host "Check modules installed:"
Import-Module PnP.PowerShell
Import-Module Az.KeyVault
Get-Module PnP.PowerShell
Get-Module Az.KeyVault
Write-Host "Check command available:"
Get-Command -Name Connect-PnPOnline -Module PnP.PowerShell
At first, we might see warning like “The first managed dependency download is in progress, function execution will continue when it’s done. Depending on the content of requirements.psd1, this can take a few minutes. Subsequent function executions will not block and updates will be performed in the background.”. So we’d just wait.
After some time the function will be able to use required modules. Here is the invocation output example:
App Registration To get unattended access to SharePoint (or Teams or Exchange etc.) as a service (daemon) application – we need credentials. It is done via “App Registration” under Entra Id (Azure AD) – here is the detailed step-by-step guide on registering apps in Azure.
Finally, we’d have a service principal with permissions we need:
We do not hard-code secrets, so let us create a key vault to keep secrets safe:
We’d leave all other configuration option by default, including “Azure role-based access control”, Allow access from: All Networks etc.
Here is the trick: even if I created this key vault and have an admin-level access, I’m not able to work with secrets values. So I need to provide additional access to myself. Being at the key vault, navigate to “Access control (IAM)” and add role assignment…
We’d select “Key Vault Secrets Officer”, next, select members… and select own name:
From this moment you should be able to CRUD secrets with values.
Now, let us generate a secret under App registration and copy secret value:
Then navigate to the Key Vault -> Object/Secrets -> Generate/Import – and save the secret value there:
Now you can get this secret… but the function cannot reach the secret… Here is the proof. Let us update the function code with:
and check the function output. You’d see something like (which is not very descriptive):
ERROR: Run Connect-AzAccount to login. Exception : Type : System.Management.Automation.PSInvalidOperationException ErrorRecord : Exception : Type : System.Management.Automation.ParentContainsErrorRecordException Message : Run Connect-AzAccount to login. HResult : -2146233087 CategoryInfo : InvalidOperation: (:) [], ParentContainsErrorRecordException FullyQualifiedErrorId : InvalidOperation TargetSite : Name : get_DefaultContext DeclaringType : [Microsoft.Azure.Commands.ResourceManager.Common.AzureRMCmdlet] MemberType : Method Module : Microsoft.Azure.PowerShell.Clients.ResourceManager.dll Message : Run Connect-AzAccount to login. Source : Microsoft.Azure.PowerShell.Clients.ResourceManager HResult : -2146233079 StackTrace : at Microsoft.Azure.Commands.ResourceManager.Common.AzureRMCmdlet.get_DefaultContext() at Microsoft.Azure.Commands.KeyVault.Models.KeyVaultCmdletBase.get_DataServiceClient() at Microsoft.Azure.Commands.KeyVault.GetAzureKeyVaultSecret.ExecuteCmdlet() at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.<>c__31.<ExecuteSynchronouslyOrAsJob>b__3_0(T c) at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.ExecuteSynchronouslyOrAsJob[T](T cmdlet, Action1 executor) at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.ExecuteSynchronouslyOrAsJob[T](T cmdlet) at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord() CategoryInfo : CloseError: (:) [Get-AzKeyVaultSecret], PSInvalidOperationException FullyQualifiedErrorId : Microsoft.Azure.Commands.KeyVault.GetAzureKeyVaultSecret InvocationInfo : MyCommand : Get-AzKeyVaultSecret ScriptLineNumber : 19 OffsetInLine : 13 HistoryId : 1 ScriptName : C:\home\site\wwwroot\myScheduledFunction\run.ps1 Line : $kvSecret = Get-AzKeyVaultSecret -VaultName $vaultName -Name $secretName -AsPlainText Statement : Get-AzKeyVaultSecret -VaultName $vaultName -Name $secretName -AsPlainText PositionMessage : At C:\home\site\wwwroot\myScheduledFunction\run.ps1:19 char:13 + $kvSecret = Get-AzKeyVaultSecret -VaultName $vaultName -Name $secretN … + ~~~~~~~~~~~~~ PSScriptRoot : C:\home\site\wwwroot\myScheduledFunction PSCommandPath : C:\home\site\wwwroot\myScheduledFunction\run.ps1 InvocationName : Get-AzKeyVaultSecret CommandOrigin : Internal ScriptStackTrace : at , C:\home\site\wwwroot\myScheduledFunction\run.ps1: line 19 PipelineIterationInfo : 0 1
So, how’d we allow key vault access for the function app? It’s as simple as that: – first, we’d need some identity assigned to function app – second, we’d provide access to the key vault to this identity
Managed Identity: assign an identity to the function. For this, you’d go to function Settings/Identity, and under System Assigned, you’d switch status to On and Save settings.
Then, you’d go to your key vault, Access Control (IAM) and add role assignment. But this time, you’d select the role “Key Vault Secrets User” (more about roles), and “Assign access to” Managed Identity, and select members – select your function app identity (notice, identity is assigned to function app, so all functions under the app will be able to use the identity):
Now, we’d check the next function invocation detail, and voila:
You can see that azure function was able on the fly to pull secret from the key vault, so now we should be able to use these credentials to access SharePoint.
As you know, having client id and client secret would allow you to call MS Graph API. But calling SharePoint API would require authentication with a certificate. “PnP.PowerShell” module use both APIs under the hood, so we’d need a certificate to connect to tenant and work with SharePoint using “PnP.PowerShell” module. Please refer to this article on how to run Connect-PnPOnline with Certificate stored in the Key Vault.
Securing Azure Function
Out-of-the-box Azure Function App is created in a not very secure manner. Here is whan we’d need to do to secure Azure Functions:
Finally, it’d be more secure not to have a key vault, but provide permissions to the managed identity of the Azure Function App.
This post is dedicated to one specific subject: implementing Microsoft 365 groups lifecycle (expiration) policy in large Microsoft 365 environments.
But this post is also a part of a bigger problem – dealing with ownerless resources in Large Microsoft 365 environments. Please refer to the umbrella post.
Scenario
You administer a large Microsoft 365 environment. Let say you have 100k users or more, 50K or more sites. Environment is not new, so after some years you have a lot of ownerless groups and sites (thousands probably), and a lot of inactive groups and sites (probably tens of thousands). You are getting more and more ownerless groups – hundreds each month. You are thinking of stopping bleeding and cleaning this up…
Implementing Microsoft 365 groups expiration policy
If you are thinking of activating in an existing environment – you would probably have a spike – all the old groups will be subject to policy. The ide is to avoid situation when a specific person – group owner will get dozens of email. It would be better if a person will receieve, let say one email per week.
Here is my 4 possible approaches to avoid this spike, distribute notifications evenly across the time and ease the pain:
By changing Group Lifetime
You would need to change the policy every, e.g. week or month, specifying group lifetime in days starting with maximum period. Consider – calculate number of days between the oldest group created an today, plus 35 days – it’ll be your first “group lifetime” – activate the policy with this number of days in “group lifetime” – and within a week you will get notifications on the oldest group/groups – after a week or two – change the “group lifetime” decreasing it by e.g. 30-60 days and reactivate the policy… and so on
You can easily calculate it all and choose your pace depending on how many groups you have to renew, how much time you need to clean-up. You got the idea.
Downside – in the email notification it will be said “otherwise the group will be deleted on …”, but you joggling with lifetime period – so these dates might confuse users
By renewing groups as admin
As an admin, you can use PowerShell “Reset-PnPMicrosoft365GroupExpiration” or graph API “POST /groups/{id}/renew“ to renew any group.
So depending on total number of groups, number of active/inactive groups, number of ownerless groups in your organization – you can come up with a strategy, using one or more of the following techniques:
renew all active and known and important groups (build list of groups to re-activate based on your own criteria) and then trigger the policy
build list of definitely inactive groups and renew all other groups
split groups into chunks and every day or week (depending on numbers) renew groups in a chunk… after that you can activate groups expiration policy, the policy will be triggered against small number of groups every day/week (with this trick you would avoid policy triggering against large number of groups and sending thousands of emails at one)
By sending customized e-mails to users
Another technique to avoid surge in your e-mail system (and most importantly – avoid sudden influx of support requests) – you can send emails to group owners with a link to renew a group. Surely you’d need to build your custom solution for that, but this does not seem like a complicated task. Some PowerShell scripting – and you are good.
“Renew group” link would look like: https://account.activedirectory.windowsazure.com/Group/RenewGroup?tenantId=<tenantId>&id=<groupId> where <tenantId> is tenant id and <groupId> is group Id. So with PowerShell you’d just pull groups in question, their owners and send email with the link dynamically built. Consider spreading this activity across the time to avoid spikes.
By sending users to the groups page
Probably the easies option to smooth the expiration policy implementation is to send group owners to a Microsoft “Groups I own” page – where they could renew or delete their groups. The page is:
Groups I Own (https://myaccount.microsoft.com/groups/groups-i-own)
The downside is – the policy should be in active state (otherwise owners will not see when the group is expiring and option to renew it). So consider a maximum lifetime period in policy, then you’d decrease this period).