Category Archives: Azure

Your SharePoint tenant admin doesn’t allow site collection admins…

Scenario

You are trying to register an application at SharePoint site with appregnew.aspx page and you are getting an error or notification message “Your SharePoint tenant admin doesn’t allow site collection admins to create an Azure Access Control (ACS) principal“.

Your SharePoint tenant admin doesn't allow site collection admins to create an Azure Access Control (ACS) principal. Please contact your SharePoint tenant administrator

Or you are trying to provide ACS-based permissions for an application to SharePoint site with appinv.aspx page and you are getting “Your SharePoint tenant admin doesn’t allow site collection admins to update app permissions. Please contact your SharePoint administrator.

Your SharePoint tenant admin doesn't allow site collection admins to update app permissions. Please contact your SharePoint tenant administrator

You can still view and even delete your apps permissions from /_layouts/15/appprincipals.aspx page:

Reason

This is due to a recent update to Microsoft 365 (tenant governance security measures enhancement MC660075) implemented by Microsoft in Aug/Sep 2023. According to the update, only tenant administrators can create or update ACS service principal by default.

The root cause for this is that the Microsoft is pushing developers out of that legacy ACS-based SharePoint Apps-only service principals towards Azure-registered applications with Sites.Selected API permissions as they are more secure etc.
In Nov 2023 Microsoft announcement retirement of ACS principals.

Key differences between ASC and Sites.Selected are:

ACS-based SharePoint app/permissionsApps registered in Azure with Sites.Selected API permissions
Support authentication with client secret only, Secrets are valid for 1 year exactly.Support authentication with client secret and/or certificate, custom expiration time.
Support granular access to SharePoint site, e.g. to site collection or web (subsite) or a specific list or library.Support only access to entire site collection (but Microsoft says granular access is coming)
Granular permissions are available – ‘Lists.SelectedOperations.Selected’, ‘ListItems.SelectedOperations.Selected’ and ‘Files.SelectedOperations.Selected’ permissions allows to provide application access to list, library or list item or particular documents
Support only classic SharePoint REST API and CSOMSupport both – classic SharePoint REST API and CSOM and Microsoft Graph API
App id (client id) is created via appregnew.aspx at a specific SharePoint site by site collection administrator (disabled in Sep 2023).App id (client id) is created in Azure portal (Entra Id), API Sites.Selected permissions are configured via Azure portal (Entra Id) and require tenant admin consent.
Permissions for the app to a site are provided at the site by site collection administrator via appinv.aspx page (disabled in Sep 2023).Permissions for the App to to a specific SharePoint site are provided via Graph API by SharePoint admin with PowerShell script.

Solution #1 – switch to Sites.Selected

  1. Register an application in Azure (via Entra Id – Azure portal GUI, PowerShell script or your company’s specific helpdesk/servicedesk request)
  2. Update the app so both – MS Graph API Sites.Selected and SharePoint Sites.Selected permissions are configured, then
  3. API permissions must be consented – so you’d seek/request your tenant admin consent
  4. Obtain and upload client certificate (recommended) or generate client secret
    (at this moment you should be able to authenticate to tenant)
  5. Request access for the app to a specific SharePoint site – your SharePoint service admin should be able to do that
    (at this moment you should be able to authorize to your SharePoint site).
  6. Validate your app has access to the target SharePoint site with PowerShell
    (check validation scripts below under References).
  7. Use recommended by Microsoft technique, code samples are available for the most popular languages/platforms – Python, C#, Java etc. (check below under References).
  8. Secure your certificate and/or secret. It is not a good idea to use hard-coded secrets, so consider using special services/storages for secrets (aka Vaults)

If you are hosting your application in Azure – consider using managed identity.

Step-by-step guide with screenshot – how to get app with Sites.Selected permissions

Video guide on using Sites.Selected to access SharePoint as application:

There are 3-rd party (and Microsoft) apps developed using classic approach (examples – Azure data Factory, Alteryx). So in some cases Sites.Selected permissions are not enough to get access to SharePoint.

Solution #2 – admin to register/update an ACS app

This option is acceptable if you have existing application that require ACS-based access.
This option is not recommended for new development, as ACS is deprecated and scheduled for retirement.

Microsoft (MC660075 in Message Center): “site collection admin will be unable to register app or update app permissions through above pages unless authorized explicitly by the SharePoint tenant admin” and “With this update site owners will not be able to register/update apps unless the tenant admin explicitly allows it.”

That is incorrect. Site collection admin cannot register app (appregnew) or provide permissions to the app (appinv) anymore. Tenant admin does not authorize site collection admins. Instead tenant (or SharePoint) admin can register an app or provide permissions to the app at a specific site (not changing the entire default behavior back…). But there was no such option (!) in the middle of October 2023, when this feature was enabled at all tenants. Even having a SharePoint admin or tenant admin permissions – if you tried to register an app with AppRegNew.aspx – you got the same error message “Your SharePoint tenant admin doesn’t allow site collection admins to…”.

Later (Checked today – Nov 6, 2023) it seems like Microsoft has implemented it! E.g. now SharePoint or tenant admin is able to register an app with AppRegNew.aspx or update it with AppInv.aspx at any specific site collection. SharePoint or tenant admin must also be among this site collection admins.

It is ok (and I’d say the preferred way) to provide ACS permissions to the app registered in Azure, so do not register apps in SharePoint anymore (do not use AppRegNew.aspx).

Bottom line: if ACS-based permissions are required for app here you go:

  • register application in Azure (Entra id)
  • activate your SharePoint service/tenant admin role
  • ensure you are also target site collection administrator
  • navigate to the site appinv.aspx page – e.g.
    “https://yourtenant.sharepoint.com/sites/yoursite/_layouts/15/appinv.aspx”
    and us Azure registered app (client) Id. E.g. for lookup provide
    • Azure registered app (client) Id for – click lookup
    • localhost as app domain
    • https://localhost as redirect url
    • Permission Request XML – depending on permissions you need, e.g. for full app access to entire site collection:
<AppPermissionRequests AllowAppOnlyPolicy="true">  
   <AppPermissionRequest Scope="http://sharepoint/content/sitecollection" 
    Right="FullControl" />
</AppPermissionRequests>

Solution #3 – step back (not recommended)

It is possible to switch back this new default behavior that prevents site collection admin to register/update apps at SharePoint. This is done with PowerShell command

Set-SPOTenant -SiteOwnerManageLegacyServicePrincipalEnabled $true

To run this command – you’d need to be a SharePoint service or tenant admin.

But this will be a step back on your journey in improving m365 tenant safety, as after that you’ll have a self-registered service principals out of control again. So devs will be using it not being aware of ACS retirement and when Microsoft switch off ACS – it will be a disaster, as all app will stop working. That is why Microsoft implemented this feature to soft-disable ACS and allowed us 2 years to redesign or apps and migrate from ACS to Entra Id apps with Sites.Selected. So this solution is not recommended.

In case you really need an exception to provide an ACS-based service principal – there is Solution number 2.

Full text of Microsoft’s MC660075 message

(Updated) SharePoint admin control for App registration / update

Tag
MAJOR UPDATE ADMIN IMPACT FEATURE UPDATE

Message Summary
Updated August 30, 2023: We have updated the content below for clarity. Thank you for your patience.

This is an enhancement to the security measures for administrative governance that modifies the default procedures for SharePoint app registration via AppRegNew.aspx page and permission updates via AppInv.aspx page. Following the implementation of this change, site collection admin will be unable to register app or update app permissions through above pages unless authorized explicitly by the SharePoint tenant admin.

Upon attempting to register an application on AppRegnew.aspx page, a notification will be displayed stating “Your SharePoint tenant admin doesn’t allow site collection admins to create an Azure Access Control (ACS) principal. Please contact your SharePoint tenant administrator.”

Similarly, upon attempting to update app permissions on AppInv.aspx page, a notification will be displayed stating “Your SharePoint tenant admin doesn’t allow site collection admins to update app permissions. Please contact your SharePoint tenant administrator.”

Kindly note that app registration and permission update via Microsoft Azure portal are not impacted by this change.

When this will happen:

The rollout process is scheduled to commence in late August and is expected to conclude in mid-September.

How this will affect your organization:

With this update site owners will not be able to register/update apps unless the tenant admin explicitly allows it.

To modify the default behavior, the tenant administrator must execute the following shell command to explicitly establish the flag as TRUE, thereby superseding the default value of FALSE. The service principal can only be created or updated by the tenant administrator by default. However, when the flag is set to TRUE, both the SharePoint tenant admin and site collection admin will be able to create or update the service principal through SharePoint.

The shell command is: Set-SPOTenant -SiteOwnerManageLegacyServicePrincipalEnabled $true

Note: The property ‘SiteOwnerManageLegacyServicePrincipalEnabled’ becomes visible in tenant settings after SharePoint Online Management shell is updated to 16.0.23710.12000 or a later version. But before this rollout, the value will always be TRUE even explicitly set to FALSE. It will only automatically be switched to FALSE as the default value after the rollout is launched.

What you need to do to prepare:

No proactive measures are required to prepare for this change. Nevertheless, it is advisable to inform your users of this modification and update any relevant documentation as necessary.

References

Sites.Selected API permissions for SharePoint access

Sites.Selected permissions are required for the non-interactive applications to get access to a specific SharePoint site using Microsoft Graph API and/or SharePoint API.
(Since Microsoft announced EOL of SharePoint App-only service principals, Sites.Selected is the only option going forward). Below are

Brief overview of Sites.Selected

Historically, we utilized so called SharePoint app-only service principals to get unattended (daemon/service) access to one specific site programmatically. Initially in on-prem, later in SPO. SharePoint app-only service principals use ACS-based authentication and allow calls to SharePoint (REST) API and usage of SharePoint CSOM.

Then Microsoft started developing Graph API. You’d need to register your app in Azure to get App Id and App secret to authenticate to Microsoft Graph API. You’d also configure specific API permissions for this app to get access to services you need. Unfortunately, for a long time there were no options to get access to only one specific site with Graph API. Available API permissions allowed access to entire SharePoint only.

Then, in 2021 Microsoft introduced Graph API “Sites.Selected” application permissions. Hooray! The problem was dev had to have two service principals – new Sites.Selected to call Graph API and classic SP-App-only to call SharePoint API. Later, in 2022 Microsoft implemented SharePoint “Sites.Selected” API permissions… More on this

Long story short, below are the detailed steps to configure Sites.Selected for you unattended app access to SharePoint site.

Steps to get and configure Sites.Selected permissions

1. Register an application in Azure (Entra Id) via Azure portal GUI, PowerShell script or helpdesk/servicedesk request. E.g. with GUI you’d login to portal.azure.com,
the search for “App registrations” and select “+ New registration”:

With PowerShell you’d do it with e.g. Register-PnPEntraIDApp cmdlet.

If you are not allowed to register an Entra Id app due to permissions restrictions in your company – connect with your IT/admins, as there must me some way to request an app.

Once you get an application registration – you are this app owner now – you should be able to navigate to your app registration and configure it (see Step 2 and below).

2. Update the app “API permissions” – so both – MS Graph API Sites.Selected and SharePoint Sites.Selected application API permissions are configured:

Request tenant admin consent for your API permissions. Finally your app registration “API permissions” should look like:

3. App Secret or Certificate
Under Certificates and secrets – generate client secret, copy secret value to safe location.

Or you can obtain trusted (or create a self-signed) certificate, and upload it to your app registration. Certificates are considered as more secure option then secrets.

4. At the Overview page – grab your app client id and tenant id :

At this moment, having tenant id, app (client) id and client secret (or certificate) – you should be able to authenticate against Microsoft 365 tenant with app-only authentication path.

But! Having just Sites.Selected API permissions configured for app does not mean your app has access to any SharePoint site. Access for the app to a specific site is provided by SharePoint team via Graph API calls. That leads us to the next step.

5. Application access to SharePoint site
You need to request this from your SharePoint service admin (or if you are an admin – DIY), but access needs to be provided for the specific app to the specific site with specified permissions (Read-Only or Read/Write or Manage or Full Control)
Here is the Graph API
Here is PowerShell PNP cmdlet

Interesting that MS Graph advertises 3 possible roles – read, write and owner, but PNP team says you can select from 4 roles – Read, Write, Manage or FullControl.

Obviously, Read role allows an app to read site content;
Write role is similar to “Contributor” user permissions – it allows CRUD operations against list items (library documents and metadata), but does not allow create/update/delete lists/libraries (for this – you’d need Manage role).

Use Sites.Selected permissions

Once your SharePoint tenant/service admin confirmed that access has been provided – you can use app client id and client secret (or certificate) to work with SharePoint from your code using Graph API. There are some good tutorials published:

Generally, this Sites.Selected permissions allows you to make calls that are documented under “Files” and “Sites and Lists” Graph API documentation. I.e. get site details, get site lists/libraries, create lists and libraries, CRUD operations against list items, download/upload library documents – all within the specific site. Sites.Selected permissions does not allow search operations, anything related to group or team etc.

If you have concerns if permissions for your app were provided correctly or not – you can validate your app access to the target SharePoint site with simple PowerShell scripts: here is the sample code

Note: Sites.Selected API permissions allows you call Microsoft Graph API with client Id and client secret. Calling SharePoint API with client secret is not supported. You have to use client id and certificate to call SharePoint API having app with Sites.Selected permissions.

Call SharePoint API with client Id and client secret is possible only if ACS-based permissions are provided for the app to the site, which is not recommended due to announced retirement (see below).

Secure your credentials

You do not want to hard-code your client secret as you do not want your credentials be leaked. So you need to secure your secrets in production. Solutions for secrets are included in cloud providers offerings, you can also use GitHub environment variables. If you are hosting your application in Azure – consider using key vault to keep your secrets. You can configure managed identity for your application and provide access to the key vault for you application managed id.

Govern Sites.Selected permissions

(For SharePoint admins).

Existing admins API/cmdlets allows yo to provide Sites.Selected permissions for specific app to specific site, and to get Sites.Selected permissions provided to the specific site. But there is no API/cmdlet for the specific app to get all sites (with permissions) this app has access to. Meantime as SharePoint admin if you keep providing permissions upon users/devs requests – after some time you have no idea what app has access to what site with which level of access, especially in large organizations.

Surely you can (and should) pull reports on all registerd apps with access to SharePoint, but…

There is a solution developed by Joe Rodgers (Microsoft). This solution use SharePoint list as an inventory/storage and Power Automate flows to pull data from Entra Id and SharePoint and provides kind of dashboard so you can review details of all app registrations in the tenant with at SharePoint Online permission. Cool!

Note: you would not provide Sites.Selected permissions just upon user/developer request. You’d always get an approval from target site owner. Target site owner must understand that application will have permanent unattended access to entire SharePoint site with permissions specified (read or write or manage or full control).

Sites.Selected permissions provisioning automation

(for SharePoint admins)

Generally, to provide an Application with Sites.Selected API permissions configured access to a specific site, SharePoint admin would run a set of PowerShell commands (or C# program or…) to ensure the client id exists, API permissions are configured and consented, to get app owners, target site owners, to get existing app permissions etc. Finally, admin would provide permissions and validate that permissions were provided correctly. It does not take long…

But in medium and large environments number of requests could be significant enough to start thinking of automation. I do have a separate article and video on Sites.Selected permissions provisioning automation.

Classic ACS permissions vs Sites.Selected permissions

Note: ACS-based permissions are deprecated:
Your SharePoint admin doesn’t allow site owners to create/update ACS principal ⋆ Vladilen Microsoft 365 engineer

Though Sites.Selected is our choice going forward, old classic ACS-based App-only permissions have some advantages (unique features) Sites.Selected does not have, e.g. ability to provide permissions not to entire site, but to specific list or subsite only. You can get more details checking this article for Comparison between Azure Apps and Entra Id Sites.Selected API permissions vs SharePoint app-only spn and ACS-based permissions.

Update:
Microsoft announced decommissioning of ACS permissions. So using ACS for any new development is not recommended.

It may be acceptable to grant ACS permissions to existing custom applications or third-party or Microsoft apps/web apps (e.g. Alteryx, Azure Data Factory) – applications that only support a client ID and secret and use the SharePoint API under the hood – but only to avoid disruption to business processes and keeping in mind that ACS will expire soon, so these applications must be replaced/updated before 2026.

Update: Microsoft implemented granular (permissions to list, item or file) alongside with Sites.Selected permissions. Original implementations of Sites.Selected allowed access to entire site collection only. With new ‘Lists.SelectedOperations.Selected’, ‘ListItems.SelectedOperations.Selected’ and ‘Files.SelectedOperations.Selected’ permissions it is possible to provide application permissions to list, library or list item or particular document (reference).

References

Connecting to Microsoft 365 SharePoint and Graph API from Azure Function App

Let say you need to run some code against Microsoft 365 on a scheduled basis. For this, you can use Azure Function App and timer-triggered Functions. From the code you’d call Microsoft Graph API and/or SharePoint API, so you’d need your Function to get credentials on the fly and use it to call APIs. For this – you’d have a key vault where credentials will be be stored and retrieved by functions when needed. Here we’ll create, configure and secure a scheduled Azure Function and Azure Key Vault.

First, we’ll create a resource group, e.g. “Azure-Func-Secured”.

Next, we’ll create a function app:

We’ll avoid the default “Flex Consumption” service plan (as it is Linux-only and does not support dependencies) and select “Consumption” a hosting option for now:

Runtime stack we’ll be using is PowerShell Core 7.4, but you can choose your own (options are – Python, .Net (C#), Node.js, Java). Let us leave other configuration settings by default (e.g. Enable Public access: On) for now, we’ll fix (secure) it later.

Ok, the function app has been created. Notice that app service plan, application insights and storage account, as well as some other services were created automatically under the same resource group:

Now we can create one or more functions under this function app, also we’d need to create a key vault and a registered app. Let us start with function and VS Code would be our environment of choice.

Let us start Visual Studio Code in a separate folder. Ensure you have “Azure Functions”, “Azure Resources” and PowerShell extensions by Microsoft. You’d sign-in to Azure:

and create a function project:

You’d choose “Timer triggered” function (as we need the function running on schedule), the function name would be MyScheduledFunct01 and we’d leave the default schedule as “0 */5 * * * *” – so our function will be triggered every 5 minutes.

Let us deploy the function right away and see if it works. For this, we can use “deploy” icon under “WORKSPACE” panel:

or “Deploy to function app” option from the function app context pop-up menu under the Azure “RESOURCES” panel:

After successful deployment give it some time, then check function invocations and ensure function is triggered an running:

Next, we’d update “requirements.psd1” to include Azure Key Vault and PnP PowerShell modules as it takes some time for the function app to pull in and install dependencies. Requirements.psd1:

# This file enables modules to be automatically managed by the Functions service.
# See https://aka.ms/functionsmanageddependency for additional information.
#
@{
    # For latest supported version, go to 'https://www.powershellgallery.com/packages/Az'. Uncomment the next line and replace the MAJOR_VERSION, e.g., 'Az' = '5.*'
    # 'Az' = 'MAJOR_VERSION.*'
    'Az.KeyVault' = '6.*'
    'PnP.PowerShell' = '2.*'
}

And we’d update function itself to monitor if dependencies are installed, than we’d deploy the function again so time would work for us. MyScheduledFunction/run.ps1:

# Input bindings are passed in via param block.
param($Timer)
$currentUTCtime = (Get-Date).ToUniversalTime()
if ($Timer.IsPastDue) {
    Write-Host "PowerShell timer is running late!"
}
Write-Host "PowerShell timer trigger function ran! TIME: $currentUTCtime"
#############################################################################################
Write-Host "Check modules installed:"
Import-Module PnP.PowerShell 
Import-Module Az.KeyVault
Get-Module PnP.PowerShell
Get-Module Az.KeyVault
Write-Host "Check command available:"
Get-Command -Name Connect-PnPOnline -Module PnP.PowerShell

At first, we might see warning like “The first managed dependency download is in progress, function execution will continue when it’s done. Depending on the content of requirements.psd1, this can take a few minutes. Subsequent function executions will not block and updates will be performed in the background.”. So we’d just wait.

After some time the function will be able to use required modules. Here is the invocation output example:

App Registration
To get unattended access to SharePoint (or Teams or Exchange etc.) as a service (daemon) application – we need credentials. It is done via “App Registration” under Entra Id (Azure AD) – here is the detailed step-by-step guide on registering apps in Azure.

Finally, we’d have a service principal with permissions we need:

We do not hard-code secrets, so let us create a key vault to keep secrets safe:

We’d leave all other configuration option by default, including “Azure role-based access control”, Allow access from: All Networks etc.

Here is the trick: even if I created this key vault and have an admin-level access, I’m not able to work with secrets values. So I need to provide additional access to myself. Being at the key vault, navigate to “Access control (IAM)” and add role assignment…

We’d select “Key Vault Secrets Officer”, next, select members… and select own name:

From this moment you should be able to CRUD secrets with values.

Now, let us generate a secret under App registration and copy secret value:

Then navigate to the Key Vault -> Object/Secrets -> Generate/Import – and save the secret value there:

Now you can get this secret… but the function cannot reach the secret… Here is the proof. Let us update the function code with:

Write-Host "Get secret from the key vault:"
$vaultName = "azure-func-secrets"
$secretName = "azure-func-secured-01"
$kvSecret = Get-AzKeyVaultSecret -VaultName $vaultName -Name $secretName -AsPlainText
Write-Host "Secret:" $kvSecret.Substring(0,3)

and check the function output. You’d see something like (which is not very descriptive):

So, how’d we allow key vault access for the function app? It’s as simple as that:
– first, we’d need some identity assigned to function app
– second, we’d provide access to the key vault to this identity

Managed Identity: assign an identity to the function.
For this, you’d go to function Settings/Identity, and under System Assigned, you’d switch status to On and Save settings.

Then, you’d go to your key vault, Access Control (IAM) and add role assignment. But this time, you’d select the role “Key Vault Secrets User” (more about roles), and “Assign access to” Managed Identity, and select members – select your function app identity (notice, identity is assigned to function app, so all functions under the app will be able to use the identity):

Now, we’d check the next function invocation detail, and voila:

You can see that azure function was able on the fly to pull secret from the key vault, so now we should be able to use these credentials to access SharePoint.

As you know, having client id and client secret would allow you to call MS Graph API. But calling SharePoint API would require authentication with a certificate. “PnP.PowerShell” module use both APIs under the hood, so we’d need a certificate to connect to tenant and work with SharePoint using “PnP.PowerShell” module. Please refer to this article on how to run Connect-PnPOnline with Certificate stored in the Key Vault.

References:

Implementing Microsoft 365 group expiration policy in large companies

This post is dedicated to one specific subject: implementing Microsoft 365 groups lifecycle (expiration) policy in large Microsoft 365 environments.

But this post is also a part of a bigger problem – dealing with ownerless resources in Large Microsoft 365 environments. Please refer to the umbrella post.

Scenario

You administer a large Microsoft 365 environment. Let say you have 100k users or more, 50K or more sites. Environment is not new, so after some years you have a lot of ownerless groups and sites (thousands probably), and a lot of inactive groups and sites (probably tens of thousands). You are getting more and more ownerless groups – hundreds each month. You are thinking of stopping bleeding and cleaning this up…

Implementing Microsoft 365 groups expiration policy

If you are thinking of activating in an existing environment – you would probably have a spike – all the old groups will be subject to policy. The ide is to avoid situation when a specific person – group owner will get dozens of email. It would be better if a person will receieve, let say one email per week.

Here is my 4 possible approaches to avoid this spike, distribute notifications evenly across the time and ease the pain:

By changing Group Lifetime

You would need to change the policy every, e.g. week or month, specifying group lifetime in days starting with maximum period. Consider
– calculate number of days between the oldest group created an today, plus 35 days – it’ll be your first “group lifetime”
– activate the policy with this number of days in “group lifetime” – and within a week you will get notifications on the oldest group/groups
– after a week or two – change the “group lifetime” decreasing it by e.g. 30-60 days and reactivate the policy… and so on

You can easily calculate it all and choose your pace depending on how many groups you have to renew, how much time you need to clean-up. You got the idea.

Downside – in the email notification it will be said “otherwise the group will be deleted on …”, but you joggling with lifetime period – so these dates might confuse users

By renewing groups as admin

As an admin, you can use PowerShell “Reset-PnPMicrosoft365GroupExpiration” or graph API “POST /groups/{id}/renew“ to renew any group.

So depending on total number of groups, number of active/inactive groups, number of ownerless groups in your organization – you can come up with a strategy, using one or more of the following techniques:

  • renew all active and known and important groups (build list of groups to re-activate based on your own criteria) and then trigger the policy
  • build list of definitely inactive groups and renew all other groups
  • split groups into chunks and every day or week (depending on numbers) renew groups in a chunk… after that you can activate groups expiration policy, the policy will be triggered against small number of groups every day/week (with this trick you would avoid policy triggering against large number of groups and sending thousands of emails at one)

By sending customized e-mails to users

Another technique to avoid surge in your e-mail system (and most importantly – avoid sudden influx of support requests) – you can send emails to group owners with a link to renew a group. Surely you’d need to build your custom solution for that, but this does not seem like a complicated task. Some PowerShell scripting – and you are good.

“Renew group” link would look like:
https://account.activedirectory.windowsazure.com/Group/RenewGroup?tenantId=<tenantId>&id=<groupId>
where <tenantId> is tenant id and <groupId> is group Id. So with PowerShell you’d just pull groups in question, their owners and send email with the link dynamically built. Consider spreading this activity across the time to avoid spikes.

By sending users to the groups page

Probably the easies option to smooth the expiration policy implementation is to send group owners to a Microsoft “Groups I own” page – where they could renew or delete their groups. The page is:

Groups I Own (https://myaccount.microsoft.com/groups/groups-i-own)

The downside is – the policy should be in active state (otherwise owners will not see when the group is expiring and option to renew it). So consider a maximum lifetime period in policy, then you’d decrease this period).

Groups expiration policy technical details

Check this KBA for a deep dive into the groups lifecycle policy technical details.

Microsoft 365 group expiration policy deep dive

Nobody likes garbage, including Microsoft 365 administrators. If any user can create a team or yammer community – they create, but then they leave company and we are getting more and more abandoned groups, teams and SharePoint sites. So we need a way to clean up environment. There is a Microsoft 365 groups expiration policy that can help remove unused groups from the system, but since all Teams and Yammer sites are group-based – it also helps SharePoint admins make things cleaner. In a nutshell what this policy does is it sends notifications to group owners so a group owner can renew the group, otherwise the group will expire and be deleted.

Who can configure the policy and how

The policy lives under Azure Portal, Azure Active Directory, Groups, Expiration:

Microsoft 365 groups expiration policy can be configured by Groups Admin or Global Admin (tenant admin) only. Microsoft 365 Teams or SharePoint admin cannot configure it. Microsoft says that User administrator can do it – so I need to verify it.

Here is the policy config screen:

Microsoft documented it well in the “Microsoft 365 group expiration policy“, but I completed some tests in my lab environment and here is what I found and what is not covered by Microsoft. Let me share it with Questions and Answers format:

Questions and Answers

General questions

Q: How long it takes for policy to start generating notification emails after activation?
A: Immediately, i.e. minutes, maybe up to one hour (in case there groups that are subject for the policy).

Q: Can I customize email that is send to group owners?
A: No, there is no such option at the moment.

Q: What is the email address notifications come from?
A: It’s “msgroupsteam@microsoft.com” with the display name “Microsoft Groups Team”

Q: What does a notification email look like?
A: Please find some examples below, in the end of this article.

Q: Are there other ways to get notifications? Teams?
A: I have not seen any official Microsoft’s documentation on this, but yes – notifications are coming via Teams too: “TeamName is expiring soon. Renew now”:

though it is not clear what exactly should used do to renew the group, as after clicking on that alert a regular teams settings page is opened:

and I got just a few notification in teams, though e-mails notifications I got many.


Q: What happens when a user clicks “Renew group” button in the email notification?
A: User will be sent to a Microsoft’s page and the following “Do you want to renew the group?” window will be shown:

On Yes, it says”<groupName> was successfully renewed. You can close this window now”:

And the group expiration date will be set up as current date.
On “No” it says “Group was not renewed. You can close this window now.”:

And an expiration day will not be changed. No more notifications will be generated. The group will be active until expiration date. Then the group will be deleted.

Q: What if two owners choose opposite?
A: The last action will take effect.

Q: what if one user choose “delete group” but the other one later decided “Renew group”?
A: The one who click “Renew group” will see “<Group Name> successfully renewed. Because the group was deleted, it might take up to 24 hours to be fully restored. You can close this window now.”

Q: What if the group does not have owners?
A: If the group is orphan (ownerless), the expiration emails will go to the email specified in policy configuration. Usually it is a distribution list with admins or other responsible team.

Q: What if the group does have a non-mail-enabled owner?
A: I have tested 2 types of entities with no email:
– just a contact in Outlook
– user with no Exchange license assigned
Results are: Outlook contact cannot be added to team, so there should be no contacts as teams/groups owners; a user with no Exchange licens can be added to team/group and Microsoft does not consider this group ownerless, so notification should be sent to group owners, but since there is no email associated to a group owner – e-mail are not sent, so we are having an issue here.

Q: What if I deactivate the policy – will email notifications sent earlier still be actionable?
In other words, would users still be able to renew the group clicking on the “Renew group” button?
A: Yes. Actually “Renew group” button is just a link to the Url:
https://account.activedirectory.windowsazure.com/Group/RenewGroup?tenantId=<tenantId>&id=<groupId>
where a group owner can renew group.

Q: If one of the owners renewed the group – what will happen with notifications sent to other owner? What if other owner click “Renew group” or “delete group”?
A: Notifications sent will stay. Since buttons in the email are just links (not actionable buttons) – user will be redirected to a web-page where he/she will be able to renew or delete the group.

Q: As per MS: “Groups that are actively in use are renewed automatically around 35 days before the group expires. In this case, the owner does not get any renewal notifications. Any of the following actions will automatically renew a group…<list of actions>”. So, what exactly does “Groups that are actively in use” mean?
A: This is not disclosed by Microsoft. They only say “Azure Active Directory (Azure AD), part of Microsoft Entra, uses intelligence to automatically renew groups based on whether they have been in recent use. This renewal decision is based on user activity in groups across Microsoft 365 services like Outlook, SharePoint, Teams, Yammer, and others.” Btw, <list of actions> includes almost all user actions – so basically any action – even just visit site/team is considered as activity.

Q: Can I track the policy in action via audit log?
A: There is no “activity type” for this policy’s specific actions… You also cannot specify user “msgroupsteam@microsoft.com” to get all activities. So no tracks on the policy “before action” – i.e. at the detection and e-mailing stage.
If a user clicks “renew” button or “delete group” link – this should be logged as this user action with Category “GroupManagement” and activity: “Update group” and “RenewedDateTime” as property modified.
If it happens that the group is deleted by policy – this should be logged under policy’s account – see below.

Automatically renewed group appears as audit log event with
– Workload: AzureActiveDirectory
– RecordType: 8 “AzureActiveDirectory”
– Activity: “Update group”
– Properties modified would be “RenewedDateTime”

Automatically deleted group appears as audit log event with
– Workload: AzureActiveDirectory
– RecordType: 8 “AzureActiveDirectory”
– Activity: “Delete group.”

Microsoft groups lifetime policy operates on behalf of Actor (first-party Microsoft service principal):

  • AppName: Microsoft.ApprovalManagement
  • AppId: 65d91a3d-ab74-42e6-8a2f-0add61688c74
  • Object Id: f64c9eca-18fd-4652-bafe-897fd2d46798

more on first-party Microsoft service principals

Q: After the group is deleted, who can restore it?
A: MS says: “A deleted Microsoft 365 group can be restored within 30 days by a group owner or by an Azure AD administrator”.
In fact, SharePoint admin (and maybe some other roles like Teams admin or Exchange admin) can restore group. SharePoint admin can restore site from recycle bin – and the group will be restored as well.

Q: My org is using retention policies. Will the lifecycle policy delete site if it contradicts with retention policy?
A: Lifecycle policy respects retention policy, so if the site should not be deleted according to retention policy or legal hold – the site will not be deleted (TBC – need to be validated).

Q: What if a user forward this e-mail notification to other user? Can this other user renew or delete the group?
A: When a user receive a notification email forwarded, and he/she click “Renew group” button – his/her experience will be the same if he/she is also a group owner. If a user is not a group owner – he/she will get “You don’t have permission to renew this group because you’re not an owner. To renew , contact a group owner. You can close this window now.”:

Note: if a user with active groups administration permissions receives email and try to renew or delete the group – he/she will also be able to do that.

Q: Can user get information on groups he/her owns, groups expiration data? Can user renew the group before the policy trigger email notification?
A: yes, all that can be done from the page: https://myaccount.microsoft.com/groups/groups-i-own

Q: What if I activate m365 groups lifecycle policy for the selected groups only?
Any insight on policy behavior?
A: The policy will work as usual, but for the selected groups only. Separate from the policy – under “my groups” users will be able to see “Expiration date” and “Renew” option for groups in policy’s scope only:

Scenario with many existing inactive groups

Let say we have a large Microsoft 365 environment with many inactive groups, some of them are inactive for a long time – e.g. 1 or 2 years. We want to implement groups expiration policy, but we want to understand better the policy behavior.

Microsoft says: “The expiration period begins when the group is created, or on the date it was last renewed” and “When you change the expiration policy, the service recalculates the expiration date for each group. It always starts counting from the date when the group was created, and then applies the new expiration policy.”
So in case we implement the policy first time, we know that Renewal Date for all groups is just a Group Creation Date.

See also Strategy of implementing group expiration policy in large Microsoft 365 environments

Q: What will happen if I activate the policy – will the policy start generating emails immediately for all groups?
A: Yes. Once activated – policy starts detecting expired groups and sending notifications to groups owners. So if you have 3k expired groups with 6k owners in it – expect policy will generate 6k e-mail notifications.

Q: Which groups the policy will be triggered against? All or Inactive only?
A: As per Microsoft, if at around 35 days before expiration it will be determined that group is actually active, the policy can renew the group automatically.
But definition of this activity is not disclosed and might be not the same as group activity status 90 days based on MS Graph data you can see at CA.
(I got notifications for groups that were not active recently but with Active status).

Q: In the case above – what would be the deadline? When the policy will delete the group?
A: If the group expiration period is passed, but the policy was just activated – it does not delete the group immediately. Policy allows ~30-35 days for owners to renew the group.
E.g. My test policy was activated May 3 and I got message for old group immediately, but it said that the group will be deleted on June 7.

Q: What if there are more than 10K emails – will it trigger Exchange throttling?
A: Most likely emails not sent will be sent next day.

Q: Can I specify a distribution list in the policy as an “Email contact for groups with no owners”?
A: Yes

Q: Can I specify an external e-mail address as an “Email contact for groups with no owners”?
A: TBC

Q: Can admin ask user to renew or delete the group by some other custom solution (skipping the policy)?
A: yes. Actually, “Renew group” button is just a link to the following Url:
https://account.activedirectory.windowsazure.com/Group/RenewGroup?tenantId=<tenantId>&id=<groupId>
where <tenantId> is tenant id and <groupId> is group Id. So basically anyone

Microsoft 365 Groups object model

Let me explain the policy behavior in m365 group object model terms.

There is a group property “RenewedDateTime”. When group is created – this property is set up to group created date/time (same as group CreatedDateTime property value).
For the notification purposes the policy calculates “Expected Expiration DateTime” as RenewedDateTime plus “Group LifeTime” (number of days specified in policy, e.g. 180). First notification is triggered about 30 days before “Expected Expiration DateTime”, so the policy simply selects groups with RenewedDateTime property value less then current DateTime minus “Group LifeTime days” minus 30 days and sends notification starting from oldest group:

RenewedDateTime < Today - GroupLifeTime -30

When owner confirms group is still needed – RenewedDateTime is setup to current DateTime.

Q: When a user chose to “Renew group” – will it impact group activity?
A: No. If a user did not visit group – but just clicked “Renew group” – it will not trigger group last activity date. E.g. inactive group will still be inactive.

Q: Is there an API to configure Microsoft 365 groups expiration policy programmatically?
A: Yes, in MS Graph API it is called Group Lifecycle Policy: groupLifecyclePolicy

Q: Can I programmatically renew the group (all groups) as an admin?
A: Yes, consider using Microsoft Graph API or PowerShell 7 with PnP.PowerShell module.
PnP Doc says Reset-PnPMicrosoft365GroupExpiration command “Renews the Microsoft 365 Group by extending its expiration with the number of days defined in the group expiration policy set on the Azure Active Directory” – but that does not seem accurate. This command sets up “RenewedDateTime” group property to the current datetime, not related to current policy settings (the policy might even not have been activated).
Microsoft Graph API entry point: “POST /groups/{id}/renew
Group.ReadWrite.All permissions required.

Q: Is it possible to setup “RenewedDateTime” property to another date/time of my choice (not the current date)?
A: Apparently that is not possible. I could not find a way so far… It says
Property 'renewedDateTime' is read-only and cannot be set.

Q: What permissions are required to renew the group with Reset-PnPMicrosoft365GroupExpiration?
A: Group.ReadWrite.All – delegated or application

Q: What exactly is behind the automatic groups renewal?
A: Actually, the is a separate process in parallel with groups expiration policy – and this process starts monitoring groups activity ~35 days before expiration and once activity is detected – the process resets group RenewedDateTime property. And the moment this date is reset – the group is excluded from policy.

Q: If I activate the policy not for all but for a selected groups only, will I still be able to renew other groups programmatically?
A: Yes, as an admin – you can resets group RenewedDateTime property programmatically all alone. It does not matter – whether this policy is activated or not.

Screenshots

Notification e-mail that comes to group owners “as is” – web outlook view:

Notification e-mail that comes to group owners when content is unblocked (web outlook):

Notification e-mail that comes to group owners when pictures are loaded (desktop Outlook):

Notification e-mail that comes to group owners some key areas:

And I’d add that e-mail says how many members in this group (number of members, not including owners… i.e. if you are the only owner – it’ll be zero members).
Correction: “Renew group” is not an actionable button – it is just a html button with a link.

Screenshot of the notification that comes to email specified in policy for the groups that does not have owners:

  • Outlook icon link sends user to group mailbox
  • SharePoint icon is the link to the associated SharePoint site
  • Clicking on Teams icon will transfer user to a default team channel chat page
  • the last one – group icon – is the link to a Microsoft’s groups management page where user can edit group, manage membership, renew group or delete group (see screenshot below):

Renew group button is visible if the expiration policy is activated:

Deleted group

When the not renewed group reaches expiration date – the policy deletes the group and group owners get an e-mail notification like this:

Email subject would be “Attention: <group name> was deleted. Restore it by Thursday, August 10, 2023” and in the body “

<group name> expired on Monday, July 10, 2023. It was deleted, along with all associated communications, files, calendar events, and tasks. You have 30 days from the expiration date to restore Test Priv team – ownerless groups policy and its content. You received this email because you’re an owner of the group”.

Owner can restore group within 30 days by simply clicking “Restore group” button. Then owner would be redirected to the “https://myaccount.microsoft.com/groups/action?groupId=<groupId>&action=Restore” and get a message “The group was successfully restored. It might take up to 24 hours before you can access all associated content. You can close this window now.”

References

FastAPI on Azure Functions with Azure API Management

Following Pamela Fox tutorial “FastAPI on Azure Functions with Azure API Management“.

The idea is to deploy FastAPI to Azure Functions the way auto-generated interactive documentation would be public, but actual API would be protected. Pamela solved it with Azure API Management and subscription keys:

“One of my goals was to have the documentation be publicly viewable (with no key) but the FastAPI API calls themselves require a subscription key. That split was the trickiest part of this whole architecture, and it started at the API Management level.”

Pamela published it in 3 parts:
– the idea and solution explained under her blog: FastAPI on Azure Functions with Azure API Management
– code and some initial steps at GitHub: pamelafox/fastapi-azure-function-apim
– video with more deploying details at YouTube: Deploying FastAPI app to Azure Functions + API Management

I will just repeat all the steps in this one-pager.

Environment I use: Linux Ubuntu + VS Code with “Dev Containers” extension and azd

  1. Clone https://github.com/pamelafox/fastapi-azure-function-apim
  2. Start visual studio code and reopen the project with container
  3. ensure it works locally with
    PYTHON_ISOLATE_WORKER_DEPENDENCIES=1 func host start
  4. Deploy it to Azure functions with (you’d answer questions):
    $ azd auth login
    $ azd init
    $ azd up
  5. Go to API management Service, Subscriptions, Add subscription, copy the key (secure it)
  6. From API management service, Overview – open Gateway URL and append it with “/public/docs”
  7. Try GET /generate_name as is – you’ll get “401”
  8. Try the same with subscription key – you’ll get “200”
  9. Save Request Url to call the API from your front-end app

Nest steps:

  • calling other APIs
  • connecting to Databases
  • using secrets

PowerShell scripts for Microsoft 365 SharePoint

After many years working with SharePoint I wrote a lot of PowerShell scripts that help me support, troubleshoot, administer and secure SharePoint. So I’m sharing my scripts with you.

It’s here: https://github.com/VladilenK/Manage-m365-with-PowerShell