Since Microsoft announced retirement of legacy Azure ACS – all SharePoint admins are working against the clock. ACS permissions were here for 10+ years and there are tons of videos and blogs guiding users how to get these permissions and use it in apps. And imagine how many ACS apps are still there accessing SharePoint sites. We do not want business screams “My critical process stopped working! Do something right now!”. So we would get the list of such apps that are still using ACS to access SharePoint, get apps owners, sites and sites owners. The ultimate goal is to communicate to right people and let them know that ACS is deprecated now and engage them to update their solutions to use only modern authentication.
What kind of apps we are talking about, specifically? I can see the following options:
- Apps that were registered in SharePoint via AppRegNew.aspx (aka SharePoint app-only service principals) and provided with permissions in SharePoint via AppInv.aspx
- Apps that were registered in Azure (Entra Id) and provided with permissions in SharePoint via AppInv.aspx
Techniques we can use to get data:
- analyze audit log to get events where apps are accessing sites
- analyze audit log to get events where ACS permissions were provided to sites
- get data from system that tracks request for new ACS permissions
- use reports from admin center
- use the PnP Microsoft 365 Assessment Tool
- get report on apps owners and permissions from from Entra Id
Let us deep dive into each data source to see if it is actually helps us to get ACS apps in use…
Audit log: apps accessing sites
Microsoft 365 audit log is supposed to save all events happening in Microsoft 365. It is available for admins via GUI, PowerShell Exchange Module and Graph API. GUI Search m365 audit log now lives under Microsoft Purview – Solutions – Audit.
GUI search Audit Log under Purview
Unfortunately, when an App registered in Azure and provided with ACS access (via appinv) is accessing SharePoint sites – no events are saved in m365 audit log.
SharePoint app-only principals (apps registered in SharePoint via appregnew) are tracked in m365 audit log. Events would have a UserId “app@sharepoint” (yes, single user id for all apps). Other event details would include activity/operation (PageViewed, FileModified etc.), Item (full Url of a document or page etc.), AppAccessContext (includes ClientAppid, ClientAppName), ApplicationId (yes, this is how we know what app access what url on the site), and many other details
Get Audit Log via Microsoft Graph API
The following reports are available in preview only (under beta):
Service principal sign-in activity
This report is available through the servicePrincipalSignInActivity resource type and details the sign-in activity for a service principal in your tenant. The sign-in activity can be delegated or application-only scenarios. For application-only scenarios, the application credential activity provides additional information on the credential usage.
Service principal sign-in activity report provides the following details for every service principal:
- id,
- appId,
- lastSignInActivity,
- delegatedClientSignInActivity,
- delegatedResourceSignInActivity,
- applicationAuthenticationClientSignInActivity,
- applicationAuthenticationResourceSignInActivity
More on Service principal sign-in activity
Application credential sign-in activity
This report is available through the appCredentialSignInActivity resource type and details the usage of an app credential (secret, certificate, or federated identity credential) in your tenant.
Application credential sign-in activity report provides the following details for every service principal credential:
- id, keyId, keyType, keyUsage,
- appId, appObjectId, servicePrincipalObjectId,
- resourceId,
- credentialOrigin,
- createdDateTime,
- expirationDateTime,
- signInActivity
More on Application credential sign-in activity
Application sign-in
Evaluate the usage of application sign-ins in your tenant using either a summary report or a report that provides details of sign-ins, such as the number of sign-ins and whether any errors occurred during sign-in.
Application sign-in report provides the following details for every service principal:
aggregatedEventDateTime, appDisplayName, appId, id, signInCount, status
More on Application sign-in
Audit log ACS permissions provided events
This is relatively easy. There are just 3 kinds of events that might help us to understand ACS usage in tenant:
- pull audit logs with record type is SharePointAppPermissionOperation so you’d get events where permissions were provided to apps (e.g. operation type (activity) would be like AppPermissionGrant)
- pull audit logs with record type is SharePoint and activity type (operation) is PageViewed and keyword for free search is appregnew – to get events when there was an attempt to register a new SharePoint app-only service principal.
- pull audit logs with record type is SharePoint and activity type (operation) is PageViewed and keyword for free search is appinv – to get events when there was an attempt to provide ACS permissions for a SharePoint app-only service principal or for an Azure App registration
System that tracks request for new ACS permissions
In case you have a process of providing ACS permissions… Process might include tickets to service desk or similar kind of system… Anyway – check if you can get data from that system – like who requested for what app to what site etc…
Reports available at admin center
So far the only report that might help is in development (see Microsoft 365 Roadmap – feature Id 417481) and scheduled to be available in March 2025.
“Enterprise Application Insights is a powerful report which helps SharePoint Administrators to discover all the SharePoint sites that are allowed access by third-party applications registered in your tenant. The report also provides details on the application’s permission and requests count to help admins take further action to strengthen the security of the site. It is part of SharePoint Advanced Management capabilities.”
The feature is already documented here: Generate App insights reports and is seems like the report will not be available for all tenants – but just for tenants with Microsoft SharePoint Premium (SharePoint Advanced Management) or Copilot license assigned.
PnP Microsoft 365 Assessment Tool
Microsoft 365 Assessment Tool is an utility designed by PnP team a while ago and since then serves SharePoint admins very well. In particular, it helps helps us identify and evaluate the Azure ACS usage for tenant by providing the usage data of ACS principals, and even generating a Power BI reports.
If you run this tool specifying AddInsACS mode, it provides you with:
- classicacsprincipals report that includes apps with Allow AppOnly permissions.
Details are: App Ids, if the app has Tenant or Site Collection Scoped Permissions, RedirectUri, AppDomains and ValidUntil
If the ValidUntil field contains specific date – that means the app was registered via appregnew
If the ValidUntil field contains “01/01/0001 00:00:00” date – that means the app was registered in EntraId
- classicacsprincipalsites – sites these apps have access to
Details are: AppIdentifier, ServerRelativeUrl
- classicacsprincipalsitescopedpermissions – list of apps permissions to sites
Details: AppIdentifier, ServerRelativeUrl, SiteId, WebId, ListId, Right (Read/Write/FullControl/Guest etc.)
If the WebId field equals zeros, that means rights were provided to entire site collection
- some other details
Unfortunately, this tool does not provide when the app was last time authenticated or when the app accessed the site
I use the following PowerShell to start the tool, get status and export reports:
$tenantDomain = "" # "contoso.sharepoint.com"
$clientid = "" #
$certThumbprint = ""
$certPath = "My|CurrentUser|" + $certThumbprint
./microsoft365-assessment.exe start --mode AddInsACS --authmode application --tenant $tenantDomain --applicationid $clientid --certpath $certPath
./microsoft365-assessment.exe status
./microsoft365-assessment.exe report --id <report id> --mode CsvOnly --path ".\ACS-reports"
Highly recommended: SharePoint Add-In and Azure ACS Assessment
Report on apps owners and permissions from from Entra Id
Using all the methods above – you’d get a list of active service principals that use legacy ACS authentication. But to whom we need communicate to regarding this service principals? Obviously, we need this service principals owners. There are multiple options how to get an app owner from Azure (Entra Id):
More Observations
Test scenario 1
DisableCustomAppAuthentication is true, i.e. ACS are not allowed in tenant.
SiteOwnerManageLegacyServicePrincipalEnabled -s false, i.e. site owners cannot register apps at sites or provide permissions to app on sites.
It is not possible for admin to go to appregnew.aspx and create an app (app-only spn).
I registered apps in Azure.
It is possible for admin to go to appinv.aspx and “provide” permissions to the azure app registrations.
An app is shown under appprincipals.aspx only in case if ACS access was provided to app but Sites.Selected access was not provided. The moment you provide Sites.Selected access for the app to the site – the app disappears from list of apps under appprincipals.aspx page. It does not help if you remove Sites.Selected permissions.
Connect-PnPOnline works with certificates or with secrets.
Get-PnPSite works only if connection was made with a Certificate (if connection was made with secret – it gives 401 unauthorized).
Test scenario 2
DisableCustomAppAuthentication is false, i.e. ACS are allowed in tenant.
SiteOwnerManageLegacyServicePrincipalEnabled -s false, i.e. site owners cannot register apps at sites or provide permissions to app on sites.
Connect-PnPOnline and Get-PnPSite works with certificates or secrets if ACS access was provided for an app to at least one site.
If there was no ACS permissions provided for the app – Get-PnPSite gives “Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))”
References