Tag Archives: Audit Log

Azure ACS retirement: Track down ACS apps

Since Microsoft announced retirement of legacy Azure ACS – all SharePoint admins are working against the clock. ACS permissions were here for 10+ years and there are tons of videos and blogs guiding users how to get these permissions and use it in apps. And imagine how many ACS apps are still there accessing SharePoint sites. We do not want business screams “My critical process stopped working! Do something right now!”. So we would get the list of such apps that are still using ACS to access SharePoint, get apps owners, sites and sites owners. The ultimate goal is to communicate to right people and let them know that ACS is deprecated now and engage them to update their solutions to use only modern authentication.

What kind of apps we are talking about, specifically? I can see the following options:

  1. Apps that were registered in SharePoint via AppRegNew.aspx (aka SharePoint app-only service principals) and provided with permissions in SharePoint via AppInv.aspx
  2. Apps that were registered in Azure (Entra Id) and provided with permissions in SharePoint via AppInv.aspx

Techniques we can use to get data:

  • analyze audit log to get events where apps are accessing sites
  • analyze audit log to get events where ACS permissions were provided to sites
  • get data from system that tracks request for new ACS permissions
  • use reports from admin center
  • use the PnP Microsoft 365 Assessment Tool 
  • get report on apps owners and permissions from from Entra Id

Let us deep dive into each data source to see if it is actually helps us to get ACS apps in use…

Audit log: apps accessing sites

Microsoft 365 audit log is supposed to save all events happening in Microsoft 365. It is available for admins via GUI, PowerShell Exchange Module and Graph API. GUI Search m365 audit log now lives under Microsoft Purview – Solutions – Audit.

GUI search Audit Log under Purview

Unfortunately, when an App registered in Azure and provided with ACS access (via appinv) is accessing SharePoint sites – no events are saved in m365 audit log.

SharePoint app-only principals (apps registered in SharePoint via appregnew) are tracked in m365 audit log. Events would have a UserId “app@sharepoint” (yes, single user id for all apps). Other event details would include activity/operation (PageViewed, FileModified etc.), Item (full Url of a document or page etc.), AppAccessContext (includes ClientAppid, ClientAppName), ApplicationId (yes, this is how we know what app access what url on the site), and many other details

Get Audit Log via Microsoft Graph API

The following reports are available in preview only (under beta):

Service principal sign-in activity

This report is available through the servicePrincipalSignInActivity resource type and details the sign-in activity for a service principal in your tenant. The sign-in activity can be delegated or application-only scenarios. For application-only scenarios, the application credential activity provides additional information on the credential usage.

Service principal sign-in activity report provides the following details for every service principal:

  • id,
  • appId,
  • lastSignInActivity,
  • delegatedClientSignInActivity,
  • delegatedResourceSignInActivity,
  • applicationAuthenticationClientSignInActivity,
  • applicationAuthenticationResourceSignInActivity

More on Service principal sign-in activity

Application credential sign-in activity

This report is available through the appCredentialSignInActivity resource type and details the usage of an app credential (secret, certificate, or federated identity credential) in your tenant.

Application credential sign-in activity report provides the following details for every service principal credential:

  • id, keyId, keyType, keyUsage,
  • appId, appObjectId, servicePrincipalObjectId,
  • resourceId,
  • credentialOrigin,
  • createdDateTime,
  • expirationDateTime,
  • signInActivity

More on Application credential sign-in activity

Application sign-in

Evaluate the usage of application sign-ins in your tenant using either a summary report or a report that provides details of sign-ins, such as the number of sign-ins and whether any errors occurred during sign-in.

Application sign-in report provides the following details for every service principal:
aggregatedEventDateTime, appDisplayName, appId, id, signInCount, status

More on Application sign-in

Audit log ACS permissions provided events

This is relatively easy. There are just 3 kinds of events that might help us to understand ACS usage in tenant:

SharePointAppPermissionOperation

Pull audit logs with record type is SharePointAppPermissionOperation so you’d get events where permissions were provided to apps. Operation type (activity) would be like AppPermissionGrant.

Microsoft started logging this record type not long ago and there is no documentation found (as of Feb 2025). So the only I noticed that might help is:

  • if user id is “app@sharepoint” – that indicates Sites.Selected permissions were provided to the app
    (e.g. via Grant-PnPAzureADAppSitePermission )
    under “AppId” you’d have an app (client) id of the client app (permissions provided to) in the form of
    “i:0i.t|ms.sp.ext|<appId>@<tenantId>”
    under “ApplicationId” and “AppAccessContext – ClientAppId” – you’d have an app (client) id of the admin app (permissions provided via)
    ApplicationDisplayName would contain the display name of the admin app
    Other fields: RecordType 205, UserType 5, AuthenticationType OAuth
  • if user id is one of your actual user’s account in tenant – that indicates ACS permissions were provided to the app (e.g. via appinv.aspx page at SharePoint site)
    under “AppId” you’d have an app (client) id of the client app (permissions provided to) in the form of
    “i:0i.t|ms.sp.ext|<appId>@<tenantId>”
    there would be no “ApplicationId” field and under “AppAccessContext” no ClientAppId
    ApplicationDisplayName Unknown
    Other fields: RecordType 205, UserType 0, AuthenticationType FormsCookieAuth

Appregnew.aspx and appinv.aspx

You can pull audit logs with record type is SharePoint and activity type (operation) is PageViewed and keyword for free search is appregnew. You’d get events when there was an attempt to register a new SharePoint app-only service principal.

The same but with appinv as a search keyword – to get events when there was an attempt to provide ACS permissions for a SharePoint app-only service principal or for an Azure App registration.

In both cases we know that there was an intention to have a principal with an ACS access. We can reach these people to inform that ACS is deprecated and so and so. Worst scenario – we notify somebody who already know that.

Some time ago (around mid – 2023) Microsoft by default disabled ability for site owners registering apps and providing permissions in SharePoint via Appregnew.aspx and appinv.aspx. So since then only SharePoint service admins could provide ACS permissions to apps. In this case you’d check with your request tracking system – to whom ACS were provided.

System that tracks request for new ACS permissions

In case you have a process of providing ACS permissions… Process might include tickets to service desk or similar kind of system… Anyway – check if you can get data from that system – like who requested for what app to what site etc…

Reports available at admin center

So far the only report that might help is in development (see Microsoft 365 Roadmap – feature Id 417481) and scheduled to be available in March 2025.

“Enterprise Application Insights is a powerful report which helps SharePoint Administrators to discover all the SharePoint sites that are allowed access by third-party applications registered in your tenant. The report also provides details on the application’s permission and requests count to help admins take further action to strengthen the security of the site. It is part of SharePoint Advanced Management capabilities.”

The feature is already documented here: Generate App insights reports and is seems like the report will not be available for all tenants – but just for tenants with Microsoft SharePoint Premium (SharePoint Advanced Management) or Copilot license assigned.

PnP Microsoft 365 Assessment Tool 

Microsoft 365 Assessment Tool is an utility designed by PnP team a while ago and since then serves SharePoint admins very well. In particular, it helps helps us identify and evaluate the Azure ACS usage for tenant by providing the usage data of ACS principals, and even generating a Power BI reports.

If you run this tool specifying AddInsACS mode, it provides you with:

  • classicacsprincipals report that includes apps with Allow AppOnly permissions.
    Details are: App Ids, if the app has Tenant or Site Collection Scoped Permissions, RedirectUri, AppDomains and ValidUntil
    If the ValidUntil field contains specific date – that means the app was registered via appregnew
    If the ValidUntil field contains “01/01/0001 00:00:00” date – that means the app was registered in EntraId
  • classicacsprincipalsites – sites these apps have access to
    Details are: AppIdentifier, ServerRelativeUrl
  • classicacsprincipalsitescopedpermissions – list of apps permissions to sites
    Details: AppIdentifier, ServerRelativeUrl, SiteId, WebId, ListId, Right (Read/Write/FullControl/Guest etc.)
    If the WebId field equals zeros, that means rights were provided to entire site collection
  • some other details

Unfortunately, this tool does not provide when the app was last time authenticated or when the app accessed the site

I use the following PowerShell to start the tool, get status and export reports:

$tenantDomain = "" # "contoso.sharepoint.com"
$clientid = "" # 
$certThumbprint = ""
$certPath = "My|CurrentUser|" + $certThumbprint

./microsoft365-assessment.exe start --mode AddInsACS --authmode application --tenant $tenantDomain --applicationid $clientid --certpath $certPath

./microsoft365-assessment.exe status

./microsoft365-assessment.exe report --id <report id> --mode CsvOnly --path ".\ACS-reports"

Highly recommended: SharePoint Add-In and Azure ACS Assessment

Report on apps owners and permissions from from Entra Id

Using all the methods above – you’d get a list of active service principals that use legacy ACS authentication. But to whom we need communicate to regarding this service principals? Obviously, we need this service principals owners. There are multiple options how to get an app owner from Azure (Entra Id):


More Observations

Test scenario 1
DisableCustomAppAuthentication is true, i.e. ACS are not allowed in tenant.
SiteOwnerManageLegacyServicePrincipalEnabled -s false, i.e. site owners cannot register apps at sites or provide permissions to app on sites.
It is not possible for admin to go to appregnew.aspx and create an app (app-only spn).
I registered apps in Azure.
It is possible for admin to go to appinv.aspx and “provide” permissions to the azure app registrations.
An app is shown under appprincipals.aspx only in case if ACS access was provided to app but Sites.Selected access was not provided. The moment you provide Sites.Selected access for the app to the site – the app disappears from list of apps under appprincipals.aspx page. It does not help if you remove Sites.Selected permissions.
Connect-PnPOnline works with certificates or with secrets.
Get-PnPSite works only if connection was made with a Certificate (if connection was made with secret – it gives 401 unauthorized).

Test scenario 2
DisableCustomAppAuthentication is false, i.e. ACS are allowed in tenant.
SiteOwnerManageLegacyServicePrincipalEnabled -s false, i.e. site owners cannot register apps at sites or provide permissions to app on sites.
Connect-PnPOnline and Get-PnPSite works with certificates or secrets if ACS access was provided for an app to at least one site.
If there was no ACS permissions provided for the app – Get-PnPSite gives “Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))”

Error messages and possible fixes

Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))” – happens if you try to access SharePoint API with an Entra Id app registration that have an API permissions but do not have legacy ACS permissions being authenticated with a secret.
Solution option 1: try authentication with a certificate.
Solution option 2: use Microsoft Graph API.

The remote server returned an error: (401) Unauthorized.” – happens if you try to access SharePoint API being authenticated with a certificate with an app registration that do not have modern API permissions correctly provided.
Solution: ensure app registration is configured with SharePoint API permissions.

(403) Forbidden” – happens if you try to access SharePoint API being authenticated with a secret with an app registration that do not have modern API permissions correctly provided.
Solution: ensure app registration is configured with Graph API permissions.

AccessDenied”,”Either scp or roles claim need to be present in the token.” – happens if you try to access Graph API being authenticated with a secret with an app registration that do not have modern API permissions correctly provided.
Solution: ensure app registration is configured with Graph API permissions.



References

Search Unified Audit Log Daemon Job

How to run “Search-UnifiedAuditLog” in unattended way, i.e. non-interactive.
What are the minimal permissions required?

The PowerShell code:

$clientId = ""
$cPwd = ConvertTo-SecureString -String "" -AsPlainText -Force

$cPath = ""C:\Users\UserName\Certificates\Cert.pfx""
$organization = "contoso.onmicrosoft.com"

Connect-ExchangeOnline -CertificateFilePath $cPath -CertificatePassword $cPwd -AppID $clientId -Organization $organization

[DateTime]$start = [DateTime]::UtcNow.AddMinutes(-45)
[DateTime]$end = [DateTime]::UtcNow
$resultSize = 1000

$results = $null
$results = Search-UnifiedAuditLog -StartDate $start -EndDate $end -ResultSize $results.Count
$results | Select-Object RecordType, CreationDate, UserIds, Operations -First 3

Disconnect-ExchangeOnline -Confirm:$false

Troubleshooting

The error “The term ‘Search-UnifiedAuditLog’ is not recognized”:

Search-UnifiedAuditLog: C:\scripts\PowerShell.auth\Search-AuditLog-w-App.ps1:16:12
Line |
16 | $results = Search-UnifiedAuditLog -StartDate $start -EndDate $end -Re …
| ~~~~~~
| The term 'Search-UnifiedAuditLog' is not recognized as a name of a cmdlet, function, script file, or executable
| program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

means a proper administrative role (e.g. “Exchange administrator”) is not assigned.

References

Track SharePoint App-only Service Principals in Microsoft 365

Update (Feb 2025):
A new KBA: Azure ACS retirement: Track down ACS apps

Update (May 2023):
You can use Get-PnPAzureACSPrincipal to returns the lists of all Azure ACS principals installed in your Tenant including subsites.

Update: Sites.Selected API MS Graph permissions was introduced by Microsoft in 2021. It was a good move towards site-level development, but still developers were limited with only what MS Graph API provides for SharePoint dev.
So devs had to use AppInv.aspx at site level to provide ACS permissions to their apps to be able to use SharePoint CSOM and REST APIs.
Recently Microsoft introduced Sites.Selected SharePoint API permissions for registered Azure Apps! So now devs should be fully happy without ACS-based permissions.

Update: I’m not sure when Microsoft implemented that, but I recently found another Record Type – “SharePointAppPermissionOperation”. There is an Activity type – AppPermissionGrant…
To be explored…

After all the updates the original article below is probably obsolete…

Scenario

Developers in the organization can use both – Azure Apps and SharePoint Apps to work with SharePoint sites in their “daemon” applications.

It is recommended to use Azure apps so, you want to know – what are SharePoint Apps registered and their owners, who registered SharePoint Apps. Eventually you would disable SharePoint Apps-only principal but before that you’d move Devs from SP-App-only to Azure App (see Disable Custom App Authentication).

(SharePoint App-only service principals aka SP-App-Only are SPN or App registered from within SharePoint using AppRegNew.aspx system page).

One of the approaches is to track Apps/Owners (or who register an app) with Unified Audit Log

Use Unified Audit Logs

The following PowerShell code:

$operations = 'Add service principal.'
$recordType = 'AzureActiveDirectory'
Search-UnifiedAuditLog -StartDate $start -EndDate $end -ResultSize $resultSize -Formatted -Operations $operations -RecordType $recordType

returns events with operation = ‘Add service principal.’ Nice, but…
if an app was registered in Azure – event contains an UPN under UserIds property:

Unfortunately, in case with registering app in SharePoint, an audit log event will be like:

i.e. UserId registered is “spo_service@support.onmicrosoft.com”, so we do not know who registered a SharePoint-only app

In theory – we could use events recorded immediately before and after “Add service principal” event to track a user and site who has registered a SharePoint-only app… But for me it seems like too complicated for automation.

Instead we can do simple search through audit log for events “AppRegNew.aspx page visited”. This gives us a good approximation of who registered SP-App-only principal. Worst scenario – we reach more people than we really need (including those who started registering sp-app-only but did not complete) but all of them would be definitely our target auditory.

Consider the following code:

$freeText = "appregnew"
$operations = 'PageViewed'
$recordType = 'SharePoint'

$results = Search-UnifiedAuditLog -StartDate $start -EndDate $end -ResultSize $resultSize -FreeText $freeText -Operations $operations -RecordType $recordType 

this would give you all users who loaded “/_layouts/15/appregnew.aspx” page

References