Category Archives: Office 365

Office 365 (Microsoft 365) is a set of cloud-based services provided by Microsoft for collaboration, including e-mail, video and audio-calls, chats, social network and collaboration on documents.

Search M365 content from code: use-cases

Why do we need to implement search in our applications?

Use-cases for search on behalf of current user

Along with the usual ones – where you just need your app to search for some data and bring it to user – there is one different scenario I’d like to share:

You need to quickly detect content in SharePoint that is open for everyone

Brute force solution – getting detailed permissions report for all SharePoint sites might not be a feasible option, especially in large environments – it is a very resource-consuming task and might take days and weeks. So consider the following…

Since search is security-trimmed – a user can get only search results he/she already has access to; but what if we create an account and do not grant any SharePoint permissions or group memberships to this account, and then we’d search for everything on behalf of this account? That would mean that all what search returns represent content that is shared with everyone. There are some tricks and gotchas – here is the separate article on the same.

Use-cases for unattended search

What are the use-cases when you need to search in your daemon app or background job? Be aware that when you search on behalf of application credentials – search is NOT security-trimmed and your query would run against ALL SharePoint content… Here are some possible scenarios.

  • Content detection/Investigation
    • Let say you want some data is never shared with anyone and never appeared in search for anyone
    • Or you might want to investigate what is the location some specific data is available at
  • Imagine you are building sites classification system and
    you use indexed custom site properties – so you are able to refine search results based on site metadata to get list of specific sites (adaptive scopes used in retention policy are based on the same mechanics)
  • Automation – let say you have a requirement to configure every tenant site in some ways – for instance – add some hosts to allowed domains to embed video or set some site properties based on who created the site or activate or deactivate some features and so on – how would you do that? You’d probably have a scheduled job that runs let say every hour against only new sites – sites created during that last hour. How would you get these recently created sites? Search with Graph API is the only nice solution today.

Index of other articles on the subject:

  • Search Microsoft 365 content programmatically: Index
  • Search Microsoft 365 content programmatically: Use-case scenarios
  • Authentication to Microsoft Graph: Azure Registered Apps Certificates and Secrets
  • Authorization to Microsoft Graph: Azure Registered Apps API permissions
  • Calling Microsoft Graph Search API from code as current user
  • Calling Microsoft Graph Search API from daemon/service app
  • Using Microsoft.Graph PowerShell module to Search in Microsoft 365
  • Using PnP.PowerShell module to Search in Microsoft 365

Search m365 SharePoint and Teams content programmatically via MS Graph API

In the articles below I’m sharing my techniques on searching in Microsoft 365 SharePoint and Teams from application using Microsoft Graph API.
Specifically I’m covering

  • Microsoft Graph search API
  • Microsoft.Graph PowerShell module
  • PnP.PowerShell module

In two flavors:

  • Search on behalf of currently authenticated user
  • Unattended Search with daemon (also called service) applications

Index of my articles on the subject:

Video tutorials

Search through Microsoft 365 SharePoint from code

Below is how do I search Microsoft 365 content programmatically from PowerShell using MS Graph API, PowerShell PnP, Microsoft Graph module, MSAL library being authenticated as user or daemon application. Let me focus on SharePoint content here but you can use the same technique to search through other Microsoft 365 services. Also, I’ll be using PowerShell but same ideas should work for other platforms/languages – Python, C#, node.js etc.

First, we need to be authenticated

Here is how to authenticate to Microsoft 365 Graph API.

Second, we need to be authorized

To search on behalf of currently authenticated user we need delegated “Sites.Read.All” API permissions. I recommend you to add both Graph API and SharePoint API permissions as different libraries might use different API’s under the hood. Ensure you add delegated “Sites.Read.All” even if you already have “Sites.FullControl.All” as by some reason “Sites.FullControl.All” does not always work for search.

Here is how an app API permissions to search as current user should look like:

app API permissions to search as current user

Actually, for interactive authentication app ownership is not required, so we can surely use our own registered app, but also it is possible to use any other app registered in Azure and properly configured, e.g. Enterprise “PnP Management Shell” app id: “31359c7f-bd7e-475c-86db-fdb8c937548e”
How do I create and configure Azure App with delegated permissions to SharePoint for PowerShel usage

For unattended search – e.g. search on behalf of daemon app – we need application “Sites.Read.All” API permissions. Again, I suggest both Graph API and SharePoint API permissions added. Here is how an app API permissions to search as daemon app should look like:

Ensure you got admin consent for API permissions.

In case you have incorrect permissions in your app – Microsoft Graph will be kind enough to inform you exactly what you need. Example:

“Access to ChatMessage in Graph API requires the following permissions: Chat.Read or Chat.ReadWrite, ChannelMessage.Read.All. However, the application only has the following permissions granted: Sites.Read.All, User.Read”

Assuming we have configured apps – let us get started with

Microsoft Graph API

Microsoft Graph API allows search through all the Microsoft 365 content – including Exchange e-mail messages, Yammer (Viva Engage) and Teams chat messages and surely OneDrive and SharePoint content (please refer to the original doc).

Authenticate as current user to Search with Graph API

I use MSAL.PS PowerShell module to get token, then I build a headers variable

# Prerequisites
Get-Module MSAL.PS -ListAvailable | ft name, Version, Path 
# Install-Module MSAL.PS -Force -Scope CurrentUser -AcceptLicense
Import-Module MSAL.PS

# Interactive Authentication
$clientid = 'd82858e0-ed99-424f-a00f-cef64125e49c'
$TenantId = '7ddc7314-9f01-45d5-b012-71665bb1c544'
$token = Get-MsalToken -TenantId $TenantId -ClientId $clientid -Interactive
$headers = @{Authorization = "Bearer $($token.AccessToken)" }

Authenticate as service/daemon app

You’d need to update the script providing Tenant id, client (app) id and client (app) secret:

# App Authentication
$clientID = ""
$clientSc = ""
$TenantId = ""

# Construct URI and body needed for authentication
$uri = "$tenantId/oauth2/v2.0/token"
$body = @{
    client_id     = $clientID
    client_secret = $clientSc
    scope         = ""
    grant_type    = "client_credentials" 

# Get OAuth 2.0 Token
$tokenRequest = Invoke-WebRequest -Method Post -Uri $uri -ContentType "application/x-www-form-urlencoded" -Body $body -UseBasicParsing
$token = ($tokenRequest.Content | ConvertFrom-Json).access_token
$headers = @{Authorization = "Bearer $token" }

Search m365 SharePoint and OD content with Microsoft Graph API

In this sample I limited search scope to list items only ($entityTypes = “[‘listItem’]”). Check other entity types here.


# Search
$entityTypes = "['listItem']"
$apiUrl = ""
$query = "*"
# body for interactive search
$body = @"
  "requests": [
      "entityTypes": $entityTypes,
      "query": {
        "queryString": "$query"

$res = Invoke-RestMethod -Headers $Headers -Uri $apiUrl -Body $Body -Method Post -ContentType 'application/json'

If you are getting error message “SearchRequest Invalid (Region is required when request with application permission.)”:

that’s OK, just modify your body to include region like this (“region”: “NAM” for North America or “GBR” or …). Also, I can modify body with from/size for paging (technique used to iterate through search results if there are many) and return just specific fields to decrease traffic and improve performance:

# Search
$entityTypes = "['driveItem','listItem','list','drive','site']"
$entityTypes = "['driveItem','listItem']"

$query = "LastModifiedTimeForRetention<2021-01-01"
$apiUrl = ""
$query = "test*"
$body = @"
  "requests": [
      "entityTypes": $entityTypes,
      "query": {
        "queryString": "$query"
      "from" : 0,
      "size" : 5,
      "fields": ["WebUrl","lastModifiedBy","name" ],
      "region": "NAM"

$res = Invoke-RestMethod -Headers $Headers -Uri $apiUrl -Body $Body -Method Post -ContentType 'application/json'

We’d not use region for interactive calls or we’ll get “Region is not supported when request with delegated permission.”.

Microsoft.Graph PowerShell module

There is a Microsoft.Graph PowerShell module provided by Microsoft which simplifies authentication and search operations.

Interactive authentication code sample:

# Prerequisites
Get-Module Microsoft.Graph.Authentication -ListAvailable 
Get-Module Microsoft.Graph.Search -ListAvailable 

# Interactive Authentication
$clientid = '31359c7f-bd7e-475c-86db-fdb8c937548e'
$clientid = 'd82858e0-ed99-424f-a00f-cef64125e49c'
$TenantId = '7ddc7314-9f01-45d5-b012-71665bb1c544'
Connect-MgGraph -ClientId $clientid -TenantId $TenantId

For daemon app authentication we need a certificate configured in Azure App and installed on the user machine. Daemon app authentication code sample (please specify your tenant id, app (client) id and certificate thumbprint:

# App Authentication
$clientID = ""
$certThumbprint = ""
$TenantId = ""
Connect-MgGraph -ClientId $clientid -TenantId $TenantId -CertificateThumbprint $certThumbprint

Code sample for SharePoint search with Microsoft.Graph PowerShell module

As currently authenticated user

# Search in the current user context
$params = @{
	requests = @(
			entityTypes = @(
			query = @{
				queryString = "lorem"
			from = 0
			size = 25
			fields = @(

$res = Invoke-MgQuerySearch -Body $params

Again, in case with app authentication – an additional parameter – region – is required:

# Search
$params = @{
	requests = @(
			entityTypes = @(
			query = @{
				queryString = "lorem"
			from = 0
			size = 25
			fields = @(
                        region = "NAM"

$res = Invoke-MgQuerySearch -Body $params

It’s a good idea to explore returning object.

PnP.PowerShell module

PnP.PowerShell allows you to search through Microsoft 365 SharePoint content with PowerShell style – using command and options.

Let us authenticate interactively:

# Interactive Authentication
$clientid = 'd82858e0-ed99-424f-a00f-cef64125e49c'
$TenantId = '7ddc7314-9f01-45d5-b012-71665bb1c544'
$siteUrl = ""
Connect-PnPOnline -ClientId $clientid -Tenant $TenantId -Url $siteUrl -Interactive

Authentication on behalf of service/daemon app would require certificate installed on the machine and configured in the app and look like:

# Application (daemon) Authentication
$clientID = ""
$certThumbprint = ""
$TenantId = ""
$siteUrl = ""
Connect-PnPOnline -ClientId $clientid -Tenant $TenantId -Url $siteUrl -Thumbprint $certThumbprint

There are no differences in Microsoft 365 SharePoint Search with PowerShell code samples for interactive and daemon apps (no region parameter).


# search
$query = "test*"
$res = Submit-PnPSearchQuery -Query $query 

# examples of query:
$query = "test*"
$query = "* contentclass:STS_ListItem_DocumentLibrary"
$query = "* author:Patti"
$query = "test* site:"

# examples of submitting request
$res = Submit-PnPSearchQuery -Query $query 
Submit-PnPSearchQuery -Query $query -All 
Submit-PnPSearchQuery -Query $query -MaxResults 5
Submit-PnPSearchQuery -Query $query -SortList @{"LastModifiedTime" = "ascending"} 

# exploring result object:

Video tutorials

Video tutorials (playlist) on how to authenticate to Microsoft 365 and Search through Microsoft 365 content from code


SharePoint Add-Ins and ASC retirement

Microsoft announced retirement for SharePoint Add-Ins and ASC-based app permissions (SharePoint app-only principals). Let me summarize here:

  • SharePoint Add-Ins, ASC-based app permissions and SharePoint app-only principals
  • Timeline of retirement and future plans
  • How to detect if there are legacy service principals used in the environment
  • Migration guidance


Known key retirement milestone dates:

  • Mar 2024 – Microsoft will stop accepting new Add-In submission to app store
  • Jul 2024 – Microsoft will stop ability for customers to acquire add-ins from App Store (*)
  • using ACS and Add-ins will be blocked for new tenants since Nov 2024
  • using ACS and Add-ins will be blocked for all tenants since Apr 2026

(*) SPFx based solutions will continue to be available

So timeline is generous, and we have plenty of time to

Detect if there are any legacy service principals used in the environment


Microsoft Azure Data Factory connect to SharePoint

Below is the my guide on how to connect Azure Data Factory to SharePoint and how to deal with connection error code 23201 “Failed to get metadata of odata service, please check if service url and credential is correct and your application has permission to the resource


You are configuring Azure Data Factory pipeline. You want to connect to SharePoint List as a data source.

To establish connection to SharePoint site you need to provide
Site Url, tenant Id, service principal Id and service principal key:

Service principal

Service principal here could be

  • SharePoint app-only service principal registered at SharePoint site via appregnew.aspx
  • Azure registered app (app registered in EntraId)

In both cases you get “service principal” – which is App Id or Client Id and “service principal key” which is app secret (client secret).

Note: in Sep 2023 Microsoft implemented update to all Microsoft 365 tenants. According to the update, by default only tenant administrators can create or update ACS service principal by default. If site collection admin starting from Oct 2023 can register SharePoint app-only spn via appregnew.aspx or provide ACS-based permissions via appinv.aspx – that means tenant admins switched this back.

So, if registering a new SharePoint app-only service principal is still available for your tenant – you can get service principal Id and key from SharePoint via appregnew and/or provide ACS-based permissions via appinv – and there should be no problem connecting to SPO list from ADF (June 2024), but please review special note below.

If a site collection administrator or owner tries to register app in SharePoint with appregnew.aspx or provide permissions to the app with appinv.aspx – and he/she gets:
Your SharePoint tenant admin doesn’t allow site collection admins to create (update) app permissions. Please contact your SharePoint administrator:

Your SharePoint tenant admin doesn’t allow site collection admins to create an Azure Access Control (ACS) principal

that means registering service principal in SharePoint is disabled. In this case Microsoft recommend using Azure application registration – with MS Graph API Sites.Selected and SharePoint Sites.Selected API permissions configured, consented by tenant admin and with access to specific SharePoint site provided by SharePoint admins (refer to this article for more details).


If you obtained service principal Id and key as Azure Registered App – connection to SharePoint site from Azure Data Factory does not always works. When you configure connection and click test – you might get an error:

error code: 23201

error details:

Failed to get metadata of odata service, please check if service url and credential is correct and your application has permission to the resource. Expected status code: 200, actual status code: BadRequest, response is : 
    <title>Request Error</title>
      <p class="heading1">Request Error</p>
      <p>The server encountered an error processing the request. See server logs for more details.</p>


So the issue above is a combination of two controversial circumstances:

  • Microsoft discourages using SharePoint app-only service principals and disabled ability for site owners to register SharePoint app-only service principals and provide ACS-based permissions in favor of Azure Registerd Apps with Sites.Selected based permissions.
  • Azure Data Factory still require ACS-based permissions (Upd: June 2024 – still true)


If Microsoft disabled ability for site owners to to provide ACS-based permissions for the app – that does not mean it is fully disabled. It turns out – SharePoint admins are still able to register SharePoint app-only principals and provide ACS-based permissions.

The recommended steps are:

  1. register Application in Azure (in EntraId, not in SharePoint) to get App (client) Id
    this could be done by user from Azure App Registrations (or, if this ability is disabled by tenant admin – there must be a way for users to request an application registered in Azure)
  2. provide to this App Id ACS-based permissions at the target SharePoint site via appinv.aspx – this is done by a person who got at the same time two roles
    • SharePoint admin role enabled and
    • The specific site collection administrator permissions
      so if your role is a regular user or developer (not an admin) – you’d request this service from your admins
  3. provide Sites.Selected permissions for the App to the target Site
    again, this is something you’d need to request from your admins –
    tenant admin should be able to provide admin consent to SharePoint and Graph API Sites.Selected permissions for your app and SharePoint admin should be able to provide actual permissions for the app to the site.

Technical Details

(as per June 2024 – still true) Actually, the only you need is to provide any ACS-based access for the application. Even to another site, web or list. You can also remove this just provided ACS-based access.
It seems like the moment you click “Trust” when you provide access via AppInv.aspx – something is triggered in Microsoft Identity Management token issuing mechanics so Azure Data factory connection starts working (assuming Sites.Selected access was provides).

Surely connection will work if you provide only ACS-based permissions (with no Sites.Selected permissions), but this is what we all want to avoid by any means.

More fun! Connection to entire site will start working even if you provide SharePoint app-only (ACS-based) permissions to some specific list. Though later, when you try to ingest data – you will be able to ingest only this list data.

Environment this is tested:

Powershell module used to enable/disable
16.0.24120.12000 Microsoft.Online.SharePoint.PowerShell

SiteOwnerManageLegacyServicePrincipalEnabled = False (almost always)
SiteOwnerManageLegacyServicePrincipalEnabled = True (when provide ACS)

LegacyAuthProtocolsEnabled = True
LegacyBrowserAuthProtocolsEnabled = True
BlockAppAccessWithAuthenticationContext = False
DisableCustomAppAuthentication = False

About Sites.Selected

Microsoft implemented Sites.Selected API permissions for Azure registered apps in 2021-2022 as a preferred way to access specific SharePoint site with application credentials. Microsoft recommend using Azure registered apps instead of SharePoint App-Only service principals and “softly” push developers toward Azure registered apps. Microsoft recently (Aug-Sep 2023) implemented an update and pushed it to all existing Microsoft 365 tenants – so that ability for site admins to register service principals at sites is turned off by default.

So starting Aug-Sep 2023 site owners/admins cannot register and provide ACS-based permissions for apps to their SharePoint sites.

In 2024 Microsoft announced EOL for SharePoint app-only spns and ACS-based permissions.

Special note

This article is written in 2023 with the sole purpose to help you resolve the issue. And it all is still true (validated in June 2024). But! I assume that sooner or later (before April 2026) Microsoft will address it’s own issue and update Azure Data Factory so ADF will be accepting permissions provided via Sites.Selected only. That is why – at the moment – I strictly recommend:

  1. Use Azure Registered App (not a SharePoint app-only spn)
  2. Get both types of permissions for this app:
    • modern – Sites.Selected SharePoint and Graph API permissions
    • old/classic – ACS-based permissions

if so – your data pipeline should continue working smoothly when MS implement modern authentication in ADF.



  • How to use KeyVault with Azure Data Factory connection to SharePoint
  • How to configure REST connection to SharePoint

Microsoft 365 admin center: Manage ownerless Microsoft 365 groups and teams

There is a new feature published at Microsoft roadmap site:

Microsoft 365 admin center: Manage ownerless Microsoft 365 groups and teams

Teams, Outlook groups, Team Sites etc. powered by Microsoft 365 Groups supports two roles: members and owners. Members can collaborate with others in the group through files, emails, messages etc. Owners manage the group membership and monitor content and conversations. When employees leave an organization or switch projects internally, it results in their existing user accounts getting deleted. If such employees were group owners, keeping track of their groups becomes critical to ensure accountability within the organization. We have introduced a new ownership governance policy to help automate the management of ownerless groups by requesting active members to become owners of the group. Admins can define who is eligible for these notifications and configure what notifications and how often these notifications are sent to active group members. Users, who are members of the ownerless groups can simply accept or decline request via the actionable email message.

  • Feature ID: 180749
  • Added to roadmap: 10/10/2023
  • Last modified: 10/10/2023
  • Product(s): Microsoft 365 Admin Center
  • Cloud instance(s): GCC
  • Platform(s): Web
  • Release phase(s): General Availability

But based on the feature description – all looks exactly as what we already have for years as “Microsoft 365 ownerless groups policy” which you can configure under Microsoft 365 Admin Center -> Settings -> Org settings -> Microsoft 365 groups

More on Microsoft 365 ownerless groups

Massive Microsoft 365 groups update with PowerShell

What if you need to bulk update Microsoft 365 groups membership e.g. to add a group owner or member for tens of thousands m365 groups? Iterating through groups one-by-one is unproductive and could take days. Can we do it faster? Here is what I found.

In my case, it was Microsoft 365 ownerless groups policy implementation for large tenant… Skipping details – I needed to update ownership for 10,000 Microsoft 365 groups and I was looking for a best/fastest possible option maybe some kind of bulk update or with multiple threads. And I figured out that the fastest way is to use PnP.PowerShell that calls Microsoft Graph API but run it against list of groups with PowerShell parallel trick. Here is the sample PowerShell code:

$groups | ForEach-Object -Parallel {
    $owner = ""
    Add-PnPMicrosoft365GroupOwner -Identity $_.Id -Users $owner
} -ThrottleLimit 50

That worked for me perfectly and it took ~8 seconds per 1,000 groups.

Sites.Selected permissions provisioning automation


You administer Microsoft 365 SharePoint Online. Part of your daily activities is providing Microsoft Graph and SharePoint Sites.Selected API permissions to other users (developers).

In Aug/Sep 2023 Microsoft pushed an update that prevents site collection admins to create or update an Azure Access Control (ACS) principal (that was the way most of developers used to get Client Id and Client secret to access SharePoint site). So your users are probably getting something like Your SharePoint tenant admin doesn’t allow site collection admins to create or update an Azure Access Control (ACS) principal message attempting to create or update SharePoint App-only principal at AppRegNew.aspx or AppInv.aspx pages. Here are more details on the issue.

Microsoft and MVPs shared some technique how to provide Sites.Selected API permissions, but dealing with scripts manually, elevating individual permissions every time you need to run the script – it all takes time and not very efficient. More and more devs are reaching you on the app. So you want to automate this process.


Solution architecture

My way to automate it includes:

  • SharePoint list as a frontend
    here you can accept intake requests, organize approval workflow and display automation results
  • Azure Function App as a backend
    here will be your PowerShell script hosted that runs on scheduled basis and takes care of actual permissions provisioning

Solution details

High-level, getting application permissions to some specific SharePoint site is a two-step process:

  1. get application registration in Azure and properly configure it
  2. get permissions for this application to a specific SharePoint site

For the first step – check this and this articles. I’ll focus on the second step below.

You can provide Sites.Selected permissions for the app to a site with

I will be using second one one. Also PnP.PowerShell will be used to get access to SharePoint intake site and read/update requests from SharePoint list and so on.

Azure App Registration

I registered an admin Application in Azure – “SharePoint Automation App”, added Graph Sites.FullControl.All and SharePoint Sites.FullControl.All permissions, then added Microsoft Graph Directory.Read.All permissions and got tenant admin consent:

I generated a self-signed certificate and added it to the app:

This app will be used to call provide permissions, and to connect to the SharePoint front-end.

Users will register their applications in Azure, add Graph Sites.Selected and SharePoint Sites.Selected permissions, got tenant admin consent, then request permissions to the specific site by creating an intake request – new list item.

Front-End SharePoint Site

I created a SharePoint site for automation. This site will play a front-end role for users. I created a list “Sites.Selected” and updated list columns so I have the following fields:

  • Target Site Url
  • Application Id
  • Permissions (read/write)
  • Automation Output

In real-world (Prod) – You can (should) also implement approval workflow as you’d provide permissions for the application to the site only with this site owner approval. The PowerShell code behind should also validate site owner’s consent with app access to site. But for the sake of simplicity I’ll skip this in my demo.

Azure Function App

I created an Azure Function App with the following parameters:
– Runtime stack: PowerShell Core
– Version: 7.2.
– OS: Windows
– Hosting plan: Consumption

And then PowerShell timer-triggered function in Visual Studio Code.

Function requirements.psd1 (it takes a few hours for Azure to install modules; while modules are installing – you might see “[Warning] The first managed dependency download is in progress, function execution will continue when it’s done. Depending on the content of requirements.psd1, this can take a few minutes. Subsequent function executions will not block and updates will be performed in the background.”):

    'Az' = '10.*'
    'PnP.PowerShell' = '2.*'

Azure Az module to access other Azure resources. PnP.PowerShell module will be used to access SharePoint.

I will keep my admin Azure registered app in a key vault, so need somehow to let the key vault know that this specific app can access this specific credentials. So I enabled system assigned managed Identity for the Function App:

MS: “This resource is registered with Azure Active Directory. The managed identity can be configured to allow access to other resources…”.
I’m going to use an object (principal) Id of this function to grant access to keyvault.

Azure key vault

Surely we do not hard-code app secrets. So we need a key vault o store app credentials.

I created a key vault under the same resource group in Azure and named it “SharePointAutomationDemo”. Then I added a roles assignment – “Key Vault Secret User” and “Key vault Reader” to the Function App via it’s managed identity:

I also assigned “Key Vault Administrator” role to the user (developer) who will add certificates/secrets to this key vault and develop Azure function code.

Code repository


Part 1: Getting Azure App Registration with Sites.Selected API Permissions

Part 2: SharePoint and Microsoft Graph API Sites.Selected permissions provisioning automation