Category Archives: SharePoint

Calling Microsoft Graph API from Python

Below is how I authenticate and call Microsoft Graph API to work with SharePoint from Python application.

Plain

no MSAL or Azure libraries used:

import requests
import json
from secrets import clientSc 

clientId = "7e60c372-ec15-4069-a4df-0ab47912da46"
# clientSc = "<imported>" 
tenantId = "7ddc7314-9f01-45d5-b012-71665bb1c544"

apiUri = "https://login.microsoftonline.com/" + tenantId + "/oauth2/v2.0/token"

body = {
    "client_id"     : clientId,
    "client_secret" : clientSc,
    "scope"         : "https://graph.microsoft.com/.default",
    "grant_type"    : "client_credentials" 
}

response = requests.post(apiUri, data=body)
token = json.loads(response.content)["access_token"]

graph_url = 'https://graph.microsoft.com/v1.0/sites/root'
site = requests.get(
    graph_url,
    headers={'Authorization': 'Bearer {0}'.format(token)}
)

print(site.content)
print(json.loads(site.content)["webUrl"])

secrets is a Python file where I assign client secret to variable clientSc (so my secret is not shared on github). This is ok for demo purposes but generally, you should not hard-code secrets but keep secrets somewhere safe (Vault).

MSAL

Using MSAL library to get bearer token:
https://github.com/VladilenK/m365-with-Python/tree/main/Graph-API-MSAL

References

Using Microsoft.Graph PowerShell to Search in Microsoft 365

There is a Microsoft.Graph PowerShell module provided by Microsoft which simplifies usage of Microsoft Graph API. Below is how to authenticate to MS Graph and how to search within SharePoint and Teams Microsoft 365 content using Microsoft.Graph PowerShell module.

Authentication

Interactive authentication code sample:

# Prerequisites
Get-Module Microsoft.Graph.Authentication -ListAvailable 
Get-Module Microsoft.Graph.Search -ListAvailable 

# Interactive Authentication
$clientid = '31359c7f-bd7e-475c-86db-fdb8c937548e'
$clientid = 'd82858e0-ed99-424f-a00f-cef64125e49c'
$TenantId = '7ddc7314-9f01-45d5-b012-71665bb1c544'
Connect-MgGraph -ClientId $clientid -TenantId $TenantId

For daemon app authentication we need a certificate configured in Azure App and installed on the user machine. Daemon app authentication code sample (please specify your tenant id, app (client) id and certificate thumbprint:

# App Authentication
$TenantId = ""
$clientID = ""
$certThumbprint = ""
Connect-MgGraph -ClientId $clientid -TenantId $TenantId -CertificateThumbprint $certThumbprint

Search with Microsoft.Graph

# Search
$params = @{
	requests = @(
		@{
			entityTypes = @(
				"driveItem"
			)
			query = @{
				queryString = "test*"
			}
			from = 0
			size = 50
			fields = @(
				"title"
				"description"
			)
                        region = "NAM"
		}
	)
}

$res = Invoke-MgQuerySearch -Body $params
$res.HitsContainers[0].Hits

Note: when you are calling MS Graph Search API authenticated as user – you need to remove “region” parameter.

Code samples: https://github.com/VladilenK/m365-PowerShell/tree/main/KBA/Search

Search Microsoft 365 content programmatically: all articles index

Video tutorial:

Providing ACS permissions for app to access SharePoint

Microsoft retires ACS

Let me quote Microsoft just to start (Dec 18, 2023):

  • “SharePoint App-Only is the older, but still very relevant, model of setting up app-principals.”
  • “… we will be retiring the use of Azure ACS (Access Control Services) for SharePoint Online auth needs and believe Microsoft 365 customers will be better served by modern auth…”
  • “Azure ACS will stop working for new tenants as of November 1st, 2024 and it will stop working for existing tenants and will be fully retired as of April 2nd, 2026…
    There will not be an option to extend using Azure ACS with SharePoint Online beyond April 2nd, 2026″
  • “… we recommend switching those applications to use Microsoft Entra ID for authorization and authentication needs…”

So, for new development it is strictly recommended to use Azure Registered Apps to access Microsoft 365 resources programmatically.

You still need ACS in some cases

But, as always, it all is not so simple, as

  • there are still plenty of 3-rd party applications written and used widely that require ACS-based permissions. Moreover, there are still some 1-st party applications (Microsoft apps and services) that require ACS-based permissions
  • though Microsoft Graph API is good and provide a lot of functionality and is developing rapidly, it cannot cover all SharePoint dev’s needs, so using SharePoint REST API could be unavoidable… and that is where some complications are coming
  • permissions to specific SharePoint sites (not to all tenant sites, but to one or several SharePoint sites in tenant) for apps is done via Sites.Selected, but this works to entire site collection only. E.g. via Sites.Selected you cannot provide granular permissions (e.g. to specific list) for an app, which might be crucial in some cases, so you’d still have to use ACS-based permissions

Hopefully, Microsoft will resolve all the issues above before April 2026… But for now we have to live with both – Azure Registered applications and API permissions configured in Entra ID and with SharePoint app-only service principals and ACS-based permissions.

Azure Apps and Entra Id vs SharePoint app-only spn and ACS

Comparison between Azure Apps and Entra Id API permissions vs SharePoint app-only spn and ACS-based permissions

ACS-based SharePoint app/permissionsApps registered in Azure with Sites.Selected API permissions
support authentication with client secret only, secret is valid for 1 year exactlysupport authentication with client secret and/or certificate, custom expiration time
support granular access to SharePoint site content – e.g. to entire site collection or web (subsite) or a specific listsupport only access to entire site collection (but Microsoft is working on granular access)
support only classic SharePoint REST API and CSOMsupport both – classic SharePoint REST API and CSOM and Microsoft Graph API
app id (client id) is created via appregnew.aspx at a specific SharePoint site by site collection administratorapp id (client id) is created in Azure portal, API Sites.Selected permissions are configured via Azure portal and require tenant admin consent
permissions for the app to a site are provided at the site by site collection administrator via appinv.aspx pagepermissions for the App to to a specific SharePoint site are provided by SharePoint admin with PowerShell script or Graph API calls
logging audit log

SharePoint app-only service principal and ACS-based permissions

Since SharePoint app-only service principals and ACS-based permissions were introduced for SharePoint 2013 as part of Add-Ins feature – there are plenty of articles from Microsoft and MVPs and SharePoint gurus on this. But I would like to highlight one thing:

  • AppRegNew page creates service principal and allows authentication
  • AppInv page provides permissions and allows authorization to SharePoint

Check SharePoint AppRegNew.aspx and AppInv.aspx for details

Recommended transition tactics

WIP…

References

Using SharePoint REST API from Python code

Using Microsoft Graph API is a preferred and recommended way to connect to SharePoint Online and other Microsoft 365 resources programmatically from Python code. But if by some reason you are required to use classic SharePoint REST API, here is how it is done.

Prerequisites:

  • Azure Registered Application
    you must register your application in Azure and configure it properly
    • Authentication blade must be configured for authenticate as current user
    • Certificates and/or secrets must be configured for daemon app (unattended access)
  • API permissions
    your Azure app registration must have API permissions configured based on resources you need access to and authentication method chosen
    here is how to configure API permissions for your app
  • Office365-REST-Python-Client library installed

Using client Id and client secret to access SharePoint REST API from Python

Errors

Possible errors and diagnostic messages

HTTPError: 401 Client Error: Unauthorized for url…
The provided client secret keys for app … are expired. Visit the Azure portal to create new keys for your app or consider using certificate credentials for added security

Code samples at GitHub

WIP…

References

Search in SharePoint using Microsoft Graph API with application credentials

Microsoft Graph API allows you to work with all the Microsoft 365 content – including search through Exchange e-mail messages, Yammer (Viva Engage) and Teams chat messages and surely OneDrive and SharePoint content (please refer to the original doc). Let me focus on searching in SharePoint Online and OD here but you can use the same technique to search through other Microsoft 365 services. I will use PowerShell but same ideas should work for other platforms/languages – Python, C#, node.js etc.

Assuming we have a registered Azure app configured correctly, including Secrets/Certificates blade and API permissions provided – we should be ready to authenticate and call Graph API unattended – on behalf of application itself.

Let us authenticate as a service/daemon app with client id and client secret:

# Authenticate to M365 as an unattended application

# specify your app id. app secret, tenant id:
$clientID = ""
$clientSc = ""
$TenantId = ""

# Construct URI and body needed for authentication
$uri = "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token"
$body = @{
    client_id     = $clientID
    client_secret = $clientSc
    scope         = "https://graph.microsoft.com/.default"
    grant_type    = "client_credentials" 
}

# Get OAuth 2.0 Token
$tokenRequest = Invoke-WebRequest -Method Post -Uri $uri -ContentType "application/x-www-form-urlencoded" -Body $body -UseBasicParsing
$token = ($tokenRequest.Content | ConvertFrom-Json).access_token
$headers = @{Authorization = "Bearer $token" }

Below is how I search Microsoft 365 content programmatically from PowerShell using MS Graph API being authenticates as user.

# Search
$entityTypes = "['driveItem','listItem','list','drive','site']"
$entityTypes = "['driveItem','listItem']"

$query = "LastModifiedTimeForRetention<2021-01-01"
$apiUrl = "https://graph.microsoft.com/beta/search/query"
$query = "test*"
$body = @"
{ 
  "requests": [
    {
      "entityTypes": $entityTypes,
      "query": {
        "queryString": "$query"
      },
      "from" : 0,
      "size" : 5,
      "fields": ["WebUrl","lastModifiedBy","name" ],
      "region": "NAM"
    }
  ]
}
"@

$res = Invoke-RestMethod -Headers $Headers -Uri $apiUrl -Body $Body -Method Post -ContentType 'application/json'
$res.value[0].searchTerms
$res.value[0].hitsContainers[0].hits
$res.value[0].hitsContainers[0].hits.Count
$res.value[0].hitsContainers[0].moreResultsAvailable

Notice we use “region” – it is required to search with Graph API under application credentials. Otherwise you will get an error message “SearchRequest Invalid (Region is required when request with application permission.)”:

Parameter “fields” allows you to request only fields you need to be returned. As returning object will be smaller your request will perform faster.

There might be a big number of objects found in m365 upon your request. Graph will not always return to you all the results. AFAIK currently the limit is 500, so if there are more than 500 objects found – only first 500 will be returned. You can specify how many objects you need to be returned per call with “size” parameter.

You can check value of $res.value[0].hitsContainers[0].moreResultsAvailable property and if it’s True – that means there are more results. The value above and parameters “from” and “size” would allow you to organize a loop so you can call search API many times to return all results.

Other articles index:
Search m365 SharePoint and Teams content programmatically via MS Graph API

Using Microsoft Graph Search API: user context

Assuming we have a registered Azure app configured correctly, including Authentication and API permissions provided – we should be ready to authenticate and call Graph API on behalf of a user.

Microsoft Graph API allows you to work with all the Microsoft 365 content – including search through Exchange e-mail messages, Yammer (Viva Engage) and Teams chat messages and surely OneDrive and SharePoint content (please refer to the original doc).

Let me focus on searching in SharePoint Online and OD here but you can use the same technique to search through other Microsoft 365 services. I will use PowerShell but same ideas should work for other platforms/languages – Python, C#, node.js etc.

Let us authenticate first. We’d need a MSAL.PS module for that.

# Ensure we have MSAL.PS module installed
Get-Module MSAL.PS -ListAvailable | ft name, Version, Path 
# Install-Module MSAL.PS -Force -Scope CurrentUser -AcceptLicense
Import-Module MSAL.PS

# Authenticate to Microsoft Interactively 
$clientid = 'd82858e0-ed99-424f-a00f-cef64125e49c' # your client id
$TenantId = '7ddc7314-9f01-45d5-b012-71665bb1c544' # your tenant id
$token = Get-MsalToken -TenantId $TenantId -ClientId $clientid -Interactive
$headers = @{Authorization = "Bearer $($token.AccessToken)" }

Below is how I search Microsoft 365 content programmatically from PowerShell using MS Graph API being authenticates as user.

# Search
# MS Graph Search API url (beta or v1.0):
$apiUrl = "https://graph.microsoft.com/beta/search/query"

# specify where to search - entity types
$entityTypes = "['driveItem','listItem','list','drive','site']"
$entityTypes = "['driveItem','listItem']"

# query
$query = "test*"

# build a simple request body
$body = @"
{ 
  "requests": [
    {
      "entityTypes": $entityTypes,
      "query": {
        "queryString": "$query"
      }
    }
  ]
}
"@

# call Graph API:
$res = Invoke-RestMethod -Headers $Headers -Uri $apiUrl -Body $Body -Method Post -ContentType 'application/json'

# explore returned object
$res.value[0].searchTerms
$res.value[0].hitsContainers[0].hits
$res.value[0].hitsContainers[0].hits.Count
$res.value[0].hitsContainers[0].moreResultsAvailable

References

Search M365 content from code: use-cases

Why do we need to implement search in our applications?

Use-cases for search on behalf of current user

Along with the usual ones – where you just need your app to search for some data and bring it to user – there is one different scenario I’d like to share:

You need to quickly detect content in SharePoint that is open for everyone

Brute force solution – getting detailed permissions report for all SharePoint sites might not be a feasible option, especially in large environments – it is a very resource-consuming task and might take days and weeks. So consider the following…

Since search is security-trimmed – a user can get only search results he/she already has access to; but what if we create an account and do not grant any SharePoint permissions or group memberships to this account, and then we’d search for everything on behalf of this account? That would mean that all what search returns represent content that is shared with everyone. There are some tricks and gotchas – here is the separate article on the same.

Use-cases for unattended search

What are the use-cases when you need to search in your daemon app or background job? Be aware that when you search on behalf of application credentials – search is NOT security-trimmed and your query would run against ALL SharePoint content… Here are some possible scenarios.

  • Content detection/Investigation
    • Let say you want some data is never shared with anyone and never appeared in search for anyone
    • Or you might want to investigate what is the location some specific data is available at
  • Imagine you are building sites classification system and
    you use indexed custom site properties – so you are able to refine search results based on site metadata to get list of specific sites (adaptive scopes used in retention policy are based on the same mechanics)
  • Automation – let say you have a requirement to configure every tenant site in some ways – for instance – add some hosts to allowed domains to embed video or set some site properties based on who created the site or activate or deactivate some features and so on – how would you do that? You’d probably have a scheduled job that runs let say every hour against only new sites – sites created during that last hour. How would you get these recently created sites? Search with Graph API is the only nice solution today.

Index of other articles on the subject:

  • Search Microsoft 365 content programmatically: Index
  • Search Microsoft 365 content programmatically: Use-case scenarios
  • Authentication to Microsoft Graph: Azure Registered Apps Certificates and Secrets
  • Authorization to Microsoft Graph: Azure Registered Apps API permissions
  • Calling Microsoft Graph Search API from code as current user
  • Calling Microsoft Graph Search API from daemon/service app
  • Using Microsoft.Graph PowerShell module to Search in Microsoft 365
  • Using PnP.PowerShell module to Search in Microsoft 365

Search m365 SharePoint and Teams content programmatically via MS Graph API

In the articles below I’m sharing my techniques on searching in Microsoft 365 SharePoint and Teams from application using Microsoft Graph API.
Specifically I’m covering

  • Microsoft Graph search API
  • Microsoft.Graph PowerShell module
  • PnP.PowerShell module

In two flavors:

  • Search on behalf of currently authenticated user
  • Unattended Search with daemon (also called service) applications

Index of my articles on the subject:

Video tutorials