Tag Archives: PowerShell

Connecting to Microsoft 365 SharePoint and Graph API from Azure Function App

Let say you need to run some code against Microsoft 365 on a scheduled basis. For this, you can use Azure Function App and timer-triggered Functions. From the code you’d call Microsoft Graph API and/or SharePoint API, so you’d need your Function to get credentials on the fly and use it to call APIs. For this – you’d have a key vault where credentials will be be stored and retrieved by functions when needed. Here we’ll create, configure and secure a scheduled Azure Function and Azure Key Vault.

First, we’ll create a resource group, e.g. “Azure-Func-Secured”.

Next, we’ll create a function app:

We’ll avoid the default “Flex Consumption” service plan (as it is Linux-only and does not support dependencies) and select “Consumption” a hosting option for now:

Runtime stack we’ll be using is PowerShell Core 7.4, but you can choose your own (options are – Python, .Net (C#), Node.js, Java). Let us leave other configuration settings by default (e.g. Enable Public access: On) for now, we’ll fix (secure) it later.

Ok, the function app has been created. Notice that app service plan, application insights and storage account, as well as some other services were created automatically under the same resource group:

Now we can create one or more functions under this function app, also we’d need to create a key vault and a registered app. Let us start with function and VS Code would be our environment of choice.

Let us start Visual Studio Code in a separate folder. Ensure you have “Azure Functions”, “Azure Resources” and PowerShell extensions by Microsoft. You’d sign-in to Azure:

and create a function project:

You’d choose “Timer triggered” function (as we need the function running on schedule), the function name would be MyScheduledFunct01 and we’d leave the default schedule as “0 */5 * * * *” – so our function will be triggered every 5 minutes.

Let us deploy the function right away and see if it works. For this, we can use “deploy” icon under “WORKSPACE” panel:

or “Deploy to function app” option from the function app context pop-up menu under the Azure “RESOURCES” panel:

After successful deployment give it some time, then check function invocations and ensure function is triggered an running:

Next, we’d update “requirements.psd1” to include Azure Key Vault and PnP PowerShell modules as it takes some time for the function app to pull in and install dependencies. Requirements.psd1:

# This file enables modules to be automatically managed by the Functions service.
# See https://aka.ms/functionsmanageddependency for additional information.
#
@{
    # For latest supported version, go to 'https://www.powershellgallery.com/packages/Az'. Uncomment the next line and replace the MAJOR_VERSION, e.g., 'Az' = '5.*'
    # 'Az' = 'MAJOR_VERSION.*'
    'Az.KeyVault' = '6.*'
    'PnP.PowerShell' = '2.*'
}

And we’d update function itself to monitor if dependencies are installed, than we’d deploy the function again so time would work for us. MyScheduledFunction/run.ps1:

# Input bindings are passed in via param block.
param($Timer)
$currentUTCtime = (Get-Date).ToUniversalTime()
if ($Timer.IsPastDue) {
    Write-Host "PowerShell timer is running late!"
}
Write-Host "PowerShell timer trigger function ran! TIME: $currentUTCtime"
#############################################################################################
Write-Host "Check modules installed:"
Import-Module PnP.PowerShell 
Import-Module Az.KeyVault
Get-Module PnP.PowerShell
Get-Module Az.KeyVault
Write-Host "Check command available:"
Get-Command -Name Connect-PnPOnline -Module PnP.PowerShell

At first, we might see warning like “The first managed dependency download is in progress, function execution will continue when it’s done. Depending on the content of requirements.psd1, this can take a few minutes. Subsequent function executions will not block and updates will be performed in the background.”. So we’d just wait.

After some time the function will be able to use required modules. Here is the invocation output example:

App Registration
To get unattended access to SharePoint (or Teams or Exchange etc.) as a service (daemon) application – we need credentials. It is done via “App Registration” under Entra Id (Azure AD) – here is the detailed step-by-step guide on registering apps in Azure.

Finally, we’d have a service principal with permissions we need:

We do not hard-code secrets, so let us create a key vault to keep secrets safe:

We’d leave all other configuration option by default, including “Azure role-based access control”, Allow access from: All Networks etc.

Here is the trick: even if I created this key vault and have an admin-level access, I’m not able to work with secrets values. So I need to provide additional access to myself. Being at the key vault, navigate to “Access control (IAM)” and add role assignment…

We’d select “Key Vault Secrets Officer”, next, select members… and select own name:

From this moment you should be able to CRUD secrets with values.

Now, let us generate a secret under App registration and copy secret value:

Then navigate to the Key Vault -> Object/Secrets -> Generate/Import – and save the secret value there:

Now you can get this secret… but the function cannot reach the secret… Here is the proof. Let us update the function code with:

Write-Host "Get secret from the key vault:"
$vaultName = "azure-func-secrets"
$secretName = "azure-func-secured-01"
$kvSecret = Get-AzKeyVaultSecret -VaultName $vaultName -Name $secretName -AsPlainText
Write-Host "Secret:" $kvSecret.Substring(0,3)

and check the function output. You’d see something like (which is not very descriptive):

So, how’d we allow key vault access for the function app? It’s as simple as that:
– first, we’d need some identity assigned to function app
– second, we’d provide access to the key vault to this identity

Managed Identity: assign an identity to the function.
For this, you’d go to function Settings/Identity, and under System Assigned, you’d switch status to On and Save settings.

Then, you’d go to your key vault, Access Control (IAM) and add role assignment. But this time, you’d select the role “Key Vault Secrets User” (more about roles), and “Assign access to” Managed Identity, and select members – select your function app identity (notice, identity is assigned to function app, so all functions under the app will be able to use the identity):

Now, we’d check the next function invocation detail, and voila:

You can see that azure function was able on the fly to pull secret from the key vault, so now we should be able to use these credentials to access SharePoint.

As you know, having client id and client secret would allow you to call MS Graph API. But calling SharePoint API would require authentication with a certificate. “PnP.PowerShell” module use both APIs under the hood, so we’d need a certificate to connect to tenant and work with SharePoint using “PnP.PowerShell” module. Please refer to this article on how to run Connect-PnPOnline with Certificate stored in the Key Vault.

References:

SPO Site LastContentModifiedDate vs LastItemModifiedDate vs LastItemUserModifiedDate vs Graph LastModifiedDateTime

How do we know when the SharePoint site was last updated?

We have multiple “when the site was modified last time” properties – e.g. some we can retrieve with SharePoint CSOM:

  • Site LastContentModifiedDate
  • Web LastItemModifiedDate
  • Web LastItemUserModifiedDate

Also we can get

  • MS Graph site object with LastModifiedDateTime property
  • get usage reports via Microsoft Graph (activity reports), and
  • use “Last activity” field via Admin Center GUI

On the other hand – we can view and modify site in multiple ways – visit site home page, open and/or update document/list item, change site/library settings, configure site permissions, assign site sensitivity label, setup site property and so on.

Question: which site “last modified” or “last activity” properties reflect what events/actions?

This might be important if we think of retention policies, or any kind of clean-up processes… Let say, we are getting report on abandoned sites (inactive sites), but we are also assigning sites sensitivity labels, or we are updating site custom properties (e.g. for adaptive scopes), we have an ownerless groups policy working etc.

What if we assign site sensitivity label to an old inactive (5 years old) site – would it affect retention policy since site was updated this way?

Results

So i did some tests and based on detailed results below, it seems like

  • Web LastItemModifiedDate is triggered when user just visited site (but property LastItemUserModifiedDate is not triggered)
  • If a document or list Item updated by user or app – all properties are triggered
  • MS Graph site property LastModifiedDateTime, root web property LastItemModifiedDate and Site LastContentModifiedDate – same values
  • If site custom property is updated – it does not affect any site “last modified” property
  • The same for sensitivity label updated by app – it does not affect any site “last modified” property
  • The same for Microsoft ownerless groups policy – when user accept or decline group membership – no site “last modified” properties are changed (the same is true for Microsoft 365 group last modified date/time property).

Please refer to the table below

Detailed test results

Test results if the event triggers property update:

EventLast Content Modified DateLast Item Modified DateLast Item User Modified DateGraph Last Modified DateTimeGUI Last activity
Page viewed by userYesYesNoYes
Home Page viewed by user
Site Page viewed by user
Document or list item updated by userYesYesYesYes
Document or list item updated by appYesYesYesYes
Site config settings updated by user
Site config settings updated by app
Site custom property updated by appNoNoNoNo
Site Sensitivity label updated by user via SharePointYesNoNoNo
Site/Group Sensitivity label updated by user via Teams
Site/Group Sensitivity label updated by user via AzureNoNoNoNo
Site Sensitivity label updated by appNoNoNoNo
Site collection admin updated by userYesYesNoYes
Site collection admin updated by appYesYesNoYes
SharePoint group membership updated by userYesYesNoYes
Standalone Site connected to a group by userYesYesYesYes
Add Microsoft Teams to Site by UserYesYesYesYes
Update m365 group membership via M365 admin console by adminYesYesNoYes
Update m365 group membership via Azure by admin
Update m365 group membership via Teams by userNoNoNoYes
Update m365 group membership via App
Accept group ownership invitation sent by ownerless groups policyNoNoNoNo
Decline group ownership invitation sent by ownerless groups policyNoNoNoNo

PowerShell scripts for Microsoft 365 SharePoint

After many years working with SharePoint I wrote a lot of PowerShell scripts that help me support, troubleshoot, administer and secure SharePoint. So I’m sharing my scripts with you.

It’s here: https://github.com/VladilenK/Manage-m365-with-PowerShell

Get all SharePoint and Teams sites owners report with PowerShell

This PowerShell script pulls all tenant sites and all sites owners. The script require app authentication with Sites.FullControl.All and Directory.Read.All permissions.
PnP.PowerShell for PowerShell 7 is used.

The script generates two reports

  • Owners report: one user per line, include: Site Url, Title, Owner e-mail, name and type
  • Sites report: one site per line, include: Site Url, Title, list of owners e-mails

Here is the script:


$connAdmin = Connect-PnPOnline -ReturnConnection -Tenant $tenantId  -Url $adminUrl -ClientId $clientid -Thumbprint $certThumbprint
$allTenantSites = Get-PnPTenantSite -Connection $connAdmin | Sort-Object Url
$allTenantSites.count

$sitesReport = @()
$ownersReport = @()
foreach ($tenantSite in $allTenantSites) {
    Write-Host $tenantSite.Url
    $connSite = Connect-PnPOnline -ReturnConnection -Tenant $tenantId  -Url $tenantSite.Url -ClientId $clientid -Thumbprint $certThumbprint
    $site = Get-PnPSite -Connection $connSite -Includes RootWeb, GroupId, Owner
    $siteOwnerEmail = ''
    $siteOwnersReport = @()
    if ($site.GroupId.Guid -eq '00000000-0000-0000-0000-000000000000') {
        $siteAdmins = Get-PnPSiteCollectionAdmin -Connection $connSite | ? { $_.PrincipalType -eq 'User' }
        $ownerType = 'Site Collection Administrator'
        $isGroupSite = $false
    }
    else {
        $siteAdmins = Get-PnPAzureADGroupOwner -Connection $connAdmin -Identity $site.GroupId.Guid
        $ownerType = 'Group Owner'
        $isGroupSite = $true
    }
    foreach ($siteAdmin in $siteAdmins) {
        if (!$siteAdmin.UserPrincipalName) {
            Get-PnPProperty -Connection $connAdmin -ClientObject $siteAdmin -Property UserPrincipalName | Out-Null
        }
        $aadUser = Get-PnPAzureADUser -Connection $connAdmin -Identity $siteAdmin.UserPrincipalName
        if ($aadUser.AccountEnabled) {
            $siteOwnerEmail += $aadUser.Mail + '; '
        }
        $siteOwnersReport += [PSCustomObject]@{
            SiteUrl     = $site.Url
            SiteTitle   = $site.RootWeb.Title
            IsGroupSite = $isGroupSite
            OwnerEmail  = $aadUser.Mail
            OwnerName   = $aadUser.DisplayName
            OwnerType   = $ownerType
            Enabled     = $aadUser.AccountEnabled
        }
    }
    $ownersReport += $siteOwnersReport
    $sitesReport += [PSCustomObject]@{
        SiteUrl     = $site.Url
        SiteTitle   = $site.RootWeb.Title
        IsGroupSite = $isGroupSite
        OwnerEmail  = $siteOwnerEmail
    }
}

$ownersReport.count
$sitesReport.count

Source code: https://github.com/VladilenK/Manage-m365-with-PowerShell

Microsoft 365 SharePoint: prevent throttling with RateLimit headers

Bert Jansen (Microsoft) revealed some details on throttling when you access Microsoft 365 programmatically – via Microsoft Graph or CSOM and guided developers on how to regulate request traffic for optimized throughput using RateLimit headers (Here).

Demystifying SharePoint throttling

Throttling is necessary to ensure that no single user or application consumes too many resources compromising the stability of the entire system, which is used by many clients.

Throttling happens at

  • User (there are user request limits. Microsoft counts all requests linked to user
  • Application (Delegated or Application permissions)
    • Resource units per app per minute
    • Resource units per app per day
  • Farm – Spike protection

Very common reason for throttling – when an Application (Delegated or Application permissions) reaches “Resource units per app per minute” threshold.

Usually you catch HTTP errors 429 or 503, wait for some time (respect Retry-after header) and try again.

SharePoint provides various APIs. Different APIs have different costs depending on the complexity of the API, but Microsoft favor Graph API over SharePoint REST/CSOM. The cost of APIs is normalized by SharePoint and expressed by resource units. Application’s limits are also defined using resource units.

Quota depends on tenant size.

Resource unit limits for an application in a tenant (please refer to the Microsoft article)

Predefined costs for Microsoft Graph calls:

Assuming 2 resource units per request is a safe bet.

Links

Update Large Number of SharePoint Sites with PowerShell Parallel

WIP

Here I’m trying to figure out – how much PowerShell Parallel option is beneficial and how to avoid throttling…

Let us test, how long would it take to create a SharePoint site, if we use regular (sequential) loop or parallelism (I’m creation a sample set of 50 SharePoint Sites in a row):

Regular
(Sequential)
seconds per site
Parallel,
100 sites in batch
seconds per site
Parallel,
500 sites in batch
seconds per site
Regular (Sequential)3.0
Parallel,  ThrottleLimit = 21.600.91
Parallel,  ThrottleLimit = 50.69
Parallel,  ThrottleLimit = 100.2 – 0.3
Parallel,  ThrottleLimit = 200.17

Interesting, but I did not get even one (throttling or any other) error during creation 500 sites.

Get sites details

Now let us test, how long it takes to get sites details with Get-PnPTenantSite (I use a sample set of 500 sites):

Test typeRegular
(Sequential),
seconds per site
Parallel
sample = 100 sites,
seconds per site
Parallel
sample = 200 sites,
seconds per site
Parallel
sample = 500 sites,
seconds per site
Regular (Sequential)0.65
Parallel,  ThrottleLimit = 20.400.330.31
Parallel,  ThrottleLimit = 50.170.140.36 (errors)
Parallel,  ThrottleLimit = 100.11 (errors)0.11 (errors)0.34 (errors)
Parallel,  ThrottleLimit = 200.12 errors+0.07 errors+0.52 (errors)

(errors) means there were small number of errors during test… e.g.

SharePoint Site Template, Look Book and PnP Provisioning Engine

Aug 2024 Update: you cannot apply site template from Look Book. You must use PowerShell to apply a Look Book template to your site.

Here is the article: Applying PnP Templates to SharePoint Sites

Some templates can be applied by regular users (site admins) and some templates would require SharePoint tenant admin permissions. But now it’s only via PowerShell. You can get an idea how templates look like at

PnP provisioning engine is something that us used under the hood.

If you are interested in automation of provisioning templates – please let me know in comments below or via site feedback.

===============================

So the information below is obsolete and I will keep it just for the sake of history of SharePoint:

SharePoint Look Book

SharePoint Look Book – a site with a collection of modern SharePoint site templates. You can browse through dozens of good-looking templates… but how do you apply chosen template to your site?

Gotcha #1

There is a button “Add to your tenant>” and it says “You must be a tenant administrator to deploy this template.” Really? No… but
Actually, SharePoint Administrator role is required to apply template from lookbook.
So yes, tenant-level admin role but just SharePoint service admin role.
Site admin role is not enough…

Gotcha #2

Next, when you try to get template by clicking “Add to your tenant>” button, it actually offers you to create a new site. But it also says “…can use existing URL”. Really? No.
When you type existing site Url into the “Relative URL to be used for the site” field –
You can get “Can’t add this template. The provided site is already in use and the current template cannot be provisioned onto an already existing site. Please provide a different URL” message:

Or, if you managed to enter existing Url, you might get: “Unfortunately your site provisioning at least partially failed!”:


References:

Token – SharePoint API compatibility matrix

If I get token with (Graph, MSAL, PnP) and use this token for (Graph API, SharePoint CSOM API, SharePoint REST API) matrix.

An App used in this tests has Sites.FullControl.All MS Graph API and SharePoint API permissions, as well as FullControl ACS based permissions to SharePoint (AppInv.aspx).

Token/APIMS Graph
/v1.0/sites
SharePoint CSOM
PnP.PowerShell
Get-PnPSite
Get-PnPTenantSite
SharePoint REST API
PnP.PowerShell
Invoke-PnPSPRestMethod
Invoke-RestMethod
MS Graph
/oauth2/v2.0/token
secret
OK(401) UnauthorizedAudienceUriValidationFailedException
MSAL.PS
Get-MsalToken
with secret
OK(401) UnauthorizedAudienceUriValidationFailedException
MSAL.PS
Get-MsalToken
with certificate
OK(401) UnauthorizedAudienceUriValidationFailedException
PnP.PowerShell
Get-PnPAccessToken
with Certificate
OKOK
OK
OK
AudienceUriValidationFailedException
PnP.PowerShell
Get-PnPGraphAccessToken
with Certificate
OKOK
OK
OK
AudienceUriValidationFailedException
PnP.PowerShell
Get-PnPAppAuthAccessToken
with Certificate or secret
InvalidAuthenticationTokenOK
OK
OK
OK
PnP.PowerShell
Request-PnPAccessToken
with Certificate
InvalidAuthenticationTokenOK
OK
OK
AudienceUriValidationFailedException
PnP.PowerShell
Request-PnPAccessToken
with Secret
InvalidAuthenticationTokenOK
OK
OK
OK
AudienceUriValidationFailedException = Exception of type ‘Microsoft.IdentityModel.Tokens.AudienceUriValidationFailedException’ was thrown