Tag Archives: PnP

Get list of new SPO sites with PowerShell

Scenario

Let say you administer Microsoft 365 SharePoint Online and you want to get a list of new SharePoint sites (e.g. sites created during last week/month).

With GUI it’s done easily: SharePoint Admin Center -> Active Sites -> sort based on “Date Created” – done.

With PowerShell – not so simple.
“Get-PnPTenantSite” cmdlet returns site objects but the object does not have “Created” field. You have to connect separately to each site and get root web object where you can check when the web was created. For small environments it is possible, for large environments it can take days… And still not nice.
-Filter option would help, but “…Currently, you can filter by these properties: Owner, Template, LockState, Url.”

Get-SPOSite – similar experience.

Solution

Microsoft Graph API helps. It returns result in seconds. There are some pros and cons for each method though.

Option #1: Microsoft Graph Search API.

Entry point: https://graph.microsoft.com/v1.0/search/query

Microsoft Graph Search API allows KQL in queries. So we can form a query with something like “created>=1/1/2021” and use entity type = ‘[“site”]’. Search should return only sites created after Jan 01, 2021.

Check PowerShell script sample here: Get-NewSites.ps1
https://github.com/VladilenK/PowerShell/blob/main/reports/SharePoint/Get-NewSites.ps1

If you are getting more than 500 results – think of paging.

Option #2: Microsoft Graph Sites API

Entry point: https://graph.microsoft.com/v1.0/sites

This option is also based on Microsoft Graph API, but sites entry point, which allows search too and sort results by property “createdDateTime”. So we will just search for everything and select how many results we need based on createdDateTime property.

Check PowerShell script sample here: Get-NewSites.ps1
https://github.com/VladilenK/PowerShell/blob/main/reports/SharePoint/Get-NewSites.ps1

References

Retrieve programmatically SharePoint Online system page html content

How can I get HTML content of a SharePoint online page from code, e.g. PowerShell?

Invoke-WebRequest returns “Sign in to your account” page, not a real page, even with -Token option.

Thanks to Denis Molodtsov, the solution is found. It turns out the “Invoke-PnPSPRestMethod” PnP cmdlet works not only against /api endpoints, but also against site pages and system pages.

But (as per my experience) it works only with PnP.PowerShell and with -UseWebLogin authentication option and with -raw parameter.

Connect-PnPOnline -url $siteUrl -UseWebLogin
Invoke-PnPSPRestMethod -url /_layouts/15/viewlsts.aspx -Raw

Other combination of authentication options ( -interactive, -clientId, -Token, -SPOManagementShell, -PnPManagementShell ) – worked well, but only for /_api endpoints, and gave me “401 UNAUTHORIZED” against system/site pages.
Unattended authentication (with clientId, clientSecret and certificate) – same.

Legacy PnP module SharePointPnPPowerShellOnline did not work at all: “EXCEPTION,PnP.PowerShell.Commands.Admin.InvokeSPRestMethod”.

I tested it with
– SharePointPnPPowerShellOnline v 3.29.2101.0 (under Windows PowerShell 5.1) and
– PnP.PowerShell 1.8.0. (both Windows PowerShell 5.1 and .net core PowerShell 7.1.5)

PowerShell Script to Fetch All Alerts from SharePoint Online Site

PowerShell Script to get All Alerts of all Users from a specific SharePoint Online Site Collection, including subsites:

https://github.com/VladilenK/PowerShell/blob/main/reports/Site/Fetch-All-Alerts-from-SPO-Site.ps1

https://raw.githubusercontent.com/VladilenK/PowerShell/main/reports/Site/Fetch-All-Alerts-from-SPO-Site.ps1

based on Salaudeen Rajack:
SharePoint Online: Get All Alerts from a Site Collection using PowerShell

Connect to SharePoint Online with PnP.PowerShell Interactively with Client App and msal token

Scenario

You use PnP.PowerShell and you need to connect to SharePoint Online via Connect-PnPOnline interactively (on behalf of a user).

Solution

  • register an Azure App (single tenant)
  • configure authentication blade:
    – add platform – “Mobile and Desktop app”
    select “https://login.microsoftonline.com/common/oauth2/nativeclient”
    add custom Redirect URI: “http://localhost”
  • configure API permissions blade:
    – add delegated permissions you need (refer to specific API you’ll use)
    e.g. Microsoft Graph Sites.FullControl.All and SharePoint AllSites.FullControl
  • use the following code to connect to your site:
$siteUrl = "https://contoso.sharepoint.com/teams/myTeamsSite"
$appId = "" # Client Id
$connection = Connect-PnPOnline -ClientId $appId -Url $adminUrl -Interactive -ReturnConnection # -ForceAuthentication
$connection

A pop-up window will appear to authenticate interactively. If you are already authenticated with another credentials (or single-sigh-on) – an interactive window might pop up and disappear – so you are not able to enter your admin id.
To ensure Connect-PnPOnline prompts you for your credentials – use ” -ForceAuthentication” option.

If you are a SharePoint tenant admin – you can connect to a tenant with:

$orgName = "yourTenantPrefix" 
$adminUrl = "https://$orgName-admin.sharepoint.com" 
$appId = "" # Client Id 
$connection = Connect-PnPOnline -ClientId $appId -Url $adminUrl -Interactive -ReturnConnection # -ForceAuthentication 
$connection 

The other option is to use MSAL.PS module to get an msal token. This might help with Microsoft graph-based requests:

$tenantId = ""
$clientid = ""
$url = ""
$token = Get-MsalToken -ClientId $clientid -TenantId $tenantId -Interactive
Connect-PnPOnline -AccessToken $token -Url $url 

By default token expires in ~ 1 hour. But you can refresh it silently.
This helps if you run heavy PowerShell script and it takes hours to complete.
So you can include something like this in the loop:

if ($token.ExpiresOn.LocalDateTime -lt $(get-date).AddMinutes(10)) {    
  $token = Get-MsalToken -ClientId $clientid -TenantId $tenantId -ForceRefresh -Silent    
  Write-Host "Token will expire on:" $token.ExpiresOn.LocalDateTime
}

NB: For delegated permissions, the effective permissions of your app are the intersection of the delegated permissions the app has been granted (via consent) and the privileges of the currently signed-in user. Your app can never have more privileges than the signed-in user.

Connect-PnPOnline with a certificate stored in Azure Key Vault

Scenario

You need to run some PnP PowerShell code unattended (daemon app, with no user interaction) against SharePoint and/or Azure AD. PnP require authentication with a certificate. You want certificate stored securely in Azure Key Vault.

Solution

  • create a self-signed certificate
  • register an application in Azure
  • add API application permissions to the app
  • upload the certificate to the app
  • create an Azure Key Vault
  • provide permissions to the Key Vault for the user
  • run Connect-AzAccount
  • upload certificate to the Key Vault manually (with GUI)

now you can run this code and it will not ask you to login:

# set parameters:
$orgName = "orgname" 
$clientID = "" # Client ID
$VaultName = "" # Azure Key Vault Name
$certName = "" # Certificate Name as in Azure Key Vault
$tenant = "$orgName.onmicrosoft.com"
$adminUrl = "https://$orgName-admin.sharepoint.com"
# run the following
$secretSecureString = Get-AzKeyVaultSecret -VaultName $vaultName -Name $certName 
$secretPlainText = ConvertFrom-SecureString -AsPlainText -SecureString $secretSecureString.SecretValue
Connect-PnPOnline -Url $adminUrl -ClientId $clientID -CertificateBase64Encoded $secretPlainText -Tenant $tenant 

The same PowerShell code in GitHub: https://github.com/VladilenK/PowerShell/blob/main/PnP/Connect-PnPOnline-with-certificate.ps1

References:

https://docs.microsoft.com/en-us/powershell/module/az.keyvault/get-azkeyvaultcertificate?view=azps-5.3.0

https://stackoverflow.com/questions/43837362/keyvault-generated-certificate-with-exportable-private-key

PnP PowerShell 7.1 Parallel

(WIP)

Can I run something like

$items | ForEach-Object -Parallel {
    $listItem = Set-PnPListItem -List "LargeList" -Identity $_ -Values @{"Number" = $(Get-Random -Minimum 100 -Maximum 200 ) }
} 

Unfortunately, no. Instead, use batching!

$batch = New-PnPBatch
1..100 | ForEach-Object{ Add-PnPListItem -List "ItemTest" -Values @{"Title"="Test Item Batched $_"} -Batch $batch }
Invoke-PnPBatch -Batch $batch

More:
Batching in PnP PowerShell | PnP PowerShell
– https://www.youtube.com/watch?v=0-DSwZyK2Zo

SharePoint PnP roadmap

Good news!
On Sep, 18 during the SIG community call, PnP Team shared their plans on PnP Sites Core library and PnP Core SDK.
“PnP Sites Core v4” library and “PnP Core SDK v1” with .net core support (.net Standard 2.0) – expected in December 2020!

PnP PowerShell v4 for SPO library built for .Net Standard 2.0 / PowerShell 7 will be released in Dec 2020 as well.

Fastest way to delete all items in a large SPO list: PnP.PowerShell batches

Scenario: You have a large (>5k items) list in SharePoint Online.
You need to delete this list. “Remove-PnPList” fails with a message “The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator“. Deleting with UI fails too.

Try this PowerShell command with ScriptBlock:

Get-PnPListItem -List $list -Fields "ID" -PageSize 100 -ScriptBlock { Param($items) $items | Sort-Object -Property Id -Descending | ForEach-Object{ $_.DeleteObject() } } 

or this PowerShell with batches:

$batch = New-PnPBatch
1..12000 | Foreach-Object { Remove-PnPListItem -List $list -Identity $_ -Batch $batch }
Invoke-PnPBatch -Batch $batch

for me both methods gave same good result: ~17 items per second ( ~7 times faster than regular).

PnP.PowerShell batches

With a new PnP.PowerShell we can perform some operations against an SPO list with batches!
How fast PnP batches are? My measurements:

Time elapsed, secondswith batcheswith scriptBlockwithout batches
Add-PnPListItem (100 items)4.33 seconds42 seconds
Add-PnPListItem (500 items)21 seconds234 seconds
Add-PnPListItem (7000 items)314 seconds
Remove-PnPListItem (1000 items)58 seconds58 seconds429 seconds
Remove-PnPListItem (7000 items)395 seconds397 seconds

i.e. with batches your pnp.powershell code runs 7-10 times faster!

References: