Tag Archives: PowerShell

How to create an old document in SharePoint

Sometimes, mostly during PoC or testing policies like retention policy or lifecycle policy you would need some documents created and updated weeks, months or even years ago.

But if you create or upload a document in SharePoint library – it will be just a regular new document. So, how to get old documents in the new environment?

I see two options:

  1. Sync with OneDrive
    If you sync a library with your local folder (done Microsoft by OneDrive desktop app) and put some old document in your synced folder – the doc will be synchronized back to SharePoint library with Created and Modified properties preserved.
  2. Make the document older with PowerShell
    With “Set-PnPListItem” PowerShell command you can update not only such properties like Title, but also “Created By”, “Modified By” and even date and time document was created and modified via “Created” and “Modified”.
    Optionally you can play with document history with “-UpdateType” parameter.
    UpdateType possible values are:
    • Update: Sets field values and creates a new version if versioning is enabled for the list
    • SystemUpdate: Sets field values and does not create a new version. Any events on the list will trigger.
    • UpdateOverwriteVersion: Sets field values and does not create a new version. No events on the list will trigger

Manage Microsoft 365 groups membership with PowerShell and Graph API

As SharePoint or Teams admin you can manage Microsoft 365 groups (create, update, delete, manage membership etc.) having your admin role activated. I use Azure registered app with “Group.ReadWrite.All” Microsoft Graph API delegated permission and Microsoft.Graph PowerShell module.

When a user was not a group member or group owner – and is added to the group members – user gets notification “You’ve joined the <Group Name> group” via e-mail that comes from a group e-mail address.

When a user is added to the group owners (or elevated to group owner if user was a group member) – user does not get notification.

When a user was a group owner and now is added to the group members – user does not get notification.

All the actions are logged into Microsoft 365 audit log under your personal Id.

Script samples:

# This script is just a sample to demonstrate basic technique on deletion m365 groups with PowerShell and MS Graph
# please do not run this script as is, but update it upon your needs

# authentication with personal Id
#  app must have as minimum "Group.ReadWrite.All" Microsoft Graph API delegated permission
#  user must have SharePoint admin (or Teams admin) roles activated
Connect-MgGraph -ClientId $clientid -TenantId $tenantId 
Get-MgContext | Select-Object Scopes -ExpandProperty Scopes

# sample data
$groups = @()
$groups += [PSCustomObject]@{GroupId = '443d22ae-683a-4fe4-8875-7bd78227a026' }
$groups += [PSCustomObject]@{GroupId = 'e5805388-c18c-48c0-b42d-6223cf8f3d82' }

# Get Groups
foreach ($group in $groups) {
    Get-MgGroup -GroupId $group.GroupId
}

# add members to the group
$groupId = '443d22ae-683a-4fe4-8875-7bd78227a026'
$userId = 'df74e0d3-d78c-495b-b47a-549437d93cf7' # Adele
New-MgGroupMember -GroupId $groupId -DirectoryObjectId $userId

# add Owner to the group
$groupId = '443d22ae-683a-4fe4-8875-7bd78227a026'
$userId = 'eacd52fb-5ae0-45ec-9d17-5ded9a0b9756' # Megan
New-MgGroupOwner -GroupId $groupId -DirectoryObjectId $userId

# Delete group
# add Owner to the group
$groupId = '443d22ae-683a-4fe4-8875-7bd78227a026'
Remove-MgGroup -GroupId $groupId

https://github.com/VladilenK/m365-PowerShell/tree/main/KBA/Ownerless-Groups

Sites.Selected API permissions for SharePoint access

Sites.Selected permissions are needed for the non-interactive application to get access to a specific SharePoint site.

Steps to get access to SharePoint site with Sites.Selected:

1. Register an application in Azure (via Azure portal GUI, PowerShell script or helpdesk/servicedesk request)

2. Update the app so both – MS Graph API Sites.Selected and SharePoint Sites.Selected permissions are configured and consented by tenant admin

3. Obtain and upload client certificate (recommended) or generate client secret

4. Request access for the app to a specific SharePoint site (your SharePoint service admin should be able to do that via PowerShell script or Graph API calls )
Here is the Graph API

5. Validate your app has access to the target SharePoint site with PowerShell
tbp

6. Secure your certificate and/or secret

Consider using vault to keep certificate/secret. If you host your application in Azure – consider using managed identity

ACS-based permissions are going to be deprecated soon:
Your SharePoint admin doesn’t allow site owners to create/update ACS principal ⋆ Vladilen Microsoft 365 engineer

SPO Site LastContentModifiedDate vs LastItemModifiedDate vs LastItemUserModifiedDate vs Graph LastModifiedDateTime

How do we know when the SharePoint site was last updated?

We have multiple “when the site was modified last time” properties – e.g. some we can retrieve with SharePoint CSOM:

  • Site LastContentModifiedDate
  • Web LastItemModifiedDate
  • Web LastItemUserModifiedDate

Also we can get

  • MS Graph site object with LastModifiedDateTime property
  • get usage reports via Microsoft Graph (activity reports), and
  • use “Last activity” field via Admin Center GUI

On the other hand – we can view and modify site in multiple ways – visit site home page, open and/or update document/list item, change site/library settings, configure site permissions, assign site sensitivity label, setup site property and so on.

Question: which site “last modified” or “last activity” properties reflect what events/actions?

This might be important if we think of retention policies, or any kind of clean-up processes… Let say, we are getting report on abandoned sites (inactive sites), but we are also assigning sites sensitivity labels, or we are updating site custom properties (e.g. for adaptive scopes), we have an ownerless groups policy working etc.

What if we assign site sensitivity label to an old inactive (5 years old) site – would it affect retention policy since site was updated this way?

Results

So i did some tests and based on detailed results below, it seems like

  • Web LastItemModifiedDate is triggered when user just visited site (but property LastItemUserModifiedDate is not triggered)
  • If a document or list Item updated by user or app – all properties are triggered
  • MS Graph site property LastModifiedDateTime, root web property LastItemModifiedDate and Site LastContentModifiedDate – same values
  • If site custom property is updated – it does not affect any site “last modified” property
  • The same for sensitivity label updated by app – it does not affect any site “last modified” property
  • The same for Microsoft ownerless groups policy – when user accept or decline group membership – no site “last modified” properties are changed (the same is true for Microsoft 365 group last modified date/time property).

Please refer to the table below

Detailed test results

Test results if the event triggers property update:

EventLast Content Modified DateLast Item Modified DateLast Item User Modified DateGraph Last Modified DateTimeGUI Last activity
Page viewed by userYesYesNoYes
Home Page viewed by user
Site Page viewed by user
Document or list item updated by userYesYesYesYes
Document or list item updated by appYesYesYesYes
Site config settings updated by user
Site config settings updated by app
Site custom property updated by appNoNoNoNo
Site Sensitivity label updated by user via SharePointYesNoNoNo
Site/Group Sensitivity label updated by user via Teams
Site/Group Sensitivity label updated by user via AzureNoNoNoNo
Site Sensitivity label updated by appNoNoNoNo
Site collection admin updated by userYesYesNoYes
Site collection admin updated by appYesYesNoYes
SharePoint group membership updated by userYesYesNoYes
Standalone Site connected to a group by userYesYesYesYes
Add Microsoft Teams to Site by UserYesYesYesYes
Update m365 group membership via M365 admin console by adminYesYesNoYes
Update m365 group membership via Azure by admin
Update m365 group membership via Teams by userNoNoNoYes
Update m365 group membership via App
Accept group ownership invitation sent by ownerless groups policyNoNoNoNo
Decline group ownership invitation sent by ownerless groups policyNoNoNoNo

Get all SharePoint and Teams sites owners report with PowerShell

This PowerShell script pulls all tenant sites and all sites owners. The script require app authentication with Sites.FullControl.All and Directory.Read.All permissions.
PnP.PowerShell for PowerShell 7 is used.

It generates two reports

  • Owners report: one user per line, include: Site Url, Title, Owner e-mail, name and type
  • Sites report: one site per line, include: Site Url, Title, list of owners e-mails

Here is the script:


$connAdmin = Connect-PnPOnline -ReturnConnection -Tenant $tenantId  -Url $adminUrl -ClientId $clientid -Thumbprint $certThumbprint
$allTenantSites = Get-PnPTenantSite -Connection $connAdmin | Sort-Object Url
$allTenantSites.count

$sitesReport = @()
$ownersReport = @()
foreach ($tenantSite in $allTenantSites) {
    Write-Host $tenantSite.Url
    $connSite = Connect-PnPOnline -ReturnConnection -Tenant $tenantId  -Url $tenantSite.Url -ClientId $clientid -Thumbprint $certThumbprint
    $site = Get-PnPSite -Connection $connSite -Includes RootWeb, GroupId, Owner
    $siteOwnerEmail = ''
    $siteOwnersReport = @()
    if ($site.GroupId.Guid -eq '00000000-0000-0000-0000-000000000000') {
        $siteAdmins = Get-PnPSiteCollectionAdmin -Connection $connSite | ? { $_.PrincipalType -eq 'User' }
        $ownerType = 'Site Collection Administrator'
        $isGroupSite = $false
    }
    else {
        $siteAdmins = Get-PnPAzureADGroupOwner -Connection $connAdmin -Identity $site.GroupId.Guid
        $ownerType = 'Group Owner'
        $isGroupSite = $true
    }
    foreach ($siteAdmin in $siteAdmins) {
        if (!$siteAdmin.UserPrincipalName) {
            Get-PnPProperty -Connection $connAdmin -ClientObject $siteAdmin -Property UserPrincipalName | Out-Null
        }
        $aadUser = Get-PnPAzureADUser -Connection $connAdmin -Identity $siteAdmin.UserPrincipalName
        if ($aadUser.AccountEnabled) {
            $siteOwnerEmail += $aadUser.Mail + '; '
        }
        $siteOwnersReport += [PSCustomObject]@{
            SiteUrl     = $site.Url
            SiteTitle   = $site.RootWeb.Title
            IsGroupSite = $isGroupSite
            OwnerEmail  = $aadUser.Mail
            OwnerName   = $aadUser.DisplayName
            OwnerType   = $ownerType
            Enabled     = $aadUser.AccountEnabled
        }
    }
    $ownersReport += $siteOwnersReport
    $sitesReport += [PSCustomObject]@{
        SiteUrl     = $site.Url
        SiteTitle   = $site.RootWeb.Title
        IsGroupSite = $isGroupSite
        OwnerEmail  = $siteOwnerEmail
    }
}

$ownersReport.count
$sitesReport.count

Source code: https://github.com/VladilenK/Manage-m365-with-PowerShell

Microsoft 365 SharePoint: prevent throttling with RateLimit headers

Bert Jansen (Microsoft) revealed some details on throttling when you access Microsoft 365 programmatically – via Microsoft Graph or CSOM and guided developers on how to regulate request traffic for optimized throughput using RateLimit headers (Here).

Demystifying SharePoint throttling

Throttling is necessary to ensure that no single user or application consumes too many resources compromising the stability of the entire system, which is used by many clients.

Throttling happens at

  • User (there are user request limits. Microsoft counts all requests linked to user
  • Application (Delegated or Application permissions)
    • Resource units per app per minute
    • Resource units per app per day
  • Farm – Spike protection

Very common reason for throttling – when an Application (Delegated or Application permissions) reaches “Resource units per app per minute” threshold.

Usually you catch HTTP errors 429 or 503, wait for some time (respect Retry-after header) and try again.

SharePoint provides various APIs. Different APIs have different costs depending on the complexity of the API, but Microsoft favor Graph API over SharePoint REST/CSOM. The cost of APIs is normalized by SharePoint and expressed by resource units. Application’s limits are also defined using resource units.

Quota depends on tenant size.

Resource unit limits for an application in a tenant (please refer to the Microsoft article)

Predefined costs for Microsoft Graph calls:

Assuming 2 resource units per request is a safe bet.

Links

Update Large Number of SharePoint Sites with PowerShell Parallel

WIP

Here I’m trying to figure out – how much PowerShell Parallel option is beneficial and how to avoid throttling…

Let us test, how long would it take to create a SharePoint site, if we use regular (sequential) loop or parallelism (I’m creation a sample set of 50 SharePoint Sites in a row):

Regular
(Sequential)
seconds per site
Parallel,
100 sites in batch
seconds per site
Parallel,
500 sites in batch
seconds per site
Regular (Sequential)3.0
Parallel,  ThrottleLimit = 21.600.91
Parallel,  ThrottleLimit = 50.69
Parallel,  ThrottleLimit = 100.2 – 0.3
Parallel,  ThrottleLimit = 200.17

Interesting, but I did not get even one (throttling or any other) error during creation 500 sites.

Get sites details

Now let us test, how long it takes to get sites details with Get-PnPTenantSite (I use a sample set of 500 sites):

Test typeRegular
(Sequential),
seconds per site
Parallel
sample = 100 sites,
seconds per site
Parallel
sample = 200 sites,
seconds per site
Parallel
sample = 500 sites,
seconds per site
Regular (Sequential)0.65
Parallel,  ThrottleLimit = 20.400.330.31
Parallel,  ThrottleLimit = 50.170.140.36 (errors)
Parallel,  ThrottleLimit = 100.11 (errors)0.11 (errors)0.34 (errors)
Parallel,  ThrottleLimit = 200.12 errors+0.07 errors+0.52 (errors)

(errors) means there were small number of errors during test… e.g.