Category Archives: SharePoint

PowerShell Script for Files Deduplication

Leave a reply

If you think you have a lot of duplicated files that consumes your hard drive storage space, this article is for you. Personally, I have a lot of video and pictures on my working hard drive and on a backup HDD. While working with photos and videos I can rename files, copy or move them from folders to folders. As a result, I end up with gigabytes and terabytes occupied with duplicated files. So I need a tool to a) find duplicated files and b) remove duplications. I tried to find good scripts but somehow I was not happy with what I found, so I wrote scripts myself.

Surely you can buy/try a 3-rd party toot with GUI, but if you are comfortable with PowerShell – consider the following.

…

References

Restricted SharePoint Search Deep Dive

Leave a reply

Restricted SharePoint Search is a new Microsoft feature to mitigate sites oversharing issue when you are implementing Copilot. The feature is documented here, but still I have some questions, e.g.:

  • How about external data? Copilot can use external data to learn from via agents and connectors. But would Restricted SharePoint Search if implemented allow data from external connectors to be used in copilot?
  • “Users’ OneDrive files, chats, emails, calendars they have access to” – means own data for every single user or all shared OD data?
  • What exactly is “Files from their frequently visited SharePoint sites”? I mean, how frequently user needs to visit site for this?
  • What exactly means “Files that the users viewed, edited, or created.”
  • What about teams chat messages, e-mails, viva engage messages?
  • “Files that were shared directly with the users” – does that mean “individual files shared” or can include folders, libraries, sites?
  • If user is a member of a teams – would all team content included?
  • It says “Files…” but would site pages be included? Or list items? Or list items attachments? Pages is something that people use to create wiki to share knowledge.
  • How long it takes for Microsoft 365 to start restricting results after Restricted SharePoint Search is enabled
  • How to deal with “You do not have the required license to perform this operation”

Here I’m going to answer the questions above.

So far I build a test scenario using my dev tenant that includes multiple collaborated users and content in the form of files, pages, list items and messages spreaded across multiple sites falling into different categories of Restricted SharePoint Search allowed content.

You do not have the required license…

If you are getting “You do not have the required license to perform this operation” when you are trying Get-SPOTenantRestrictedSearchMode or Get-PnPTenantRestrictedSearchMode – that means there is no Copilot for Microsoft 365 licenses assigned to tenant yet. This feature – Restricted SharePoint Search – works only when at least one Copilot license is assigned to tenant.

… TBC

References

SharePoint Governance

Leave a reply

Governance in IT is establishing rules, policies, tools and practices that helps you manage and protect your enterprise resources. SharePoint governance (or wider – Collaboration governance) covers

  • resources ownership and lifecycle
  • users’ access to resources
  • compliance with your business standards
  • security of your data

References

Granular Application Permissions to SharePoint

Leave a reply

In 2021 Microsoft implemented “Sites.Selected” Graph API permissions to allow application access (without a signed in user) to specific sites (entire site only). In 2024 Microsoft implemented granular access – to specific list/libraries, as well as to specific documents/files and list items. Now name convention is *.SelectedOperations.Selected.
Permissions come in two flavors – delegated and application:

  • Files.SelectedOperations.Selected – Allow the application to access a subset of files (files explicitly permissioned to the application). The specific files and the permissions granted will be configured in SharePoint Online or OneDrive.
  • ListItems.SelectedOperations.Selected – Allow the application to access a subset of lists. The specific lists and the permissions granted will be configured in SharePoint Online.
  • Lists.SelectedOperations.Selected – Allow the application to access a subset of lists. The specific lists and the permissions granted will be configured in SharePoint Online.


Update SharePoint Site Title: GUI vs PowerShell

Leave a reply

If you need to update a SharePoint site title (site name) programmatically (e.g. with PowerShell), and if this site is a group-based site (e.g. Microsoft Teams team site or Viva Engage community site or…) – you should not update SharePoint site title, but you should update group display name instead. Here is why.

In Microsoft 365 there is no sync from SharePoint site title to a group name. When you are updating SharePoint site title with GUI – you can see that new site title becomes new group/team name as well. So you might think that if you update SharePoint site title – Microsoft synchronizes it to connected group name. That’s not true. Actually when you are updating a group-based (e.g. teams-connected) SharePoint site title with GUI – Microsoft updates group first, then syncs updated group display name to SharePoint site name (title).

Here is the proof:

That’s a network trace I got with browser dev tools when I renamed site (updated site title) with GUI. So you can see the first API call is to update group, then group properties are synced back to site.

When we are updating a standalone site title – we are not seeing these calls.

So, if you need to update group-based site title programmatically – you must update group instead. 

# does not work for group-based (e.g. Teams) sites:
Set-PnPTenantSite -Identity ... -Title "New Site Title"

# instead, you'd update group display name 
Set-PnPMicrosoft365Group -Identity ... -DisplayName "New Display Name"
# and site title will be updated accordingly

References:

Calling Microsoft Graph API from Python

Leave a reply

Below is how I authenticate and call Microsoft Graph API to work with SharePoint from Python application.

Plain

no MSAL or Azure libraries used:

import requests
import json
from secrets import clientSc 

clientId = "7e60c372-ec15-4069-a4df-0ab47912da46"
# clientSc = "<imported>" 
tenantId = "7ddc7314-9f01-45d5-b012-71665bb1c544"

apiUri = "https://login.microsoftonline.com/" + tenantId + "/oauth2/v2.0/token"

body = {
    "client_id"     : clientId,
    "client_secret" : clientSc,
    "scope"         : "https://graph.microsoft.com/.default",
    "grant_type"    : "client_credentials" 
}

response = requests.post(apiUri, data=body)
token = json.loads(response.content)["access_token"]

graph_url = 'https://graph.microsoft.com/v1.0/sites/root'
site = requests.get(
    graph_url,
    headers={'Authorization': 'Bearer {0}'.format(token)}
)

print(site.content)
print(json.loads(site.content)["webUrl"])

secrets is a Python file where I assign client secret to variable clientSc (so my secret is not shared on github). This is ok for demo purposes but generally, you should not hard-code secrets but keep secrets somewhere safe (Vault).

MSAL

Using MSAL library to get bearer token:
https://github.com/VladilenK/m365-with-Python/tree/main/Graph-API-MSAL

References

Using Microsoft.Graph PowerShell to Search in Microsoft 365

1 Reply

There is a Microsoft.Graph PowerShell module provided by Microsoft which simplifies usage of Microsoft Graph API. Below is how to authenticate to MS Graph and how to search within SharePoint and Teams Microsoft 365 content using Microsoft.Graph PowerShell module.

Authentication

Interactive authentication code sample:

# Prerequisites
Get-Module Microsoft.Graph.Authentication -ListAvailable 
Get-Module Microsoft.Graph.Search -ListAvailable 

# Interactive Authentication
$clientid = 'd82858e0-ed99-424f-a00f-cef64125e49c'
$TenantId = '7ddc7314-9f01-45d5-b012-71665bb1c544'
Connect-MgGraph -ClientId $clientid -TenantId $TenantId

For daemon app authentication we need a certificate configured in Azure App and installed on the user machine. Daemon app authentication code sample (please specify your tenant id, app (client) id and certificate thumbprint:

# App Authentication
$TenantId = ""
$clientID = ""
$certThumbprint = ""
Connect-MgGraph -ClientId $clientid -TenantId $TenantId -CertificateThumbprint $certThumbprint

Search with Microsoft.Graph

# Search
$params = @{
	requests = @(
		@{
			entityTypes = @(
				"driveItem"
			)
			query = @{
				queryString = "test*"
			}
			from = 0
			size = 50
			fields = @(
				"title"
				"description"
			)
                        region = "NAM"
		}
	)
}

$res = Invoke-MgQuerySearch -Body $params
$res.HitsContainers[0].Hits

Note: when you are calling MS Graph Search API authenticated as user – you need to remove “region” parameter.

Code samples: https://github.com/VladilenK/m365-PowerShell/tree/main/KBA/Search

Search Microsoft 365 content programmatically: all articles index

Video tutorial:

Providing ACS permissions for app to access SharePoint

3 Replies

Microsoft retires ACS

Let me quote Microsoft just to start (Dec 18, 2023):

  • “SharePoint App-Only is the older, but still very relevant, model of setting up app-principals.”
  • “… we will be retiring the use of Azure ACS (Access Control Services) for SharePoint Online auth needs and believe Microsoft 365 customers will be better served by modern auth…”
  • “Azure ACS will stop working for new tenants as of November 1st, 2024 and it will stop working for existing tenants and will be fully retired as of April 2nd, 2026…
    There will not be an option to extend using Azure ACS with SharePoint Online beyond April 2nd, 2026″
  • “… we recommend switching those applications to use Microsoft Entra ID for authorization and authentication needs…”

So, for new development it is strictly recommended to use Azure Registered Apps to access Microsoft 365 resources programmatically.

You still need ACS in some cases

But, as always, it all is not so simple, as

  • there are still plenty of 3-rd party applications written and used widely that require ACS-based permissions. Moreover, there are still some 1-st party applications (Microsoft apps and services) that require ACS-based permissions
  • though Microsoft Graph API is good and provide a lot of functionality and is developing rapidly, it cannot cover all SharePoint dev’s needs, so using SharePoint REST API could be unavoidable… and that is where some complications are coming
  • permissions to specific SharePoint sites (not to all tenant sites, but to one or several SharePoint sites in tenant) for apps is done via Sites.Selected, but this works to entire site collection only. E.g. via Sites.Selected you cannot provide granular permissions (e.g. to specific list) for an app, which might be crucial in some cases, so you’d still have to use ACS-based permissions

Hopefully, Microsoft will resolve all the issues above before April 2026… But for now we have to live with both – Azure Registered applications and API permissions configured in Entra ID and with SharePoint app-only service principals and ACS-based permissions.

Azure Apps and Entra Id vs SharePoint app-only spn and ACS

Comparison between Azure Apps and Entra Id API permissions vs SharePoint app-only spn and ACS-based permissions

ACS-based SharePoint app/permissionsApps registered in Azure with Sites.Selected API permissions
support authentication with client secret only, secret is valid for 1 year exactlysupport authentication with client secret and/or certificate, custom expiration time
support granular access to SharePoint site content – e.g. to entire site collection or web (subsite) or a specific listsupport only access to entire site collection (but Microsoft is working on granular access)
Now (Sep 2024) Microsoft supports granular (list, library, Item, Document level) access for an app to a site.
support only classic SharePoint REST API and CSOMsupport both – classic SharePoint REST API and CSOM and Microsoft Graph API
app id (client id) is created via appregnew.aspx at a specific SharePoint site by site collection administratorapp id (client id) is created in Azure portal, API Sites.Selected permissions are configured via Azure portal and require tenant admin consent
permissions for the app to a site are provided at the site by site collection administrator via appinv.aspx pagepermissions for the App to to a specific SharePoint site/list/item are provided by SharePoint admin with PowerShell script or Graph API calls
logging audit log

SharePoint app-only service principal and ACS-based permissions

Since SharePoint app-only service principals and ACS-based permissions were introduced for SharePoint 2013 as part of Add-Ins feature – there are plenty of articles from Microsoft and MVPs and SharePoint gurus on this. But I would like to highlight one thing:

  • AppRegNew page creates service principal and allows authentication
  • AppInv page provides permissions and allows authorization to SharePoint

Check SharePoint AppRegNew.aspx and AppInv.aspx for details

Recommended transition tactics

For developers

  • Prioritizing using Microsoft Graph API.
  • In cases Graph API does not provide required functionality – it’s ok to use SharePoint API, but please ensure certificate is used (not secret).

For SharePoint admins

  • Encourage users register applications in Azure (not in SharePoint)
  • Disable ability for site owners register service principals in SharePoint via appregnew.aspx
    Your users will start seeing “Your SharePoint tenant admin doesn’t allow site collection admins…” message (see details), but that’s ok.
  • Create a process so users can request application permissions to SharePoint. Provide Sites.Selected permissions by default. Consider automation.
    In rare cases when 3-rd party apps require legacy ACS-based permissions, it would be you (SharePoint service admin) who will provide ACS-based access to sites.
    Track this activity (so you know for whom this ACS-based permissions were provided).
    Inform every developer that ACS will be gone.
  • Keep audit logs
    Starting today and until it’s over you’d get audit logs from Microsoft 365 purview center – consider selecting all events anyone visited appinv.aspx page.
  • In March-April 2025 (1 year before) ACS EOL, start notifying developers who use ACS.
    You can get list of developers combining
    – audit log data
    – report from Entra Id on apps owners
  • In advance ( let say, starting September 2025) you can try to temporary switch off ACS (“scream test”).

References