Tag Archives: Microsoft Search

When a user searches, Microsoft Search processes the query and parses search intent from larger phrases, using Artificial Intelligence (AI) to learn common superfluous phrases users add to their queries that don’t impact their search intent. The search results that the user has permission to see are presented on the search results page. Microsoft Search uses intelligent ranking algorithms to order results based on relevance.

Search m365 SharePoint and Teams content programmatically via MS Graph API

In the articles below I’m sharing my techniques on searching in Microsoft 365 SharePoint and Teams from application using Microsoft Graph API.
Specifically I’m covering

  • Microsoft Graph search API
  • Microsoft.Graph PowerShell module
  • PnP.PowerShell module

In two flavors:

  • Search on behalf of currently authenticated user
  • Unattended Search with daemon (also called service) applications

Index of my articles on the subject:

Video tutorials

Search through Microsoft 365 SharePoint from code

Below is how do I search Microsoft 365 content programmatically from PowerShell using MS Graph API, PowerShell PnP, Microsoft Graph module, MSAL library being authenticated as user or daemon application. Let me focus on SharePoint content here but you can use the same technique to search through other Microsoft 365 services. Also, I’ll be using PowerShell but same ideas should work for other platforms/languages – Python, C#, node.js etc.

First, we need to be authenticated

Here is how to authenticate to Microsoft 365 Graph API.

Second, we need to be authorized

To search on behalf of currently authenticated user we need delegated “Sites.Read.All” API permissions. I recommend you to add both Graph API and SharePoint API permissions as different libraries might use different API’s under the hood. Ensure you add delegated “Sites.Read.All” even if you already have “Sites.FullControl.All” as by some reason “Sites.FullControl.All” does not always work for search.

Here is how an app API permissions to search as current user should look like:

app API permissions to search as current user

Actually, for interactive authentication app ownership is not required, so we can surely use our own registered app, but also it is possible to use any other app registered in Azure and properly configured, e.g. Enterprise “PnP Management Shell” app id: “31359c7f-bd7e-475c-86db-fdb8c937548e”
How do I create and configure Azure App with delegated permissions to SharePoint for PowerShel usage

For unattended search – e.g. search on behalf of daemon app – we need application “Sites.Read.All” API permissions. Again, I suggest both Graph API and SharePoint API permissions added. Here is how an app API permissions to search as daemon app should look like:

Ensure you got admin consent for API permissions.

In case you have incorrect permissions in your app – Microsoft Graph will be kind enough to inform you exactly what you need. Example:

“Access to ChatMessage in Graph API requires the following permissions: Chat.Read or Chat.ReadWrite, ChannelMessage.Read.All. However, the application only has the following permissions granted: Sites.Read.All, User.Read”

Assuming we have configured apps – let us get started with

Microsoft Graph API

Microsoft Graph API allows search through all the Microsoft 365 content – including Exchange e-mail messages, Yammer (Viva Engage) and Teams chat messages and surely OneDrive and SharePoint content (please refer to the original doc).

Authenticate as current user to Search with Graph API

I use MSAL.PS PowerShell module to get token, then I build a headers variable

# Prerequisites
Get-Module MSAL.PS -ListAvailable | ft name, Version, Path 
# Install-Module MSAL.PS -Force -Scope CurrentUser -AcceptLicense
Import-Module MSAL.PS

# Interactive Authentication
$clientid = 'd82858e0-ed99-424f-a00f-cef64125e49c'
$TenantId = '7ddc7314-9f01-45d5-b012-71665bb1c544'
$token = Get-MsalToken -TenantId $TenantId -ClientId $clientid -Interactive
$headers = @{Authorization = "Bearer $($token.AccessToken)" }

Authenticate as service/daemon app

You’d need to update the script providing Tenant id, client (app) id and client (app) secret:

# App Authentication
$clientID = ""
$clientSc = ""
$TenantId = ""

# Construct URI and body needed for authentication
$uri = "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token"
$body = @{
    client_id     = $clientID
    client_secret = $clientSc
    scope         = "https://graph.microsoft.com/.default"
    grant_type    = "client_credentials" 
}

# Get OAuth 2.0 Token
$tokenRequest = Invoke-WebRequest -Method Post -Uri $uri -ContentType "application/x-www-form-urlencoded" -Body $body -UseBasicParsing
$token = ($tokenRequest.Content | ConvertFrom-Json).access_token
$headers = @{Authorization = "Bearer $token" }
$headers

Search m365 SharePoint and OD content with Microsoft Graph API

In this sample I limited search scope to list items only ($entityTypes = “[‘listItem’]”). Check other entity types here.

Code:

# Search
$entityTypes = "['listItem']"
$apiUrl = "https://graph.microsoft.com/beta/search/query"
$query = "*"
# body for interactive search
$body = @"
{ 
  "requests": [
    {
      "entityTypes": $entityTypes,
      "query": {
        "queryString": "$query"
      }
    }
  ]
}
"@

$res = Invoke-RestMethod -Headers $Headers -Uri $apiUrl -Body $Body -Method Post -ContentType 'application/json'
$res.value[0].hitsContainers[0].hits

If you are getting error message “SearchRequest Invalid (Region is required when request with application permission.)”:

that’s OK, just modify your body to include region like this (“region”: “NAM” for North America or “GBR” or …). Also, I can modify body with from/size for paging (technique used to iterate through search results if there are many) and return just specific fields to decrease traffic and improve performance:

# Search
$entityTypes = "['driveItem','listItem','list','drive','site']"
$entityTypes = "['driveItem','listItem']"

$query = "LastModifiedTimeForRetention<2021-01-01"
$apiUrl = "https://graph.microsoft.com/beta/search/query"
$query = "test*"
$body = @"
{ 
  "requests": [
    {
      "entityTypes": $entityTypes,
      "query": {
        "queryString": "$query"
      },
      "from" : 0,
      "size" : 5,
      "fields": ["WebUrl","lastModifiedBy","name" ],
      "region": "NAM"
    }
  ]
}
"@

$res = Invoke-RestMethod -Headers $Headers -Uri $apiUrl -Body $Body -Method Post -ContentType 'application/json'
$res.value[0].searchTerms
$res.value[0].hitsContainers[0].hits
$res.value[0].hitsContainers[0].hits.Count

We’d not use region for interactive calls or we’ll get “Region is not supported when request with delegated permission.”.

Microsoft.Graph PowerShell module

There is a Microsoft.Graph PowerShell module provided by Microsoft which simplifies authentication and search operations.

Interactive authentication code sample:

# Prerequisites
Get-Module Microsoft.Graph.Authentication -ListAvailable 
Get-Module Microsoft.Graph.Search -ListAvailable 

# Interactive Authentication
$clientid = '31359c7f-bd7e-475c-86db-fdb8c937548e'
$clientid = 'd82858e0-ed99-424f-a00f-cef64125e49c'
$TenantId = '7ddc7314-9f01-45d5-b012-71665bb1c544'
Connect-MgGraph -ClientId $clientid -TenantId $TenantId

For daemon app authentication we need a certificate configured in Azure App and installed on the user machine. Daemon app authentication code sample (please specify your tenant id, app (client) id and certificate thumbprint:

# App Authentication
$clientID = ""
$certThumbprint = ""
$TenantId = ""
Connect-MgGraph -ClientId $clientid -TenantId $TenantId -CertificateThumbprint $certThumbprint

Code sample for SharePoint search with Microsoft.Graph PowerShell module

As currently authenticated user

# Search in the current user context
$params = @{
	requests = @(
		@{
			entityTypes = @(
				"driveItem"
			)
			query = @{
				queryString = "lorem"
			}
			from = 0
			size = 25
			fields = @(
				"title"
				"description"
			)
		}
	)
}

$res = Invoke-MgQuerySearch -Body $params
$res.HitsContainers[0].Hits
   

Again, in case with app authentication – an additional parameter – region – is required:

# Search
$params = @{
	requests = @(
		@{
			entityTypes = @(
				"driveItem"
			)
			query = @{
				queryString = "lorem"
			}
			from = 0
			size = 25
			fields = @(
				"title"
				"description"
			)
                        region = "NAM"
		}
	)
}

$res = Invoke-MgQuerySearch -Body $params
$res.HitsContainers[0].Hits

It’s a good idea to explore returning object.

PnP.PowerShell module

PnP.PowerShell allows you to search through Microsoft 365 SharePoint content with PowerShell style – using command and options.

Let us authenticate interactively:

# Interactive Authentication
$clientid = 'd82858e0-ed99-424f-a00f-cef64125e49c'
$TenantId = '7ddc7314-9f01-45d5-b012-71665bb1c544'
$siteUrl = "https://s5dz3.sharepoint.com"
Connect-PnPOnline -ClientId $clientid -Tenant $TenantId -Url $siteUrl -Interactive

Authentication on behalf of service/daemon app would require certificate installed on the machine and configured in the app and look like:

# Application (daemon) Authentication
$clientID = ""
$certThumbprint = ""
$TenantId = ""
$siteUrl = "https://contoso.sharepoint.com"
Connect-PnPOnline -ClientId $clientid -Tenant $TenantId -Url $siteUrl -Thumbprint $certThumbprint

There are no differences in Microsoft 365 SharePoint Search with PowerShell code samples for interactive and daemon apps (no region parameter).

Examples:

# search
$query = "test*"
$res = Submit-PnPSearchQuery -Query $query 

# examples of query:
$query = "test*"
$query = "* contentclass:STS_ListItem_DocumentLibrary"
$query = "* author:Patti"
$query = "test* site:https://s5dz3.sharepoint.com/teams/sxc"

# examples of submitting request
$res = Submit-PnPSearchQuery -Query $query 
Submit-PnPSearchQuery -Query $query -All 
Submit-PnPSearchQuery -Query $query -MaxResults 5
Submit-PnPSearchQuery -Query $query -SortList @{"LastModifiedTime" = "ascending"} 

# exploring result object:
$res.ResultRows.Count
$res.ResultRows[0]
$res.ResultRows.Title
$res.ResultRows.OriginalPath
$res.ResultRows.LastModifiedTime

Video tutorials

Video tutorials (playlist) on how to authenticate to Microsoft 365 and Search through Microsoft 365 content from code

References

Microsoft 365 Search: built-in people search by nickname

Did Microsoft silently implement nickname search?

E.g.

I have created a user “Robert Dylan” and never used name “Bob”, but search understands that I’m looking for Robert Dylan when I’m searching for Bob or Bob Dylan or Bobby.

Important thing – result is shown under “All” and “People” verticals.

I know Microsoft claimed m365 search is backed by Turing technology – it understands you, answers your question but not just blindly display keyword occurences (announcement at Ignite Spring, more on Ignite Fall 2021)… So is that it?

But the same trick did not work with Dick for Richard or (of course) Syd for Roger Barrett:

Please check “How to configure Microsoft 365 People Search by Nickname and Full Name

Using Path property in Microsoft 365 Search Query

Using Path property in Microsoft 365 Search Query was kind of ambiguous. But now Microsoft implemented update and clarified some details. So below are some tips and tricks on filtering by site Url (path) in query field in Microsoft 365 Search verticals.

Path filter with trailing slash (“/”)

In November 2022 Microsoft rolled out an update for multiple search features, including checks on the path managed property for a trailing slash. Previously path filters were valid with and without trailing slashes.

Consider the following scenario.

Given the path filter with the contain operator (“:”)

Path:https://contoso.sharepoint.com/sites/MySite

These path could be matched with:

Path:https://contoso.sharepoint.com/sites/MySite
Path:https://contoso.sharepoint.com/sites/MySite/subsite
Path:https://contoso.sharepoint.com/sites/MySite2
Path:https://contoso.sharepoint.com/sites/MySite2/subsite
Path:https://contoso.sharepoint.com/sites/MySite3

Obviously, the match intent is unclear. Adding a trailing slash clarifies that only MySite (and below) matches. So intended matches would be only:

Path:https://contoso.sharepoint.com/sites/MySite
Path:https://contoso.sharepoint.com/sites/MySite/subsite

Using SPSiteUrl property

The other option – use the SPSiteUrl property with the full path:

SPSiteUrl:https://contoso.sharepoint.com/sites/MySite 

SPSiteUrl and Path properties use different matching strategies. When using contains operator (colon sign “:”) – SPSiteUrl will match the full value, while Path will do a “starts with” match.

DepartmentId

DepartmentId is a search managed property used under Hub sites and propagated through all associated sites content.

That means if we want to scope down search to hub site with it’s content – we can use DepartmentId property, e.g.

DepartmentId=4965d9be-929b-411a-9281-5662f5e09d49

instead of iteration through all hub sites and using path: property.

clearCache or cacheClear

Each time we need to validate a recent change in Microsoft Search configuration, we need to use query parameter that clears cache, and each time I’m not sure is it clearCache or cacheClear 🙂

So, correct is “cacheClear=true”, e.g:

https://www.office.com/search/sites?auth=2&cacheClear=true&q=bird*

Reference:

Microsoft 365 People Search by Nickname and Full Name

Scenario

Nickname (or alias or moniker or hypocorism etc.) is some another name – usually shorter than original name and widely used in maybe all countries in the world. E.g. Bob is a nickname for Robert in the US and GB, Checo is the other name for Sergio in Mexico. There might be personal names also – e.g. Bapu (father) is an word that is usually associated with Mohandas Karamchand Gandhi, also known as the Mahatma Gandhi. More examples: David “Noodles” Aaronson, Roger Keith “Syd” Barrett etc.

In Microsoft 365 we want to search for a person’s name we know – and in many cases it’s a nickname – e.g. Beth (Bethany) or Alex (Alexander). Can m365 search do that?

Unfortunately, not… At least at 19:38 on 2023-03-30
… though please check Microsoft 365 Search: built-in people search by nickname.

Solution

It takes a few steps to implement search by nickname:

  • create a custom property under SharePoint Online User Profiles service
  • fill this property with values
  • configure Search Schema – map crawled property to managed property

Step-by-Step

Create a custom property under SharePoint Online User Profiles service

  • Ensure you have a SharePoint Administrator role activated
  • Navigate to SharePoint Admin Center – more features – User Profiles – Manage user properties – New Property
  • Configure custom property according to your needs,
    – ensure “Policy Settings” “Default Privacy Setting:” Everyone is selected
    – ensure “Search Settings” “Indexed” is selected

Fill SharePoint Online User Profile Properties with values

That would be a custom solution – e.g. manual work from SPO Admin Center GUI or PowerShell script ( e.g. with some dictionary). This is required for search to pick it up, crawl the property and create crawled property – so you could proceed with search schema mapping.

Configure Search Schema – map crawled property to managed property

For the nickname to be generally available in full-text-search – i.e.
user simply enter value in a search bar and gets results – here are the steps:

  • Ensure you have a SharePoint Administrator role activated
  • Navigate to SharePoint Admin Center – More features – Search – Manage Search Schema
  • Select Crawled Properties and ensure search picked up your custom property and crawled it – check your crawled property name under Category: People.
  • Under Managed Properties – create a new managed property
    • select “Searchable”
    • under Advanced searchable settings – select Full-text index: PeopleIdx
  • map this managed property to crawled property

========================================

References

SharePoint Search: Site vs Tenant Scope, Placeholder Text, Answers and Verticals

Service vs Site search

If you configure Verticals Query at the tenant level – i.e., Microsoft 365 Administration -> Settings -> Search and intelligence ->  Customizations -> Verticals
then search results will be trimmed everywhere  – SharePoint Landing Page, Office landing page (Office.com), Office App, Bing search (but not other sites).

Teams search will not be affected as from Teams you only search for teams content. Same for Onedrive and Yammer. Sites with site or hub search scope will not be affected too.

If you configure verticals at site level: Site Settings -> Microsoft Search -> Configure search settings -> Verticals
and want this be in effect – ensure site search scope is set to site or hub scope. But in this case you will loose answers functionality.

Global search settings – like acronyms, bookmarks and verticals – works only at tenant level search or at site leve if the site search scope is set to tenant.
If site search scope is site or hub – then site-level search verticals will apply (and no answers functionality will be possible).