In the articles below I’m sharing my techniques on searching in Microsoft 365 SharePoint and Teams from application using Microsoft Graph API. Specifically I’m covering
Microsoft Graph search API
Microsoft.Graph PowerShell module
PnP.PowerShell module
In two flavors:
Search on behalf of currently authenticated user
Unattended Search with daemon (also called service) applications
Below is how do I search Microsoft 365 content programmatically from PowerShell using MS Graph API, PowerShell PnP, Microsoft Graph module, MSAL library being authenticated as user or daemon application. Let me focus on SharePoint content here but you can use the same technique to search through other Microsoft 365 services. Also, I’ll be using PowerShell but same ideas should work for other platforms/languages – Python, C#, node.js etc.
To search on behalf of currently authenticated user we need delegated “Sites.Read.All” API permissions. I recommend you to add both Graph API and SharePoint API permissions as different libraries might use different API’s under the hood. Ensure you add delegated “Sites.Read.All” even if you already have “Sites.FullControl.All” as by some reason “Sites.FullControl.All” does not always work for search.
Here is how an app API permissions to search as current user should look like:
For unattended search – e.g. search on behalf of daemon app – we need application “Sites.Read.All” API permissions. Again, I suggest both Graph API and SharePoint API permissions added. Here is how an app API permissions to search as daemon app should look like:
Ensure you got admin consent for API permissions.
In case you have incorrect permissions in your app – Microsoft Graph will be kind enough to inform you exactly what you need. Example:
“Access to ChatMessage in Graph API requires the following permissions: Chat.Read or Chat.ReadWrite, ChannelMessage.Read.All. However, the application only has the following permissions granted: Sites.Read.All, User.Read”
Assuming we have configured apps – let us get started with
Microsoft Graph API
Microsoft Graph API allows search through all the Microsoft 365 content – including Exchange e-mail messages, Yammer (Viva Engage) and Teams chat messages and surely OneDrive and SharePoint content (please refer to the original doc).
Authenticate as current user to Search with Graph API
I use MSAL.PS PowerShell module to get token, then I build a headers variable
If you are getting error message “SearchRequest Invalid (Region is required when request with application permission.)”:
that’s OK, just modify your body to include region like this (“region”: “NAM” for North America or “GBR” or …). Also, I can modify body with from/size for paging (technique used to iterate through search results if there are many) and return just specific fields to decrease traffic and improve performance:
For daemon app authentication we need a certificate configured in Azure App and installed on the user machine. Daemon app authentication code sample (please specify your tenant id, app (client) id and certificate thumbprint:
I have created a user “Robert Dylan” and never used name “Bob”, but search understands that I’m looking for Robert Dylan when I’m searching for Bob or Bob Dylan or Bobby.
Important thing – result is shown under “All” and “People” verticals.
I know Microsoft claimed m365 search is backed by Turing technology – it understands you, answers your question but not just blindly display keyword occurences (announcement at Ignite Spring, more on Ignite Fall 2021)… So is that it?
But the same trick did not work with Dick for Richard or (of course) Syd for Roger Barrett:
Using Path property in Microsoft 365 Search Query was kind of ambiguous. But now Microsoft implemented update and clarified some details. So below are some tips and tricks on filtering by site Url (path) in query field in Microsoft 365 Search verticals.
Path filter with trailing slash (“/”)
In November 2022 Microsoft rolled out an update for multiple search features, including checks on the path managed property for a trailing slash. Previously path filters were valid with and without trailing slashes.
Consider the following scenario.
Given the path filter with the contain operator (“:”)
SPSiteUrl and Path properties use different matching strategies. When using contains operator (colon sign “:”) – SPSiteUrl will match the full value, while Path will do a “starts with” match.
DepartmentId
DepartmentId is a search managed property used under Hub sites and propagated through all associated sites content.
That means if we want to scope down search to hub site with it’s content – we can use DepartmentId property, e.g.
DepartmentId=4965d9be-929b-411a-9281-5662f5e09d49
instead of iteration through all hub sites and using path: property.
where you can use objects: “Site Url”, “Site Name” and “Refinable String 0″..”Refinable String 99”. Conditions would be “is equal to”, “is not equal to”, “starts with” and “not starts with”. Or you can select “Advanced query builder” and enter KQL query.
Advanced query builder
Advanced query builder allows us to use more site properties then “Site Url”, “Site Name” and “Refinable Strings” and more conditions than “is (not) equal to” and “(not) starts with”.
E.g. we can use “Title”, “Created”, “Modified” site properties and “=”,”:”,”<“, “>”, “<=”, “>=” conditions.
Working queries examples:
created>=2022-07-21
modified>1/31/2023
created>12/31/2021 AND modified>=7/31/2022
created<=2020-11-15 OR modified>2023-02-06 (?)
created<=2020-1-15 OR modified>2023-01-31 (?)
created<=11/15/2020 OR modified>1/31/2023
title:test
SiteTitle:test
RefinableString09:Test*
RefinableString09<>Test
RefinableString09=Birding AND RefinableString08<>Included
Not working queries examples:
site:https://contoso.sharepoint.com/sites/test*
RefinableString11 = Birds # (do not use spaces in advanced query)
Path:https://contoso-my.sharepoint.com
Template:STS
Template:"SITEPAGEPUBLISHING#0"
Template:SITEPAGEPUBLISHING*
? RefinableString09<>Birding AND RefinableString08:Official
modified>31/1/2023 (should be like modified>2023-01-31
)
Query against custom site property (aka property bag value)
You can create custom site property and assign value to the property with Set-PnPAdaptiveScopeProperty or Set-PnPPropertyBagValue. Property must be with “Indexed” parameter. Once the property is set up, m365 search crawls site and creates crawled property. Then you map crawled property to some pre-created refinable string managed property. You can assign alias to this managed property.
In my test scenario I used RefinableString09 with alias SiteCustomSubject.
Site property value
Query
result
Birding
RefinableString09:Bird
does not work
Birding
SiteCustomSubject:Bird
does not work
Birding
RefinableString09:Bird*
works
Birding
SiteCustomSubject:Bird*
does not work
Birding
RefinableString09:Birding
works
Birding
SiteCustomSubject:Birding
does not work
Birding
RefinableString09:Birding*
works
Birding
RefinableString09=Birding
works
Birding
RefinableString09=Bird
does not work
Birding
RefinableString09=Bird*
does not work
Birding
SiteCustomSubject=Birding
does not work
RefinableString09<>Birding
works
RefinableString09=Birding AND RefinableString08<>Included
works
Query against multi-value property.
Site property value
Query
result
TestA TestB
RefinableString09:TestA
works
TestA TestB
RefinableString09 = ‘TestA TestB’
does not work
TestA TestB
??? RefinableString09=’Test10 Test5′
does not work
TestA TestB
RefinableString09:TestB
?
TestA,TestB
RefinableString09:Test*
works
TestA,TestB
RefinableString09=Test*
does not work
TestA,TestB
RefinableString09:Test
does not work
TestA,TestB TestA;TestB TestB TestA TestA TestB
RefinableString09:TestB
works
TestA, TestB TestB,TestA TestA TestB
RefinableString09=TestA
does not work
TestA,TestB
(basic) RefinableString09 starts with test
works
Some more findings
Modify adaptive scope
If you need to modify adaptive scope – you’d better delete it and create a new one. The reason – if you want to validate what sites are included in scope with GUI – via button “Scope details” – you want to see only sites that are in scope, but that’s not the case when you modify the scope, because if you modify the scope – you’d see sites that are not in scope with “Removed” status.
Alternatively you can use filter to filter out removed from scope sites.
what else?
What is the takeaway from this for SharePoint administrators? We would be asked to configure SharePoint the way compliance…
After many years working with SharePoint I wrote a lot of PowerShell scripts that help me support, troubleshoot, administer and secure SharePoint. So I’m sharing my scripts with you.
Manage result layouts for SharePoint results in Microsoft Search
We’re making changes to Microsoft Search. This update will allow Microsoft Search administrators to change result layouts for select SharePoint content using adaptive cards with Result Type feature in Microsoft Search administration.
The default result layouts for SharePoint sites, pages, list items and Portable document format (PDF) results can now be replaced with layouts built using adaptive cards. The changes can be made for Organization level search applicable to Office.com and SharePoint home as well as site level search on SharePoint sites. Changes for Microsoft Search in Bing will be rolled out soon. Note that the feature does not support changing of Office file search results.
This message is associated with Microsoft 365 Roadmap ID 81952
Before the change, when you add a new result type under “Search and intelligence” Customizations – it looked like this:
So there was no built-in “SharePoint” content source as an option – only custom “external” data sources.
But with the new feature implemented list of content sources for the result type will look like this:
If you choose “SharePoint and OneDrive” content source – the next option would be to select type of content:
You also can create different result types for different types of content based on properties-based rules (e.g. one result type for all sites – and a separate result type for a specific site or hub) with optional “Set rules for this type of content”:
Default site result experience would look like
Search results with modified SharePoint result type might look like:
When you modify template via Layout Designer – it is essential to know available object properties.
You can get properties from the “Available properties” below – there is also search through properties feature.
Or you can use SharePoint Search Query Tool to get metadata on search results.
It might take hours and even days for your search to start showing new layouts, but “&cacheClear=true” should help.
DepartmentId
If your sites are organized in hierarchy under Hub site – you can use DepartmentId managed property to include all hub-associated sites content
Each time we need to validate a recent change in Microsoft Search configuration, we need to use query parameter that clears cache, and each time I’m not sure is it clearCache or cacheClear 🙂