Using Path property in Microsoft 365 Search Query

1 Reply

Using Path property in Microsoft 365 Search Query was kind of ambiguous. But now Microsoft implemented update and clarified some details. So below are some tips and tricks on filtering by site Url (path) in query field in Microsoft 365 Search verticals.

Path filter with trailing slash (“/”)

In November 2022 Microsoft rolled out an update for multiple search features, including checks on the path managed property for a trailing slash. Previously path filters were valid with and without trailing slashes.

Consider the following scenario.

Given the path filter with the contain operator (“:”) 

Path:https://contoso.sharepoint.com/sites/MySite

These path could be matched with:

Path:https://contoso.sharepoint.com/sites/MySite
Path:https://contoso.sharepoint.com/sites/MySite/subsite
Path:https://contoso.sharepoint.com/sites/MySite2
Path:https://contoso.sharepoint.com/sites/MySite2/subsite
Path:https://contoso.sharepoint.com/sites/MySite3

Obviously, the match intent is unclear. Adding a trailing slash clarifies that only MySite (and below) matches. So intended matches would be only:

Path:https://contoso.sharepoint.com/sites/MySite
Path:https://contoso.sharepoint.com/sites/MySite/subsite

Using SPSiteUrl property

The other option – use the SPSiteUrl property with the full path:

SPSiteUrl:https://contoso.sharepoint.com/sites/MySite

SPSiteUrl and Path properties use different matching strategies. When using contains operator (colon sign “:”) – SPSiteUrl will match the full value, while Path will do a “starts with” match.

DepartmentId

DepartmentId is a search managed property used under Hub sites and propagated through all associated sites content.

That means if we want to scope down search to hub site with it’s content – we can use DepartmentId property, e.g.

DepartmentId=4965d9be-929b-411a-9281-5662f5e09d49

instead of iteration through all hub sites and using path: property.

Microsoft 365 Retention Policies SharePoint Adaptive Scopes Advanced Query

1 Reply

Basic query is available as GUI:

where you can use objects: “Site Url”, “Site Name” and “Refinable String 0″..”Refinable String 99”. Conditions would be “is equal to”, “is not equal to”, “starts with” and “not starts with”. Or you can select “Advanced query builder” and enter KQL query.

Advanced query builder

Advanced query builder allows us to use more site properties then “Site Url”, “Site Name” and “Refinable Strings” and more conditions than “is (not) equal to” and “(not) starts with”.

E.g. we can use “Title”, “Created”, “Modified” site properties and “=”,”:”,”<“, “>”, “<=”, “>=” conditions.

Working queries examples: 

created>=2022-07-21
modified>1/31/2023
created>12/31/2021 AND modified>=7/31/2022
created<=2020-11-15 OR modified>2023-02-06 (?)
created<=2020-1-15 OR modified>2023-01-31 (?)
created<=11/15/2020 OR modified>1/31/2023
title:test
SiteTitle:test
RefinableString09:Test*
RefinableString09<>Test
RefinableString09=Birding AND RefinableString08<>Included

Not working queries examples:

site:https://contoso.sharepoint.com/sites/test* 
RefinableString11 = Birds # (do not use spaces in advanced query)
Path:https://contoso-my.sharepoint.com
Template:STS
Template:"SITEPAGEPUBLISHING#0"
Template:SITEPAGEPUBLISHING*
? RefinableString09<>Birding AND RefinableString08:Official
modified>31/1/2023 (should be like modified>2023-01-31
)

Query against custom site property (aka property bag value)

You can create custom site property and assign value to the property with
Set-PnPAdaptiveScopeProperty or Set-PnPPropertyBagValue.
Property must be with “Indexed” parameter. Once the property is set up, m365 search crawls site and creates crawled property. Then you map crawled property to some pre-created refinable string managed property. You can assign alias to this managed property.

In my test scenario I used RefinableString09 with alias SiteCustomSubject.

Site property valueQueryresult
BirdingRefinableString09:Birddoes not work
BirdingSiteCustomSubject:Birddoes not work
BirdingRefinableString09:Bird*works
BirdingSiteCustomSubject:Bird*does not work
BirdingRefinableString09:Birdingworks
BirdingSiteCustomSubject:Birdingdoes not work
BirdingRefinableString09:Birding*works
BirdingRefinableString09=Birdingworks
BirdingRefinableString09=Birddoes not work
BirdingRefinableString09=Bird*does not work
BirdingSiteCustomSubject=Birdingdoes not work
RefinableString09<>Birdingworks
RefinableString09=Birding AND RefinableString08<>Includedworks

Query against multi-value property.

Site property valueQueryresult
TestA TestBRefinableString09:TestAworks
TestA TestBRefinableString09 = ‘TestA TestB’does not work
TestA TestB??? RefinableString09=’Test10 Test5′does not work
TestA TestBRefinableString09:TestB ?
TestA,TestBRefinableString09:Test*works
TestA,TestBRefinableString09=Test*does not work
TestA,TestBRefinableString09:Testdoes not work
TestA,TestB
TestA;TestB
TestB TestA
TestA TestB		RefinableString09:TestBworks
TestA, TestB
TestB,TestA
TestA TestB		RefinableString09=TestAdoes not work
TestA,TestB(basic) RefinableString09 starts with testworks

Some more findings

Modify adaptive scope

If you need to modify adaptive scope – you’d better delete it and create a new one. The reason – if you want to validate what sites are included in scope with GUI – via button “Scope details” – you want to see only sites that are in scope, but that’s not the case when you modify the scope, because if you modify the scope – you’d see sites that are not in scope with “Removed” status.

Alternatively you can use filter to filter out removed from scope sites.

what else?

What is the takeaway from this for SharePoint administrators? We would be asked to configure SharePoint the way compliance…

References

Adaptive scopes Retention Policies Data Lifecycle Purview

SharePoint Sites Lookup

Leave a reply

That’s a very common problem in SharePoint world. You are looking for a site owner but there is no tool available for regular user to find who owns the site.

Scenarios.

You get a link to some SharePoint site, but you do not have access to it. You requested access but nobody has responded. You need to find who is the site owner.

(To be continued)

PowerShell scripts for Microsoft 365 SharePoint

Leave a reply

After many years working with SharePoint I wrote a lot of PowerShell scripts that help me support, troubleshoot, administer and secure SharePoint. So I’m sharing my scripts with you.

It’s here: https://github.com/VladilenK/Manage-m365-with-PowerShell

Get all SharePoint and Teams sites owners report with PowerShell

1 Reply

This PowerShell script pulls all tenant sites and all sites owners. The script require app authentication with Sites.FullControl.All and Directory.Read.All permissions.
PnP.PowerShell for PowerShell 7 is used.

The script generates two reports

  • Owners report: one user per line, include: Site Url, Title, Owner e-mail, name and type
  • Sites report: one site per line, include: Site Url, Title, list of owners e-mails

Here is the script:


$connAdmin = Connect-PnPOnline -ReturnConnection -Tenant $tenantId  -Url $adminUrl -ClientId $clientid -Thumbprint $certThumbprint
$allTenantSites = Get-PnPTenantSite -Connection $connAdmin | Sort-Object Url
$allTenantSites.count

$sitesReport = @()
$ownersReport = @()
foreach ($tenantSite in $allTenantSites) {
    Write-Host $tenantSite.Url
    $connSite = Connect-PnPOnline -ReturnConnection -Tenant $tenantId  -Url $tenantSite.Url -ClientId $clientid -Thumbprint $certThumbprint
    $site = Get-PnPSite -Connection $connSite -Includes RootWeb, GroupId, Owner
    $siteOwnerEmail = ''
    $siteOwnersReport = @()
    if ($site.GroupId.Guid -eq '00000000-0000-0000-0000-000000000000') {
        $siteAdmins = Get-PnPSiteCollectionAdmin -Connection $connSite | ? { $_.PrincipalType -eq 'User' }
        $ownerType = 'Site Collection Administrator'
        $isGroupSite = $false
    }
    else {
        $siteAdmins = Get-PnPAzureADGroupOwner -Connection $connAdmin -Identity $site.GroupId.Guid
        $ownerType = 'Group Owner'
        $isGroupSite = $true
    }
    foreach ($siteAdmin in $siteAdmins) {
        if (!$siteAdmin.UserPrincipalName) {
            Get-PnPProperty -Connection $connAdmin -ClientObject $siteAdmin -Property UserPrincipalName | Out-Null
        }
        $aadUser = Get-PnPAzureADUser -Connection $connAdmin -Identity $siteAdmin.UserPrincipalName
        if ($aadUser.AccountEnabled) {
            $siteOwnerEmail += $aadUser.Mail + '; '
        }
        $siteOwnersReport += [PSCustomObject]@{
            SiteUrl     = $site.Url
            SiteTitle   = $site.RootWeb.Title
            IsGroupSite = $isGroupSite
            OwnerEmail  = $aadUser.Mail
            OwnerName   = $aadUser.DisplayName
            OwnerType   = $ownerType
            Enabled     = $aadUser.AccountEnabled
        }
    }
    $ownersReport += $siteOwnersReport
    $sitesReport += [PSCustomObject]@{
        SiteUrl     = $site.Url
        SiteTitle   = $site.RootWeb.Title
        IsGroupSite = $isGroupSite
        OwnerEmail  = $siteOwnerEmail
    }
}

$ownersReport.count
$sitesReport.count

Source code: https://github.com/VladilenK/Manage-m365-with-PowerShell

Manage result layouts for SharePoint results in Microsoft Search

1 Reply

Microsoft is improving Search (MC489165):

Manage result layouts for SharePoint results in Microsoft Search

We’re making changes to Microsoft Search. This update will allow Microsoft Search administrators to change result layouts for select SharePoint content using adaptive cards with Result Type feature in Microsoft Search administration.

The default result layouts for SharePoint sites, pages, list items and Portable document format (PDF) results can now be replaced with layouts built using adaptive cards. The changes can be made for Organization level search applicable to Office.com and SharePoint home as well as site level search on SharePoint sites. Changes for Microsoft Search in Bing will be rolled out soon. Note that the feature does not support changing of Office file search results.

This message is associated with Microsoft 365 Roadmap ID 81952

Before the change, when you add a new result type under “Search and intelligence” Customizations – it looked like this:

result type content sources

So there was no built-in “SharePoint” content source as an option – only custom “external” data sources.

But with the new feature implemented list of content sources for the result type will look like this:

SharePoint and OneDrive content source

If you choose “SharePoint and OneDrive” content source – the next option would be to select type of content:

Select type of content and set rules

You also can create different result types for different types of content based on properties-based rules (e.g. one result type for all sites – and a separate result type for a specific site or hub) with optional “Set rules for this type of content”:

Default site result experience would look like

Search results with modified SharePoint result type might look like:

When you modify template via Layout Designer – it is essential to know available object properties.

You can get properties from the “Available properties” below – there is also search through properties feature.

Or you can use SharePoint Search Query Tool to get metadata on search results.

It might take hours and even days for your search to start showing new layouts, but “&cacheClear=true” should help.

DepartmentId 

If your sites are organized in hierarchy under Hub site – you can use DepartmentId managed property to include all hub-associated sites content

DepartmentId is just a hub site Id

… to be continued …

References

clearCache or cacheClear

Leave a reply

Each time we need to validate a recent change in Microsoft Search configuration, we need to use query parameter that clears cache, and each time I’m not sure is it clearCache or cacheClear 🙂

So, correct is “cacheClear=true”, e.g:

https://www.office.com/search/sites?auth=2&cacheClear=true&q=bird*

Reference:

Microsoft 365 SharePoint: prevent throttling with RateLimit headers

Leave a reply

Bert Jansen (Microsoft) revealed some details on throttling when you access Microsoft 365 programmatically – via Microsoft Graph or CSOM and guided developers on how to regulate request traffic for optimized throughput using RateLimit headers (Here).

Demystifying SharePoint throttling

Throttling is necessary to ensure that no single user or application consumes too many resources compromising the stability of the entire system, which is used by many clients.

Throttling happens at

  • User (there are user request limits. Microsoft counts all requests linked to user
  • Application (Delegated or Application permissions)
    • Resource units per app per minute
    • Resource units per app per day
  • Farm – Spike protection

Very common reason for throttling – when an Application (Delegated or Application permissions) reaches “Resource units per app per minute” threshold.

Usually you catch HTTP errors 429 or 503, wait for some time (respect Retry-after header) and try again.

SharePoint provides various APIs. Different APIs have different costs depending on the complexity of the API, but Microsoft favor Graph API over SharePoint REST/CSOM. The cost of APIs is normalized by SharePoint and expressed by resource units. Application’s limits are also defined using resource units.

Quota depends on tenant size.

Resource unit limits for an application in a tenant (please refer to the Microsoft article)

Predefined costs for Microsoft Graph calls:

Assuming 2 resource units per request is a safe bet.

Links