Tag Archives: Restricted SharePoint Search

With Restricted SharePoint Search an admin can restrict both organization-wide search and Copilot experiences to a curated set of SharePoint sites – sites that you have checked the permissions and applied data governance for. Sites from this “allowed list” can participate in organization-wide search and Copilot experiences.

Restricted SharePoint Search Deep Dive

Leave a reply

Restricted SharePoint Search is a new Microsoft feature to mitigate sites oversharing issue when you are implementing Copilot. The feature is documented here, but still I have some questions, e.g.:

  • How about external data? Copilot can use external data to learn from via agents and connectors. But would Restricted SharePoint Search if implemented allow data from external connectors to be used in copilot?
  • “Users’ OneDrive files, chats, emails, calendars they have access to” – means own data for every single user or all shared OD data?
  • What exactly is “Files from their frequently visited SharePoint sites”? I mean, how frequently user needs to visit site for this?
  • What exactly means “Files that the users viewed, edited, or created.”
  • What about teams chat messages, e-mails, viva engage messages?
  • “Files that were shared directly with the users” – does that mean “individual files shared” or can include folders, libraries, sites?
  • If user is a member of a teams – would all team content included?
  • It says “Files…” but would site pages be included? Or list items? Or list items attachments? Pages is something that people use to create wiki to share knowledge.
  • How long it takes for Microsoft 365 to start restricting results after Restricted SharePoint Search is enabled
  • How to deal with “You do not have the required license to perform this operation”

Here I’m going to answer the questions above.

So far I build a test scenario using my dev tenant that includes multiple collaborated users and content in the form of files, pages, list items and messages spreaded across multiple sites falling into different categories of Restricted SharePoint Search allowed content.

You do not have the required license…

If you are getting “You do not have the required license to perform this operation” when you are trying Get-SPOTenantRestrictedSearchMode or Get-PnPTenantRestrictedSearchMode – that means there is no Copilot for Microsoft 365 licenses assigned to tenant yet. This feature – Restricted SharePoint Search – works only when at least one Copilot license is assigned to tenant.

… TBC

References

Restricted SharePoint Search rationale

Leave a reply

Restricted SharePoint Search is a new (2024) Microsoft 365 feature that should help Copilot and general search results be more relevant, especially in large Microsoft 365 environments.

The problem background

When you have a really big number of sites – it is very difficult to keep them all in a well-managed state, e.g. to have reasonable (minimal) permissions provided to each site. So the typical situation (unfortunately) is: we have a lot of overshared sites. There are also a lot of ownerless sites where permissions are not managed. We know that search is security-trimmed, i.e. a user can get search results from content he/she already has access to. But with overshared sites – users get results they should not be able to see. With regular search experience – a user can see with his own eyes the source of the content he/she gets results from – so user can understand that results are coming from sites user should not have access to (overshared sites). But when it comes to AI-based search (Copilot) – user is getting answers, but he/she do not always know the source of that data.

So the problem is – we want to ensure Copilot is trained on a proper set of data and results are curated to users needs and access permissions. So for Copilot we really need to exclude from search scope such sites we are not sure content is valid, accurate and properly secured. We do not want users to get garbage or exposed sensitive information as an authoritative answer from Copilot.

The solution

This is where Restricted SharePoint Search feature should help, as with this feature your can restrict organization-wide search (and Copilot) to a curated list of SharePoint sites – “allowed sites” – public sites that passed attestation and where permissions are checked and data governance policies are applied, and content user work with on daily basis – his/her own documents and content shared with user directly (check details on Microsoft’s How does Restricted SharePoint Search work) – e.g. content user is supposed to have access to normally.

Excluded from search scope would be sites shared with user indirectly, e.g. something that was shared with everyone.

The root cause

Interesting, that with this feature Microsoft is not solving the real issue, but hiding (concealing) the real issue and just making Microsoft 365 to look more secure.

The real problem (root cause) is over-sharing data. But Microsoft already sold us SharePoint (and then Microsoft 365). And now Microsoft is trying to sell us Copilot, so they “solved” the over-sharing issue with “let us limit search” solution instead of “let’s fix oversharing”.

Note 1: Restricted SharePoint Search feature is free – i.e. it is included in standard Microsoft 365 license. Do not be confused with site access restriction policy – feature that require SharePoint Premium license and allows to restrict access to some SharePoint sites with specific groups only.

Note 2: I know that Microsoft is trying to address over-sharing issue as part of their SharePoint Premium (SharePoint Advanced Management) package, e.g. with AI Insights and Data access governance insights – reports that can help prevent oversharing by detecting sites that contain potentially overshared or sensitive content. With Manage content lifecycle we’d decrease amount of “garbage” or outdated content.
But SharePoint Advanced Management is licensed separately, when Restricted SharePoint Search is free.

Note 3: I know that users are an even more real problem because they tend to simplify and share information irresponsibly.

References

Dealing with Ownerless Groups in large Microsoft 365 environments

1 Reply

Microsoft 365 groups is a key concept in today’s collaboration landscape that includes Microsoft Teams, Viva Engage, SharePoint etc. Access to resources is organized via groups. It is essential that every Microsoft 365 group has an owner (owners) so we have somebody to enforce Collaboration governance through.

Scenario

Let say you administer a large Microsoft 365 environment (e.g. ~100k+ users and/or ~50K+ sites) and after some years you have a lot of ownerless groups and sites (around 5k probably), and a lot of inactive groups and sites (maybe 15k). You are getting more and more ownerless groups – dozens each week. You are thinking of stopping bleeding and cleaning this up…

Out-of-the-box we have Microsoft 365 groups expiration policy and Microsoft 365 ownerless groups policy. You might also have some 3-rd party tools implemented – e.g. ShareGate, SysKit Point.

If you do not care – you might just activate both OotB Microsoft policies – via GUI – they are simple to activate. But once you activated policies – they will trigger thousands of emails. Now imagine a person is getting dozens of emails asking him/her to be an owner or to renew the group that probably he/she has no idea about… What will happen next? People will probably ignore these alerts. Then? Groups and sites will be automatically deleted. And then? Right, there will be a huge noise and many angry users and high-priority tickets and you will have to restore sites/teams and finally you’ll have to deal with all that mess manually.

So, what is the right way to clean-up a large Microsoft 365 environment from ownerless and inactive teams, groups sites? Not a trivial question, hah?

Solution

Disclaimer: I’m sharing here my personal opinion with no obligations or warranty etc., so you’d dig into all the technologies used and based on your particular situation build your own plan. But my personal opinion is based on my 15+ years experience with SharePoint, including really large environments.

Note: It is always a good idea to discuss your plans with you org’s communication team and helpdesk/service-desk to adjust clean-up activities with other initiatives and let other people be prepared.

High-level steps for group-based Sites:

  • consider implementing Minimum 2 owners per group policy to stop bleeding. Currently Microsoft 365 does not have such functionality, so consider 3-rd party tool like SysKit Point or custom PowerShell script that sends notifications
    • apply this policy to groups where you already have 2+ owners – it’ll be safe
    • apply this policy to all other groups by chanks
  • consider custom PowerShell clean-up, e.g. you can simply delete groups with no owners and no members and/or inactive groups with no content and/or groups that are inactive for a long time (this must be aligned with business and legal)
  • implement Microsoft’s Ownerless groups policy in “Clean-Up” configuration; there are some tricks and gotchas worth a separate post, but in short
    • avoid scoping down this policy via people (security groups)
    • implement it for all groups all users with 6-7 weeks and custom e-mail template
  • implement Microsoft groups expiration policy in “Clean-Up” configuration… again, there are a few different strategies – see this article
  • change Microsoft Ownerless groups policy configuration to a “Permanent” mode configuration set
  • (or) change Microsoft 365 groups expiration policy with a “Permanent” mode configuration
  • (or) develop and implement custom staged decommissioning process – kind of “last chance” set of scripts to discontinue groups that are still ownerless after all efforts above. Staged means we do not just delete these groups, but e.g. we can
    – rename ownerless groups
    – convert groups from public to private
    – set teams to archived mode
    – exclude sites from copilot search with “Restricted SharePoint Search” etc.
    – set site to no-access mode
    – remove members from the group
    – and finally delete the group with connected team team and site
    I have a separate article on custom staged decommissioning process

Note: There will always be ownerless groups in large environment. We have to live with it. So all steps above – think of it as a processes – we’d need to do it on regular basis.

All above was mostly about group-based sites (as we have OotB Microsoft policies for groups), but we probably have the same problem (or even worth) with standalone sites (that would be a separate topic).

DepartmentId for Enterprise Intranet Portal – Home Site and Search Scope and Relevance

Leave a reply

Scenario

Your organization use Microsoft 365. You are implementing or configuring an Intranet Portal (Home Site). Search plays an important role here – you want search be relevant to the context – i.e.

  • Official Results – if a user searches something on a company’s intranet portal – user expect “official” results, not a something from somebody’s OneDrive or Yammer chat
  • Promoted Results – so information management team can adjust search with search answers – Bookmarks, Acronyms and Q&As

Problem

Microsoft Bookmarks are working only at tenant level search – i.e. if you want bookmarks work on site level search – you need to set up site search scope as tenant.

So if you configure the Intranet Portal site (Home site) search scope to “site” or “hub” to limit results with site/hub content – you will loose “answers” functionality.

Solution

The solution is very simple:

  • Keep site search scope as tenant-wide to use answers (boormarks), and
  • Configure search verticals and query to limit results to “official” sites only

Update Query field with KQL – e.g., with something like

(path:http//contoso.sharepoint.com/sites/IntranetPortal/ OR path:http//contoso.sharepoint.com/sites/CompanyNews/ OR path:http//contoso.sharepoint.com/sites/Onboarding)/)

to get results only from “Intranet Portal” and “Company News” sites.

Keep in mind that this will affect all other SharePoint search entry points – SharePoint landing page, Office.com etc. – so although you can configure All (and Files) verticals, but it’s not recommended. It will confuse users – they expect to search for everything under “All” vertical. Instead, consider custom vertical – e.g. “Official” scope.

After configuring – It might take 1-24 hours for the change to take effect, depending on tenant size.

Service vs Site search

If you configure that at the tenant level – i.e., Microsoft 365 Administration -> Settings -> Search and intelligence ->  Customizations -> Verticals
then search results will be trimmed everywhere  – SharePoint Landing Page, Office landing page (Office.com), Office App, Bing search.
Teams search will not be affected as from Teams you only search for teams content. Same for Onedrive and Yammer.

If you want the “official” search results only under Intranet Portal and leave other search entry points unaffected – then
you need to configure the same at Home (Intranet Portal) site level: Site Settings -> Microsoft Search -> Configure search settings -> Verticals
and configure site search scope to site or hub scope. But in this case you will loose answers functionality.

Global search settings – like acronyms, bookmarks and verticals – works only if site search scope is tenant.
If site search scope is site or hub – then site-level search verticals will apply (and no answers functionality will be possible).

Home site is a root site

There might be two problems with that:

  1. if a home site configured as a root site – you KQL will look like(path:http//contoso.sharepoint.com/ OR path:http//contoso.sharepoint.com/sites/CompanyNews)and that query will not work as any site Url will match the root site Url.
  2. if you need to mention many sites in KQL – like 50 sites with an Official Information – you might hit the “number of allowed character” limit

The solution is DepartmentId property:

DepartmentId

Use DepartmentId={<Hub Site ID>} in the KQL qury and your search results will be limited to your hub content while answers will still be working too. You can even combine DepartmentId with other conditions to add more sites (that are not in hub) to search scope:

(DepartmentId={4965d9be-929b-411a-9281-5662f5e09d49} OR path:http//contoso.sharepoint.com/sites/Onboarding OR path:http//contoso.sharepoint.com/sites/CompanyNews)

It worth to mention, that DepartmentId is the property that propagated from the root of the hub site to all associated sites and their content – list items, documents and pages.

Site Property Bag

Another possible option would be – site property bag…
The ultimate goal is to provide users with “Official” results only. But official sites might not be all part of one Home site hub. We can include in All search vertical query 10, 50, 100 sites, but what if we have 10k official sites in enterprise – e.g. operated by different departments – and all of them might want to be present in search results.
So, how about – if the site is considered official – we create an indexed site property, e.g. “SiteIsOfficial” with a value “Yes”. Then we map the crawled property to a managed property – e.g. RefinableString89 – and use this managed search property in query – e.g. (SiteIsOfficial=Yes).

This is actually clever idea, but this does not work… This query would only return sites, not sites content. E.g. what was indexed as site object – will be included (including home page). But site items – i.a. documents, lists, other pages – all site content – will not be included…

So let’s get back to DepartmentId…

Rename All Vertical

Again – the ultimate goal is to give users option to have “Official” results. But they still might want to be able to search through all content.
What if we rename the default “All” vertical to “Home Site” and configure query for official results only.
Then we can create a custom vertical called “Everything” or “All” with no query limitations to give users all reasults

Update: not a good idea either… If the home site search scope is tenant – so verticals are configured and be visible at tenant level – i.e. everywhere…

Separate Official Vertical

My personal preference is to keep All vertical as real All, and create a custom Vertical “Official” for official results only where we would use query trick.

In addition, it would be nice to highlight results from official sources by using custom result type – check “Manage result layouts for SharePoint results in Microsoft Search”

Update: Restricted SharePoint Search

There is a new feature in Microsoft 365 SharePoint Online – “Restricted SharePoint Search“. With Restricted SharePoint Search you can restrict organization-wide search to a curated set of “allowed” SharePoint sites – sites that you have checked the permissions and applied data governance for.

Resources: