Category Archives: Microsoft 365

Restricted SharePoint Search rationale

Restricted SharePoint Search is a new (2024) Microsoft 365 feature that should help Copilot and general search results be more relevant, especially in large Microsoft 365 environments.

The problem background

When you have a really big number of sites – it is very difficult to keep them all in a well-managed state, e.g. to have reasonable (minimal) permissions provided to each site. So the typical situation (unfortunately) is: we have a lot of overshared sites. There are also a lot of ownerless sites where permissions are not managed. We know that search is security-trimmed, i.e. a user can get search results from content he/she already has access to. But with overshared sites – users get results they should not be able to see. With regular search experience – a user can see with his own eyes the source of the content he/she gets results from – so user can understand that results are coming from sites user should not have access to (overshared sites). But when it comes to AI-based search (Copilot) – user is getting answers, but he/she do not always know the source of that data.

So the problem is – we want to ensure Copilot is trained on a proper set of data and results are curated to users needs and access permissions. So for Copilot we really need to exclude from search scope such sites we are not sure content is valid, accurate and properly secured. We do not want users to get garbage or exposed sensitive information as an authoritative answer from Copilot.

The solution

This is where Restricted SharePoint Search feature should help, as with this feature your can restrict organization-wide search (and Copilot) to a curated list of SharePoint sites – “allowed sites” – public sites that passed attestation and where permissions are checked and data governance policies are applied, and content user work with on daily basis – his/her own documents and content shared with user directly (check details on Microsoft’s How does Restricted SharePoint Search work) – e.g. content user is supposed to have access to normally.

Excluded from search scope would be sites shared with user indirectly, e.g. something that was shared with everyone.

The root cause

Interesting, that with this feature Microsoft is not solving the real issue, but hiding (concealing) the real issue and just making Microsoft 365 to look more secure.

The real problem (root cause) is over-sharing data. But Microsoft already sold us SharePoint (and then Microsoft 365). And now Microsoft is trying to sell us Copilot, so they “solved” the over-sharing issue with “let us limit search” solution instead of “let’s fix oversharing”.

Note 1: Restricted SharePoint Search feature is free – i.e. it is included in standard Microsoft 365 license. Do not be confused with site access restriction policy – feature that require SharePoint Premium license and allows to restrict access to some SharePoint sites with specific groups only.

Note 2: I know that Microsoft is trying to address over-sharing issue as part of their SharePoint Premium (SharePoint Advanced Management) package, e.g. with AI Insights and Data access governance insights – reports that can help prevent oversharing by detecting sites that contain potentially overshared or sensitive content. With Manage content lifecycle we’d decrease amount of “garbage” or outdated content.
But SharePoint Advanced Management is licensed separately, when Restricted SharePoint Search is free.

Note 3: I know that users are an even more real problem because they tend to simplify and share information irresponsibly.

References

New Microsoft Graph Connector service plan

Microsoft Graph connectors allow your organization to index third-party data into Microsoft Graph. Microsoft Graph connectors enable Microsoft 365 Copilot better as it has more information relevant to your organization to answer prompts.

According to Microsoft, Microsoft 365 will soon include a new service plan, Graph Connectors Search with Index, offering a 50 million item index limit per tenant at no cost. Rollout starts September 2024.

Microsoft Search and Intelligence - Data Sources. 
You can build and customize connections managed by your organization. These can index data from apps such as Salesforce, Oracle SQL and Azure DevOps. Connections listed as Search under Connected experiences count toward your search connection quota utilization.

Previously, to index third-party data into Microsoft Graph through Microsoft Graph connectors, you either needed to have a built-in entitlement through specific licenses (e.g., 500 items of index quota per Microsoft Copilot for Microsoft 365 license) or purchase add-on quota. With this change, the index quota per license entitlement is removed, as is add-on cost for additional quota. You now receive an entitlement of 50 million items for each tenant.

Each entity (or record) from the source data system that you add to Microsoft Graph can be considered an item which then shows up as a unique citation in Copilot’s responses, as a unique search result in Microsoft Search, etc. Depending on the type of data source, 1 item is –

  • 1 document (word, excel, ppt, pdf, etc.) in file share
  • 1 wiki page in Confluence
  • 1 webpage in a website
  • 1 ticket/issue in Jira

Total quota utilized is calculated in terms of total items stored in the index. Updates/changes to an item are not counted in any manner. There are no cost implications of updating an item multiple times. It still counts as 1 item only.

Applicable to subscriptions: Office 365 E1, Office 365 E3, Office 365 E5, Microsoft 365 E3, Microsoft 365 E5, Microsoft 365 F1, Microsoft 365 F3, Office 365 F3, Microsoft 365 Business Basic, Microsoft 365 Business Standard, Microsoft 365 Business Premium, Office 365 G1, Office 365 G3, Office 365 G5, Microsoft 365 G3, Microsoft 365 G5, Office 365 A3, Office 365 A5, Microsoft 365 A3, Microsoft 365 A5

Update SharePoint Site Title: GUI vs PowerShell

If you need to update a SharePoint site title (site name) programmatically (e.g. with PowerShell), and if this site is a group-based site (e.g. Microsoft Teams team site or Viva Engage community site or…) – you should not update SharePoint site title, but you should update group display name instead. Here is why.

In Microsoft 365 there is no sync from SharePoint site title to a group name. When you are updating SharePoint site title with GUI – you can see that new site title becomes new group/team name as well. So you might think that if you update SharePoint site title – Microsoft synchronizes it to connected group name. That’s not true. Actually when you are updating a group-based (e.g. teams-connected) SharePoint site title with GUI – Microsoft updates group first, then syncs updated group display name to SharePoint site name (title).

Here is the proof:

That’s a network trace I got with browser dev tools when I renamed site (updated site title) with GUI. So you can see the first API call is to update group, then group properties are synced back to site.

When we are updating a standalone site title – we are not seeing these calls.

So, if you need to update group-based site title programmatically – you must update group instead.

# does not work for group-based (e.g. Teams) sites:
Set-PnPTenantSite -Identity ... -Title "New Site Title"

# instead, you'd update group display name 
Set-PnPMicrosoft365Group -Identity ... -DisplayName "New Display Name"
# and site title will be updated accordingly

References:

Who is Microsoft forms form owner

It is a very common situation in Microsoft 365 when someone creates a form and the form works perfectly, but then the form stopped working and nobody knows who was (or who is) this Microsoft form owner. Below is how to detect the form owner based only on existing form link. You can find out is it a group or a user.

How to find Microsoft forms form owner

the steps are:

1. Use form link

You can use collaborator form link that looks like:
https://forms.office.com/Pages/DesignPage.aspx?FormId=<formId>
or responder form link that looks like: https://forms.office.com/Pages/ResponsePage.aspx?id=<formId>
(or short one: https://forms.office.com/r/kDKaHDauj7)

so just follow the link (use it in your browser)

2. Open browser developers tool – just hit F12 in browser or select “open developer tool” from menu. Inspect the network traces.

You need to find a request Url that starts with https://forms.office.com/formapi/api/…
(you can use filter as below for “formapi”).

You’d refresh your page, or complete and submit the form until this url appears under network traces like this:

3. Then copy request Url to notepad as text. Bingo!

In the url example below:
https://forms.office.com/formapi/api/tc05faac-c82a-5b9d-b0c5-1f64b6755421/groups/f28f8c19-52cb-435c-948c-4c5619c943b7/forms...

The “tc05faac-c82a-5b9d-b0c5-1f64b6755421” id is the form owner’s tenant id
“/groups/” indicates that this specific form is owned by group, and
the “f28f8c19-52cb-435c-948c-4c5619c943b7” is the owner group id in EntraId

In case the form is owned by user, the Url would look like
https://forms.office.com/formapi/api/tc05faac-c82a-5b9d-b0c5-1f64b6755421/users/f6351c57-e247-528e-90ab-5i3d50c235b6
where
“/users/” indicates that the form belongs to a user and
“f6351c57-e247-528e-90ab-5i3d50c235b6” is the id of the user who owns the form

This hack works also for users who already left the company (account is disabled).

Note:
If you have an SSO in your org and cannot find this call under network – try different browser or incognito mode or logging out before the call – as what you need appears at early stages – even before authentication or when you submit the form

Some other tricks:

Having a collaborator or long responder link – I can say the form is owned by a person if the form id is 80 characters length, and the form is owned by group – if the form id is 88 characters length

References

Microsoft 365 SharePoint Archive: deep dive

Microsoft announced SharePoint Archive in 2023 and make the feature generally available in Apr 2024. Though there are good Microsoft’s articles on how to enable and configure SharePoint Archive, as well as some FAQ pages, there are still a lot of questions regarding behavior details, e.g.

  • what happens with Team content if the group-based site is Archived
  • is there an API or how do we archive/restore sites programmatically
  • would MS Graph Search API work for archived sites

I have just activated the feature and I’m planning updating this page with my gotchas and findings…

Why Archive?

  • If the site is not used, but you are not ready to delete it (or cannot delete it for compliance) – you can save money on storage by archiving site.
    – Regular SharePoint Storage = $0.2 per GB per month
    – Archived Storage = $0.05 per GB per month

Reactivation fee

How much is to restore a site from Archive?
Microsoft says restore is free within 7 days. After 7 days it’ll cost $0.6 per GB.
In the example below Microsoft charges me $1 to restore a simple OotB site with no documents:

This amount is based on the retail price for reactivations. Your actual charges may be lower, and can be seen in Microsoft 365 Archive bill

Microsoft says “This amount is based on the retail price for reactivations. Your actual charges may be lower, and can be seen in Microsoft 365 Archive bill.”

Reactivate site. 

You'll be charged a reactivation fee. This reactivation fee is based on the retail price for reactivations. Your actual charges may be lower, and can be seen in Microsoft 365 Archive bill.

The site will move back to Active sites page and start consuming active storage. This action can't be cancelled once it starts.
Estimated reactivation fee
$1

Another confirmation is requested:

Reactivate site.

You’ll be charged a reactivation fee. This reactivation fee is based on the retail price for reactivations. Your actual charges may be lower, and can be seen in Microsoft 365 Archive bill.

The site will move back to Active sites page and start consuming active storage. This action can’t be cancelled once it starts.
Estimated reactivation fee
$1

Reactivation request submitted
It will take up to 24 hours for the site to reactivate and move to the active sites page

After a few days I saw cost “<0.01$”

===========

To be continued…

What if you get “The site is archived” message?

Microsoft recently (Apr 2024) announced general availability for it’s new SharePoint Archive feature (learn more). So if you are seeing “The site is archived” and “A SharePoint Administrator archived this site. If you need access, ask an admin to reactivate it.” error message and the page in your browser is “sharepointerror.aspx?scenario=SiteArchived” then… guess what… your site was archived.

And if you need this site – please reach your SharePoint admin as soon as possible, as reactivating the site within 7 days is free, otherwise it might cost your company some dollars.

Though the page Url is “https://yourtenant.sharepoint.com/_layouts/15/sharepointerror.aspx?scenario=SiteArchived” and the page title is “Error”:

this is not an error but just a new SharePoint feature 🙂

Dealing with Ownerless Groups in large Microsoft 365 environments

Microsoft 365 groups is a key concept in today’s collaboration landscape that includes Microsoft Teams, Viva Engage, SharePoint etc. Access to resources is organized via groups. It is essential that every Microsoft 365 group has an owner (owners) so we have somebody to enforce Collaboration governance through.

Scenario

Let say you administer a large Microsoft 365 environment (e.g. ~100k+ users and/or ~50K+ sites) and after some years you have a lot of ownerless groups and sites (around 5k probably), and a lot of inactive groups and sites (maybe 15k). You are getting more and more ownerless groups – dozens each week. You are thinking of stopping bleeding and cleaning this up…

Out-of-the-box we have Microsoft 365 groups expiration policy and Microsoft 365 ownerless groups policy. You might also have some 3-rd party tools implemented – e.g. ShareGate, SysKit Point.

If you do not care – you might just activate both OotB Microsoft policies – via GUI – they are simple to activate. But once you activated policies – they will trigger thousands of emails. Now imagine a person is getting dozens of emails asking him/her to be an owner or to renew the group that probably he/she has no idea about… What will happen next? People will probably ignore these alerts. Then? Groups and sites will be automatically deleted. And then? Right, there will be a huge noise and many angry users and high-priority tickets and you will have to restore sites/teams and finally you’ll have to deal with all that mess manually.

So, what is the right way to clean-up a large Microsoft 365 environment from ownerless and inactive teams, groups sites? Not a trivial question, hah?

Solution

Disclaimer: I’m sharing here my personal opinion with no obligations or warranty etc., so you’d dig into all the technologies used and based on your particular situation build your own plan. But my personal opinion is based on my 15+ years experience with SharePoint, including really large environments.

Note: It is always a good idea to discuss your plans with you org’s communication team and helpdesk/service-desk to adjust clean-up activities with other initiatives and let other people be prepared.

High-level steps for group-based Sites:

  • consider implementing Minimum 2 owners per group policy to stop bleeding. Currently Microsoft 365 does not have such functionality, so consider 3-rd party tool like SysKit Point or custom PowerShell script that sends notifications
    • apply this policy to groups where you already have 2+ owners – it’ll be safe
    • apply this policy to all other groups by chanks
  • consider custom PowerShell clean-up, e.g. you can simply delete groups with no owners and no members and/or inactive groups with no content and/or groups that are inactive for a long time (this must be aligned with business and legal)
  • implement Microsoft’s Ownerless groups policy in “Clean-Up” configuration; there are some tricks and gotchas worth a separate post, but in short
    • avoid scoping down this policy via people (security groups)
    • implement it for all groups all users with 6-7 weeks and custom e-mail template
  • implement Microsoft groups expiration policy in “Clean-Up” configuration… again, there are a few different strategies – see this article
  • change Microsoft Ownerless groups policy configuration to a “Permanent” mode configuration set
  • (or) change Microsoft 365 groups expiration policy with a “Permanent” mode configuration
  • (or) develop and implement custom staged decommissioning process – kind of “last chance” set of scripts to discontinue groups that are still ownerless after all efforts above. Staged means we do not just delete these groups, but e.g. we can
    – rename ownerless groups
    – convert groups from public to private
    – set teams to archived mode
    – exclude sites from copilot search with “Restricted SharePoint Search” etc.
    – set site to no-access mode
    – remove members from the group
    – and finally delete the group with connected team team and site
    I have a separate article on custom staged decommissioning process

Note: There will always be ownerless groups in large environment. We have to live with it. So all steps above – think of it as a processes – we’d need to do it on regular basis.

All above was mostly about group-based sites (as we have OotB Microsoft policies for groups), but we probably have the same problem (or even worth) with standalone sites (that would be a separate topic).

You cannot use Power BI to visualize this list issue

If you are working with SharePoint Online list and select Integrate – Power BI – Visualize the list, but it gives you error message “You cannot use Power BI to visualize this list”, “Looks like the feature for visualizing lists is turned off. Please contact your admin to enable this feature”:

You cannot use Power BI to visualize this list

The issue appears to be not in SharePoint, but in Power BI. Note it says “You cannot use Power BI to visualize this list” and “Looks like the feature for visualizing lists is turned off. Please contact your admin to enable this feature.”

Also the url of this page is Power BI Url:
“https://app.powerbi.com/sharepointlist?spListId=%7Bd3b56”, so you’d need contact Power Platform Administrators, not SharePoint administrators.

Power BI administrator would go to Microsoft Fabric Admin portal

and ensure “Integration with SharePoint and Microsoft Lists” is Enabled for the entire organization or for specific security groups. In the last case – ensure user who is getting “You cannot use Power BI to visualize this list” is added to at least one of the groups but not added to “Except specific groups”.

If the user is allowed under “Integration with SharePoint and Microsoft Lists” so “Users in the organization can launch Power BI from SharePoint lists and Microsoft Lists. Then they can build Power BI reports on the data in those lists and publish them back to the lists.” then, normally, user would see:

and something like: