Tag Archives: SharePoint Online

Office 365 Search scopes

Search is everywhere in Microsoft 365. You can search from SharePoint, Teams, Delve, Yammer etc.

But! You cannot search for anything from everywhere!

Search for your Teams chat messages works only in Teams.
But from Teams you cannot search for regular (non-group) sites and public teams sites
All descriptions are totally out of search (e.g. site description, library/list description – including Yammer groups, Teams and regular sites).
Public Team Sites content is not searchable from Teams and Yammer

So, what are the scopes of each search entry point in Office 365 and is there an entry point you can search for everything?

Search scopes	SharePoint Search center	SharePoint home Office portal Office desktop app Delve	Teams	Bing
SharePoint content	Yes	Yes		Yes
Teams content	Yes	Yes	Yes	Yes
Teams chats		(*1)	Yes	Yes
Yammer content	Yes	Yes		Yes
Yammer chat		(*1)		Yes
User profiles	Yes	Yes
Email

(*1) Microsoft announced they are working on bringing conversations (both Teams chats and Yammer) to SharePoint landing page first, then to Office home page.

Detailed:

	Scope	Out of Scope
SharePoint Search Center	– all sites content (Teams, Yammer, regular), – user profiles – OneDrive	Teams chat Yammer chat
SharePoint Landing Page	same as SharePoint Search center but Teams chats and Yammer Conversations are coming	same as SharePoint Search Center
Office.com	same as SharePoint (Teams chats and Yammer Conversations are coming after SharePoint)	same as SharePoint
Delve
Teams	Teams content Teams chat	OneDrive Yammer User Profiles regular SharePoint sites
Bing	Everything*	* except people profiles content (e.g. about me)

Seems like the only tool you can search for EVERYTHING with is Microsoft Bing:

After Microsoft add Teams chats and Yammer conversations to SharePoint landing page search scope (then to Office home page) – it’ll be the best place to search from for everything.

More on Microsoft Search vs SharePoint Search and Microsoft Search RoadMap

Microsoft Office 365 Search: Find what you need with Microsoft Search in Bing

It is possible customize Modern Microsoft Search pages with PnP Modern Search

SharePoint PnP roadmap

Good news!
On Sep, 18 during the SIG community call, PnP Team shared their plans on PnP Sites Core library and PnP Core SDK.
“PnP Sites Core v4” library and “PnP Core SDK v1” with .net core support (.net Standard 2.0) – expected in December 2020!

PnP PowerShell v4 for SPO library built for .Net Standard 2.0 / PowerShell 7 will be released in Dec 2020 as well.

How to delete a large SPO list and/or all items in a large SPO list

Scenario 1: You have a large (>5k items) list in SharePoint Online.
You need to delete this list.

Scenario 2: You have a large (>5k items) list in SharePoint Online.
You need to delete all the list items, but keep the list.

Deleting a large SharePoint Online list

GUI: Microsoft improved SharePoint, so now it takes ~1 second to delete any SharePoint list, including 5000+ items list via GUI.

PowerShell: “Remove-PnPList -Identity $list” command works very fast – ~1 second to delete entire list with >5000 items.

Delete all items in a large SharePoint Online list

In this scenario we need to keep the list, but make it empty (clean it out).

GUI: You can change the list view settings “Item Limit” to <5000, but (at least in my experience) when you try to select, let say, 1000 items and delete them via GUI – it says “775 items were not deleted from large list”:

so this option seems like not a good one.

ShareGate: 3-rd party tools like Sharegate, SysKit give a good results too.

PowerShell

Try this PowerShell command with ScriptBlock:

Get-PnPListItem -List $list -Fields "ID" -PageSize 100 -ScriptBlock { Param($items) $items | Sort-Object -Property Id -Descending | ForEach-Object{ $_.DeleteObject() } }

or this PowerShell with batches:

$batch = New-PnPBatch
1..12000 | Foreach-Object { Remove-PnPListItem -List $list -Identity $_ -Batch $batch }
Invoke-PnPBatch -Batch $batch

for me both methods gave same good result: ~17 items per second ( ~7 times faster than regular).

PnP.PowerShell batch vs ScriptBlock

How fast are PnP batches? What is better in terms of performance – ScriptBlock or Batching? Here are my measurements:

Time elapsed, seconds	with batches	with scriptBlock	without batches
Add-PnPListItem (100 items)	4-10 seconds		42-120 seconds
Add-PnPListItem (500 items)	20-40 seconds		230-600 seconds
Add-PnPListItem (7000 items)	314-560 seconds
Remove-PnPListItem (1000 items)	58-103 seconds	58 seconds	430-1060 seconds
Remove-PnPListItem (7000 items)	395-990 seconds	397-980 seconds

both – PnP PowerShell batches and ScriptBlocks are 7-10 times faster than plain PnP PowerShell!

Note… For the sake of history: It used to be like that for 5k+ lists:
“Remove-PnPList” fails with a message “The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator”. Deleting with GUI fails too.

References:

How to create a Sub-Site if subsites creation is disabled

Update: Microsoft is deploying an updated version of “Disable Subsites” feature:

This update makes the setting options for new subsite creation easier to understand and prevents users from being able to create subsites using alternate paths when the subsite setting is disabled.

Admins in the SharePoint admin center can choose to either enable or disable subsite creation across sites or enable for classic sites only. Now, when disabling subsite creation, not only will the subsite option be hidden from the command bar including classic but also users will not be able to create new subsites directly through an URL or API.

The option: Hide the Subsite command has been renamed to Disable subsite creation for all sites and will also hide the subsite creation command (including classic) and disable users from being able to create new subsites through a URL or API.
The option: Show the Subsite command only for classic sites, has been renamed to Enable subsite creation for classic sites only.
The option: Show the Subsite command for all sites, has been renamed to Enable subsite creation for all sites.

Update is applied. What’s Next?

After this update is applied, if you have “Subsite Creation” set to “Disable subsite creation for all sites”, then if any attempt to create a subsite – you’ll get an error message “Sorry, something went wrong. New subsites are not available for your organization. Create a new site instead.”

Site is a new folder

Microsoft recommend “flat structure”, i.e. no subsites. So SPO admins are disabling subsites creation at tenant level. Did you know that you still can create subsite? Let me explain how it is done.

If creation subsites is allowed, you should be able to see it like this:

But actually subsites are not always best practice. Microsoft recommend “flat structure”, i.e. instead of subsite you should have a separate site collection, and if you need a hierarchy and navigation – use Hub sites. So, in Office 365 SharePoint admins usually “disable” SubSites creation:

Now, you see, SubSites are not really disabled, but only the button to create subsites is hidden: “This controls whether the Subsite command appears on the New menu on the Site contents page”.

Anyway, the result is: you are not able to create a SubSite (web) in SharePoint Online:

Actually there are at least 3 options to create a SubSite:

Option 1. Create a SubSite in Classic mode.

Step 1: Select “Site Contents” page
Step 2: Click “Return to classic SharePoint”
Step 3: Create SubSite

Option 2. Create a SubSite from “Sites and Workspaces” page

Step 1: Go to “Site Settings”
Step 2: Select “Sites and Workspaces” page (site/_layouts/15/mngsubwebs.aspx)
Step 3: Create a SubSite

Option 3: use PowerShell PnP

Step 1: Install PowerShell PnP
Step 2: Connect to your site with PnP
Step 3: create a SubSite

Install-Module SharePointPnPPowerShellOnline
Connect-PnPOnline -Url <your site Url> -UseWebLogin
New-PnPWeb -Url "<new Web Url>" -Title "MySubSite" -Template "STS#3"

References:

Office 365 retention labels and policies for SharePoint

As I am a SharePoint person, and retention policies and labels are not a SharePoint engineer responsibility, I do not go to the m365 Compliance Center frequently. Below are My notes for myself on key moments – how to create and configure Office 365 retention labels and Policies at Compliance Center and use labels in SharePoint Online (SPO).

In SPO at each site collection level you can still work with retention policies the old way – create policies under Site Collection Settings – Content Type Policy – and apply policies at library level under Library Settings/Information Management Policy Settings. There is also Site Retention Policy.

But Microsoft is making efforts to centralize and unify such things – so you can specify retention policies in one place and apply them across all Office 365 content (not only SharePoint). That place was called Office 365 Security and Compliance Center (SCC). Later Microsoft separated Security Center and Compliance Center. So currently Retention Policies are under “Microsoft Purview” (former Microsoft Compliance Center) -> Solutions -> “Data lifecycle management”:

To get access to “Data lifecycle management” solution – you need to have a “” or “” roles. SharePoint or Teams administrator cannot access Purview. Even having “Global reader” or “Security reader” an admin will not be able to see “Data lifecycle management” blade. Here is how Microsoft Purview looks like for a Global reader:

Although SharePoint admins usually do not have access to SCC and do not go to Site content, we still need to know how it all works. And labels are recommended way to specify retention in SharePoint, so here we are.

Labels are applied to documents, documents are kept in libraries, and at each library you can “Apply a label to items in this library”.

Create Labels

Labels are created in SCC under Classification. The main part looks familiar to SharePoint people:

You can

Retain Content forever or for a specified number of days/months/years and then
– delete it or trigger a disposition review or do nothing
Delete content if it’s older than specified number of days/months/years

after it was created/modified/labelled

Apply labels

Now you need to publish created labels – and that is how you create a policy. I.e. policies are where you specify which labels to which content (Exchange, OneDrive, SharePoint, Office 365 groups)

You can also auto-apply labels based on conditions, like

content that contains sensitive info
content that contains specific words or phrases, or properties
content that matches a trainable classifier

but as per Microsoft, “It will take up to 7 days to automatically apply the label to all items that match your conditions.”

Note: “trainable classifier” means an AI ML will be used, and as per Microsoft “Creating machine learning rules requires an Office 365 E5 subscription for your organization”

SharePoint admin center

You can do nothing with labels at SharePoint admin center. Labels are created, published and auto-applied at SCC. At each site collection levels site administrators can apply labels.

SharePoint site

At site collection settings you can still see “Content Type Policy Templates” and “Site Policy”, but that is not the case. Labels are applied at library level under Library Settings/Apply label to items in this list or library.

where you can select a label to apply for all new items in the library. With

You can also apply the label to items that already exist in the library.

You can also apply (change) label for each single item or multiple selected items under Details pop-up page:

or from under Contect Menu/More/Compliance details:

Adaptive retention policies and scopes

Microsoft recently implemented “Adaptive” retention policies. At step 2 of “Create retention policy” you’ll be asked “Choose the type of retention policy to create”: “A policy can be adaptive or static. Advantage of an adaptive policy will automatically update where it’s applied based on attributes or properties you’ll define. A static policy is applied to content in a fixed set of locations and must be manually updated if those locations change.”

And if you selected “Adaptive” – on the next step you will need to provide the adaptive scope (so at this moment you should already have created your adaptive scopes):

So, let us create your adaptive scopes.
What type of scope do you want to create? SharePoint sites…

And then you’ll have nothing more then set of conditions:

where you can use objects: “Site Url”, “Site Name” and “Refinable String 0″..”Refinable String 99”. Conditions would be “is equal to”, “is not equal to”, “starts with” and “not starts with”. Or you can select “Advanced query builder” and enter LQL query.

What is the takeaway from this for SharePoint administrators? We would be asked to configure SharePoint the way compliance/retention people can use Refinable Strings.

References

SPO: Description, Owner fields are not searchable

Scenario:

You have a list (or a document library) in SharePoint Online.
You can search through the list but some fields like “Description”, “Owner”, “AssignedTo”, “Created”, “CreatedBy” are not searchable.

Cause:

This is frustrating, but expected behavior. Check Microsoft’s “Overview of crawled and managed properties in SharePoint Server“. You see some pre-created managed properties do not have “Searchable” option enabled.

Solution:

(The following is still not finished:)

Prove:

I have created a new SPO site test78, a new list Test11 and created (not added from existing) a custom field “Description” to the list:

I also created “Description2” column the same way. No data is added to the list so far.

Search schema looks like:

for Description managed property:

mapping:

Notice that “Description” managed property is not searchable and “ows_Description” crawled property is mapped to “Description” managed property.

Searching for “ows_Description” crawled property gives me:

and that’s OK, as we have no data in the list so “ows_Description2” crawled property does not exist.

Now let me add some data to the list:

and wait a few minutes while continues crawl grabs data.

You can see:

Title and Description2 are searchable, but we are not able to search through “Description” field content.

Explanation

Actually this is by design.
Microsoft: “The index only includes content and metadata from the managed properties”.
(? Maybe Microsoft tries to protect their resources from overloading or) maybe they protect us from irrelevant results, but by default list column “Description” is mapped to non-searchable managed property.

“Searchable” means: “…The content of this managed property is included in the full-text index.” I.e. if the property is not searchable – “The content of this managed property is not included in the full-text index.” => that’s by design.

But the property is queryable!
Queryable “Enables querying against the specific managed property”. E.e.
“Description:Descr1*” query should work. And it works:

“Description2:Descr*” query should not work as we did not map Description2 property to any managed properties, so we can find content via full-text search but cannot find under managed property:

Solution

Option #1.
Use queries like “Description:TextToSearch” (check also SharePoint KQL).

Option #2.
Do not use name “Description”.
Choose something else like “Short Description” or “Case Description”

Option #3.
Use existing site column “Description” from group:Custom Columns. It’s “single line of text” though. Note: “SharePoint Server Standard Site Collection features” must be activated.

The thing is it’s internal name is “CategoryDescription” and display name is “Description”. So if you add this column to the list – the content will be searchable:

Option #4
Create a new site column, name it e.g. “DescriptionSrchblClmn”.
Add this column to the list from existing site columns.
Rename it to “Description”.

Option #5 (under testing… TBU)
Change default mapping.
e.g. unmap “ows_Description” crawled property from “Description” managed property. This should be enough, as “ows_Description” crawled property has “Include in full-text index” option ON:

NB: if you unmap “ows_Description” crawled property from “Description” managed property, it’ll affect other site lists.

Option #6.
In addition to option #5 you can create your own managed property (e.g. “DescriptionSearchable”), make it searchable and map it to “ows_Description” crawled property.

Remember: if you made a change in search schema, run “Reindex site” under Site Settings -> “Search and Offline Availability”. It’s like on-prem “Full crawl” but works at web level.

================================

Links.

Microsoft: Manage the search schema in SharePoint
Microsoft: Keyword Query Language (KQL) syntax reference
Vladilen: Search for a crawled property name with wildcards
Microsoft: Overview of crawled and managed properties in SharePoint Server

Allow commenting on modern pages in SharePoint Online

Microsoft: “You can also select to allow or prevent commenting on modern pages. If you allow commenting, it can be turned on or off at the page level“

By default both
– Allow users to create new modern pages
– Allow commenting on modern pages
are turned on (enabled)

Tenant or SharePoint admin can find settings under
SharePoint Admin Center -> Settings -> Pages

When you are creating a modern site page – there is an option “Comments” turned On by default:

And page with comments looks like:

Comments on site pages (aka modern pages) can be enabled or disabled at each of the levels:
– Tenant level
– Site (aka site collection) level
– Web (aka subsite ) level
– Page level
Here is how it is done:

Level	How it’s done	Who can do it
Tenant	GUI ( SharePoint Administration) or PowerShell	Global Administrator or SharePoint Administrator
Site (Site Collection)	PowerShell	Global Administrator or SharePoint Administrator
Web (Subsite)	PowerShell	Site Collection Administrator or Owner (Full Control rights to web)
Page	GUI (Page Editing screen)	Site Member (Edit right to page)

If commenting on modern pages disabled at higher level – lower level settings do not work. E.g. If you disable “Allow commenting on modern pages” at tenant level (it takes minutes) – the functionality will gone from all modern pages of all sites.

When you switch page comments Off – all existing comments will be hidden (but not deleted).
If you later turn comments On – comments will reappear, including Likes.

If “Allow commenting on modern pages” disabled at tenant or web level – you will not see “Comments On/Off” switch while editing page.
If “Allow commenting on modern pages” disabled at site collection level – you will see “Comments On/Off” switch while editing page, but you will not be able to turn it ON.

PowerShell

When you disable “Allow commenting on modern pages” at tenant level –
PowerShell Object (site/web) property “CommentsOnSitePagesDisabled” will not be changed for any site/web.
You can still with PowerShell set it to True/False:
“Set-PnPWeb -CommentsOnSitePagesDisabled:$false”
but it does not take any effect.

If you enable “Allow commenting on modern pages” at tenant level (it takes ~10 minutes) – the functionality will return to all modern pages and
all webs and sites properties “CommentsOnSitePagesDisabled” will ???.
You can change it with PowerShell:
“Set-PnPWeb -CommentsOnSitePagesDisabled:$false”.

# having Site Collection Admin Permissions:
# disable Comments On Site Pages for a subsite:
$webName = "SubSite_02"
Set-PnPWeb -Web $webName -CommentsOnSitePagesDisabled:$true 
# enable Comments On Site Pages for a subsite 
# (only if comments enabled at tenant level):
Set-PnPWeb -Web $webName -CommentsOnSitePagesDisabled:$false


# having global admin or SharePoint admin permissions:
# site collection:
Set-PnPTenantSite -Url $siteUrl -CommentsOnSitePagesDisabled:$true
# tenant-level Comments:
Set-PnPTenant -CommentsOnSitePagesDisabled:$true # disable
 comments
Set-PnPTenant -CommentsOnSitePagesDisabled:$false # enable comments

# does not work:
Set-PnPSite -CommentsOnSitePagesDisabled:$true

How do I know if if the page is modern page or classic page (PowerShell)?

$list = Get-PnPList "Site Pages" -Includes ContentTypes, Fields
$list.ContentTypes | ft -a
$cType = Get-PnPContentType -List $list | ?{$_.Name -eq 'Site Page'};
$cType.id
$queryString = "<View><Query><Where><Eq><FieldRef Name='ContentTypeId'/><Value Type='Text'>" + $cType.Id.StringValue + "</Value></Eq></Where></Query></View>"  
$modernPages = Get-PnPListItem -List $list -Query $queryString
$modernPages.count
$modernPages | ft -a

How do I know if the page is a Home Page (PowerShell)?

# web object contains relative link to the web's Home Page:
$web = Get-PnPWeb -Connection $siteConnection -includes WelcomePage
$web.WelcomePage

References

– Microsoft “Allow users to create and comment modern pages“

Note:
We did not discuss “Wiki Pages” or “Web part Pages”, we discussed only “Modern Pages” (aka Site Pages).
I have tested it all personally using Communication sites.
MS-Team (group-based) and standalone SharePoint (no-group) sites – TBP.

MS Teams vs SharePoint

… by teams you are bringing in collaboration but you are losing discoverability…
… you put everything in one document library which becomes overloaded…
… you have to rely on search to try to find it…
… challenge for teams to become scalable (not everything can be added to tabs) – @DarceHess, Microsoft 365 & SharePoint PnP Weekly – Episode 91