It seems like Microsoft implemented search by short name (nick)…
It seems like Microsoft implemented search by short name (nick)…
What is the Microsoft Search KQL query field limits for a verticals? Is there limited number of characters or lines?
You know what is Microsoft 365 Search Vertical and what is KQL query in vertical configuration, right?
Under Microsoft 365 admin center Search and intelligence you can configure search verticals. There are some out-of-the-box verticals – like All, Files, Sites, People and you can configure custom one.
As a part of vertical configuration – you can specify KQL query – if you want e.g. limit search with some sites or content types etc.
The question is – how many sites I can specify in this query field? E.g. can I specify 1000 sites? 10k sites?
And the answer is: It does not matter, because the limit is not in number of characters or lines.
In my dev environment I was able to save 50,000 lines (~3M characters). But attempt to save 100K lines (6M symbols) has failed (due to timeout, I believe:
Again, as I said the problem is not here.
The problem is time required for search to apply query. I.e. when you ask search to bring you something – after it gets results from index and before display results to you it applies KQL query configured for the vertical. And this time is the bottleneck.
Here is what I got measuring search response time depending on query size:
|KQL query |
# of lines
|KQL query size,|
# of symbols
(can’t save KQL query
(can’t save KQL query)
Which means that after ~ 1000 lines (50,000 characters) KQL query size – query becomes too slow, and after ~3000 lines (180k chars) – can fail (due to timeout I’d say).
What if you want specific users do not appear in Microsoft 365 SharePoint, Teams or Delve search results? Examples are:
a) secondary/admin accounts
e.g. a person have several roles and several accounts under the same name, e.g.
regular user: John Smith John.Smith@contoso.com
administrative account: John Smith John.Smith.firstname.lastname@example.org
b) role or service accounts like SharePoint_Support@contoso.com
Getting multiple results for the same one person might confuse users and even lead to miscommunication and broken processes.
There is a good article by Tania Menice (Microsoft): Exclude Users From Delve and SharePoint Online People Search with the latest updates stating that currently it is not possible, but Microsoft is working on it.
Basically, the article says:
It works perfect for classic search. The problem is it does not work as expected in modern Microsoft Search.
“People” vertical is not customizable so far. So we cannot change query to do the same trick. But… it seems like Microsoft is working on it so finally it should be done by ootb means.
Here is the current situation on how different services or search entry points respect SPS-HideFromAddressLists property:
|Microsoft 365 Service or Search Entry point||respect SPS-HideFromAddressLists |
|web Outlook “New message” user picker||Yes|
|web Outlook “Contacts”||Yes|
|Office.com “All” vertical||Yes|
|Office.com “People” vertical||No|
|SharePoint landing page “All” vertical||Yes|
|SharePoint landing page “People” vertical||No|
|Bing Work All Vertical||Yes|
|Bing Work People Vertical||Yes|
So only “People” vertical in Microsoft search does not respect SPS-HideFromAddressLists (msExchHideFromAddressLists).
What about cloud-based accounts (not synchronized from local AD)?
There is a configuration setting “Show in global address list” that does the same job. It’s under Microsoft 365 admin center -> Active Users -> User – Edit -> Mail -> Show in global address list:
And another configuration settings “Hide from global address list (GAL)” under Exchange Admin Center:
|m365 Admin Center: Show in Global Address List||n/a||No||No||Yes||No|
|Exchange Admin Center: Hide from global address list (GAL)||n/a||On||On||Off||On|
|Get-AzureADUser -ObjectId … | Select -ExpandProperty ShowInAddressList||null||null||null||null||False|
|SPO UPA ‘SPS-HideFromAddressLists‘ value||False||False||False||False||True|
|Outlook Address List “All Users”||Shown|
|Office.com Search: Vertical “All”||Shown|
|Office.com Search: Vertical “People”||Shown||Shown||Shown||Shown|
|Bing Work Search: All/People verticals||Shown||Shown|
|Teams Search: “All” Vertical||Shown||Shown|
|Teams Search: “People” vertical||Shown||Shown||Shown||Shown|
|Microsoft 365 Profile card – Organization||Shown||Shown|
|Teams Profile card – Organization||Shown||Shown||Shown||Shown|
|Teams People Picker||Shown||Shown||Shown||Shown|
|SharePoint People Picker||Shown|
|Outlook People Picker:||Shown|
Does it seem confused we have properties:
Are these properties related to each other?
(WIP) Let’s test it:
(immediate reaction – minutes if not other mentioned)
|“Show in Global Address List” |
under m365 Admin Center
|“Hide from global address list (GAL)” |
under Exchange Admin Center
Azure AD User object property
SharePoint User Profile property
|New user created, license assigned||Yes||Off||null||False|
|Uncheck “Show in my organization address list” under Microsoft 365 admin center||No||On||after one minute: null|
after 24 hours: null
|after one minute: False|
after 24 hours: False
Azure AD User object property to “True”
Azure AD User object property to “False”
Note: Az module works fine too. I.e. Get-AzADUser instead of Get-AzureADUser and Set-AzADUser instead of Set-AzureADUser.
Let say you administer Microsoft 365 SharePoint Online and you want to get a list of new SharePoint sites (e.g. sites created recently – during last day/week/month).
With GUI it’s done easily: SharePoint Admin Center -> Active Sites -> sort based on “Date Created” – done.
With PowerShell – not so simple.
“Get-PnPTenantSite” cmdlet returns a site object but the object does not have “Created” field. It’s a web property. But to get a web object – you have to connect separately to each site and get root web object to check when the web was created. For small environments it is possible, for large environments it can take days… And still not nice.
“Get-PnPTenantSite” with “-Filter” option would help, but “…Currently, you can filter by these properties: Owner, Template, LockState, Url.”
Get-SPOSite – similar experience.
Microsoft Graph API helps. It returns result in seconds and you can sort or filter results based on created date . Below are two methods: Option 1 is based on Search and filtering and Option 2 is based on Sites Search and sorting. So there are some pros and cons for each method.
Microsoft Graph Search API allows KQL in queries. So we can form a query with something like “created>=1/1/2021” and use entity type = ‘[“site”]’. Search should return only sites created after Jan 01, 2021.
Check PowerShell script sample here: Get-NewSites.ps1
If you are getting more than 500 results – think of paging.
This option is also based on Microsoft Graph API, but sites entry point, which allows search too and sort results by property “createdDateTime”. So we will just search for everything and select how many results we need based on createdDateTime property.
Check PowerShell script sample here: Get-NewSites.ps1
updated: Feb 13, 2022
(Old/Classic) SharePoint Search: content-centric (SharePoint Search Center)
(New/Modern) Microsoft Search: people-centric (Teams, Office, OneDrive, Delve etc.)
Office graph = codename for collective set of services and insights we generate on top of the infrastructure that fast office graph group developed
= social Intel concepts (SharePoint home, Delve, OneDrive Discoverview) are derivatives of Office graph
Microsoft Graph = API ( including universal search API)
The Graph Search API went General Availability (GA):
– Microsoft Search API in Microsoft Graph
– Use the Microsoft Search API to query data
– Microsoft Search API Code samples, Tutorials at github
Microsoft Search API provides one unified search endpoint that you can use to query data in the Microsoft cloud – messages and events in Outlook mailboxes, and files on OneDrive and SharePoint – that Microsoft Search already indexes.
Turing technology – understands you, answers your question e.g. hover over doc -> doc summary (based on “deep speed” AI model)
announcement at Ignite Spring, more on Ignite Fall 2021
Modern Search: MS nailed the fundamentals, now start bringing it everywhere – to Teams first, then SharePoint (said Nov 2020).
Modern Search Customizations – we’ll take the best from Classic SharePoint Search,
a lot will retire – investing in more flexibility
PnP modern Search
– custom result pages, webparts, branding theme; filters, refiners, scoping control )
pnp modern search – webparts (video)
Core idea behind Microsoft search is coherence
“People use search in a different ways
1) you have organisations who have a well-established intranet built around set of governance controls, a very clean architecture and they want to build a search into that intranet scenario; that’s why a lot of SharePoint capabilities are going to come along with Microsoft search for that particular endpoint
2) then you have other people who live their day in teams“
Shared search engine results page (developed once – transitioned everywhere)
Ctrl-F to search through teams (chats?) (contextual search)
Natural language search (starting from Outlook)
Image search (before eoy), +
Metadata content type search Syntex customers will be able to use content type to search in the advanced search flyout in document libraries that contain content types beyond the default types. Syntex only.
Conversations: teams chats, outlook groups conversations, yammer conversation can be found under Conversations vertical in Bing search. Later – in Office.com and SharePoint landing page.
E-mail messages are added to Conversations vertical in Bing search
Bookmarks (new promoted results), acronyms, Q&A – all under “Answers”
Bookmarks Targeting – for the specific audience based on device/OS, Country/Region, security groups…
SharePoint Search Admin Center -> will be migrated from SharePoint admin center to to Microsoft Search Admin Center transitioning (Search and Intelligence Admin Center) – long-running project custom dictionaries, spelling suggestions – will retire, (move to a graph-driven speller)
+ Viva Topics – based search capabilities
Graph Connectors are generally available (ADLS – Azure Data Lake Storage Gen2, Azure DevOps, Azure SQL and Microsoft SQL Server, Enterprise websites, MediaWiki, File share, Oracle SQL, Salesforce, Jira, Confluence, ServiceNow + 100+ from partners; New connectors coming to Microsoft Search: Jira Graph connector, Confluence Graph connector).
Graph Connector allows to connect external source of information to Microsoft 365 and makes that data available across all m365 apps and services so you can find what you need wherever you’re working, whether in one of your favorite productivity apps or one of the many Microsoft 365 services such as SharePoint or Office.com
Graph Connectors roadmap:
Search results on select Graph connectors will soon support actions that will allow users to interact with the result and perform changes to the Connector content within the Search application.
The results shown in a result cluster are grouped together based on the search vertical configuration.
Profile Query variables
Define any attribute from the user’s Profile, as a query variable and it would be resolved during query evaluation (This feature is currently in preview)
Profile enrichment with Graph connectors
…you will soon be able to enrich Microsoft 365 profile properties like Job title, Phone numbers, Skills etc. with data from HRMS systems using graph connectors. …then surface this rich profile information on people experiences like profile cards.
federation capabilities will allow enterprises build and integrate their custom LOB search experiences, customized search providers, into the overall Microsoft Search. With federated search, you can make information from systems where the data cannot leave the systems boundaries available to search across in Microsoft 365 productivity apps and services, without indexing its data with Microsoft Search.
Azure Cognitive Search Federation
PowerBI search vertical
Custom verticals and custom refiners
Standalone Search – AAD identity – Graph connector – Ingest your data – use Search = in Windows 10, Office.com ( e.g. for those who have their data in other productivity suite, have no intent to use m365, but want to search)
Bill Baer “Making the most of Microsoft Search” @ MS Ingnite fall 2021
There is a known problem in SharePoint – complicated permissions system. Site owners/administrators provide access, site contributors upload documents and nobody knows – who has access to their sites. As a result – sometimes sensitive documents become overshared (over-exposed).
The biggest concern is sites content shared with “Everyone”. How do we find sites shared with “Everyone” in a large Microsoft 365 environment?
NB. When I say “shared with Everyone” – I actually mean 3 possible “everyone” logins:
We can get full permissions report at tenant level (or permissions provided only to “Everyone”). There are 3-rd party tools (e.g. ShareGate, SysKit, AvePoint, Metalogix etc.), or you can run PowerShell script…
Sounds easy? Well, if you have less than 1000 sites – probably it will work. But if your environment is 10K+ sites – it will take forever. Permission report might run hours for an average site with site/subsite, list/library and list item details level. So the approach will not work for large enterprise environments.
One might say – we can limit report with root web permissions only to get it faster. But this would not be accurate. And what is not accurate in the IT security – lead to even bigger risks. So, we need report detailed up to every item level deep, as even one file with sensitive info shared with everyone can cause security issue. (3-rd party tools usually by default limit it to libraries level.)
Ok, if this approach is not really working – what’s working?
Clever idea: why do we need to iterate through all the tenant documents/items if all the content is already crawled by search? Search is also respect permissions. Can we just use search to get files shared with Everyone? Let us see.
What if we use some dummy user account with no specific permissions provided and no group membership and try to search content on behalf of that account. The idea is if this user can see data – it means that data is open for everyone.
Check this and this articles. Can we get results programmatically (e.g. with PowerShell)? Can we use Microsoft Graph search API? Sure.
Check this article “How to search against SharePoint Online Content with Microsoft Graph search API with PowerShell”.
But! We have some problems here.
Search Problem #1. Again, for small environments or if there are not much “Open” sites – it would work. But for large enterprise environments the problem is the same as in “brute force”. Search returns too many results – it’ll take weeks to get all of them. (There are team sites “legally” shared with everyone, public Office 365 group based sites, communication sites… ).
Search Problem #2. Even if we get all search results – we do not know – at what level permissions are provided to everyone. So we will need to build list of sites based on the search results – ant then still need to run permissions report against these sites.
Search Problem #3. We are getting results with paging. But recently Microsoft started limiting number of returning results. E.g. your search request result might say like “there are 3659735 total hits” but after result number 1000 it just stops returning anything, even with paging.
The idea: why do we need to get all search results if even one result from a site would be enough to add the site to the list of sites require permissions review. In other words, we do not need all results from site, we only need one to know the site is open.
So, consider (imho, the best) approach (Solution):
Note: consider there are resources like “Styles Library” shared with everyone by default.
Note: You can refine the list you get at step 1 – e.g., excluding sites connected to public teams or known communication sites…
Note: consider implementing sensitivity labels. At least you can start with high-sensitive sites. Site owners/member will know – what kind of site they are working on.
Pro: the only fast and accurate enough to rely on
Con 1 : crawling and indexing takes time, so search-based reports can miss recent changes in data and permissions
Con 2: this approach cannot be automated (since we need an interactive authentication).
The Next step would be “How to let site owners know that there are resources shared with Everyone… on their sites”.
If you have country-specific content – Microsoft Search allows bookmarks to be configured to pop-up only for users from a specific country.
And “Use Azure AD locations” flag is a new option that make it actually works…
For a long time country settings were the same but without “Use Azure AD locations” flag. So what does “Use Azure AD locations” flag do?
“Use Azure AD locations” flag is a straight-forward configuration settings. It says: “This bookmark will only appear for users with Azure AD locations that match selected countries or regions. If cleared, the user’s IP address will be used to determine location. This checkbox can be altered from both Country or region setting and Targeted variations setting.”
I have tested this new “Use Azure AD locations” flag – it works. Once you configure user’s country in AAD and country-targeted bookmarks – all works good. Bookmarks appear for the user.
to be provided
What was before Microsoft implemented this “Use Azure AD locations” option? How did Microsoft understand “this user is from that country”. What was the criteria to correlate User <-> Country? License assigned country? Windows locale? Browser settings? Azure AD properties?
How does Microsoft define “this user is from that country”. What are the criteria to correlate User <-> Country? Physical IP address? License assigned country? Locale? Browser settings? Azure AD properties?
It turned out, the way it was designed previously:
– configure Microsoft 365 integration with Bing
– in Bing -> Settings : select Country/Region
– search from Bing
that was the only way to make it work! So yes, do not leave “Use Azure AD locations” option unchecked. Microsoft confirmed it was poor design.
Always “Use Azure AD locations” flag.
Search is everywhere in Microsoft 365. You can search from SharePoint, Teams, Delve, Yammer etc.
But! You cannot search for anything from everywhere!
So, what are the scopes of each search entry point in Office 365 and is there an entry point you can search for everything?
Office desktop app
|Scope||Out of Scope|
|SharePoint Search Center||– all sites content|
(Teams, Yammer, regular),
– user profiles
|SharePoint Landing Page||same as SharePoint Search center|
but Teams chats and Yammer Conversations are coming
|same as SharePoint Search Center|
|Office.com||same as SharePoint|
(Teams chats and Yammer Conversations are coming after SharePoint)
|same as SharePoint|
regular SharePoint sites
|Bing||Everything*||* except people profiles content |
(e.g. about me)
Seems like the only tool you can search for EVERYTHING with is Microsoft Bing:
After Microsoft add Teams chats and Yammer conversations to SharePoint landing page search scope (then to Office home page) – it’ll be the best place to search from for everything.
Microsoft Office 365 Search: Find what you need with Microsoft Search in Bing
It is possible customize Modern Microsoft Search pages with PnP Modern Search
Quick and simple answer: use SharePoint Search center or Microsoft Search, (or Bing if it is integrated).
In Microsoft Office 365, under MS Teams, there are 3 team types:
Private team: you can only join the team if you are invited or know the team code.
SharePoint site behind the private team is shared only to members – not for everyone. You cannot see team name or description or content until you are team member (details). You are not able to search for the team name or content.
Public team: you can join the public team if you wish. The site behind the public team is shared with everyone except external users, so you can see public team name and description, but from MS Teams (desktop or web application) you cannot see public team content until you are team member.
Org-wide team: you are joined the team automatically (details)
From Teams – you can click on “Join or create a team” and you should be able to see some public teams (but not all):
There is a “Search teams” box at the top right,
so what if you are looking for a specific public team (not in the list) …
You know exact team name or at least some first letters.
Solution: You are lucky. Just start typing team name in search bar at top right and hit “enter”- you will see shortened list of public teams matching your search criteria:
NB: do not use wildcards, it will not work:
NB: do not use top search bar, it will not work:
You want to join a public team, but you do not know exact team name.
You know (or guess) something about the team, like
Unfortunately, in this case both great Microsoft technologies – Search and Team – fail. You will not be able to find a public team:
Solution: use SharePoint search
SharePoint site is created once a team is created.
For public teams – SharePoint site has “Everyone except external users” by default in “Members” group:
which means literally “Everyone except external users” has access to the site with “Edit” permissions.
SharePoint search is security-trimmed, i.e. you will see the site content in search results only if you do have access to the site. So just go to the SharePoint or SharePoint search center and search for what you know or guess about the team:
You can use all the power of SharePoint search (wildcards, refiners, keyword query language KQL etc)
Once you found something – you can go to the SharePoint site:
Now from the site – look at the site name and hover the mouse over the site name – you’ll see pop-up window.
Now you know exact team name – and you can search for the team under Teams,
or, if you are so lucky you see “Join” button – just join the team.click site title or hove over the site title:
One moment – you cannot see team’s chat messages in SharePoint, as chats are kept in Azure. But you can search for chat content after you joint the team.
Somehow both – SharePoint Search and Teams Search are not working against site/team description. Hopefully this bug will be addressed.
You can also search for site Url in teams.
When you create a team – Office 365 generates a short team name (removes spaces and adds numbers if the team name is not unique; e.g. if the team name “Test” you might have “test381” as a short name, but if the team name is “This Is My Unique Team” – short name might be “ThisIsMyUniqueTeam”).
After you can change team name and/or SharePoint site name.
Team search under MS teams work for both names – short name initially assigned (kept as site specific Url) and new team name. But only starting with the beginning of the string.
p.s. Thanks to “Birds of Kazakhstan” for pictures
btw, there is a good video tutorial on how to find a public team in Office 365 using full-text search
You have a list (or a document library) in SharePoint Online.
You can search through the list but some fields (or document properties) like “Description”, “Subject”, “Author”, “Owner”, “AssignedTo”, “Created”, “CreatedBy” are not searchable.
Crawled properties are mapped to non-searchable managed properties. So this is by design. Check Microsoft’s “Overview of crawled and managed properties in SharePoint Server” (we do not have this document for SharePoint Online, so we have to rely on this doc; though you can go to your Search schema in SPO to verify). You see some pre-created managed properties do not have “Searchable” option enabled.
(See below for details, as this is still not finished:)
I have created a new SPO site test78, a new list Test11 and created (not added from existing) a custom field “Description” to the list:
I also created “Description2” column the same way. No data is added to the list so far.
Search schema looks like:
for Description managed property:
Notice that “Description” managed property is not searchable and “ows_Description” crawled property is mapped to “Description” managed property.
Searching for “ows_Description” crawled property gives me:
and that’s OK, as we have no data in the list so “ows_Description2” crawled property does not exist.
Now let me add some data to the list:
and wait a few minutes while continues crawl grabs data.
You can see:
Title and Description2 are searchable, but we are not able to search through “Description” field content.
Actually this is by design.
Microsoft: “The index only includes content and metadata from the managed properties”.
(Maybe Microsoft tries to protect their resources from overloading or maybe they protect us from irrelevant results, but including entire document content in full-text search and at the same time not including properties like Document Subject – this does not make sense to me). So the sad fact is list column “Description” is mapped to non-searchable managed property by default.
“Searchable” means: “…The content of this managed property is included in the full-text index.” I.e. if the property is not searchable – “The content of this managed property is not included in the full-text index.” => that’s by design.
But – the good news – the property is Queryable!
Queryable “Enables querying against the specific managed property”.
E.e. “Description:Descr1*” query should work. And it works:
“Description2:Descr*” query should not work as we did not map Description2 property to any managed properties, so we can find content via full-text search but cannot find under managed property:
Use queries like “Description:TextToSearch” (check also SharePoint KQL).
Do not use name “Description”.
Choose something else like “Short Description” or “Case Description”
Use existing site column “Description” from group:Custom Columns. It’s “single line of text” though. Note: “SharePoint Server Standard Site Collection features” must be activated.
The thing is it’s internal name is “CategoryDescription” and display name is “Description”. So if you add this column to the list – the content will be searchable:
Create a new site column, name it e.g. “DescriptionSrchblClmn”.
Add this column to the list from existing site columns.
Rename it to “Description”.
Create your own managed property (e.g. “DescriptionSearchable”), make it searchable and map it to “ows_Description” crawled property.
ensure crawled property has “Include in full-text index” option ON:
NB: changing search schema affects other site lists/libraries.
Remember: if you made a change in search schema, run “Reindex site” under Site Settings -> “Search and Offline Availability”. It’s like on-prem “Full crawl” but works at web level.
Microsoft: Manage the search schema in SharePoint
Microsoft: Keyword Query Language (KQL) syntax reference
Vladilen: Search for a crawled property name with wildcards
Microsoft: Overview of crawled and managed properties in SharePoint Server