Let say you administer Microsoft 365 SharePoint Online and you want to get a list of new SharePoint sites (e.g. sites created during last week/month).
With GUI it’s done easily: SharePoint Admin Center -> Active Sites -> sort based on “Date Created” – done.
With PowerShell – not so simple. “Get-PnPTenantSite” cmdlet returns site objects but the object does not have “Created” field. You have to connect separately to each site and get root web object where you can check when the web was created. For small environments it is possible, for large environments it can take days… And still not nice. -Filter option would help, but “…Currently, you can filter by these properties: Owner, Template, LockState, Url.”
Get-SPOSite – similar experience.
Solution
Microsoft Graph API helps. It returns result in seconds. There are some pros and cons for each method though.
Microsoft Graph Search API allows KQL in queries. So we can form a query with something like “created>=1/1/2021” and use entity type = ‘[“site”]’. Search should return only sites created after Jan 01, 2021.
This option is also based on Microsoft Graph API, but sites entry point, which allows search too and sort results by property “createdDateTime”. So we will just search for everything and select how many results we need based on createdDateTime property.
Scenario: you administer SharePoint Online Microsoft 365 tenant business asks you to remove mobile phone numbers from SharePoint User Profiles:
Solution:
As a SharePoint administrator, you can do it: 1. Start Microsoft 365 SharePoint Admin Center 2. Navigate to More Features -> User Profiles -> Manage User Properties
3. Under “Contact Information” -> Mobile phone -> Edit
4. Uncheck “Replicable”, Save, wait a minute or two 5. – Select “Default Privacy Settings”: “Only Me” – Uncheck “User can override” – Uncheck “Allow users to edit values for this property” – Uncheck “Show in the profile properties section of the user’s profile page” – Uncheck “Indexed”
The error “The term ‘Search-UnifiedAuditLog’ is not recognized”:
Search-UnifiedAuditLog: C:\scripts\PowerShell.auth\Search-AuditLog-w-App.ps1:16:12 Line | 16 | $results = Search-UnifiedAuditLog -StartDate $start -EndDate $end -Re … | ~~~~~~ | The term 'Search-UnifiedAuditLog' is not recognized as a name of a cmdlet, function, script file, or executable | program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
means a proper administrative role (e.g. “Exchange administrator”) is not assigned to the app.
Office graph = codename for collective set of services and insights we generate on top of the infrastructure that fast office graph group developed = social Intel concepts (SharePoint home, Delve, OneDrive Discoverview) are derivatives of Office graph
Microsoft Search API provides one unified search endpoint that you can use to query data in the Microsoft cloud – messages and events in Outlook mailboxes, and files on OneDrive and SharePoint – that Microsoft Search already indexes.
Turing technology – understands you, answers your question e.g. hover over doc -> doc summary (based on “deep speed” AI model) announcement at Ignite Spring, more on Ignite Fall 2021
Modern Search: MS nailed the fundamentals, now start bringing it everywhere – to Teams first, then SharePoint (said Nov 2020).
Modern Search Customizations – we’ll take the best from Classic SharePoint Search, a lot will retire – investing in more flexibility
Bill Baer: “People use search in a different ways 1) you have organisations who have a well-established intranet built around set of governance controls, a very clean architecture and they want to build a search into that intranet scenario; that’s why a lot of SharePoint capabilities are going to come along with Microsoft search for that particular endpoint 2) then you have other people who live their day in teams“
Updates
Shared search engine results page (developed once – transitioned everywhere) Ctrl-F to search through teams (chats?) (contextual search) Natural language search (starting from Outlook) Image search (before eoy), + teams chats, outlook groups conversations, yammer conversation -> bing, office.com, sharepoint
Bookmarks (new promoted results), acronyms, Q&A – all under “Answers”
BookmarksTargeting – for the specific audience based on device/OS, Country/Region, security groups…
SharePoint Search Admin Center -> will be migrated from SharePoint admin center to to Microsoft Search Admin Center transitioning (Search and Intelligence Admin Center) – long-running project custom dictionaries, spelling suggestions – will retire, (move to a graph-driven speller)
+ Viva Topics – based search capabilities
Create Topic Answers with Microsoft Viva Topics to bring together people, content, and information (including synonyms and acronyms)
Knowledge answers provide a direct answer to questions authoritative information in an organization across SharePoint and OneDrive content
Files/Calendars/Links answers
Graph Connectors Graph Connectors are generally available (ADLS – Azure Data Lake Storage Gen2, Azure DevOps, Azure SQL and Microsoft SQL Server, Enterprise websites, MediaWiki, File share, Oracle SQL, Salesforce, Jira, Confluence, ServiceNow + 100+ from partners; New connectors coming to Microsoft Search: Jira Graph connector, Confluence Graph connector).
Graph Connector allows to connect external source of information to Microsoft 365 and makes that data available across all m365 apps and services so you can find what you need wherever you’re working, whether in one of your favorite productivity apps or one of the many Microsoft 365 services such as SharePoint or Office.com
Graph Connectors roadmap:
Actionable experiences Search results on select Graph connectors will soon support actions that will allow users to interact with the result and perform changes to the Connector content within the Search application.
Results clusters The results shown in a result cluster are grouped together based on the search vertical configuration.
Profile Query variables Define any attribute from the user’s Profile, as a query variable and it would be resolved during query evaluation (This feature is currently in preview)
Profile enrichment with Graph connectors …you will soon be able to enrich Microsoft 365 profile properties like Job title, Phone numbers, Skills etc. with data from HRMS systems using graph connectors. …then surface this rich profile information on people experiences like profile cards.
Search Federation federation capabilities will allow enterprises build and integrate their custom LOB search experiences, customized search providers, into the overall Microsoft Search. With federated search, you can make information from systems where the data cannot leave the systems boundaries available to search across in Microsoft 365 productivity apps and services, without indexing its data with Microsoft Search.
Standalone Search – AAD identity – Graph connector – Ingest your data – use Search = in Windows 10, Office.com ( e.g. for those who have their data in other productivity suite, have no intent to use m365, but want to search)
configure API permissions blade: – add delegatedpermissions you need (refer to specific API you’ll use) e.g. Microsoft Graph Sites.FullControl.All and SharePoint AllSites.FullControl
A pop-up window will appear to authenticate interactively. If you are already authenticated with another credentials (or single-sigh-on) – an interactive window might pop up and disappear – so you are not able to enter your admin id. To ensure Connect-PnPOnline prompts you for your credentials – use ” -ForceAuthentication” option.
If you are a SharePoint tenant admin – you can connect to a tenant with:
By default token expires in ~ 1 hour. But you can refresh it silently. This helps if you run heavy PowerShell script and it takes hours to complete. So you can include something like this in the loop:
NB: For delegated permissions, the effective permissions of your app are the intersection of the delegated permissions the app has been granted (via consent) and the privileges of the currently signed-in user. Your app can never have more privileges than the signed-in user.
There is a know problem in SharePoint – it’s complicated permissions system. As a result, many sites are overshared (over-exposed) and site owners/administrators even do not know – who has access to their sites…
The most concern is sites shared with “Everyone”, “Everyone except external users” and “All users”. How do we find sites shared with “Everyone” in a large Microsoft 365 tenant?
Approach #1 (Brute force)
We can get full permissions report at tenant level (or permissions provided to “Everyone”). There are 3-rd party tools (e.g. ShareGate, SysKit, AvePoint, Metalogix etc.), or you can run PowerShell script…
Sounds easy? Well, if you have 1000 sites – probably it will work. But if your environment 10K+ sites – it will take forever. Permission report might run hours for an average site with site/subsite, list/library and list item details level. So the approach will not work for large enterprise environments.
We cannot limit report with root web only – we need report detailed up to every item level deep, as even one file with sensitive info shared with everyone can cause security issue.
So, if this approach is not working – what’s working?
Approach #2 (Search)
Clever idea: why do we need to iterate through all the tenant documents/items if all the content is already crawled by search? Can we just use search to get files shared with Everyone? Sure!
The idea is to use some dumb/test user account with no specific permissions provided and no group membership and try to search content on behalf of the user. Results we get are obviously from sites shared with everyone.
Check this and this articles. Can we get results programmatically (e.g. with PowerShell)? Can we use Microsoft Graph search API? Sure. Check this article “How to search against SharePoint Online Content with Microsoft Graph search API with PowerShell”.
But! We have two problems here.
Search Problem #1. The problem is the same as in “brute force”. Search returns so many results – it’ll take weeks to get all of them. (There are team sites “legally” shared with everyone, public Office 365 group based sites, communication sites… ).
Search Problem #2. Even if we get all search results – we do not know – what is the exact Url of the resource shared with all users. So we will need to build list of sites based on the search results – ant then still need to run permissions report against these sites.
Approach # 3 Hybrid
The idea: why do we need to get all search result if even one result from a site would be enough to add the site to the list of sites require permission review.
So, consider (imho, the best) approach.
You get list of sites in tenant. Here you can refine the list excluding, e.g. sites connected to public teams or known communication sites… Using sensitivity labels you can start with high-sensitive sites. Finally you’ll have a list of sites you want to check – if there are resources on this site shared with “Everyone…”
You run search against each site in the loop (e.g. consider KQL option “Site: https://yourTenant.SharePoint.com/sites/YourSite”. Once at least something found in the site – add the site to the “Open Sites” list
With this approach you will get list of sites shared with “Everyone…” in a couple of minutes.
NB: consider there are resources like “Styles Library” shared with everyone by default.
The Next step would be “How to let site owners know what are resources shared with Everyone… on their sites”.
You are a software developer. Your company uses Microsoft Office 365 (SharePoint, Teams etc.). The need is to work with a specific site collection programmatically (from code – Python, C#, Java, PowerShell, JavaScript etc.) – e.g. upload/download documents, update list items, search etc.
The code must run without user interaction (unattended, aka daemon app). Sometimes this is also called “SharePoint Automation”.
The solution is based on a new Graph API feature – Sites.Selected and a classic SP-Only app.
Solution
Register an Azure App and configure it: MS Graph API permissions: add -> Microsoft Graph -> Applications Permissions -> “sites.selected”
Ask SharePoint/Tenant admin run PowerShell code (e.g. this one) to assign proper permissions to your azure app for a specific site collection (consider site owner consent)
Provide SharePoint API permissions: (require Site Collection Owner/Admin account) – use https://YourTenant.sharepoint.com/teams/YourSite/_layouts/15/appinv.aspx to add SharePoint API permissions to your app. E.g. full control permissions to site collection would be
You are a developer or power user in a company with Microsoft 365 tenant. You need to connect to Microsoft Graph and then call Microsoft Graph API to consume some MS Graph resources on behalf of authenticated user programmatically with PowerShell – e.g. add/remove documents or list items, search for sites or documents content etc. – whatever available with Graph API.
You do not have tenant admin permissions or any tenant-level admin permissions (SharePoint, Teams, Exchange etc. ). But you can register an Azure App and request tenant admin consent.
Solution
register an Azure App
under authentication blade – add platform – “Mobile and Desktop app” add “http://localhost” (and select …/nativeclient Url ?)
under API permissions blade – add delegatedpermissions you need (refer to specific API you’ll use)
There is a known problem with Microsoft Delve. It’s not a technology problem though.
We know SharePoint site permissions are not easy to manage. E.g. you can break permissions inheritance at any level – subsite, library, list, folder, list item or specific document. Anybody with full permissions can do that. The worst thing is there is was (*1) no native ability for site owner to get full site permissions report. We must have used third-party tools or PowerShell to have all permissions in one document.
So no wonder SharePoint sites were heavily over-exposed. Especially when a site owner tired with complexity of SharePoint permissions system decided to share resource with “Everyone”. And the other person, not knowing site is shared with everyone, might save some sensitive data. That is the real issue.
Now, what is Delve? It’s a service that – get signals from allover Office 365 – who did what etc. – based on that, using AI and Office Graph, generates suggestions – “what others do”. Of course, Delve is security-trimmed, i.e. it will neve suggest you a document you do not have access to. But some sites might be overshared. Delve works as it should work – it suggests you documents it believes related to you (based on Microsoft Graph insights) and you already have access to.
Now bad thing happens – people start seeing documents they never new they have access to. Where are these documents from? Of course from sites shared with Everyone. Who to blame for the security breach? Delve? Microsoft Graph? Microsoft 365 SharePoint Online?
Strictly says, it is not Delve’s problem. It’s more human problem than technological. Delve just does it’s job, and does perfectly. Delve simply displays the information already shared widely.
How do we solve the issue?
Disable Delve?
Disable search (stop sites crawling and remove results)?
Those methods are half-measure. Methods above are just hiding the problem – not solving it. Agree it helps stop the deterioration, bud does not fix the root cause.
How do we solve the real problem and what is the root cause?
Of course, we need remove incorrectly provided permissions. How?
Only site owner (data owner) knows which content should be shared with whom with which access rights. So we need to ask sites owners to review their permissions. How?
First, we need a list of over-exposed sites. How?
There are two methods (more details – check this article)
Brute force – use PowerShell or 3-rd party tool to get permission report on all sites in tenant, select permissions provided for Everyone…
Smart move – use Microsoft search. As search is security-trimmed, we can search for available content on behalf of a user with no permissions provided.
Then we find owners for each wide-open site. How?
for group-based sites we get member of the “owners” group
for non-group based sites we get site collection administrators
We would also sort sites by “is it supposed to be public?”. I.e. if the site was born as public – e.g. Public Team or Public Yammer community – or Communication site – maybe it’s less concern.
It would be a good idea to bring DLP and/or automatic content sensitivity labelling, so we could start remediation from sites labelled as storing most sensitive data.
Finally, we need to let site owner know that his site is Open to everybody and ask to fix it. How?
e-mail communication
SharePoint list containing overshared sites with owners, configured the yaw every person can see only sites he/she owns
Search is everywhere in Microsoft 365. You can search from SharePoint, Teams, Delve, Yammer etc.
But! You cannot search for anything from everywhere!
Search for your Teams chat messages works only in Teams.
But from Teams you cannot search for regular (non-group) sites and public teams sites
All descriptions are totally out of search (e.g. site description, library/list description – including Yammer groups, Teams and regular sites).
Public Team Sites content is not searchable from Teams and Yammer
So, what are the scopes of each search entry point in Office 365 and is there an entry point you can search for everything?
Search scopes
SharePoint Search center
SharePoint home Office portal Office desktop app Delve
Teams
Bing
SharePoint content
Yes
Yes
Yes
Teams content
Yes
Yes
Yes
Yes
Teams chats
(*1)
Yes
Yes
Yammer content
Yes
Yes
Yes
Yammer chat
(*1)
Yes
User profiles
Yes
Yes
Email
(*1) Microsoft announced they are working on bringing conversations (both Teams chats and Yammer) to SharePoint landing page first, then to Office home page.
Detailed:
Scope
Out of Scope
SharePoint Search Center
– all sites content (Teams, Yammer, regular), – user profiles – OneDrive
Teams chat Yammer chat
SharePoint Landing Page
same as SharePoint Search center but Teams chats and Yammer Conversations are coming
same as SharePoint Search Center
Office.com
same as SharePoint (Teams chats and Yammer Conversations are coming after SharePoint)
same as SharePoint
Delve
Teams
Teams content Teams chat
OneDrive Yammer User Profiles regular SharePoint sites
Bing
Everything*
* except people profiles content (e.g. about me)
Seems like the only tool you can search for EVERYTHING with is Microsoft Bing:
After Microsoft add Teams chats and Yammer conversations to SharePoint landing page search scope (then to Office home page) – it’ll be the best place to search from for everything.
