Tag Archives: SPO

DepartmentId for Enterprise Intranet Portal – Home Site and Search Scope and Relevance

Scenario

Your organization use Microsoft 365. You are implementing or configuring an Intranet Portal (Home Site). Search plays an important role here – you want search be relevant to the context – i.e.

Official Results – if a user searches something on a company’s intranet portal – user expect “official” results, not a something from somebody’s OneDrive or Yammer chat
Promoted Results – so information management team can adjust search with search answers – Bookmarks, Acronyms and Q&As

Problem

Microsoft Bookmarks are working only at tenant level search – i.e. if you want bookmarks work on site level search – you need to set up site search scope as tenant.

So if you configure the Intranet Portal site (Home site) search scope to “site” or “hub” to limit results with site/hub content – you will loose “answers” functionality.

Solution

The solution is very simple:

Keep site search scope as tenant-wide to use answers (boormarks), and
Configure search verticals and query to limit results to “official” sites only

Update Query field with KQL – e.g., with something like

(path:http//contoso.sharepoint.com/sites/IntranetPortal/ OR path:http//contoso.sharepoint.com/sites/CompanyNews/ OR path:http//contoso.sharepoint.com/sites/Onboarding)/)

to get results only from “Intranet Portal” and “Company News” sites.

Keep in mind that this will affect all other SharePoint search entry points – SharePoint landing page, Office.com etc. – so although you can configure All (and Files) verticals, but it’s not recommended. It will confuse users – they expect to search for everything under “All” vertical. Instead, consider custom vertical – e.g. “Official” scope.

After configuring – It might take 1-24 hours for the change to take effect, depending on tenant size.

Service vs Site search

If you configure that at the tenant level – i.e., Microsoft 365 Administration -> Settings -> Search and intelligence -> Customizations -> Verticals
then search results will be trimmed everywhere – SharePoint Landing Page, Office landing page (Office.com), Office App, Bing search.
Teams search will not be affected as from Teams you only search for teams content. Same for Onedrive and Yammer.

If you want the “official” search results only under Intranet Portal and leave other search entry points unaffected – then
you need to configure the same at Home (Intranet Portal) site level: Site Settings -> Microsoft Search -> Configure search settings -> Verticals
and configure site search scope to site or hub scope. But in this case you will loose answers functionality.

Global search settings – like acronyms, bookmarks and verticals – works only if site search scope is tenant.
If site search scope is site or hub – then site-level search verticals will apply (and no answers functionality will be possible).

Home site is a root site

There might be two problems with that:

if a home site configured as a root site – you KQL will look like(path:http//contoso.sharepoint.com/ OR path:http//contoso.sharepoint.com/sites/CompanyNews)and that query will not work as any site Url will match the root site Url.
if you need to mention many sites in KQL – like 50 sites with an Official Information – you might hit the “number of allowed character” limit

The solution is DepartmentId property:

DepartmentId

Use DepartmentId={<Hub Site ID>} in the KQL qury and your search results will be limited to your hub content while answers will still be working too. You can even combine DepartmentId with other conditions to add more sites (that are not in hub) to search scope:

(DepartmentId={4965d9be-929b-411a-9281-5662f5e09d49} OR path:http//contoso.sharepoint.com/sites/Onboarding OR path:http//contoso.sharepoint.com/sites/CompanyNews)

It worth to mention, that DepartmentId is the property that propagated from the root of the hub site to all associated sites and their content – list items, documents and pages.

Site Property Bag

Another possible option would be – site property bag…
The ultimate goal is to provide users with “Official” results only. But official sites might not be all part of one Home site hub. We can include in All search vertical query 10, 50, 100 sites, but what if we have 10k official sites in enterprise – e.g. operated by different departments – and all of them might want to be present in search results.
So, how about – if the site is considered official – we create an indexed site property, e.g. “SiteIsOfficial” with a value “Yes”. Then we map the crawled property to a managed property – e.g. RefinableString89 – and use this managed search property in query – e.g. (SiteIsOfficial=Yes).

This is actually clever idea, but this does not work… This query would only return sites, not sites content. E.g. what was indexed as site object – will be included (including home page). But site items – i.a. documents, lists, other pages – all site content – will not be included…

So let’s get back to DepartmentId…

Rename All Vertical

Again – the ultimate goal is to give users option to have “Official” results. But they still might want to be able to search through all content.
What if we rename the default “All” vertical to “Home Site” and configure query for official results only.
Then we can create a custom vertical called “Everything” or “All” with no query limitations to give users all reasults

Update: not a good idea either… If the home site search scope is tenant – so verticals are configured and be visible at tenant level – i.e. everywhere…

Separate Official Vertical

My personal preference is to keep All vertical as real All, and create a custom Vertical “Official” for official results only where we would use query trick.

In addition, it would be nice to highlight results from official sources by using custom result type – check “Manage result layouts for SharePoint results in Microsoft Search”

Resources:

Microsoft “Manage search verticals“
Microsoft “Changing the scope of search“
Microsoft “Plan an intelligent SharePoint intranet“
Update from Microsoft: Using Path property in Microsoft 365 Search Query
Microsoft 365 Search Vertical KQL query field limits
Manage result layouts for SharePoint results in Microsoft Search

Connect to SharePoint Online and MS Graph Interactively with Client App and MSAL token

1 Reply

Scenario

You have got a Microsoft 365 subscription with SharePoint Online. You use PowerShell, PnP.PowerShell module and MS Graph API to work with SharePoint under current user’s credential. So you need to authenticate to SharePoint Online via Connect-PnPOnline and to Microsoft Graph API interactively on behalf of a user.

Problem

Unfortunately, both “Connect-PnPOnline -Interactive -Url <siteUrl>” or “Connect-PnPOnline -UseWebLogin -Url <siteUrl>” might fail with something like “Need admin approval”, “App needs permission to access resources in your organization that only an admin can grant. Please ask an admin to grant permission to this app before you can use it.” or similar

Solution

register an Azure App. Choose “single tenant”
configure authentication blade:
– add platform – “Mobile and Desktop app”
– select “https://login.microsoftonline.com/common/oauth2/nativeclient”
– add custom Redirect URI: “http://localhost”
configure API permissions blade:
– add delegated permissions you need (refer to specific API you’ll use)
e.g. Microsoft Graph Sites.FullControl.All and SharePoint AllSites.FullControl
use the following code samples

PnP.PowerShell

$siteUrl = "https://contoso.sharepoint.com/teams/myTeamsSite"
$appId = "" # Client Id
Connect-PnPOnline -ClientId $appId -Url $siteUrl -Interactive
Get-PnPSite

A pop-up window will appear to authenticate interactively. If you are already authenticated with another credentials (or single-sigh-on) – an interactive window might pop up and disappear – that prevents you enter your other id.
To ensure Connect-PnPOnline prompts you for your credentials – use ” -ForceAuthentication” option.

If you are a SharePoint tenant admin – you can connect to a tenant with:

$orgName = "yourTenantPrefix" 
$adminUrl = "https://$orgName-admin.sharepoint.com" 
$appId = "" # Client Id 
$connection = Connect-PnPOnline -ClientId $appId -Url $adminUrl -Interactive -ReturnConnection # -ForceAuthentication 
$connection

Microsoft Graph API

Use MSAL.PS module to get an msal token then use token in Microsoft graph-based requests:

$tenantId = ""
$clientid = ""
$url = ""
$token = Get-MsalToken -ClientId $clientid -TenantId $tenantId -Interactive

By default token expires in ~ 1 hour. But you can refresh it silently.
This helps you in long-running PowerShell scripts that takes hours to complete.
So you can include something like this in the loop:

if ($token.ExpiresOn.LocalDateTime -lt $(get-date).AddMinutes(10)) {    
  $token = Get-MsalToken -ClientId $clientid -TenantId $tenantId -ForceRefresh -Silent    
  Write-Host "Token will expire on:" $token.ExpiresOn.LocalDateTime
}

Application permissions

Somehow using Connect-PnPOnline with AccessToken option did not work if the token was acquired with MSAL.PS interactively. But it did work when you get msal.ps token unattended (using App credentials). So…

If you can get an Application (non Delegated) permissions to your azure-registerd-app,
you can use msal token to connect to site with PnP

=========================

NB: For delegated permissions, the effective permissions of your app are the intersection of the delegated permissions the app has been granted (via consent) and the privileges of the currently signed-in user. Your app can never have more privileges than the signed-in user.

How to Find Content Shared with Everyone in SharePoint and Teams

1 Reply

There is a known problem in SharePoint – complicated permissions system. Site owners/administrators provide access, site contributors upload documents and nobody knows – who has access to their sites. As a result – sometimes sensitive documents become overshared (over-exposed).

The biggest concern is sites content shared with “Everyone”. How do we find sites shared with “Everyone” in a large Microsoft 365 environment?

NB. When I say “shared with Everyone” – I actually mean 3 possible “everyone” logins:

Everyone
Everyone except external users
All users

Approach #1 (Brute force)

We can get full permissions report at tenant level (or permissions provided only to “Everyone”). There are 3-rd party tools (e.g. ShareGate, SysKit, AvePoint, Metalogix etc.), or you can run PowerShell script…

Sounds easy? Well, if you have less than 1000 sites – probably it will work. But if your environment is 10K+ sites – it will take forever. Permission report might run hours for an average site with site/subsite, list/library and list item details level. So the approach will not work for large enterprise environments.

One might say – we can limit report with root web permissions only to get it faster. But this would not be accurate. And what is not accurate in the IT security – lead to even bigger risks. So, we need report detailed up to every item level deep, as even one file with sensitive info shared with everyone can cause security issue. (3-rd party tools usually by default limit it to libraries level.)

Ok, if this approach is not really working – what’s working?

Approach #2 (Search)

Clever idea: why do we need to iterate through all the tenant documents/items if all the content is already crawled by search? Search is also respect permissions. Can we just use search to get files shared with Everyone? Let us see.

What if we use some dummy user account with no specific permissions provided and no group membership and try to search content on behalf of that account. The idea is if this user can see data – it means that data is open for everyone.

Check this and this articles. Can we get results programmatically (e.g. with PowerShell)? Can we use Microsoft Graph search API? Sure.
Check this article “How to search against SharePoint Online Content with Microsoft Graph search API with PowerShell”.

But! We have some problems here.

Search Problem #1. Again, for small environments or if there are not much “Open” sites – it would work. But for large enterprise environments the problem is the same as in “brute force”. Search returns too many results – it’ll take weeks to get all of them. (There are team sites “legally” shared with everyone, public Office 365 group based sites, communication sites… ).

Search Problem #2. Even if we get all search results – we do not know – at what level permissions are provided to everyone. So we will need to build list of sites based on the search results – ant then still need to run permissions report against these sites.

Search Problem #3. We are getting results with paging. But recently Microsoft started limiting number of returning results. E.g. your search request result might say like “there are 3659735 total hits” but after result number 1000 it just stops returning anything, even with paging.

Approach # 3 Hybrid

The idea: why do we need to get all search results if even one result from a site would be enough to add the site to the list of sites require permissions review. In other words, we do not need all results from site, we only need one to know the site is open.

So, consider (imho, the best) approach (Solution):

You get list of all sites in tenant.
You run search request against each site in the loop
(e.g. consider KQL option “Site: https://yourTenant.SharePoint.com/sites/YourSite”.
If at least something found in the site – add the site to the “Open Sites” list.
With this approach you will get list of sites shared with “Everyone…” in a couple of minutes.
Run permissions report against this shortlist

Note: consider there are resources like “Styles Library” shared with everyone by default.

Note: You can refine the list you get at step 1 – e.g., excluding sites connected to public teams or known communication sites…

Note: consider implementing sensitivity labels. At least you can start with high-sensitive sites. Site owners/member will know – what kind of site they are working on.

Pros and cons of the Approach # 3 Hybrid

Pro: the only fast and accurate enough to rely on

Con 1 : crawling and indexing takes time, so search-based reports can miss recent changes in data and permissions

Con 2: this approach cannot be automated (since we need an interactive authentication).

How to communicate to site owners

The Next step would be “How to let site owners know that there are resources shared with Everyone… on their sites”.

References

Access SPO Site Programmatically via MS Graph API and SharePoint API

Leave a reply

Scenario

You are a software developer. Your company uses Microsoft Office 365 (SharePoint, Teams etc.). The need is to work with a specific site collection programmatically (from code – Python, C#, Java, PowerShell, JavaScript etc.) – e.g. upload/download documents, update list items, search etc.

The code must run without user interaction (unattended, aka daemon app). Sometimes this is also called “SharePoint Automation”.

The solution is based on a new Graph API feature – Sites.Selected and a classic SP-Only app.

Solution

Register an Azure App and configure it as usual.
Select API Permissions blade and add two permissions:
– Microsoft Graph -> Applications Permissions -> “sites.selected”
– SharePoint -> Applications Permissions -> “sites.selected“
Request “Grant admin consent” from a tenant/global admin
Request SharePoint admin to run PowerShell code (e.g. this one) to assign proper permissions to your azure app for a specific site collection (consider site owner consent)
(optionally) Provide SharePoint API permissions:
(require Site Collection Owner/Admin account) – use
https://YourTenant.sharepoint.com/teams/YourSite/_layouts/15/appinv.aspx
to add SharePoint API permissions to your app. E.g. full control permissions to site collection would be

<AppPermissionRequests AllowAppOnlyPolicy="true">  
   <AppPermissionRequest Scope="http://sharepoint/content/sitecollection" 
    Right="FullControl" />
</AppPermissionRequests>

Consider minimal permissions (e.g. Right=”Read” see more with Sumit)

Problem Solved

you get access to one and only one site collection (“least privilege” principal)
you get both – SharePoint API and Microsoft Graph API permissions to SharePoint
you can use app secret or certificate to authenticate – depending on what are your security requirements

Note: if your scenario require authenticated user present – the solution would be a little different: Connect-PnPOnline Interactive with Client App Id

Update:

Sites.Selected API MS Graph permissions was introduced by Microsoft in 2021. It was a huge step forward, but still devs were limited with MS Graph API against SharePoint.
So devs had to use AppInv at site level to provide ACS permissions to their apps to use SharePoint CSOM and REST APIs.
Recently Microsoft introduced Sites.Selected SharePoint API permissions for registered Azure Apps! So now devs should be fully happy without ACS-based permissions AppInv.aspx. (See more here on disabling SP Apps Only SPNs)

Thanks to Leon Armston and Scott Murdock

References:

SharePoint site full permissions report

Leave a reply

There has always been one problem in the SharePoint world: full site permissions report. Full means across entire site – including all objects with broken permissions.
It seems like Microsoft has solved the problem: Full site permissions report is available for site owners out-of-the-box.

How to get SharePoint All Site Permissions Report

(Ensure you are site collection admin or team/group owner).
Just navigate to Site Usage, scroll to the end and run report.

Select gearbox “Settings” and then Site usage:

Or Select “Site Contents”, then “Site Usage” as shown below:

2. Scroll down to the “Shared with external users” block and click “Run report”:

Create/Select folder (*) for the report and click “Save”:
- If there are no folders in the Documents folder – you need to create one (otherwise you will not be able to save it:)

Once yo have a folder available – just click “Save”:

Give it some time (5-10 minutes) – check the folder’s content. There should be a file with a report on all site permissions.
For items shared with direct access, the report contains one row for each user / item combination.
SharePoint groups are shown in the report as groups (not individual users inside them… so you have to check group membership to get really full permissions report).

Again, you must be a site admin to run the report.

Secure the permissions report
If you don’t want other site members to see the report – secure the report’s folder – e.g. for site owners and for those who must be able see the report…
Consider creating a separate library for permissions reports and secure it instead of securing a folder under Documents.

Some more ideas on SharePoint permissions

Permissions are tricky in SharePoint. By default, you have permissions assigned to the root site of the site collection and all subsites, libraries etc. inherit root permissions.
But you can break inheritance at any level you need to provide specific (unique) permissions to the resource.
Of course you can always navigate to the resource and check resource permissions. But… what if there are hundreds of broken permissions… should you iterate everything under your site to check manually if permissions are broken or inherited?

So the real problem was – you never knew who have access to your site as there was no out-of-the-box tool to get all site permissions in one single report. There are third-party solutions – like ShareGate, Metalogix or SysKit – or you can develop PowerShell script generating report on all SPO site permissions. But… finally Microsoft solved this problem – Microsoft implemented out of the box full site permissions report.

Reference:

Microsoft Report on file and folder sharing in a SharePoint site

PnP.PowerShell Release 1.3.0

Leave a reply

Great news:

Added -Interactive login option to Connect-PnPOnline which is similar to -UseWebLogin but without the limitations of the latter. The -UseWebLogin is using cookie based authentication towards SharePoint and cannot access Graph tokens. Using -Interactive we use Azure AD Authentication and as a result we are able to acquire Graph tokens.

more changes: https://github.com/pnp/powershell/releases/tag/1.3.0

PnP.PowerShell Batches and PowerShell 7 Parallel

1 Reply

Parallelism

Can I use PowerShell 7 “-Parallel” option against SharePoint list items with PnP.PowerShell? Can I run something like:

$items | ForEach-Object -Parallel {
    $listItem = Set-PnPListItem -List "LargeList" -Identity $_ -Values @{"Number" = $(Get-Random -Minimum 100 -Maximum 200 ) }
}

Yes, sure… But! Since it’s a cloud operation against Microsoft 365 – you will be throttled if you start more than 2 parallel threads! Using just 2 threads does not provide significant performance improvements.

Batching

So, try PnP.PowerShell batches instead. When you use batching, number of requests to the server are much lower. Consider something like:

$batch = New-PnPBatch
1..100 | ForEach-Object{ Add-PnPListItem -List "ItemTest" -Values @{"Title"="Test Item Batched $_"} -Batch $batch }
Invoke-PnPBatch -Batch $batch

Measurements

Adding and setting 100 items with “Add-PnPListItem” and “Set-PnPListItem” in a large (more than 5000 items ) SharePoint list measurements:

	Add-PnPListItem Time per item, seconds	Set-PnPListItem Time per item, seconds
Regular, without batching	1.26	1.55
Using batches (New-PnPBatch)	0.10	0.80
Using “Parallel” option, with ThrottleLimit 2	0.69	0.79
Using “Parallel” option, with ThrottleLimit 3	0.44 (fails level: ~4/100)	0.53 (fails level: ~3/100)

Adding items with PnP.PowerShell batching is much faster than without batching.

SPO: Allow users to create modern pages

1 Reply

Microsoft: “Using modern pages in Microsoft SharePoint is a great way to share ideas using images, Office files, video, and more. Users can Add a page to a site quickly and easily, and modern pages look great on any device.
If you’re a global or SharePoint admin in Microsoft 365, you can allow or prevent users from creating modern pages. You can do this at the organization level by changing settings in the SharePoint admin center. If you allow the creation of site pages as the organization level, site owners can turn it on or off at the site level.“

By default both
– Allow users to create new modern pages
– Allow commenting on modern pages
are turned on (enabled)

Tenant or SharePoint admin can find settings under
SharePoint Admin Center -> Settings -> Pages

How it looks like:

Site Pages are created under “Pages” Library.

Let us test it, with:
– (tenant-level) Allow users to create new modern pages: ON
– (tenant-level) Allow commenting on modern pages: ON
– web feature “Site Pages” – “Allows users to add new site pages to a site”: Activated

User Permissions	can create Page	can edit page	can Enable/Disable page comments	can comment on Page
Full Control (Owner)	Yes	Yes	Yes	Yes
Edit (Member)	Yes	Yes	Yes	Yes
Read (Visitor)	No	No	No	Yes

There is a web feature “Site Pages” – “Allows users to add new site pages to a site”.
The feature is activated by default:

What if we disable this feature?
“New -> Page” has disappeared from “New” menu under “Site Contents” for Owners and Members…
From “Home” and “Pages” you still can see “New -> Page” options.
You can still create a new page from but if you try to create a page from Pages – “Sorry, something went wrong” “Cannot create a Site Page. Please have your administrator enable the required feature on this site.” :

Office 365 behavior, with:
– (tenant-level) Allow users to create new modern pages: ON
– (tenant-level) Allow commenting on modern pages: ON
– web feature “Site Pages” – “Allows users to add new site pages to a site”: Deactivated

User Permissions	can create Page	can edit page	can Enable/Disable page comments	can comment on Page
Full Control (Owner)	Yes, but only from “Home” not from “Site Contents” or “Pages”	Yes	Yes	Yes
Edit (Member)	Yes, but only from “Home” not from “Site Contents” or “Pages”	Yes	Yes	Yes
Read (Visitor)	No	No	No	Yes

If we disable feature “Site Pages” – “Allows users to add new site pages to a site” on the root web – it does not affect subsites (subwebs).

Can we Activate/Deactivate the feature “Site Pages” using PowerShell?

PowerShell

(TBP)

References
– Microsoft “Allow users to create and comment modern pages“

See also:
– Allow commenting on modern pages