Tag Archives: SharePoint

Track SharePoint App-only Service Principals in Microsoft 365

Scenario

Developers in the organization can use both – Azure Apps and SharePoint Apps to work with SharePoint sites in their “daemon” applications.

It is recommended to use Azure apps so, you want to know – what are SharePoint Apps registered and their owners, who registered SharePoint Apps. Eventually you would disable SharePoint Apps-only principal but before that you’d move Devs from SP-App-only to Azure App (see Disable Custom App Authentication).

(SharePoint App-only service principals aka SP-App-Only are SPN or App registered from within SharePoint using AppRegNew.aspx system page).

One of the approaches – track Apps/Owners with Unified Audit Log

Use Unified Audit Logs

The following PowerShell code:

$operations = 'Add service principal.'
$recordType = 'AzureActiveDirectory'
Search-UnifiedAuditLog -StartDate $start -EndDate $end -ResultSize $resultSize -Formatted -Operations $operations -RecordType $recordType

returns events with operation = ‘Add service principal.’ Nice, but…
if an app was registered in Azure – event contains an UPN under UserIds property:

Unfortunately, in case with registering app in SharePoint, an audit log event will be like:

i.e. UserId registerd is “spo_service@support.onmicrosoft.com”, so we do not know who registered a SharePoint-only app

In theory – we could use events recorded immediately before and after “Add service principal” event to track a user and site who has registered a SharePoint-only app… But for me it seems like too complicated for automation.

Instead we can do simple search through audit log for events “AppRegNew.aspx page visited”. This gives us a good approximation of who registered SP-App-only principal. Worst scenario – we reach more people than we really need (including those who started registering sp-app-only but did not complete) but all of them would be definitely our target auditory.

Consider the following code:

$freeText = "appregnew"
$operations = 'PageViewed'
$recordType = 'SharePoint'

$results = Search-UnifiedAuditLog -StartDate $start -EndDate $end -ResultSize $resultSize -FreeText $freeText -Operations $operations -RecordType $recordType 

this would give you all users who loaded “/_layouts/15/appregnew.aspx” page

Update: Sites.Selected API MS Graph permissions was introduced by Microsoft in 2021. It was a good move towards site-level development, but still developers were limited with only what MS Graph API provides for SharePoint dev.
So devs had to use AppInv.aspx at site level to provide ACS permissions to their apps to be able to use SharePoint CSOM and REST APIs.
Recently Microsoft introduced Sites.Selected SharePoint API permissions for registered Azure Apps! So now devs should be fully happy without ACS-based permissions.

References

How to Find Content Shared with Everyone in SharePoint and Teams

There is a known problem in SharePoint – complicated permissions system. Site owners/administrators provide access, site contributors upload documents and nobody knows – who has access to their sites. As a result – sometimes sensitive documents become overshared (over-exposed).

The biggest concern is sites content shared with “Everyone”. How do we find sites shared with “Everyone” in a large Microsoft 365 environment?

NB. When I say “shared with Everyone” – I actually mean 3 possible “everyone” logins:

  • Everyone
  • Everyone except external users
  • All users

Approach #1 (Brute force)

We can get full permissions report at tenant level (or permissions provided only to “Everyone”). There are 3-rd party tools (e.g. ShareGate, SysKit, AvePoint, Metalogix etc.), or you can run PowerShell script…

Sounds easy? Well, if you have less than 1000 sites – probably it will work. But if your environment is 10K+ sites – it will take forever. Permission report might run hours for an average site with site/subsite, list/library and list item details level. So the approach will not work for large enterprise environments.

One might say – we can limit report with root web permissions only to get it faster. But this would not be accurate. And what is not accurate in the IT security – lead to even bigger risks. So, we need report detailed up to every item level deep, as even one file with sensitive info shared with everyone can cause security issue. (3-rd party tools usually by default limit it to libraries level.)

Ok, if this approach is not really working – what’s working?

Clever idea: why do we need to iterate through all the tenant documents/items if all the content is already crawled by search? Search is also respect permissions. Can we just use search to get files shared with Everyone? Let us see.

What if we use some dummy user account with no specific permissions provided and no group membership and try to search content on behalf of that account. The idea is if this user can see data – it means that data is open for everyone.

Check this and this articles. Can we get results programmatically (e.g. with PowerShell)? Can we use Microsoft Graph search API? Sure.
Check this article “How to search against SharePoint Online Content with Microsoft Graph search API with PowerShell”.

But! We have some problems here.

Search Problem #1. Again, for small environments or if there are not much “Open” sites – it would work. But for large enterprise environments the problem is the same as in “brute force”. Search returns too many results – it’ll take weeks to get all of them. (There are team sites “legally” shared with everyone, public Office 365 group based sites, communication sites… ).

Search Problem #2. Even if we get all search results – we do not know – at what level permissions are provided to everyone. So we will need to build list of sites based on the search results – ant then still need to run permissions report against these sites.

Search Problem #3. We are getting results with paging. But recently Microsoft started limiting number of returning results. E.g. your search request result might say like “there are 3659735 total hits” but after result number 1000 it just stops returning anything, even with paging.

Approach # 3 Hybrid

The idea: why do we need to get all search results if even one result from a site would be enough to add the site to the list of sites require permissions review. In other words, we do not need all results from site, we only need one to know the site is open.

So, consider (imho, the best) approach (Solution):

  1. You get list of all sites in tenant.
  2. You run search request against each site in the loop
    (e.g. consider KQL option “Site: https://yourTenant.SharePoint.com/sites/YourSite”.
    If at least something found in the site – add the site to the “Open Sites” list.
    With this approach you will get list of sites shared with “Everyone…” in a couple of minutes.
  3. Run permissions report against this shortlist

Note: consider there are resources like “Styles Library” shared with everyone by default.

Note: You can refine the list you get at step 1 – e.g., excluding sites connected to public teams or known communication sites…

Note: consider implementing sensitivity labels. At least you can start with high-sensitive sites. Site owners/member will know – what kind of site they are working on.

Pros and cons of the Approach # 3 Hybrid

Pro: the only fast and accurate enough to rely on

Con 1 : crawling and indexing takes time, so search-based reports can miss recent changes in data and permissions

Con 2: this approach cannot be automated (since we need an interactive authentication).

How to communicate to site owners

The Next step would be “How to let site owners know that there are resources shared with Everyone… on their sites”.

References

Access SPO Site Programmatically via MS Graph API and SharePoint API

Scenario

You are a software developer. Your company uses Microsoft Office 365 (SharePoint, Teams etc.). The need is to work with a specific site collection programmatically (from code – Python, C#, Java, PowerShell, JavaScript etc.) – e.g. upload/download documents, update list items, search etc.

The code must run without user interaction (unattended, aka daemon app). Sometimes this is also called “SharePoint Automation”.

The solution is based on a new Graph API feature – Sites.Selected and a classic SP-Only app.

Solution

  1. Register an Azure App and configure it as usual.
    Select API Permissions blade and add two permissions:
    – Microsoft Graph -> Applications Permissions -> “sites.selected
    – SharePoint -> Applications Permissions -> “sites.selected
  2. Request “Grant admin consent” from a tenant/global admin
  3. Request SharePoint admin to run PowerShell code (e.g. this one) to assign proper permissions to your azure app for a specific site collection (consider site owner consent)
  4. (optionally) Provide SharePoint API permissions:
    (require Site Collection Owner/Admin account) – use
    https://YourTenant.sharepoint.com/teams/YourSite/_layouts/15/appinv.aspx
    to add SharePoint API permissions to your app. E.g. full control permissions to site collection would be
<AppPermissionRequests AllowAppOnlyPolicy="true">  
   <AppPermissionRequest Scope="http://sharepoint/content/sitecollection" 
    Right="FullControl" />
</AppPermissionRequests>

Consider minimal permissions (e.g. as per Sumit)

Problem Solved

  • you get access to one and only one site collection (“least privilege” principal)
  • you get both – SharePoint API and Microsoft Graph API permissions to SharePoint
  • you can use app secret or certificate to authenticate – depending on what are your security requirements

Note: if your scenario require authenticated user present – the solution would be a little different: Connect-PnPOnline Interactive with Client App Id

Update:

Sites.Selected API MS Graph permissions was introduced by Microsoft in 2021. It was a huge step forward, but still devs were limited with MS Graph API against SharePoint.
So devs had to use AppInv at site level to provide ACS permissions to their apps to use SharePoint CSOM and REST APIs.
Recently Microsoft introduced Sites.Selected SharePoint API permissions for registered Azure Apps! So now devs should be fully happy without ACS-based permissions AppInv.aspx. (See more here on disabling SP Apps Only SPNs)

Thanks to Leon Armston and Scott Murdock

References:

SharePoint site full permissions report

There has always been one problem in the SharePoint world: full site permissions report. Full means across entire site – including all objects with broken permissions.
It seems like Microsoft has solved the problem: Full site permissions report is available for site owners out-of-the-box.

How to get SharePoint All Site Permissions Report

(Ensure you are site collection admin or team/group owner).
Just navigate to Site Usage, scroll to the end and run report.

  1. Select gearbox “Settings” and then Site usage:

Or Select “Site Contents”, then “Site Usage” as shown below:

2. Scroll down to the “Shared with external users” block and click “Run report”:

  1. Create/Select folder (*) for the report and click “Save”:
    • If there are no folders in the Documents folder – you need to create one (otherwise you will not be able to save it:)
  • Once yo have a folder available – just click “Save”:

Give it some time (5-10 minutes) – check the folder’s content. There should be a file with a report on all site permissions.
For items shared with direct access, the report contains one row for each user / item combination.
SharePoint groups are shown in the report as groups (not individual users inside them… so you have to check group membership to get really full permissions report).

Again, you must be a site admin to run the report.

  1. Secure the permissions report
    If you don’t want other site members to see the report – secure the report’s folder – e.g. for site owners and for those who must be able see the report…
    Consider creating a separate library for permissions reports and secure it instead of securing a folder under Documents.

Some more ideas on SharePoint permissions

Permissions are tricky in SharePoint. By default, you have permissions assigned to the root site of the site collection and all subsites, libraries etc. inherit root permissions.
But you can break inheritance at any level you need to provide specific (unique) permissions to the resource.
Of course you can always navigate to the resource and check resource permissions. But… what if there are hundreds of broken permissions… should you iterate everything under your site to check manually if permissions are broken or inherited?

So the real problem was – you never knew who have access to your site as there was no out-of-the-box tool to get all site permissions in one single report. There are third-party solutions – like ShareGate, Metalogix or SysKit – or you can develop PowerShell script generating report on all SPO site permissions. But… finally Microsoft solved this problem – Microsoft implemented out of the box full site permissions report.

Reference:

Microsoft Report on file and folder sharing in a SharePoint site

Connect-PnPOnline with a certificate stored in Azure Key Vault

Scenario

You run some PnP PowerShell code unattended e.g. daemon/service app, background job – under application permissions – with no user interaction.
Your app needs to connect to SharePoint and/or Microsoft Graph API. Your organization require authentication with a certificate (no secrets). You want certificate stored securely in Azure Key Vault.

Solution (Step-by-step process)

  1. Obtain a certificate (create a self-signed or request trusted)
  2. In Azure where you have Microsoft 365 SharePoint tenant
    1. Create a new Registered App in Azure; save App (client) id, Directory (Tenant) Id
    2. Configure App: add MS Graph and SharePoint API application (not delegated) permissions
    3. Upload the certificate to the app under “Certificates & secrets”
  3. In Azure where you have paid subscription (could be same or different)
    1. Create an Azure Key Vault
    2. Upload certificate to the Key Vault manually (with GUI)
  4. While you develop/debug your custom daemon application at your local machine
    1. Provide permissions to the Key Vault via Access Control and Access Policies to your personal account
    2. Connect to Azure (the one where your Key Vault is) running Connect-AzAccount
      – so your app can get a Certificate to authenticate to SharePoint Online
  5. For your application deployed to Azure (e.g. Azure Function App )
    1. Turn On managed identity (Your Function App -> Identity -> Status:On) and Save; notice an Object (Principal) Id just created
    2. Provide for your managed identity principal Id permissions to the Key Vault via Key Vault Access Policies, so when your daemon app is running in the cloud – it could go to the key Vault and retrieve Certificate

Here is the sample PowerShell code to get certificate from Azure Key Vault and Connect to SharePoint with PnP (Connect-PnPOnline):

# ensure you use PowerShell 7
$PSVersionTable

# connect to your Azure subscription
Connect-AzAccount -Subscription "<subscription id>" -Tenant "<tenant id>"
Get-AzSubscription | fl
Get-AzContext

# Specify Key Vault Name and Certificate Name
$VaultName = "<azure key vault name>"
$certName = "certificate name as it stored in key vault"

# Get certificate stored in KeyVault (Yes, get it as SECRET)
$secret = Get-AzKeyVaultSecret -VaultName $vaultName -Name $certName
$secretValueText = ($secret.SecretValue | ConvertFrom-SecureString -AsPlainText )

# connect to PnP
$tenant = "contoso.onmicrosoft.com" # or tenant Id
$siteUrl = "https://contoso.sharepoint.com"
$clientID = "<App (client) Id>" # Azure Registered App with the same certificate and API permissions configured
Connect-PnPOnline -Url $siteUrl -ClientId $clientID -Tenant $tenant -CertificateBase64Encoded $secretValueText

Get-PnPSite

The same PowerShell code in GitHub: Connect-PnPOnline-with-certificate.ps1

References:

SPO: Allow users to create modern pages

Microsoft: “Using modern pages in Microsoft SharePoint is a great way to share ideas using images, Office files, video, and more. Users can Add a page to a site quickly and easily, and modern pages look great on any device.
If you’re a global or SharePoint admin in Microsoft 365, you can allow or prevent users from creating modern pages. You can do this at the organization level by changing settings in the SharePoint admin center. If you allow the creation of site pages as the organization level, site owners can turn it on or off at the site level.

By default both
– Allow users to create new modern pages
– Allow commenting on modern pages
are turned on (enabled)

Tenant or SharePoint admin can find settings under
SharePoint Admin Center -> Settings -> Pages

How it looks like:

Site Pages are created under “Pages” Library.

Let us test it, with:
– (tenant-level) Allow users to create new modern pages: ON
– (tenant-level) Allow commenting on modern pages: ON
– web feature “Site Pages” – “Allows users to add new site pages to a site”: Activated

User
Permissions
can create Pagecan edit pagecan Enable/Disable
page comments
can comment on Page
Full Control (Owner)YesYesYesYes
Edit (Member)YesYesYesYes
Read (Visitor)NoNoNoYes

There is a web feature “Site Pages” – “Allows users to add new site pages to a site”.
The feature is activated by default:

What if we disable this feature?
“New -> Page” has disappeared from “New” menu under “Site Contents” for Owners and Members…
From “Home” and “Pages” you still can see “New -> Page” options.
You can still create a new page from but if you try to create a page from Pages – “Sorry, something went wrong” “Cannot create a Site Page. Please have your administrator enable the required feature on this site.” :

Office 365 behavior, with:
– (tenant-level) Allow users to create new modern pages: ON
– (tenant-level) Allow commenting on modern pages: ON
– web feature “Site Pages” – “Allows users to add new site pages to a site”: Deactivated

User
Permissions
can create Pagecan edit pagecan Enable/Disable
page comments
can comment on Page
Full Control (Owner)Yes,
but only from “Home”
not from “Site Contents” or “Pages”
YesYesYes
Edit (Member)Yes,
but only from “Home”
not from “Site Contents” or “Pages”
YesYesYes
Read (Visitor)NoNoNoYes


If we disable feature “Site Pages” – “Allows users to add new site pages to a site” on the root web – it does not affect subsites (subwebs).

Can we Activate/Deactivate the feature “Site Pages” using PowerShell?

PowerShell

(TBP)

References
– Microsoft “Allow users to create and comment modern pages

See also:
Allow commenting on modern pages

Office 365 Search scopes

Search is everywhere in Microsoft 365. You can search from SharePoint, Teams, Delve, Yammer etc.

But! You cannot search for anything from everywhere!

  • Search for your Teams chat messages works only in Teams.
  • But from Teams you cannot search for regular (non-group) sites and public teams sites
  • All descriptions are totally out of search (e.g. site description, library/list description – including Yammer groups, Teams and regular sites).
  • Public Team Sites content is not searchable from Teams and Yammer

So, what are the scopes of each search entry point in Office 365 and is there an entry point you can search for everything?

Search scopesSharePoint
Search center
SharePoint home
Office portal
Office desktop app
Delve
TeamsBing
SharePoint contentYesYesYes
Teams contentYesYesYesYes
Teams chats(*1)YesYes
Yammer contentYesYesYes
Yammer chat(*1)Yes
User profilesYesYes
Email
(*1) Microsoft announced they are working on bringing conversations (both Teams chats and Yammer) to SharePoint landing page first, then to Office home page.

Detailed:

ScopeOut of Scope
SharePoint Search Center– all sites content
(Teams, Yammer, regular),
– user profiles
– OneDrive
Teams chat
Yammer chat
SharePoint Landing Pagesame as SharePoint Search center
but Teams chats and Yammer Conversations are coming
same as SharePoint Search Center
Office.comsame as SharePoint
(Teams chats and Yammer Conversations are coming after SharePoint)
same as SharePoint
Delve
TeamsTeams content
Teams chat
OneDrive
Yammer
User Profiles
regular SharePoint sites
BingEverything* * except people profiles content
(e.g. about me)

Seems like the only tool you can search for EVERYTHING with is Microsoft Bing:

After Microsoft add Teams chats and Yammer conversations to SharePoint landing page search scope (then to Office home page) – it’ll be the best place to search from for everything.

More on Microsoft Search vs SharePoint Search and Microsoft Search RoadMap

Microsoft Office 365 Search: Find what you need with Microsoft Search in Bing

It is possible customize Modern Microsoft Search pages with PnP Modern Search

SharePoint PnP roadmap

Good news!
On Sep, 18 during the SIG community call, PnP Team shared their plans on PnP Sites Core library and PnP Core SDK.
“PnP Sites Core v4” library and “PnP Core SDK v1” with .net core support (.net Standard 2.0) – expected in December 2020!

PnP PowerShell v4 for SPO library built for .Net Standard 2.0 / PowerShell 7 will be released in Dec 2020 as well.

How to create a Sub-Site if subsites creation is disabled

Update: Microsoft is deploying an updated version of “Disable Subsites” feature:

This update makes the setting options for new subsite creation easier to understand and prevents users from being able to create subsites using alternate paths when the subsite setting is disabled.

Admins in the SharePoint admin center can choose to either enable or disable subsite creation across sites or enable for classic sites only. Now, when disabling subsite creation, not only will the subsite option be hidden from the command bar including classic but also users will not be able to create new subsites directly through an URL or API.

The option: Hide the Subsite command has been renamed to Disable subsite creation for all sites and will also hide the subsite creation command (including classic) and disable users from being able to create new subsites through a URL or API.
The option: Show the Subsite command only for classic sites, has been renamed to Enable subsite creation for classic sites only.
The option: Show the Subsite command for all sites, has been renamed to Enable subsite creation for all sites.

Update is applied. What’s Next?

After this update is applied, if you have “Subsite Creation” set to “Disable subsite creation for all sites”, then if any attempt to create a subsite – you’ll get an error message “Sorry, something went wrong. New subsites are not available for your organization. Create a new site instead.”

Site is a new folder

Microsoft recommend “flat structure”, i.e. no subsites. So SPO admins are disabling subsites creation at tenant level. Did you know that you still can create subsite? Let me explain how it is done.

If creation subsites is allowed, you should be able to see it like this:

But actually subsites are not always best practice. Microsoft recommend “flat structure”, i.e. instead of subsite you should have a separate site collection, and if you need a hierarchy and navigation – use Hub sites. So, in Office 365 SharePoint admins usually “disable” SubSites creation:

Now, you see, SubSites are not really disabled, but only the button to create subsites is hidden: “This controls whether the Subsite command appears on the New menu on the Site contents page”.

Anyway, the result is: you are not able to create a SubSite (web) in SharePoint Online:

Actually there are at least 3 options to create a SubSite:

Option 1. Create a SubSite in Classic mode.

Step 1: Select “Site Contents” page
Step 2: Click “Return to classic SharePoint”
Step 3: Create SubSite

Option 2. Create a SubSite from “Sites and Workspaces” page

Step 1: Go to “Site Settings”
Step 2: Select “Sites and Workspaces” page (site/_layouts/15/mngsubwebs.aspx)
Step 3: Create a SubSite

Option 3: use PowerShell PnP

Step 1: Install PowerShell PnP
Step 2: Connect to your site with PnP
Step 3: create a SubSite

Install-Module SharePointPnPPowerShellOnline
Connect-PnPOnline -Url <your site Url> -UseWebLogin
New-PnPWeb -Url "<new Web Url>" -Title "MySubSite" -Template "STS#3"

References:

See also: How to create a SharePoint Site in Office 365 if site creation is disabled