I will be saving my personal gotchas on Microsoft 365 External Access and Guest Access in SharePoint and Teams
We configure external/guest access in AAD, m365 Admin Center, Teams Admin Center, SharePoint Admin Center, specific Group, Team or SharePoint site.
We can configure external guest access directly, or can configure sensitivity labels and policies in Purview (Compliance Admin Center). Configuring sensitivity labels for sites/groups we configure external guest access settings. Configuring sensitivity labels policies we apply labels.
External access via “All Users” group
Be careful with “All users” group created as part of the process.
Microsoft: “The dedicated All Users group includes all users in the directory, including guests and external users.” And indeed, “All Users” group by default include external users.
So here is the scenario: we have a site where external sharing is enabled, and someone is sharing a specific file1 or folder1 with some external users. The other site/group member is sharing another file2/folder2 with “All Users” assuming All Users means all this group member. This gives external users access to file2/folder2.
Remediation
Option 0: remove “All Users” group
Option 1: exclude External users or Guest users from “All Users” group:
(user.userPrincipalName -notContains "#EXT#@")
or
(user.userType -ne "Guest")
(explained here).
Option 2: schedule a job that removes “All Users” from all sites UIL. Optionally inform site owners not to use “All Users” but use “Everyone except external users”.
How to exclude a user from “Everyone Except External Users” group?
Let say, you have a public site and you indeed want to provide access to all internal users with the exception of specific relatively small group of people. E.g. 10,000 users in company and only 200 are not fully integrated yet and you do not want to consider as equals in rights to all others and you do not want to provide automatic access. Unfortunately, there is no “deny access” options in SharePoint. All the functionality is about to “allow access” to something. What are you options?
Option 1: Create a security group and include in the group 9,800 users.
In this case you’d need to review all sites with access provided to EEEU
Option 2: Change user type in AAD (Entra Id) from Member to Guest.
In this case those users will not be a part of EEEU. They will be “Internal Guests”.
You’ll still be able to provide direct access to sites and include such users in teams but they will be marked as Guests.

Think of it that there are internal and external users, and there are guests and members.
Typically all internal users are members, and all external users are guests. But that does not always reflect real life. And if you change a “User type” property for some internal user from Member to Guest – this users will be an Internal Guest. Check this MS article: Understand and manage the properties of B2B guest users