OS X Server has service access control lists (SACLs) for all its services. They are used to control which users are authorised to use the services that OS X Server offers.
In a default installation, these SACLs are not present, which results in all users having access to all services. Once you go ahead and click the button to "Manage Service Access", it creates the required SACLs. Users in external directories are now by default given no access to any of the services.
Perhaps you clicked the "Manage Service Access" and confirmed it, but would like to revert back to how it used to be. It's simple enough, just follow these easy steps!
With Server app active. Click View > Show System Accounts in the menu bar.
Select groups in the sidebar, then select "Local Groups" and type in com.apple.access in the filter box. Select all the groups starting with com.apple.access. Be sure to exclude the Screen Sharing ACL. Once you have them selected, you can click the minus button to remove them.
That's all there is to it, you'll notice that the "Manage Service Access" button has now returned and service access is back to what it was before clicking the button.
Do note that if you had any custom service access set, it will be lost after this change!
ps. Thanks to Rich Trouton (@rtrouton) for helping the thought process and help solve this question!