When creating an IServiceManagement for Dynamics 365, why does the authentication endpoint respond with an HTML sign in page?

I have some integration code that intends to use the Organization Service via the CRM SDK.

On one environment, creating an IServiceManagement<IOrganizationService>:

IServiceManagement<IOrganizationService> orgServiceManagement = ServiceConfigurationFactory.CreateManagement<IOrganizationService>(new Uri("dynamics uri")));

and then authenticating with service account credentials:

AuthenticationCredentials authCredentials = new AuthenticationCredentials();
            authCredentials.ClientCredentials.UserName.UserName = _config.GetValue<string>("Dynamics:Username");
            authCredentials.ClientCredentials.UserName.Password = _config.GetValue<string>("Dynamics:Password");
            AuthenticationCredentials tokenCredentials = orgServiceManagement.Authenticate(authCredentials);

works fine.

On another Dynamics environment, the call to GetServiceManagement fails with the following error message:

System.InvalidOperationException HResult=0x80131509 Message=Metadata contains a reference that cannot be resolved: 'https://login.microsoftonline.com/[guid]/oauth2/authorize?client_id=[some client id]&response_mode=form_post&response_type=code+id_token&scope=openid+profile&state=OpenIdConnect.AuthenticationProperties%[some base-64]RedirectTo%3dhttps%253a%252f%252ftst-success.crm4.dynamics.com%252f&nonce=[some nonce]&redirect_uri=https:%2f%2fcloudredirector.crm4.dynamics.com%2fG%2fAuthRedirect%2fIndex.aspx&max_age=86400'. Source=System.ServiceModel StackTrace: at System.ServiceModel.Description.MetadataExchangeClient.MetadataRetriever.Retrieve(TimeoutHelper timeoutHelper) at System.ServiceModel.Description.MetadataExchangeClient.ResolveNext(ResolveCallState resolveCallState) at System.ServiceModel.Description.MetadataExchangeClient.GetMetadata(MetadataRetriever retriever) at System.ServiceModel.Description.MetadataExchangeClient.GetMetadata(Uri address, MetadataExchangeClientMode mode) at Microsoft.Xrm.Sdk.Client.ServiceMetadataUtility.RetrieveServiceEndpointMetadata(Type contractType, Uri serviceUri, Boolean checkForSecondary) at Microsoft.Xrm.Sdk.Client.ServiceConfiguration`1..ctor(Uri serviceUri, Boolean checkForSecondary) at Microsoft.Xrm.Sdk.Client.ServiceConfigurationFactory.CreateConfiguration[TService](Uri serviceUri, Boolean enableProxyTypes, Assembly assembly) at Microsoft.Xrm.Sdk.Client.ServiceConfigurationFactory.CreateConfiguration[TService](Uri serviceUri) at CrmAuthTest.Program.Main(String[] args) in c:\users\t.wolverson\Source\Repos\CrmAuthTest\CrmAuthTest\Program.cs:line 18

Inner Exception 1: XmlException: CData elements not valid at top level of an XML document. Line 1, position 3.

(I have masked the bits which look identifying or cryptographic)

POSTing to this URL in PostMan yields the HTML for a browser login page, which explains the failure; this isn't what the ServiceConfigurationFactory expects. The scenario is not user-interactive, so this would never make sense, there is no browser and no user able to interact with it.

What do I have to change in Dynamics CRM Online to stop it doing this, and make it just work normally?

Do you instantiate your OrganizationServiceProxy depending on the AuthenticationProviderType right after the lines of code you have posted? Like this

        var orgServiceManagement = ServiceConfigurationFactory.CreateManagement<IOrganizationService>(new Uri(ConfigurationManager.AppSettings["CrmUrlService"]));

        var authCredentials = new AuthenticationCredentials();
        authCredentials.ClientCredentials.UserName.UserName = ConfigurationManager.AppSettings["CrmUserName"];
        authCredentials.ClientCredentials.UserName.Password = ConfigurationManager.AppSettings["CrmPassword"];
        var tokenCredentials = orgServiceManagement.Authenticate(authCredentials);

        IOrganizationService _service;
        switch (orgServiceManagement.AuthenticationType)
        {
            case AuthenticationProviderType.ActiveDirectory:
                _service = new OrganizationServiceProxy(orgServiceManagement, tokenCredentials.ClientCredentials);
                break;
            default:
                _service = new OrganizationServiceProxy(orgServiceManagement, tokenCredentials.SecurityTokenResponse);
                break;
        }

Even if this solves your problem, I recommend that you use CrmServiceClient instead. This class can be found in Microsoft.Xrm.Tooling.Connector dll. It is the go to authentication class when building Windows client applications that connect to Microsoft Dynamics 365. More information on this can be found here

Here is an example on how to initialize CrmServiceClient when connecting to Dynamics 365 online using Office 365:

        var myConnectionString = "Url=https://[YourOrganization].crm4.dynamics.com;Username=[YourUser];Password=[YourPassword];AuthType=Office365;";
        var crmClient = new CrmServiceClient(myConnectionString);

       //Do your stuff
        var response = crmClient.Execute(new WhoAmIRequest());

If you need other authentication methods in Dynamics Online check how to build your connection string here.

For on-premises check how to build your connection string here.