SCRIPT7002: XMLHttpRequest: Network Error 0x80070005, Access is denied

I'm developing an excel web add-on in visual studio. I'm trying to send an ajax request to my API, but I keep getting this error "SCRIPT7002: XMLHttpRequest: Network Error 0x80070005, Access is denied." This error only happens when I test my code with visual studio. Internet Explorer, Chrome, and Firefox all work with my request. I'm pretty sure this is not a CORS issue since I get the response headers:

• Access-Control-Allow-Headers: *
• Access-Control-Allow-Methods: GET,PUT,POST,DELETE,OPTIONS,HEAD
• Access-Control-Allow-Orgin: *

For the record, I've been able to make requests successfully to other APIs through visual studio. So I'm completely lost. I've looked at the other threads on here, but none of them have a solution to my problem. Down below is my ajax call in case that helps (I left out my API token and URL intentionally).

$.ajax({
    type: 'GET',
    url: 'https://***',
    dataType: 'json',
    headers: { "api-token": "***"},
    success: function (data) {
        console.log("success!");
    },
    error: function (jqXHR, textStatus, errorThrown) {
        console.log("error!");
    }
});