Cannot connect to sql server by IP Address but using the hostname works

After upgrading to the newest version of windows (windows 10 build 10586) I am no longer able to connect to a remote sql server by ip address.

Here is the error I see:

[SqlException (0x80131904): Connection Timeout Expired. The timeout period elapsed while attempting to consume the pre-login handshake acknowledgement. This could be because the pre-login handshake failed or the server was unable to respond back in time. The duration spent while attempting to connect to this server was - [Pre-Login] initialization=18303; handshake=14385; ]

This is not a named instance of sql server. I am able to connect without an issue if i use the hostname of the machine. What could cause this?

On Windows 10 before the November update there was no problem connecting.

Sql Server 2012 (11.0.5058) Windows 10 Build 10586