UDP Socket receives packets from one IP Address while it ignores others

I am having a issue I have a UDP socket open on port 49473 while one ip address is being recognized another is not. I see in wireshark packets coming from 18.x.x.x and 24.x.x.x only the 18.x.x.x packets are being read. Any ideas what might be happing? (Side note 18.x.x.x is not traversed but 24.x.x.x is not sure if that makes a difference..) Also I have tried increasing the buffer from 8192 to 150k it did not help. The buffer is not dropping packets so that ones out unfortunately..

Image from wireshark click to enlarge

    public UdpSocket(int port = 0)
    {
        sw.Start();

        _buffer.Client = this;

        if (port != 0)
            _buffer.IsHost = true;

        _discovery = new Discovery(_buffer);
        _buffer.NatOp = new NatOperation(_buffer, _buffer.IsHost);

        _endPoint = new IPEndPoint(IPAddress.Any, port);
        _buffer.Stream = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
        _buffer.Stream.ReceiveBufferSize = 8192;
        _buffer.Stream.SendBufferSize = 8192;
        _buffer.Stream.EnableBroadcast = true;

        //***********Magic Fairy Dust*************
        uint IOC_IN = 0x80000000;
        uint IOC_VENDOR = 0x18000000;
        uint SIO_UDP_CONNRESET = IOC_IN | IOC_VENDOR | 12;
        _buffer.Stream.IOControl((int)SIO_UDP_CONNRESET, new byte[] { Convert.ToByte(false) }, null);
        //*********** Fairy Dust End *************

        _buffer.Stream.Bind(_endPoint);

        Thread receive = new Thread(ReceiveUdp);
        Thread send = new Thread(SendThread);

        receive.IsBackground = true;
        send.IsBackground = true;

        receive.Start();
        send.Start();
    }

    private void ReceiveUdp()
    {
        while (IsRunning)
        {
            try
            {
                Packet packet = _buffer.CreatePacket();
                EndPoint _ep = new IPEndPoint(IPAddress.Any, 0);
                count = _buffer.Stream.ReceiveFrom(data, 1024, SocketFlags.None, ref _ep);

                if (((IPEndPoint)_ep).Address.ToString() != "18.x.x.x")
                    Console.WriteLine("Foreign Address Connection " + packet.RemoteEp.Address.ToString()); //never returns so 18.x.x.x only processing 

                packet.Data.MirrorData(count, data);
                packet.RemoteEp = _ep as IPEndPoint;

                packet.ReadHeader();

                lock (_inProcess)
                    _inProcess.Enqueue(packet);
            }
            catch { }
        }
    }

For anyone reading this SendTo and ReceiveFrom creates ICMP responses the magic fairy dusts disables that, this solution only works on Windows / Linux. Also I accepted the SocketOptions IP unrestricted as correct.. In my case Socket.Connect was the issue, when you do this it only allows that ip to make transfers.

This may be a NAT related problem. Do a try

_buffer.Stream.SetIPProtectionLevel(IPProtectionLevel.Unrestricted);

before or straight after Bind. UDPClient does exactly that by method AllowNatTraversal.

Or this may be a firewall/AV related issue. Turn them off temporarly and get a try.