System.Net.Sockets.SocketException when trying to connect to hosted game in Unity Mirror

This might be a noob question, I just started Mirror in the Unity3D Game Engine:

When I try to connect to a client-hosted server, this exception gets thrown.

Client Recv: failed to connect to ip=localhost port=7777 reason=System.Net.Sockets.SocketException (0x80004005): No connection could be made because the target machine actively refused it.

It works fine when I host and join the server on the same machine, but when I try to use different PCs on the same network, that happens.