File or Assembly not found when using Assembly.Load(path)

I am implementing a ASP.NET MVC 4 application which supports plugins and since today I have a strange behavior and I do not know why:
I load the Plugins via Assembly.Load(path) which worked fine a few days ago, but locked my files as shadow copying does not work as the plugin folder was outside /bin.
Because of this I used Assembly.Load(File.ReadAllBytes(path)) which also works fine, but I think is not really clean code, so today I moved my plugin folder inside /bin and switched back to Assembly.Load(path), but now I get the following exception (Translated as I get it in my locale) when executing my code:

Could not load file or assembly "PATH_TO_MY_DLL" or one of its dependencies. The given assembly name or codebase was invalid. (Exception from HRESULT: 0x80131047)

If I switch back to Assembly.Load(File.ReadAllBytes(path)) it works, but why does it not work when loading the file directly anymore?

I think you are making a mistake when you changed it back.

Assembly.Load(string name) : loads the assembly named "name".

You should use:

Assembly.LoadFile(path);