How to setup custom error pages with Razor views and ASP.NET MVC5?

I am trying to implement custom error pages for my ASP.NET (.NET Framework 4.7) MVC 5 web application.

Unfortunately, I could not get custom error pages working without the redirection. I always get the following error:

System.Web.HttpException (0x80004005): The controller for path '/test' was not found or does not implement IController.

Having the redirect setup (replace redirectMode="ResponseRewrite" with defaultRedirect="~/Error/GenericError"), pages which could not be found would redirect to the error page. However, I do not want the redirect for logical/semantic/SEO reasons (e.g. not the error page [redirect] is not found but the page the user has opened). But this is not for discussion.

How to implement custom error pages without the redirect?

---

Directory structure

• /
  • Controllers/
    • ErrorController.cs
    • HomeController.cs
  • Views/
    • Error/
      • 404.cshtml
    • Home/
      • Index.cshtml
  • Web.config

/Web.config (stripped from unrelated, e.g. log4net configuration section)

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <system.web>
        <customErrors mode="On" redirectMode="ResponseRewrite">
            <error statusCode="404" redirect="~/Error/Error404" />
        </customErrors>
    </system.web>
    <system.webServer>
        <httpErrors errorMode="Custom" existingResponse="Replace">
            <remove statusCode="404" />
            <error statusCode="404" responseMode="ExecuteURL" path="~/Error/Error404" />
        </httpErrors>
    </system.webServer>
</configuration>

/Controllers/ErrorController.cs

using System.Web.Mvc;

namespace WebApplication1.Controllers
{
    [HandleError]
    public class ErrorController : Controller
    {
        public ActionResult Error404()
        {
            return View("404");
        }
    }
}