STM32 uC after exit DFU mode does not return to normal but has breakpoints

I'm entering the DFU mode of the STM32 without using the Boot0 physical pin. I do this by sending a command via the UART and switch the uC to DFU mode. I'm using the STM32F042G6U6.

But it doesn't restart when I use the ST tools to exit from DFU mode.

The below code is what I use boot my uC into DFU mode.

#define SYSMEM_RESET_VECTOR         0x1FFFC404 //Offset +4
#define RESET_TO_BOOTLOADER_CODE    0xDEADBEEF
#define BOOTLOADER_STACK_POINTER    0x20001800

void __initialize_hardware_early(void)
{
    *((unsigned long *)0x20001600) =  0xCAFEFEED;

    void (*SysMemBootJump)(void);
    SysMemBootJump = (void (*)(void)) (*((uint32_t *)(SYSMEM_RESET_VECTOR)));

    __set_MSP(BOOTLOADER_STACK_POINTER);
    SysMemBootJump();
}

void dfu_run_bootloader()
{
    *((unsigned long *)0x20001600) = RESET_TO_BOOTLOADER_CODE;
    NVIC_SystemReset();
}

dfu_run_bootloader is called from within the UART receive function. __initialize_hardware_early was placed inside SystemInit.

void SystemInit(void)
{
    if (*((unsigned long *)0x20001600) == 0xDEADBEEF) {
        __initialize_hardware_early();
    }
... // The rest of the SystemInit() function

After click "Leave DFU Mode" I expect the device to return to the normal state. But it doesn't.

When I attach the debugger it reports a breakpoint in two places. Everything as in the pictures below. The device starts to work properly when I click the resume button.