Are ARM Cortex-M0 Stacking Registers Saved On $psp or $msp During Hardfault?

I have an issue where my Cortex-M0 is hard faulting, so I am trying to debug it. I am trying to print the contents of the ARM core registers that were pushed to the stack when the hard fault occurred.

Here is my basic assembly code:

__attribute__((naked)) void HardFaultVector(void) {
    asm volatile(
        // check LR to see whether the process stack or the main stack was being used at time of exception.
        "mov r2, lr\n"
        "mov r3, #0x4\n"
        "tst r2, r3\n"
        "beq _MSP\n"

        //process stack was being used.
        "_PSP:\n"
        "mrs r0, psp\n"
        "b _END\n"

        //main stack was being used.
        "_MSP:\n"
        "mrs r0, msp\n"
        "b _END\n"

        "_END:\n"
        "b fault_handler\n"
    );  
}  

The function fault_handler will print the contents of the stack frame that was pushed to either the process stack or the main stack. Here's my question though:

When I print the contents of the stack frame that supposedly has the saved registers, here is what I see:

Stack frame at 0x20000120:
 pc = 0xfffffffd; saved pc 0x55555554
 called by frame at 0x20000120, caller of frame at 0x20000100
 Arglist at unknown address.
 Locals at unknown address, Previous frame's sp is 0x20000120
 Saved registers:
  r0 at 0x20000100, r1 at 0x20000104, r2 at 0x20000108, r3 at 0x2000010c, r12 at 0x20000110, lr at 0x20000114, pc at 0x20000118, xPSR at 0x2000011c

You can see the saved registers, these are the registers that are pushed by the ARM core when a hard fault occurs. You can also see the line pc = 0xfffffffd; which indicates that this is the LR's EXC_RETURN value. The value 0xfffffffd indicates to me that the process stack was being used at the time of the hard fault.

If I print the $psp value, I get the following:

gdb $ p/x $psp
$91 = 0x20000328

If I print the $msp value, I get the following:

gdb $ p/x $msp
$92 = 0x20000100

You can clearly see that the $msp is pointing to the top of the stack where supposedly the saved registers are located. Doesn't this mean that the main stack has the saved registers that the ARM core pushed to the stack?

If I print the memory contents, starting at the $msp address, I get the following:

gdb $ x/8xw 0x20000100
0x20000100 <__process_stack_base__>:    0x55555555  0x55555555  0x55555555  0x55555555
0x20000110 <__process_stack_base__+16>: 0x55555555  0x55555555  0x55555555  0x55555555

It's empty...

Now, if I print the memory contents, starting at the $psp address, I get the following:

gdb $ x/8xw 0x20000328
0x20000328 <__process_stack_base__+552>:    0x20000860  0x00000054  0x00000054  0x20000408
0x20000338 <__process_stack_base__+568>:    0x20000828  0x08001615  0x1ad10800  0x20000000

This looks more accurate. But I thought the saved registers are supposed to indicate where in flash memory they are located? So how does this make sense?

The comments by old_timer under your question are all correct. The registers will be pushed to the active stack on exception entry, whether this is PSP or MSP at the time. By default, all code uses the main stack (MSP), but if you're using anything other than complete bare metal it's likely that whatever kernel you're using has switched Thread mode to using the process stack (PSP).

Most of your investigations suggest that the PSP was in use, with your memory peek around the PSP and MSP being pretty much indisputable. The only bit of evidence you have for it having been the MSP is the results of the fault_handler function, for which you have not posted the source; so my first guess would be that this function is broken in some way.

Do also remember that one common reason for entering the HardFault handler is that another exception handler has caused an exception. This can easily happen in cases of memory corruption. In these cases (assuming Thread mode uses the PSP) the CPU will first enter Handler mode in response to the original exception, pushing r0-r3,r12,lr,pc,psr to the process stack. It will start executing the original exception handler, then fault again, pushing r0-r3,r12,lr,pc,psr to the main stack while entering the HardFault handler. There's often some unravelling to do.

old_timer also mentions using real assembly language, and I agree here too. Even though the ((naked)) attribute should be removing the prologue and epilogue (between them most of the possible 'compilerisms'), your code would simply be far more readable if it was written in bare assembly language. Inline assembly language has its uses, for example if you want to do something very low-level that you can't do from C but you want to avoid a call-return overhead. But when your entire function is written in assembly language, there's no reason to use it.