Why do we increment the program counter by 4 instead of 32 in MIPS?

In MIPS, I know that the PC is incremented by 4 for each instruction. This is because the word is on a 32 bit boundary (4 bytes). This makes sense to me, and naturally we need to increment the PC by the difference in spatial location of two consecutive words.

For instance, we might have 0x00000000 as the PC value but then, when we increment the PC, it becomes 0x00000004. But, this difference is really 4 bits, not 4 bytes (32 bits). Why is it 0x00000004 instead of 0x80000000?

Binary-wise this question becomes: 0000 0000 0000 0000 0000 0000 0000 0000 to 0000 0000 0000 0000 0000 0000 0000 0010 versus 1000 0000 0000 0000 0000 0000 0000 0000. The second seems to be a difference of 32 bits (4 bytes) versus the first which is a difference of 4 bits.

The only thing I can think of is that would have to multiply the PC by 8 to get the actual starting point of a word. Adding by 4 should not be enough? Can someone please explain what I'm missing?

Because the machine is byte addressable as opposed to bit addressable; therefore adding 4 advances the pointer by 32 bits.

The address bus wants the location in bytes. There is no way to address individual bits.

It's entire possible that it's actually incrementing the PC by 1 and the low two bits don't even exist in the register. At least one processor worked that way.