The only thing working in that interaction is mouse clicks, which get interweaved with keyboard key events like it should be.
This is still the case if I just get one byte per interrupt, even if the only code running in these interrupts gets the byte and prints it.

Maybe qemu is at fault, that's why I would like to get bochs/real hardware running first.