When Setting up a new B2Qt Device you can use the option to use a specific "Private key file".

Using the "Test" function works as intended and doesn't ask for a password.

But using other functionality like "Open Remote Shell" or "Show Running Processess..." doesn't do anything or asks for the password again instead of using the key file.