How do I stop keyboard input

I have a legion go 2 and whenever I am in game mode with bazzite and using my controller if I take a short 4 minute break and resume the controls will go to recognizing a keyboard instead of my controller. Is there a way I can stop it from doing that I never use keyboard when gaming