Gemini Live could give you more control over what it hears with push-to-talk mode

https://www.androidauthority.com/gemini-live-push-to-talk-3684532