Prevent gamepad from focusing on other players' widgets?

In my local multiplayer project, I have a single viewport that can see all four players. I create widgets for each player and set the focus for each player to their own widget. The gamepad navigation (stick + dpad) navigates between all 4 players widgets for all 4 gamepads!

Is there a way to prevent this? I was happy to find “built-in” gamepad widget navigation support, but it’s pretty useless if you can’t restrict what it can navigate to.