Why when someone installs my game in Windows does it say Publisher is Unknown?

When I package my game for development (because I want to alpha test right now, not necessarily release yet), then the resulting game needs an installer so I can distribute the test version as a single self-installing file.
So I pack it up using InnoSetup, and everything’s fine. I run the installer, it installs. So far, so good.

When the user runs the game for the first time, Windows pops up a security notice saying it doesn’t recognize the publisher and asks for permission to run it or not. It lists my Game name but under that it says “Publisher: Unknown”.

How do I make the publisher known? Is this a problem specific to InnoSetup or do I have to do something called “code signing” for my users to have a proper experience installing and running the game, and showing the game is from me and not some mysterious “Unknown” publisher?

I want to avoid code signing if possible because it apparently costs $$ and that seems like a waste when I’m already putting money into publishing it in an online games store.