Can't get Steamworks up and running in our Project

We have recently been Greenlit on Steam, so after receiving the Steam Game ID I wanted to integrate the Steamworks SDK into our game. I followed the official Online Subsystem Steam documentation, as well as the Steam guide on the UE4 wiki. We are using official UE4.12 (i.e. not from GitHub), which ships with Steamworks 1.32.

Now when I re-start the UE4 editor, no messages appear in the output log (I have searched for “steam” there). Also when I run the game in standalone mode and hit Shift+Tab, nothing happens.

Any ideas where I should check for the issue, why Steam overlay is not showing up in our project?