Terrain roughness painting

I would like to paint other channels then just the diffuse color/texture of a terrain. Is it possible to paint just the roughness without manipulating the Base Color?

I just simply tried making a new Layer that I only input into the Roughness slot but it also changed the Base Color anyway.