How to get specific textures to use displacement?

I am trying to get a single texture to use displacement but every time I try to use it the way the live tutorial has it every texture in that scene gets displaced based on that one textures height map. How do I make this so it just effects a single texture for the landscape when painting?