Material heightmap on terrain not functioning properly

When trying to create a basic terrain for a map, I’ve run into a problem.
Somehow the heightmap of the terrain doesn’t scale properly. When I adjust the material to multiply the heightmap values by a large amount, I can see some weird peaks in the terrain height that don’t make much sense.

If I don’t do this, the terrain appears flat. The contour when selecting it, however, implies that the terrain does use the height map!

When playing, the terrain also flickers from time to time.

I have scaled the UV tiling for the material since it would otherwise display the material hundreds of times in each direction, but I doubt that is the problem. When using the LandscapeLayerCoords node instead of the regular TexCoords node, the same problem persists.

I have no idea what causes this problem or how to fix it, I would really appreciate some help!