What is the difference between using remove from parent and toggling the visibility of a widget?

I am somewhat confused about the difference here. I understand what both do but I have primarily used a visibility toggle rather than removing the widget from viewport (parent). Yet a lot of the examples I’ve been seeing have been using remove from parent. Are there downsides to hiding a widget as opposed to removing from viewport/parent?