How to handle UObjects inside of a TMap

I have this going on inside one of my files:

struct ItemInfo {
	int index;
	UGUIElement* guiElem; // This
};
TMap<int, ItemInfo> map;

Whenever the player does something, a new element is added to this map with map.Add(x, { i, CreateWidget(world) }), eg. the map holds the actual data. This map gets cleared with a simple map.Empty() call at some point.

The problem is memory leaks. I know/think that TMap isn’t going to be deallocating anything for me, so should I loop through it instead of just calling Empty()? I’ve thought about making guiElem a TUniquePtr instead of a raw pointer, but read that I shouldn’t be using those smart pointers for UObjects. Since this isn’t marked as a UPROPERTY() however, it’s not going to be automatically garbage collected (I think). I also don’t want to put that macro on it just for GC.

How should I properly deal with this situation?

I might be completely of with my wild guess, but here is at least something to help you get further.

Based of this post about memory allocation and deallocation for structs:

It is possible that you need to loop through each element of the map and delete guiElem before you call empty().