UE4 does not always delete assets.
Thanks for providing clear repro steps; After taking a closer look, I see that this has already been reported as UE-39208 which, if you follow the link, shows that it has not yet been resolved. You can also use this link to track the status of the bug for future reference.
Thanks for your feedback,
answered Feb 16 '17 at 06:19 PM
Follow this question
Once you sign in you will be able to subscribe for any updates here