As the answer from user146001 mentions, in your Home directory, you enable "Show Hidden Files" (or press Ctrl + H to see hidden files) and find your .VirtualBox folder. In there, go to your HardDisks folder and simply delete the .vdi files, as these are the files of your virtual drives.