To delete the dedicated folder from Finder, uninstall the Google Drive app. Everything within the Drive folder is automatically synced up to drive.google.com. So first, you need to disconnect your account to stop syncing and secure your files on the cloud.