Remove an added GPG key from the host.