All vulnerability hangover is, is an immense amount of shame and fear projecting itself upon you after you shared your heart beyond what feels safe.
In order to reach vulnerability hangover, you had to choose to be emotionally exposed and vulnerable... so, you ignored shame once.
Best to just keep ignoring him again.