Hello Tobias,
yes, I would have assumed that it's greyed out as well, but I thought it might be worth a shot.
I just tried to remove a deactivated user and had no issues seeing the "remove from account" buttons. The only other thing I could think of is that you don't have the permissions. But that also shouldn't apply as a super admin
Best
Colin