If you remove the Load and unload device drivers user right from the Print Operators group or other accounts, you could limit the abilities of users who are assigned to specific administrative roles in your environment. You should ensure that delegated tasks aren't negatively affected.