For certain applications, it can be beneficial to constrain the parameter space during model training. For example,enforcing orthogonality of the learned parameters can improve convergence for RNNs. PyTorch provides a mechanism forapplying parametrizations such as this, andfurther allows for custom constraints to be defined.