UPDATE: It works perfectly fps wise with Virtual Desktop, so the problem is with the link (I have both the official and an amazon link cable).
The issue with virtual desktop is you cannot get the same clarity as with the link cable by overriding the resolution on the oculus app.