First, let $x\in\{-1,0,1\}$. Then, $y^q\in\mathbb{Q}$ for any odd prime $q$. If $y\notin \mathbb{Q}$, then both $y^3,y^5\in\mathbb{Q}$, so $y^2\in\mathbb{Q}$, yielding $y\in\mathbb{Q}$. Hence, suppose $x,y\notin\{-1,0,1\}$. Consider $(p,q)=(31,3)$ and $(31,5)$ gives $y^5-y^3 = y^3(y^2-1)\in\mathbb{Q}$. Likewise, considering $(p,q)=(31,5),(31,7)$ gives $y^5(y^2-1)\in\mathbb{Q}$. As $y(y^2-1)\ne 0$, we get $x^2,y^2\in\mathbb{Q}$. The rest is straightforward.