Hint : Use $p^{th}$ root of unity and its minimal polynomial $Q(x)= x^{p-1} + x^{p-2} + ... +1$ . Note that $Q$ is irreducible over $\mathbb Z [x] $ .

quite useful link, see page 5. lemma 2. http://reflections.awesomemath.org/2006_1/2006_1_equi_polygons.pdf

The only if part is obvious, now we prove the if part. Let the side lengths of the polygon be $a_0,a_1,\ldots,a_{p-1}$, in that order. We know that $a_0,\ldots,a_{p-1}$ are rationals. By scaling, we may also assume that they are integers. Let $w$ be the $p$-th root of unity, hence we have $a_0+a_1w+a_2w^2+\ldots+a_{p-1}w^{p-1}=0$. The two polynomials $P(x):=a_0+a_1x+a_2x^2+\ldots+a_{p-1}x^{p-1}$ and $Q(x):=x^{p-1}+x^{p-2}+\ldots+1$ share a common root $w$. So $w$ is also a root of $P(x)-a_{p-1}Q(x)=(a_{p-2}-a_{p-1})x^{p-2}+\ldots+(a_1-a_{p-1})x+(a_0-a_{p-1})$. However it is well-known that $Q(x)$ is the minimal polynomial of $w$, hence $P(x)-a_{p-1}Q(x)\equiv0$. Thus $a_i=a_{p-1}$, which implies that the polygon is regular.