Notice that AA '^2+CC'^2=BB'^2+DD'^2

Let $O$ be the centre of $ABCD$. WLOG we may suppose that $M$ lies in $OAB$'s interior. $\textbf{Lemma}$ $AM^2+CM^2=BM^2+DM^2$, no matter how we choose a point in the plane of rectangle $ABCD$. $\textbf{Proof of lemma}$ Denote by $O$ the centre of $ABCD$. By the median law, we get $4MO^2=2(AM^2+CM^2)-AC^2$ and $4MO^2=2(BM^2+DM^2)-BD^2$. As $ABCD$ is a rectangle, $AC=BD$ and so we get the desired relation. $\blacksquare$ Note that the condition of $p$ being circumscribed is equivalent with $AA'+CC'=BB'+DD'$. Denote by $r$ the radius of the circle. Then we have $$\sqrt{AM^2-r^2}+\sqrt{CM^2-r^2}=\sqrt{BM^2-r^2}+\sqrt{DM^2-r^2};$$$$AM^2+CM^2-2r^2+2\sqrt{(AM^2-r^2)(CM^2-r^2)}=$$$$BM^2+DM^2-2r^2+2\sqrt{(BM^2-r^2)(DM^2-r^2)};$$ By our lemma, our relation is equivalent with: $$(AM^2-r^2)(CM^2-r^2)=(BM^2-r^2)(DM^2-r^2);$$$$AM\cdot CM=BM\cdot DM.$$Look at the lemma again. We get that $AM+CM=BM+DM$ and $CM-AM=DM-BM$, thus $CM=DM$ and $AM=BM$. But this means that $AA'=BB'$ and $CC'=DD'$, so indeed $p$ has an axis of symmetry.