For $p \neq 2$, we see that $3^p - (p+2)^2 \equiv 1 - 1 \equiv 0 \pmod 2$, so we must have $3^p - (p+2)^2 = 2$ if $p \neq 2$. We see $p=3$ works, and $3^p - (p+2)^2$ is monotonically increasing from $p \geq 3$. This means $p=3$ is the only solution for $p \neq 2$. If $p=2$, $3^p - (p+2)^2 = -7$, so the only possible $p$ is $\boxed{3}$.