Clearly prime $n$ doesn't work. We will show that all composite $n$ work. Let $S_i$ denote the number on the board after $i$ exclamations. ($S_0=n!$). Let $p_i$ denote the largest prime such that $v_{p_i}(S_i)$ is odd. Then the following algorithm will make the number on the board a perfect square. 1) If $p_i$ doesn't exist, then algorithm terminates. 2) Otherwise apply an exclamation mark after $p_i+1$. For a proof see that $p_i$ decreases after every application of 2nd move. So, eventually $S_i$ becomes a perfect square.

the same problem was the third one in 05.04.2019 in National Olympiad of Portugal as problem 3, see here