The answer is $n=251,1004,4016$. We will show, by strong induction on $n \geq 1$, that $x_n=\frac{4016}{n(n+1)}$. The result is true for $n=1$, so assume it is true for $1,2, \dots ,n-1$. Then $$\sum_{i=1}^{n-1} x_i=\sum_{i=1}^{n-1} \frac{4016}{i(i+1)}=4016 \sum_{i=1}^{n-1} \left(\frac{1}{i}-\frac{1}{i+1} \right)=4016\left(1-\frac{1}{n} \right)$$$$\Rightarrow x_n=\frac{\sum_{i=1}^{n-1} x_i}{n^2-1}=4016 \cdot \frac{1-\frac{1}{n}}{n^2-1}=\frac{4016}{n(n+1)}$$This gives the desired result. Note that $S_n=n^2x_n$, and so we have $$a_n=x_n+\frac{1}{n^2}S_n=(n+1)x_n=\frac{4016}{n}$$But, $4016=16 \times 251$, where $251$ is a prime. This gives the answer stated initially.