The only constant polynomials which work are $P(x) = 1$ and $P(x) = -1$ ($P(1) \mid 3$ and $P(2) \mid 4$, clearly the only integers dividing both are $1$ and $-1$). Now assume $P(x)$ is nonconstant. Then $|P(x)| > 2$ for some $x$, let $m = P(n)$ be one such number. Since $P(a) \equiv P(b) \pmod{n}$ whenever $a \equiv b \pmod{n}$, we can take a sufficiently large positive integer $x$ such that $m \mid P(x)$ and $m < x$. Finally $P(x) \mid x! + 2$ gives a contradiction, and thus there are no nonconstant solutions.

Pretty sad. Suppose $P(x)$ is nonconstant, so thus we can find that $P(n)$ is greater than $2020$ for some $n>0$. Then, if the largest prime divisor of $P(n)$ is $p$, then note that $n+p>p$ so $p\mid (n+p)!$, but $p\mid P(n+p)$ as well. This implies that $p\mid 2$, so $P(n)$ is a power of $2$ for all values. However, if $n>4$, then $n!+2\equiv 2\pmod 4$, implying that $P(n)\in\{\pm 2\}$ for large $n$, absurd unless it is constant. Now note $P(1)\mid 3$ and $P(2)\mid 4$, so as it is constant, it is $\pm 1$.

I claim the answer is $P(x)=\pm1$, which clearly works. Assume that $P(x)$ is nonconstant. Hence, there exists an integer $k>3$, for which $|P(k)|>2$. Let $p$ be any prime dividing $P(k)$. Then it follows that $$(k+p)-k\mid P(k+p)-P(k)\iff p\mid P(k+p)-P(k)\iff p\mid P(p+k).$$This means that $p\mid(p+k)!+2$, so $p=2$ and $P(k)=2^m$ for some natural $m$. Thus, $2^m\mid k!+2$, however $k!+2\equiv2\pmod{4}$, so $m=1$. Hence, $P(k)=2$ for infinitely many $k$, so $P\equiv2$. This is a contradiction because $P(x)$ is nonconstant. Now if $P(x)$ is constant, say $P(x)\equiv c$, we get that $c\mid 3$ and $c\mid 4$, so $c=\pm1$ and we are done.

Assume $P$ is nonconstant, then by Schur's there exists infinitely many primes $p>2$ that divides $P(n)$ for some $n$. Now see that \[p \mid P(n+p) \mid (n+p)!+2 \implies p \mid 2\]Which is a contradiction. When $P$ is constant, easy to check that the only solutions are $\boxed{P(X) \equiv \pm 1}$.

(Replace $f$ with $P$) Assume that $f$ is a non-constant polynomial: then, by Schur's theorem, there exists infinitely many primes $p$ such that: $p|f(n)$ for some $n \in \mathbb N$. Then: $p|P(n+p)|(n+p)!+2 \implies p|2$ which is a contradiction. Therefore: $f$ is a constant polynomial and checking for $n=1, 2$: we get $\boxed{f \equiv \pm 1}$.