We first claim that $F_k$ is reducible over each $\mathbb{Z}_p$. Indeed, let $K$ be any such field. If $-1$ is a square in $K$, then $F_k = (X^2 - (k+1))^2 + 4X^2$ is clearly reducible over $K$. If $k$ is a square in $K$, then $F_k = (X^2 + (k+1))^2 - 4kX^2$ is clearly reducible over $K$. So now assume that $-1$ and $k$ are not squares in $K$. Then $-k$ is a square in $K$, say $-k = a^2$ for $a\in K$. Then $F_k = (X^2 + (1-k+2a))(X^2 + (1-k-2a))$ is reducible. This proves the claim. So the part "such that $F_k$ is reducible over all $\mathbb{Z}_p$" is actually totally redundant... The question is merely "find all $k$ such that $F_k$ is irreducible over $\mathbb{Z}$". As noted above, if $k$ or $-k$ is a square in $\mathbb{Z}$, $F_k$ is reducible, so the answer is negative. Now assume that $k$ is not a perfect square or minus a perfect square in $\mathbb{Z}$. We claim that $F_k$ is irreducible over $\mathbb{Z}$ in that case. To prove this point, first note that $F_k = (X^2 - (k+1))^2 + 4X^2$ can not have a real root, hence certainly not a linear factor over $\mathbb{Z}$. So if $F_k$ is reducible, it must be the product of two quadratic factors, say $F_k = (X^2+aX+b)(X^2-aX+c)$ with $a,b,c\in\mathbb{Z}$. Then $0 = a(b-c)$, $bc = (k+1)^2$ and $b+c-a^2 = 2(1-k)$. The first equation shows that there are two cases. If $a = 0$ then $b+c = 2(1-k)$ and $bc = (k+1)^2$, so the equation $T^2 + 2(k-1)T + (k+1)^2$ would have a solution in $\mathbb{Z}$. However, its discriminant $4(k-1)^2-4(k+1)^2 = -16k$ is not a square because $-k$ is (supposed to) not (be) a perfect square. The second case is the one in which $b=c$; then $b = c = \varepsilon (k+1)$ for some $\varepsilon\in\{-1,1\}$ and $a^2 = 2(b+k-1) \in \{-4,4k\}$, but neither $-4$ or $4k$ is a perfect square, so this is impossible as well. We conclude that $F_k$ is irreducible over $\mathbb{Z}$.