First we may assume $a_0$ is non zero. We've $a_ {n-1}.a_{n-2}......a_1=1,-1$ So we can take $P(x)=(x-1)^a(x+1)^b(x-a_0)$ First comparing coeff of $x^{n-1}$ it's not difficult to see $a_{n-1}=1$ So $a_0=b-a-1$ Now comparing of $x$ we get $b(b-a)+a=1,0$ Now it's easy to see $a=1,b=1$ Now suppose $a_0=a_1=...a_{k-1}=0$ and $a_k$ is non zero then we get $P(x)=x^k(x+1)^2(x-1)$.

I haven't checked your proof, but clearly $(x+1)(x-1)^2 = x^3 - x^2 - x + 1$ is wrong, since the coefficients are $-1,-1,1$ while the roots are $-1,1,1$. Most likely you have made a mistake, and should have reached $(x+1)^2(x-1) = x^3 + x^2 - x - 1$, where the coefficients are $1,-1,-1$ and the roots are $-1,-1,1$.

Edited now

And of course you missed the possibility $P(x) = x^n$; all coefficients may be $0$, so there might be no $k$ such that $a_k \neq 0$.

mavropnevma wrote: And of course you missed the possibility $P(x) = x^n$; all coefficients may be $0$, so there might be no $k$ such that $a_k \neq 0$. Yes of course , thanks

The answers are as follows: 1) $P(x) = x^k$ 2) $P(x) = x^k(x+1)^2(x-1)$ 3) $P(x) = x^k(x+2)(x-1)$. Hint: Use sum of squares of roots.

There are $3$ families of solutions, the first one is $P(x)=x^n$, the second one is $P(x)=x^n(x+1)^2(x-1)$ and the final one is $P(x)=x^n (x+2)(x-1)$, where $n$ is an arbitrary nonnegative integer. Obviously we must have that $P(x)=x^n$ is a solution. Now notice that if $a_0 \neq 0$, by Vieta's relations we must have that $a_1a_2\dots a_{n-1} = (-1)^n$, meaning that $a_1,a_2 \dots a_n \in \{ -1,1 \}$. Meaning that we have that we can scale up by $x^k$, where $k$ is an arbitrary nonnegative integer. First case is when $n=2$. Then we must have that $x^2+a_1x+a_0 = x^2-(a_1+a_0)x+a_1a_0$. Since we have that $a_0 \neq 0$, we must have that $a_1=1$, and we easily have that $a_0=-2$, meaning that $P(x)=(x+1)(x-2)$, and of course we can scale up by $x^k$. The second case is when $n=2t$, where $ t > 1$. By Vieta's relations we must have that: $$a_0+a_1+\dots +a_{n-1} = -a_{n-1} \in \{ -1,1 \}$$But notice that we muse have an even number of summands on the $LHS$, and since all of them are in the set $\{1,-1\}$, we have that this is impossible. The third and final case is when $n=2t+1$, where $t \geq 0$. Now notice that: $$\left( \sum a_t \right)^2 - 2 \left( \sum a_t \sum_{k \geq t} a_k \right) = \sum a_t^2= n$$which is equivalent to: $$1-2a_{n-2} = n > 0$$this means that $a_{n-2} = -1$ since $a_{n-2} \neq 0$. Thus we have that $n=3$. We easily get that $P(x)=(x-1)(x+1)^2$, which can be scaled up by $x^k$, where $k$ is an arbitrary nonnegative integer.

We claim that the only such polynomials are $x^n, x^{n-2} (x^2+x-2)$, and $x^{n-3} (x^3+x^2-x-1)$, each of which clearly work. Note that we can always multiply and divide the polynomial by $x^k$. Thus, we will consider the case where $a_0\neq 0$. Firstly, note that by Vieta's, \[\prod_{i=0}^{n-1} (-a_i) = a_0 \Longrightarrow \prod_{i=1}^{n-1} a_i = (-1)^n\]Thus, all $a_i$ are either $-1$ or $1$. Additionally, note that $n=0,1$ clearly fail since $a_0\neq 0$. Claim: $a_0\in \{-2,-1,0,1\}$ Proof: For positive $a_0$, then we have \[\sum_{i=0}^{n} a_i x^i \geq a_0^n - a_0^{n-1} -\cdots -a_0 + a_0\]which is $>0$ for all $a_0\geq 2$. For negative $a_i$, a similar argument yields that $a_0\geq -2$. $\square$. If $a_0=-2$, then the equality case must be held, which is $x^{2k} +x^{2k-1}-x^{2k-2} + x^{2k-2}- x^{2k-3}\cdots -x^2+x-2$, but this fails by plugging in $x=-1$ for $k\geq 2$. Thus, $a_0=2\Longrightarrow f(x) = x^2+x-2$. Otherwise, if $a_0\in \{-1,1\}$, then it must be divisible by $(x+1)$ because $x^n+x^{n-1}+\cdots + 1\neq (x-1)^n$ clearly fails. It also must be divisible by $(x-1)$ because $x^n-x^{n-1}-\cdots -x-1 \neq (x+1)^n$ also fails. Now, note that $\sum_{i=1}^{n-1} = -a_{n-1} \in \{-1,1\}$. Thus, there are $M$ 1s and $M+1$ -1s, or $M+1$ 1s and $M$ -1s. Regardless, \[a_{n-2} = \binom{M}{2} + \binom{M+1}{2} - M(M+1) = -M \in\{-1,1\}\]Thus, the only two remaining choices are $(x+1)^2(x-1)=x^3+x^2-x-1$ and $(x-1)^2 (x+1) = x^3 - x^2-x+1$. The second one fails, so we have shown that the three classes we outlined at the beginning are the only choices. $\blacksquare$.