Start out by simplifying the left-hand side, and you get $x^4-6x^3+10x^2-4x-4=0$ After that, I'd recommend using the rational root theorem, and if there are any rational solutions, you can divide out the factor. I haven't done this for this problem yet, but if there are two rational solutions you can reduce down to a quadratic and find any other solutions with the quadratic formula.

RRT gives no rational roots, you'd probably have to factor it into two quadratics. Any solutions without undetermined coefficients?

yofro (via PM) wrote: What happens if x is a root of $x^2-4x-2$?

@jasperE3 AoPS User ok

jasperE3 wrote: yofro (via PM) wrote: What happens if x is a root of $x^2-4x-2$? That's pretty cool

The $x^2-4x-2$ is kind of intuitive because clearly setting $x^2-3x-2=x$ gives a solution to the above equation. We still need the other quadratic factor for 4 roots, but I'm not seeing something that is intuitive. Of course, it's pretty easy now to just factor $x^2-4x-2$ from the equation in #2 and use MUD, but I was wondering if there was a more intuitive approach on that other factor. @below you really should be thanking yofro for these amazing hints. Do you have a reason to believe the other factor does not have real roots?

@natmath AoPS User whoa that's awesome solutions to $x^2-4x-2$ are $$x=\frac{4\pm\sqrt{16+8}}{2}=\frac{4\pm2\sqrt6}{2}=2\pm\sqrt6.$$Those are real! 2 solutions to the equation already! If I'm not mistaken, these are the only real solutions and this problem is solved....?

uh I think you messed up? You're confusing $x^2-4x+2$ and $x^2-4x-2$? Is there a typo in the problem? If there's a typo in the problem and it is $x^2-3x-2$ instead of $x^2-3x+2$ we can use poly division and get our other answers to be $1\pm \sqrt 5$ and of course using the other roots, we can let $x$ be a root of $x^2-2x-4$, then $x^2-3x-2=-x+2$, and $(-x+2)^2-3(-x+2)-2-x=x^2-4x+4+3x-6-2-x=x^2-2x-4=0$

@above where are you getting $x^2-2x-4$ from?

Very sorry. Yes, the problem was incorrect. It has been edited now.

Yofro gave me a hint on $x^2-2x-4$. It's not as direct as the first one, but I can try to explain it. If $f(x)=x^2-3x-2$, then we want to find the solutions to $$f(f(x))=x$$ Of course, if there existed an $r$ s.t. $f(r)=r$, then this would clearly be a solution to our equation. That $r$ must be either of the roots to $$x^2-3x-2=x$$$$x^2-4x-2=0$$ Let's say for some constant $k$ there exists an $r$ s.t. $f(r)=k-r$. Now let's say $k-r$ also satisfies the equation $f(x)=k-x$ (i.e. $f(k-r)=k-(k-r)=r$). Then this $r$ would satisfy the original equation. However, finding this root is not as immediate. We want $r$ and $k-r$ to be the solutions of $$f(x)=k-x$$$$x^2-3x-2=k-x$$$$x^2-2x-(2+k)=0$$By viete's, the sum of roots is $$r+k-r=2$$$$k=2$$So the solutions to $$x^2-2x-4=0$$also satisfy our original equation. Also I think we know that there are no other solutions aside from those in the form $f(x)=x$ and $f(x)=k-x$ because $x,k-x$ are the only polynomial functions that satisfy $f(f(x))=x$.

Take one x in other side and set y=$x^2-3x$ and solve quadratic take one of its root equate it to x then you get a good answer