Let $ x=y=0$ then $ f^2(x)-2f(x)+1=0$ or $ f(0)=1$. Let $ y=1$ then $ f(x+1)=f(x)f(1)-f(x)+1=f(x)(f(1)-1)+1$. Now let $ x=-1$ then $ f(-1)(f(1)-1)=0$ so $ f(-1)=0$ or $ f(1)=1$. But if $ f(1)=1$ then let $ y=1$ we have $ f(x+1)=f(x)f(1)-f(x)+1=1$ or $ f(x)=1,\forall x\in\mathbb{Q}$. Now consider $ f(-1)=0$. Put $ x=y=-1$ and $ x=1;y=-2$ then we have $ f(-2)+f(1)=1$ and $ f(1)f(-2)-f(-2)+1=0$, which lead to $ f(-2)+f(1)=1$ and $ f(1)f(-2)+f(1)=0$. There are 2 cases here: 1. $ f(1)=2,f(-2)=1$ then $ f(x+1)=f(x)+1$ and by induction we got $ f(x+n)=f(x)+n,\forall x\in\mathbb{Q},n\in\mathbb{N}$ so $ f(n)=n+1,\forall n\in\mathbb{N}$. Now let $ x=n,y=\frac{1}{n}$ we have $ f(n+\frac{1}{n})=f(n)f(\frac{1}{n})-f(1)+1$ and by what said above we conclude $ f(\frac{1}{n})+n=(n+1)f(\frac{1}{n})-1$ or $ f(\frac{1}{n})=1+\frac{1}{n}$. Next, let $ x=m,y=\frac{1}{n}$ we have $ f(m+\frac{1}{n})=f(m)f(\frac{1}{n})-f(\frac{m}{n})+1$ or $ m+f(\frac{1}{n})=(m+1)(\frac{1}{n}+1)-f(\frac{m}{n})+1$ or $ f(\frac{m}{n})=\frac{m}{n}+1$, which leads to the function $ f(x)=x+1,\forall x\in\mathbb{Q}$. 2. $ f(1)=0,f(-2)=1$, then we have $ f(x+1)=-f(x)+1,\forall x\in\mathbb{Q}$, replace $ x$ with $ x+1$ we obtain $ f(x+2)=-f(x+1)+1=f(x)-1+1=f(x),\forall x\in\mathbb{Q}$. Since, we just need to express $ f(x)$ in $ [0;2]$. At here, I did not find anyway to come through. Could anybody please help me???

http://www.artofproblemsolving.com/Forum/viewtopic.php?t=149509

Dear Mashimaru, " 2. , then we have f(x+1)=-f(x)+1,\forall x\in\mathbb{Q}, replace with we obtain f(x+2)=-f(x+1)+1=f(x)-1+1=f(x),\forall x\in\mathbb{Q}. Since, we just need to express in . At here, I did not find anyway to come through. Could anybody please help me??? " You almost had the solution. From $ f(x+2) = f(x)$ and $ f(1) = 0, f(-2) = 1$, we have $ f(n) = 0$ for $ n$ odd and $ f(n) = 1$ for $ n$ even. Substituting $ x = 2k, y = \frac{1}{2k}$ in the main equation, we get $ f(2k + \frac{1}{2k}) = f(2k)f(\frac{1}{2k}) -f(1) + 1 \Rightarrow f(\frac{1}{2k}) = f(\frac{1}{2k}) -f(1) + 1$ because $ f(2k) = 1, f(2k + x) = f(x)$ This gives us $ f(1) = 1$ which is a contradiction. Hence, case 2 is not possible.

Taking $x = y = 0$, we first obtain the initial condition $f(0) = f(0)^{2} - f(0) + 1 \Rightarrow 0 = f(0)^{2} - 2f(0) + 1 \Rightarrow 0 = [f(0) - 1]^2$, or $f(0) = 1$. Let $f(x) = Ax + 1$ and substitute this expression into the functional equation to produce: $A(x+y) + 1 = (Ax + 1)(Ay + 1) - Axy - 1 + 1$; or $Ax + Ay + 1 = A^{2}xy + Ax + Ay + 1 - Axy$; or $0 = (A^{2} - A)xy$ which yields $A = 0, 1$. Therefore, the solution includes the functions $\fbox{f(x) = 1}$ and $\fbox{f(x) = x + 1}$ for all $x \in \mathbb{Q}$.