Many on my team solve it. It is not hard, just 1. prove f(x) <= 0 for all x. 2. Prove f(-big) = very small 3. Prove that f(f(x)) >= f(x). 4. Now you can prove f(a) = 0 if a < 0. 5. You show f(0) = 0, is not hard.

pythag011 wrote: wangsacl wrote: a lot, arshakus. even more than problem 2 solver, i believe. I know one South African, several British. Many on my team solve it. It is not hard, just 1. prove f(x) <= 0 for all x. 2. Prove f(-big) = very small 3. Prove that f(f(x)) >= f(x). 4. Now you can prove f(a) = 0 if a < 0. 5. You show f(0) = 0, is not hard. during the competition this problem looks like an easy one))) and i i think i did it))) but no))) i get $f(x)\le 0$ but then noticed that i get it in wrong way. can you plaese write only that part?

arshakus wrote: pythag011 wrote: wangsacl wrote: a lot, arshakus. even more than problem 2 solver, i believe. I know one South African, several British. Many on my team solve it. It is not hard, just 1. prove f(x) <= 0 for all x. 2. Prove f(-big) = very small 3. Prove that f(f(x)) >= f(x). 4. Now you can prove f(a) = 0 if a < 0. 5. You show f(0) = 0, is not hard. during the competition this problem looks like an easy one))) and i i think i did it))) but no))) i get $f(x)\le 0$ but then noticed that i get it in wrong way. can you plaese write only that part? First of all: clearly if f is not always 0 there is some a such that f(a) < 0. Then since we have f(x+a) <= xf(a) + f(f(a)), if we make x = -1000000f(a), f(x+a) will be negative. so there is some m with f(m) negative. Assume there is also some n with f(n) negative. We have f(x+m) <= xf(m) + f(f(m)). This implies that f(Very big number) = a negative big number. f(x+n) <= xf(n) + f(f(n)). This implies f(negative very big number) = a negative big number. Then, if we let y = -x and both be very big and x positive, we have f(0) <= (verybig)*(-verybig)+(-verybig), so f(0) must be negative veryveryveryhugebig, which cannot be, because f(0) is finite.

Let $P(x,y)$ be the assertion $f(x + y) \leq yf(x) + f(f(x))$. EDIT. Argh... I can't solve the $f(0)=0$ part... The below solution should be correct, but someone please help me to show that $f(0)=0$. (1) $P(0,x) \implies f(0+x)\leq xf(0)+f(f(0)) \implies f(x) \leq 0 \quad \forall x \in \mathbb R$. (2) $P(x,-x) \implies f(x-x) \leq -x f(x) + f(f(x)) {\implies} xf(x) \leq f(f(x)) \quad \forall x \in \mathbb R$. (3) $P(x,0) \implies f(x+0) \leq xf(0) + f(f(x)) \implies f(x) \leq f(f(x)) \quad \forall x \in \mathbb R$. (4) $P(x, f(x)-x) \implies f(x+f(x)-x) \leq (f(x)-x)f(x)+f(f(x)) \implies$ $xf(x) \leq (f(x))^2 \quad \forall x \in \mathbb R$. Choose $x \leq 0$ and use (1), (2) $\implies f(f(x)) \geq 0$. But we had $f(x) \leq 0 \quad \forall x \in \mathbb R$, so $f(f(x))=0 \quad \forall x \leq 0$. Use (2) to get $xf(x) \leq f(f(x))=0 \quad \forall x\leq 0$ (5). (4) $\implies f(x)=0$ or $x \geq f(x)$. If $f(x)=0 \quad \forall x \leq 0$, we're done. So suppose that $x \geq f(x) \quad \forall x \in \mathbb R$. But this is completely in contradiction with (5), because $x \geq f(x) \implies xf(x) \geq f(x)^2 \geq 0.$ The proof is complete.

@amparvardi P(0,f(0)) does not imply f(0)= 0

Too easy for a #3, IMO =)

inputing $y=0$ yields $f(x) \leq f(f(x))$ since $f(x)$ has a real value,it can be said $x \leq f(x)$ so setting up $f(x)=0$ satisfies this condition for all $x \leq 0$

Matheater1 wrote: inputing $y=0$ yields $f(x) \leq f(f(x))$ since $f(x)$ has a real value,it can be said $x \leq f(x)$ so setting up $f(x)=0$ satisfies this condition for all $x \leq 0$ something completely wrong

First of all, for $y = f(x) - x$ we get: $f(x)(f(x) - x) \geq 0 \forall x \in R$ (*). Now for $y = f(f(x)) - x$ and every $z \in R$ we have: $f(f(x)) + yf(x) \geq f(f(f(x))) \geq -zf(f(x)) + f(f(x) + z)$ and for $z = -f(x) -1$ we get: $f(f(x))(f(x) + 1) - xf(x) \geq f(f(x))(f(x) + 1) + f(-1) \Rightarrow$ $\Rightarrow -xf(x) \geq f(-1) \forall x \in \mathbb{R}$ Hence for $x$ negative: $f(x) \geq \frac{a}{x}, a = -f(-1)$ Now for any $x$, for very small $y < -x$ we have $x+y < 0$ and hence: $f(f(x)) + yf(x) \geq f(x+y) \geq \frac{a}{x+y} \rightarrow 0$ for $y \rightarrow -\infty$ and if we had $f(x) > 0$ then we would have: $f(f(x)) + yf(x) \rightarrow -\infty$ which is absurd. Hence $f(x) \leq 0 \forall x \in R$ Now if for $x < -\sqrt{|f(-1)|} \leq 0$ we had $f(x) < 0$ then $f(x) \leq x$ would hold, because otherwise $f(x)(f(x) - x) < 0$ which cannot be true (contradicts (*)). Then we would have $-x^2 \geq -xf(x) \geq f(-1) \Rightarrow |x| \leq \sqrt{|f(-1)|}$ which is a contradiction. Hence for $x < -\sqrt{|f(-1)|} \leq 0$ we get $f(x) = 0$. Consequently there exists a $b \in R$ such that $f(b)=0$ and $f(b+y) = 0 \forall y<0$, and then the given inequality for $x=b$ and $y<0$ gives $f(0) \geq 0$, and because $f(0) \leq 0$ we have $f(0)=0$. Now, for $y=-x>0$ the given inequality implies: $-xf(x) \geq -xf(x) + f(f(x)) \geq f(0) = 0 \Rightarrow f(x) \geq 0$ and because $f(x) \leq 0$ we deduce that $f(x) = 0$ for all $x<0$ and we are done. Edit 17/11/2013: improved the text

Plugg $y=f(x)-x$ into the functional inequality, $f(f(x))\leq (f(x)-x)f(x)+f(f(x))\Longleftrightarrow 0\leq f(x)(f(x)-x)$, For all $x\leq 0$, we have $f(x)\leq x$ or $0\leq f(x)$

Here is the solution I wrote during the contest: Let $P(x,y)$ be the assertion: $f(x+y)\leq yf(x)+f(f(x))$ $P(x,0)\implies f(x)\leq f(f(x)) ...(1)$ $P(0,y)\implies f(y)\leq yf(0+f(f(0)) ...(2)$ $P(x,f(x)-x)\implies f(f(x))\leq f(x)^{2}-xf(x)+f(f(x)\implies f(x)^{2}\geq xf(x)$ $P(x,f(0)-x)\implies f(f(0))\leq f(x)f(0)-xf(x)+f(f(x))\leq f(x)f(0)-xf(x)+$ $~$ $f(x)f(0)+f(f(0))\implies 2f(0)f(x)\geq xf(x) ...(3)$ Assume $f(0)\neq 0$ We consider all possible cases: $i) f(x)\geq 0$ for some $x\implies$ $2f(0)\geq x$ $ii) f(x)\leq 0$ for some $x\implies$ $2f(0)\leq x$ Now, again we check both cases for $f(0)$ $i) f(0)>0$ $P(0,\frac{-1-f(f(0))}{f(0)})\implies -1-f(f(0))\geq 2f(0)^{2}\implies f(f(0))\leq -1$, contradiction because of $(1)$ $ii) f(0)<0$ If $f(f(0))\geq 0 \longrightarrow f(0)\geq 0$, contradiction. Hence $f(f(0))<0$. Now, $(2)$ can be rewritten as $f(y)\leq yf(0)$ And $(1)$ can be rewritten as $f(x)\leq f(x)f(0)$ which implies $f(x)\leq 0 \forall x\in\mathbb{R}$. But that means that $2f(0)\leq x\forall x\in\mathbb{R}$, contradiction. Hence, $f(0)=0$. $(2)$ becomes $f(x)\leq 0\forall x$ and $(3)$ becomes $xf(x)\leq 0\forall x$. $(3)\implies f(x)=0 \forall x\leq 0$ P.S. Sorry to write so shortly and make you do many things by yourself. Reply if you don't have something clear

Most solutions probably also prove that $f(x) \leq 0$ for all $x \in \mathbb{R}$, and, if in at least one point $a > 0$ having $f(a) < 0$, that $\lim_{x\to \infty} f(x) = -\infty$. Let us present such an example, so that we are satisfied that not just the identically null function satisfies. EDIT. It transpires this requirement was part of the original formulation of the problem. Since the conclusion is $f(x) = 0$ for all $x\leq 0$ it means $f\circ f \equiv 0$, so the initial relation simply writes $f(x+y) \leq yf(x)$. Define $f(x) = -\textrm{e}^x$ for $x>0$. Then we should only check that $\bullet$ For $x>0$, $x+y \leq 0$, that $0 \leq -y\textrm{e}^x$; true since we need $y<0$. $\bullet$ For $x\leq 0$, $x+y > 0$, that $-\textrm{e}^{x+y} \leq 0$; trivially true. $\bullet$ For $x>0$, $x+y > 0$, that $-\textrm{e}^{x+y} \leq -y\textrm{e}^x$, i.e. $\textrm{e}^y \geq y$; true for all $y \in \mathbb{R}$. EDIT. Notice that this example "explains" the use of the iterated $f(f(x))$ in the RHS, since the inequality $f(x+y) \leq yf(x) + f(x)$ is fulfilled by $f(x) = -\textrm{e}^x$ defined allover $\mathbb{R}$. On a related issue, the links provided by kunny offer techniques of similar kind to be appliable to these types of "functional inequalities".

Set $y=0$, we have $f(x)\le f(f(x))\forall x\implies f(x)\ge x\forall x$. So $f(f(0))\ge f(0)\ge0$. Set $x=0,f(y)\le yf(0)+f(f(0))$. Replace $y$ by $x$ and have $x\le f(x)\le xf(0)+f(f(0))\implies f(f(0))\ge x(f(0)-1)$. Since this inequality is true for all $x$, if $f(0)\neq0$, $x\le \frac{f(f(0))}{f(0)-1}$ or $x\ge \frac{f(f(0))}{f(0)-1}\forall x$, contradiction! So we have $f(0)=0$. Then $f(x)\le x.0+0=0$. Set $y=-x$. Then $f(0)\le f(f(x))-xf(x)\implies f(f(x)\ge xf(x)$. Choose $x\le0,f(x)\le0,xf(x)\ge0$. So $f(f(x))\ge0$. But $f(f(x))\le0$. These two imply $f(f(x))=0$. Then $0\le xf(x)\le f(f(x))=0\implies f(x)=0$ since $x\le0$.

mathmdmb wrote: Set $y=0$, we have $f(x)\le f(f(x))\forall x\implies f(x)\ge x\forall x$. That conclusion is incorrect - see my example for a function with $f(x) < 0 < x$ for $x>0$.

kunny wrote: Plugg $y=f(x)-x$ into the functional inequality, $f(f(x))\leq (f(x)-x)f(x)+f(f(x))\Longleftrightarrow 0\leq f(x)(f(x)-x)$, For all $x\leq 0$, we have $f(x)\leq x$ or $0\leq f(x)$ The worst IMO 3rd question ever. Edit: See my post in the next page for an explanation of this post.

Let $P(x,y)$ be the assertion that $f(x+y) \leq yf(x) + f(f(x))$. $P(0, f(x))$ gives $f(f(x)) \leq f(x) f(0) + f(f(0))$, whence $$\begin{eqnarray} f(x+y) &\leq& f(x)(y + f(0)) + f(f(0)). \end{eqnarray}$$ Suppose for the sake of contradiction that $f(c) > 0$ for some $c$. Then by taking $y$ to be very small (i.e., negative and having large absolute value) and setting $x=c$ in (1), we get that $\lim_{x \to -\infty} f(x) = -\infty$. Thus, for sufficiently small $x$, we have $f(x)(c - x + f(0)) + f(f(0)) < 0$. But for such $x$, we get $$0 < f(c) = f(x+(c-x)) \leq f(x)(c - x + f(0)) + f(f(0)) < 0,$$ which is impossible, so $f(x) \leq 0$ for all $x$. Now substitute $y=f(0) - x$ into (1) to get $f(f(0)) \leq f(x)(2f(0) - x) + f(f(0))$, whence $f(x)(2f(0) - x) \geq 0$. Since $f(x) \leq 0$ for all $x$, we see that if $x < 2f(0)$, then $f(x) \geq 0$. Thus, $f(x) = 0$ whenever $x < 2f(0)$. Substituting $y = -f(0)$ and $x = 3f(0) - 1$ into (1) therefore yields $0 \leq f(f(0))$, so $f(f(0)) = 0$ as well. But $P(x,0)$ gives $f(x) \leq f(f(x))$, so $f(0) \leq f(f(0)) \leq f(f(f(0))) = f(0)$, so $f(0) = f(f(0)) = 0$. Thus, $f(x) = 0$ whenever $x < 2f(0) = 0$, and $f(0) = 0$, so $f(x) = 0$ whenever $x \leq 0$.

kunny wrote: Plugg $y=f(x)-x$ into the functional inequality, $f(f(x))\leq (f(x)-x)f(x)+f(f(x))\Longleftrightarrow 0\leq f(x)(f(x)-x)$, For all $x\leq 0$, we have $f(x)\leq x$ or $0\leq f(x)$ Sorry, I have been confusing you, guys. This was just my idea. I wonder, the problem is proposed from Japan or Igor Voronovich, Belarus? kunny

Here are similar problems: <1A> $$f\left(x-f(y)\right)>yf(x)+x$$ <1B> $$f(x-f(y)) > yf(x)+x$$ <2A> $$f(x+y) \ge f(x) + yf(f(x))$$ <2B> $$f(x+y) \geq f(x)+yf(f(x))$$ <3> $$f(x+y)<yf(f(x))$$

$f(f(x))\ge f(x)$, set $f(x)=a,f(a)\ge a$, since the function defined for $\mathbb R\to \mathbb R,a$ can assume all real values, so we may replace $a\to x\implies f(x)\ge x\forall x$.

Claim 1: $f(x) \le 0$ for all $x \in \mathbb{R}$ Proof: Suppose $f(z) > 0$ for some $z$. Putting $x = z$ and $y \rightarrow -y$, we have $f(z-y) \le f(f(z)) - yf(z)$. This means that there exist constants $k,c$ such that $f(x) \le k + cx$ for all sufficiently negative $x$. But now taking $x$ to be extremely negative and $y$ to be positive, we have that $f(x+y) \le yf(x) + f(f(x)) \le cxy + ky + c^2x + k(c+1)$. Replace $x$ with $-x$ here to make it clearer, we have $$f(y-x) \le -cxy - c^2x + ky + k(c+1)$$ and taking $y-x = z$ and both $x,y$ big, we have the LHS is positive but the RHS is very negative, which is impossible. So $f(x) \le 0$ for all $x$, as claimed. $\square$ Claim 2: If $f$ has a zero, we are done. Proof: Suppose $f(k) = 0$ for some $k$. Note that letting $y = k-x$, we have $0 \le (k-x)f(x) + f(f(x))$ so if $x < k$, then $f(x) = 0$ too since it can't be positive. On the other hand, we also have, putting $x = k, y = 0$ we have that $0 \le f(0)$, so $f(0) = 0$ and by the previous line, this means that if $x < 0$, then $f(x) = 0$ too, as desired. $\square$ Claim 3: $f$ has a zero. Proof: Suppose not, then put $y = f(x) - x$ in the original equation to get $f(x)(f(x) - x) \ge 0$, so if $f(x)$ is not zero, since its negative, the second term must be nonpositive too. So $f(x) \le x$ for all $x$. The original equation gives that $f(x+y) \le yf(x) + f(f(x)) < yf(x)$, so taking $y = -x$, we have $f(0) < -xf(x) \le -x^2$ for all negative $x$, which is a contradiction. $\square$. Therefore, we indeed get $f(x) = 0$ for all $x \le 0$, as desired. $\blacksquare$

Let $P(x, y)$ be the assertion. Suppose $f(0) \neq 0$. For now, assume $f(0) > 0$. $$P(-M, 0) \implies f(-M) \leq -Mf(0) + f(f(0)) \leq -N$$ where $M, N$ can get sufficiently large. This shows that $f(-BIG) \to -BIG$. Taking $f(-M, N)$ where $M, N$ are very large and $N >> M$ gives $f(N-M) \leq Nf(-M) + f(f(-M))$. Here, $N-M$ is very large, and $Nf(-M) + f(f(-M))$ is very negative. This shows that $f(BIG) = -BIG$. Finally, taking $P(-L, L+c)$ where $c$ is some constant, and $L$ is some very large number gives us $f(c) \leq (L+c)f(-L) + f(f(-L))$. The RHS is a sufficiently large negative number by increasing $L$, whereas the LHS is a constant. This is a contradiction, hence, $f(0)$ cannot be positive. A similar contradiction arises when $f(0) < 0$, thus $f(0) = 0$. Here we observe $$P(0, y) \implies f(y) \leq 0 \qquad (1)$$ for any $y$. To finish, take $P(x, -x)$ for negative $x$ to get $0 \leq -xf(x) + f(f(x)) \implies xf(x) \leq f(f(x))$. $f(f(x))$ is nonpositive by $(1)$ and $xf(x)$ is nonnegative since $x < 0$ and $f(x) \leq 0$. Thus, for the inequality to hold, $f(x) =0$ for any negative $x$, and this completes our proof.

I found this simple for a IMO-P3 (as I have solve it ), Here's goes my solution--- Let $P(x,y)$ denotes the given assertion. Now from $P(x,-x) : f(f(x)) \ge f(0) +xf(x)$ , $P(0, f(x)): f(f(x)) \le f(x)f(0) +f(f(0))$ implying $f(0) +xf(x) \le f(x)f(0) +f(f(0))$ , so we get $$\boxed{ f(x) \le \frac{f(f(0)) - f(0)}{x-f(f(0))}}$$ , changing $x \mapsto f(x)$ in the above expression , we get $f(f((x)) \le \frac{f(f(0)) - f(0)}{f(x)-f(f(0))}$ , but as $f(x) \le f(f(x))$ , we get $f(x) \le \frac{f(f(0)) - f(0)}{f(x)-f(f(0))}$ , solving we get $f(x) \in [M , N]$ , for some fixed constants $M, N$ , but from $P(x,-x)$ where $x > \frac{N-f(0)}{M}$ , we get $N \ge f(f(x)) \ge f(0) + f(f(x)) \ge f(0) +xM > N$ , a contradiction unless $M = 0$. First assume $f(0) \neq 0$. Now as we know $M$ explicitly in terms of $f(0)$ and $f(f(0))$ , from the quadratic $f(x) \le \frac{f(f(0)) - f(0)}{f(x)-f(f(0))}$ , so equating it to $0$ and doing some computations we get , then $\boxed{f(f(0)) = f(0)}$ giving $f(x) \le 0$ for all real $x$. So from above we get that there exist a $x$ such that $f(x) = x$ , as $x \neq 0$ from assumption and $f(x) \le 0$ for all real $x$ , so $x <0$ , but from $P(y , x-y)$ where $y > x+1$ , we get $x \le (x-y)f(y) + f(f(y)) \le f(f(y))(x-y+1)$ so $0 \ge f(f(y)) \ge \frac{x}{x-y+1} >0$ a contradiction. So $\boxed {f(0) = 0}$ , so again we get $f(x) \le 0$ for all real numbers $x$ , and from $P(x,-x): 0 \ge f(f(x)) > xf(x)$ , where $x <0$ , so $f(x) \ge 0$ giving $f(x) = 0$ for all $x \le 0$. Hence we are done. $\blacksquare$

Let $P(x,y)$ be the given assertion. Comparing $P(x,f(y)-x)$ and $P(y,f(x)-y)$ yields, $$xf(x)+yf(y)\leq 2f(x)f(y).$$ $y\mapsto 2f(x)\implies xf(x)\leq 0. \qquad (*)$ Claim: $f(k)\leq 0~~\forall k.$ Proof. Suppose $\exists k:f(k)>0,$ then $$f(k+y)\leq yf(k)+f(f(k)).$$ Now $y\to -\infty$ implies that $\lim_{x\to -\infty} f(x)=-\infty.$ $P(x,z-x)\implies f(z)\leq (z-x)f(x)+f(f(x)).$ Then $x\to -\infty,$ yields a contradiction. $\blacksquare$ From $(*)$ we get $f(x)=0,\forall x<0.$ $P(0,f(0))\implies f(0)\geq 0,$ thus we get $f(0)=0,$ as desired.

Let $P(x,y)$ be the assertion. $P(x,0)$ gives $f(x)\le f(f(x))$. $P(f(x),f(f(x))-f(x))$ gives $(f(f(x))-f(x))\cdot f(f(x))\ge 0$, and since $f(f(x))\ge f(x), f(f(x))\ge 0$ as well. Assume there is a nonzero output $c$ of $f$, otherwise if $f\equiv 0$ then we are done. Then in $P(x,y)$, we can fix $x$ such that $f(x)=c$ and vary $y$ to get that $f$ is unbounded below. Since $f(f(x))\ge 0$ for all $x$, this means that $f(x)\ge 0$ has arbitrarily low solutions. If there is a $z$ such that $f(z)>0$, then we can choose a sufficiently low $y$ such that $f(y+z)\ge 0$, and $P(z,y)$ does not hold. Thus $f(x)\le 0$ for all $x$, and $f(f(x))=0$ for all $x$. $f(0)=f(f(f(0)))=0$, so $P(y,-y)$ gives $-yf(y)\ge 0$. This means that for $y\le 0, f(y)\ge 0$, or $f(y)=0$, as desired. Edit: It looks like as is, this sol doesn't work because at the beginning, if $f(x)=f(f(x))$ then $f(f(x))$ could be less than $0$.

Denote the given condition by $P(x, y)$. Lemma 1: $f(f(x)) \ge f(x)$ for all $x \in \mathbb{R}$. Proof: Consider $P(x, 0)$. $\square$ Lemma 2: For all $a \in \mathbb{R}$, there exists $k \in \mathbb{R}$ such that $f(b) \le f(a)b + k$ for all $b \in \mathbb{R}$. Proof: Consider $P(a, b-a)$, or $f(b) \le (b-a)f(a) + f(f(a)) = f(a)b + f(f(a)) - af(a)$. Then setting $k = f(f(a)) - af(a)$ works. $\square$ Lemma 3: $f(x) \le 1$ for all $x \in \mathbb{R}$. Proof: Suppose otherwise and let $f(a) > 1$. Then by Lemma 2, we can construct an upper bound for $f$ that is a linear function with slope $f(a) > 1$. As a result, for sufficiently large $b$, $f(-b) < -b$. Then, there are arbitrarily large $c$ such that $-c$ is in the range of $f$, i.e. $f(d) = -c$. But this implies that $f(-c) = f(f(d)) \ge f(d) = -c$, so $f(-c) \ge -c$. But if $c$ is sufficiently large, setting $b = c$ in $f(-b) < -b$ gives a contradiction. $\square$ Lemma 4: $f(x) \le 0$ for all $x \in \mathbb{R}$. Proof: Again suppose otherwise. By Lemma 2, there exists $c > 0$ and $d$ such that $f(x) \le cx + d$ for all $x \in \mathbb{R}$. Now let $a > 0$ and consider $P(-a, a)$. We have $$f(0) \le af(-a) + f(f(-a)) \le a(-ac+d) + 1 = -ca^2 + da + 1.$$ The RHS is a quadratic with negative leading coefficient, so it can be arbitrarily negative by setting a sufficiently large $a$, giving that $f(0)$ is less than or equal to an arbitrarily negative number, contradiction. $\square$ Lemma 5: It suffices to show that there exists $x \in \mathbb{R}$ such that $f(x) = 0$. Proof: If such a $x$ exists, then $$0 \ge f(0) = f(f(x)) \ge f(x) = 0,$$ so $f(0) = 0$. Then, for any $a < 0$, if $f(a) < 0$, then $P(a, -a)$ gives $$f(0) \le -af(a) + f(f(a)) \le -af(a) < 0,$$ contradiction, so $f(a) = 0$ for all $a < 0$ as well. $\square$ From now on we will assume $f(x) < 0$ for all $x \in \mathbb{R}$. Lemma 6: For all $\epsilon > 0$, there exists $x$ such that $f(x) > -\epsilon$. Proof: Suppose otherwise, so assume there exists $\epsilon > 0$ such that for all $x$, $f(x) \le -\epsilon$. Then for all $x$, $P(x-1, 1)$ gives $$f(x) \le f(x - 1) + f(f(x-1)) \le -2\epsilon,$$ so for all $x$, $f(x) \le -2\epsilon$. This argument can be iterated to show that $f(x) \le -2^k\epsilon$ for all positive integers $k$, forcing all values of $f(x)$ to be arbitrarily negative, which is absurd. $\square$ Lemma 7: For all $\epsilon > 0$, there exists $-\epsilon < x < 0$ such that $f(x) > -\epsilon$. Proof: By lemma 6, we have $y$ such that $f(y) > -\epsilon$. Then we can set $x = f(y)$, which works since $-\epsilon < f(y) < 0$ and $f(x) = f(f(y)) \ge f(y) > -\epsilon$. $\square$ To finish, $P(0, x)$ gives that $f(x) \le xf(0) + f(f(0))$. By Lemma 7, we can choose $x$ such that both $x$ (and thus $xf(0)$) and $f(x)$ are arbitrarily close to 0. If we choose both $xf(0)$ and $f(x)$ to be much closer to 0 than $f(f(0)) < 0$ is, we get a contradiction, and we are done.

Seemed too easy for a #3. Anyway, here's a solution: Plugging in $y=f(x)-x$ we get that $f(f(x))\leq (f(x)-x)f(x)+f(f(x))\Longrightarrow 0\leq (f(x)-x)f(x)$. Therefore if $x<0$, $f(x)\leq x$ or $f(x)\geq 0$, so let $S = \{x\mid x\in\mathbb{R}^{-}, f(x)\leq x\}$. Note that if $x<0$ and $x\not\in S$, then $f(x)\geq 0$ from what we mentioned above. Claim 1. $f(x)\leq 0$ $\forall x\in\mathbb{R}$. Proof: Assume the contrary and let $f(a)>0$. Then if $C = \min\left\{-|a|, -\frac{f(f(a))}{f(a)}\right\}$ we have that for all $y<C$: $$f(a+y)\leq yf(a)+f(f(a))<0$$ Therefore $\forall y<C$ we have $a+y\in S$. This also implies that $f(a+y)\leq a+y<0$, so $f(a+y)\in S$ as well. Now, however, we can pick a large enough $N>0$ such that $-N+a<C$ and plugging $(x,y) = (-N+a, N)$ gives: $$f(a) \leq Nf(-N+a)+f(f(-N+a))\leq N(-N+a) + f(-N+a)< 0+(-N+a)<0$$ which is a contradiction with our assumption that $f(a)>0$. Claim 2. $f(x) = 0$ for all $x\leq 0$. Proof: Assume that $f(0)\neq 0$. Then, using the result from the first claim $f(x+y)\leq yf(x)+f(f(x))\leq yf(x)$. Substituting $x=-N$ and $y=N$ for $N>\sqrt{-f(0)}$ implies that $f(-N)\geq \frac{1}{N}f(0)>-N$, so $-N\not\in S$, so $f(-N)=0$ for all $N>\sqrt{-f(0)}$. Hence there exists a $z<0$ such that $f(z)=0$ and so plugging $x=z$, $y=-\sqrt{-f(0)}-1\in S$ implies that: $$0=f(z-\sqrt{-f(0)}-1)\leq f(0)$$ But we've shown that $f(0)\leq 0$ in Claim 1, so $f(0)=0$. Now simply plugging in $(x,y)=(-|x|,|x|)$ finishes as $$0=f(0)\leq |x|f(-|x|)+f(f(-|x|))\leq |x|f(-|x|)\Longrightarrow f(\mathbb{R}^{-})\geq 0\overset{\text{Claim 1}}{\Longrightarrow} f(\mathbb{R}^{-})=0$$

Let $P(x,y)$ denote the assertion then $P(x,f(2f(x))-x)$ and $P(2f(x),-f(x))$ give $$f(f(2f(x)))\le (f(2f(x))-x)f(x)+f(f(x))\le -f(x)f(2f(x))+(f(2f(x))-x)f(x)+f(f(2f(x)))$$ so $xf(x)\le 0$. Thus, for $x<0$, $f(x)\ge 0$. If for some $x$, $f(x)>0$ then for sufficiently small $y$, $f(x+y)<0$, contradiction to $f(x+y)\ge 0$. Thus, $f(x)\le 0$ for all $x$. This implies $f(x)=0$ for $x<0$. Let $x<0,y<0$ then $P(x+y)$ gives $0\le f(0)$ so $f(0)=0$. We are done.

Solved with AdventuringHobbit Plugging in $(x,y-x)$ changes the equation to the equivalent $f(y)\le yf(x)-xf(x)+f(f(x))$. Now observe that then $f(y)$ is bounded by a line in terms of $x$; if we can find $x$ such that $f(x)>0$ the $f$ is bounded from above by this line of positive slope and thus for sufficiently negative values for $x$ we get $f(x)<0$ and we get that $f$ is also bounded from above by a line of negative slope, hence bounded from above by some constant; otherwise, $f(x)\le0$ for all $x$ and we still get that $f$ is bounded from above. Hence, say $C$ is an upper bound for $f$. Now, we have $f(y)\le yf(x)-xf(x)+f(f(x))\le yf(x)-xf(x)+C$. Suppose we had $a$ such that $f(a)>0$. Then plugging in $x=a$ gives that as $y\to-\infty$, $f(y)\to-\infty$. Then letting $y=0$ and letting $x\to-\infty$, we have $f(0)\le -xf(x)+C\to-\infty$, which is absurd, so we conclude that no such $a$ can exist, hence $f(x)\le0$ for all $x$. Observe that plugging in $x=y$ gives $f(x)\le f(f(x))$. Now, if there exists some $a$ such that $f(a)=0$, then plugging in $x=y=a$ gives $f(0)\ge 0$. On the other hand, if there exists some $b\le 0$ such that $f(b)<0$, then plugging in $x=b$ gives $f(y)\le f(b)y-bf(b)+f(f(b))\le f(b)y-bf(b)$. Then plugging in $y=0$ gives $f(0)\le -bf(b)<0$ if $b<0$ or $b=0$ in which case we immediately get $f(0)<0$ anyways, so we have $f(0)<0$ either way. Hence, if there exists some $a$ such that $f(a)=0$, then we can conclude that $f(x)=0$ for all $x\le 0$ as no $b\le 0$ can exist such that $f(b)<0$. Suppose for the sake of contradiction then that $f(x)<0$ for all $x$. Then plugging in $x=0$ gives $f(y)\le yf(0)+f(f(0))$. This means that for a neighborhood around $0$, $f(y)$ has a strictly negative upper bound. Now, returning to $f(y)\le yf(x)-xf(x)+f(f(x))$, send $x\to-\infty$; then $-x\to\infty$. Consider if $y\ge0$: as $f(f(x))$ and $yf(x)$ are negative we have that $-xf(x)$ is bounded from below so $f(x)\to0$. However, from $f(x)\le f(f(x))$, we get a contradiction because for sufficiently negative $x$, we will get that $f(x)$ is within the neighborhood of $0$ we mentioned earlier on which $f$ has a strictly negative upper bound, but then we can simultaneously make $x$ sufficiently negative for $f(x)$ to be greater than this upper bound, and we get that $f(f(x))$ has $f(x)$ as a lower bound, a lower bound that is strictly greater than its upper bound, contradiction.

Nice one Let $P(x,y)$ be the assertion. $P(x,0) \implies f(x)\le f(f(x))$ $P(x,f(x)-x) \implies f(x)(f(x)-x) \ge 0 \implies f(f(x))(f(f(x))-f(x)) \ge 0 \implies f(f(x)) \ge 0$ for all $x \in \mathbb R \ \ \ (*)$ Claim: $f(0)=0$ Proof: Assume contrary: $f(0)=c\neq 0$ Case1: $c>0$ $P(0,y) \implies f(y) \le yc+f(c) \implies f(f(y)) \le f(y)c+f(c) \implies 0 \le f(y)c+f(c) \implies 0 \le (yc+f(c))c+f(c)$ for all $y \in \mathbb R \ \ \ (1)$ Setting $y=-1-\frac{f(c)(c+1)}{c^2}$ in $(1) \implies -c^2 \ge 0$ which is contradiction. Case2: $c<0$ $P(0,y) \implies f(y) < f(c)$ for $y>0 \implies$ if $f(c)>0 \implies f(f(c))<f(c) \implies$ contradiction. So $f(c) \le 0$ Since $0 \le f(y)c+f(c) \implies f(y)c \ge 0 \implies f(y) \le 0$ for all $y \in \mathbb R$ If there exists $t$ such $f(t)=0$, then $P(t,0)$ gives $c \ge 0 \implies$ contradiction $\implies f(y) < 0$ for all $y \in \mathbb R \implies$ again contradiction by $(*)$ So $f(0)=0$ Thus $P(0,x)$ gives $f(x) \le 0$. Since $f(x)^2 \ge xf(x) \implies xf(x) \le 0$ Let there exist negative $t$ such $t<0$ and $f(t) \neq 0$. Then $tf(t) <0 \implies f(t)>0 \implies$ contradiction. So for all $x<0$ $f(x)=0$ we are done)

Let $P(x,y)$ be the assertation $(x,y)$ to the given inequality. First, note that by $P(x,0),P(0,x),P(0,f(x)),P(x,-x)$, and $P(x,f(x)-x)$ gives us: $f(x)\le f(f(x))$ ...(1) $f(x)\le xf(0)+f(f(0))$ ...(2) $f(f(x))\le f(x)f(0)+f(f(0))$ ...(3) $f(0)\le -xf(x)+f(f(x))$ ...(4) $0\le f(x)(f(x)-x)$ ...(5) We'll consider some cases based on the value of $f(0)$. Case 1. $f(0)=0$ By ...(2), we obtain $f(x)\le 0$ for all $x$ real. By ...(4), we also get that $xf(x)\le f(f(x))\le 0$ Now, if we have $x<0$, it is obvious that $f(x)\ge 0$, which leads us to $f(x)=0$ Thus, we get $f(x)=0$ for all $x\le 0$. Case 2. $f(0)>1$ From ...(3), we conclude that $f(x)\ge \frac{-f(f(0))}{f(0)-1}$ for all $x\in \mathbb{R}$. Now, we choose real number $a< \frac{-f(f(0))}{f(0)-1}$. Then, $P(0,\frac{a-f(f(0))}{f(0)})$ gives $$\frac{-f(f(0))}{f(0)-1}\le f(\frac{a-f(f(0))}{f(0)})\le a \implies \frac{-f(f(0))}{f(0)-1}\le a < \frac{-f(f(0))}{f(0)-1}$$ , which gives us a contradiction. Case 3. $f(0)=1$ Note that from ...(1), we get $f(1)\ge 1$. Now, combining ...(3) and ...(4), we get that $$xf(x)+1\le f(f(x))\le f(x)+f(1) \implies f(x)(x-1)\le f(1)-1.$$ So, we must have $f(f(1)(f(1)-1)\le f(1)-1$. If $f(1)>1$, we get that $f(f(1))\le 1$. However, we must have $f(f(1))\ge f(1)\ge 1$. Hence, $f(f(1))=f(1)=1$ which is a contradiction. Therefore, we have $f(1)=1$. Now, by substituting $x=1$ on ...(4), we get $1\le 0$, which again give us a contradiction. Case 4. $0<f(0)<1$ Notice that by combining the inequality given in the problem and ...(3), we have $$f(x+y)\le yf(x)+f(f(x)) \le yf(x)+f(x)f(0)+f(f(0)) \implies f(x+y)\le f(x)(y+f(0))+f(f(0)).$$ Now, substituting $y\rightarrow f(0)-x$ to the last inequality, we get $0\le f(x)(2f(0)-x)$. Hence, if we have $x<0$, we would get $f(x)\ge 0$. Because $f(f(0))\ge f(0)>0$, by substituting $x=\frac{-f(f(0))-1}{f(0)}<0$ on ...(2) gives $f(x)\le -1$. Combining those two results, we get $-1 \ge f(x)\ge 0$, which is clearly a contradiction. Case 5. $f(0)<0$ Note that $f(f(f(0)))\ge f(f(0))$. We claim that $f(f(0))>0$. Notice that by combining the inequality given in the problem and ...(3), we have $$f(x+y)\le yf(x)+f(f(x)) \le yf(x)+f(x)f(0)+f(f(0)) \implies f(x+y)\le f(x)(y+f(0))+f(f(0)).$$ Now, by replacing $(x,y)\rightarrow (x,f(0)-x)$ where $x<2f(0)$, we would get that $0\le f(x)(2f(0)-x)$. Hence, for all $x<2f(0)$, we must have $f(x)\ge 0$. By combining ...(3) and ...(4), we get that $f(0)+xf(x)\le f(f(x))\le f(x)f(0)+f(f(0))$, which implies for all real number $x$, we have $f(x)\le \frac{-f(f(0))}{f(0)-1}.$ If $f(f(0))<0$, then $f(x)<0$ for all $x\in \mathbb{R}$. However, we already got that for all $x<2f(0)$, we must have $f(x)\ge 0$. This is a contradiction. If $f(f(0))=0$, then we have $f(x)(f(0)-1)\ge 0$. Hence, we have $f(x)\le 0$ for all $x$ real. Now, take arbitrary $s<2f(0)$. By the previous result, we have $f(s)\ge 0$ which leads to $f(s)=0$. Then, by substituting $x\rightarrow f(s)$ on ...(1), we conclude $f(f(s))\ge f(s)=0$, which implies $f(f(s))=f(0)=0$, which contradicts our assumption of $f(0)<0$. Hence, we now have $f(f(0))>0$. By substituting $x\rightarrow f(0)$ on ...(3), we get $f(f(f(0)))\le f(f(0))f(0)+f(f(0))$ which implies $0\le \frac{f(f(f(0)))-f(f(0))}{f(f(0))}\le f(0)$, which is also a contradiction. Therefore, there are no such $f$ in the case 2,3,4, and 5. The problem statement is also proved on the first case. We're done.

Ibrahim_K wrote: Nice one Let $P(x,y)$ be the assertion. $P(x,0) \implies f(x)\le f(f(x))$ $P(x,f(x)-x) \implies f(x)(f(x)-x) \ge 0 \implies f(f(x))(f(f(x))-f(x)) \ge 0 \implies f(f(x)) \ge 0$ for all $x \in \mathbb R \ \ \ (*)$ Claim: $f(0)=0$ Proof: Assume contrary: $f(0)=c\neq 0$ Case1: $c>0$ $P(0,y) \implies f(y) \le yc+f(c) \implies f(f(y)) \le f(y)c+f(c) \implies 0 \le f(y)c+f(c) \implies 0 \le (yc+f(c))c+f(c)$ for all $y \in \mathbb R \ \ \ (1)$ Setting $y=-1-\frac{f(c)(c+1)}{c^2}$ in $(1) \implies -c^2 \ge 0$ which is contradiction. Case2: $c<0$ $P(0,y) \implies f(y) < f(c)$ for $y>0 \implies$ if $f(c)>0 \implies f(f(c))<f(c) \implies$ contradiction. So $f(c) \le 0$ Since $0 \le f(y)c+f(c) \implies f(y)c \ge 0 \implies f(y) \le 0$ for all $y \in \mathbb R$ If there exists $t$ such $f(t)=0$, then $P(t,0)$ gives $c \ge 0 \implies$ contradiction $\implies f(y) < 0$ for all $y \in \mathbb R \implies$ again contradiction by $(*)$ So $f(0)=0$ Thus $P(0,x)$ gives $f(x) \le 0$. Since $f(x)^2 \ge xf(x) \implies xf(x) \le 0$ Let there exist negative $t$ such $t<0$ and $f(t) \neq 0$. Then $tf(t) <0 \implies f(t)>0 \implies$ contradiction. So for all $x<0$ $f(x)=0$ we are done) Wrong, $P(x,x-f(x))$ doesn't imply that $f(f(x))$ cannot be less than 0, as you must investigate the case $f(f(x))=f(x)$.

Claim: $$f(x) \begin{cases} \leq 0 &\text{if } x>2f(0) \\ \geq 0 & \text{if } x<2f(0) \end{cases}$$ Proof: Plugging $x,f(0)-x$ and $f(0)-x,x$ give $$f(f(x))-f(f(0))\geq xf(x)-f(x)f(0)$$ $$f(f(0))-f(f(x))\geq -f(x)f(0)$$ By adding these, we get that $2f(x)f(0)\geq xf(x)\iff f(x)(2f(0)-x)\geq 0$ hence the claim is true.$\square$ Claim: $f(x)\leq 0$ for all $x\in R$. Proof: When we take $-x^2+y^2,-y^2$ where $-x^2<2f(0),$ we have $$0\leq f(-x^2)\leq -y^2f(y^2-x^2)+f(f(y^2-x^2))$$ So if $x^2>-2f(0),$ then $f(f(y^2-x^2))\geq y^2f(y^2-x^2)$. Take $y^2-x^2=z$. $$f(f(z))\geq y^2f(z)$$ Suppose that $f(z)>0$. Then, we would have $\frac{f(f(z))}{f(z)}\geq y^2$ where $y$ goes to positive infinity. This gives a contradiction so $f(x)\leq 0$.$\square$ Note that if $x<2f(0),$ then $f(x)=0$. Claim: $f(0)\geq f(x)$ for all $x\in R$. Proof:Take $x<2f(0)$. Plugging $x,y-x$ yields $f(y)\leq (y-x)f(x)+f(f(x))=f(0)$ thus, $f(y)\leq 0$ for all reals.$\square$ Since there exists $f(x)=0,$ we conclude that $f(0)\geq f(x)=0$. If $f(0)>0,$ then $0<2f(0)$ gives $f(0)=0$ which is a contradiction.$\blacksquare$

Zhero wrote: Thus, for sufficiently small $x$, we have $f(x)(c - x + f(0)) + f(f(0)) < 0$. How did he concluded that?