Denote by $[m,n]:=\operatorname{lcm}(m,n)$, $(m,n):=\operatorname{gcd}(m,n)$, and by $P(x,y)$ the equation $[f(x),y)](f(x),f(y))=f(x)f(f(y))$. Then $P(x,f(x))$ yields $(f(x),f(f(x)))=f(f(f(x)))$ and $P(f(x),f(x))$ yields $[f(f(x)),f(x)]=f(f(f(x)))$. So combined, this yields $f(x)=f(f(x))=f(f(f(x)))$. Replacing $f(f(y))$ by $f(y)$ in $P(x,y)$ and using symmetry we obtain $[f(x),y]=[f(y),x]$ for all $x,y$ so in particular $f(x)=[f(1),x]$ for all $x$. So $f$ is of the form $f(x)=[c,x]\forall x$ for some constant $c\in\mathbb{Z}_{>0}$, and plugging this into the original equation shows that every function of this form is also a solution.

Exactly my solution.

jasperE3 wrote: Exactly my solution. Excuse me, how can i watch IMOC Problems?

IMOC Problems: 2017 2018 2019 2020 2021

$f$ constant works. Now,assume $f$ is non-constant. Then ,we from $y=x$,combined with $y=f(x)$,gives us $f(x)=f(f(x))=f(f(f(x)))$, In the original problem statement ,we nowget that $f$ is linear and so let it be $cx$. Now,if $c$ is not $1$,consider a prie $p$ dividing it. Choose $x$ coprime to $c$ and $y=p$ to get a contradiction . So $f$ is either constant or $f(x)=x$.

Wrong, see the first solution.

$$[f(x),y](f(x),f(y))=f(x)f(f(y))$$$\textbf{Claim:} \ f(f(x))=f(x)$ $P(x,f(x))\implies f(x).(f(x),f(f(x)))=f(x)f(f(f(x)))\iff (f(x),f(f(x)))=f(f(f(x)))$ Thus, $f(x)\geq f(f(x))$ $P(x,x)\implies [f(x),x].f(x)=f(x)f(f(x))\iff [f(x),x]=f(f(x))$ So $f(x)\leq f(f(x))$ \[f(f(f(x)))\geq f(f(x))\geq f(x)\geq f(f(f(x)))\]Hence $f(f(x))=f(x)$ $$[f(x),y](f(x),f(y))=f(x)f(y)$$ $\textbf{Claim:} \ x|f(x)$ $P(x,x)\implies [f(x),x]f(x)=f(x)^2\iff [f(x),x]=f(x)\implies x|f(x)$ $\textbf{Claim:} \ f(x)=[c,x]$ We have $[f(x),y].(f(x),f(y))=f(x)f(y))=[f(x),f(y)].(f(x),f(y))\implies Q:[f(x),y]=[f(x),f(y)]$ Let $f(1)=c$ and note that $f(c)=c$. $Q(x,1)\implies f(x)=[f(x),1]=[f(x),c]\implies c|f(x)$ $Q(1,y)\implies [c,y]=[c,f(y)]$ $Q(c,p)\implies [c,p]=[c,f(p)]$ If we take $p>c$, then $cp=[c,f(p)]\implies f(p)|cp \ $ Also $(c,p)=1, \ c|f(p)$ and $p|f(p)$ thus $pc|f(pc)|pc\iff f(pc)=pc$ $Q(pc,y)\implies [pc,y]=[pc,f(y)]$ Denote $f(x)=cg(x)$. We have $[pc,y]=c[p,g(y)]$ Taking $p$ sufficiently large gives that $[pc,y]=pcg(y)\implies [c,y]=cg(y)=f(y)$ Hence $f(y)=[y,c]$ as desired.$\blacksquare$

$P(f(x), f(x))$ gives $\text{lcm}(f(x), f(f(x))) = f(f(f(x))) \implies f(x), f(f(x)) \mid f(f(f(x)))$. $P(x, f(x))$ gives $\gcd(f(x), f(f(x))) = f(f(f(x))) \implies f(f(f(x))) \mid f(x), f(f(f(x))) \mid f(f(x)) \implies f(x) = f(f(x)) = f(f(f(x)))$. So, we have $\text{lcm}(f(x), y)\gcd(f(x), f(y)) = f(x)f(y) \implies \text{lcm}(f(x), y) = \text{lcm}(f(x), f(y))$. Substituting $y = 1$, we have $f(x) = \text{lcm}(f(x), f(1)) = f(x) \implies f(1) \mid f(x)$ for all $x$. Substituting $x = 1$, we have $f(y) = \text{lcm}(f(1), y)$ (so $y \mid f(y)$) and we claim that this function works. Note that $f(y) = f(f(y)) = \dots$ since $f(1) \mid f(y)$, so we simply have to show that $\text{lcm}(f(x), y) = \text{lcm}(f(x), f(y))$ for all $x, y$ to show that this function works. For primes $p$ such that $p \nmid f(1)$, $p \mid y \leftrightarrow p \mid f(y)$, and $v_p(f(y)) = \max(v_p(y), v_p(f(1)) = v_p(y)$ since $p \nmid f(1)$, so $v_p(f(x), y) = v_p(f(x), f(y))$ for all such primes. For primes $q$ such that $q \mid f(1)$, we have $q \mid \text{lcm}(f(x), y), \text{lcm}(f(x), f(y))$ since $q \mid f(x)$. If $v_q(y) \geq v_q(f(1))$, we have $v_q(f(y)) = v_q(y) \implies v_q(f(x), y) = v_q(f(x), f(y))$. If $v_q(y) < v_q(f(1))$, $v_q(f(y)) = v_q(f(1))$, and since $f(1) \mid f(x), v_q(y), v_q(f(y)) \leq v_q(f(x)) \implies v_q(f(x)) = v_q(\text{lcm}(f(x), y) = v_q(\text{lcm}(f(x), f(y))$. So, $v_p(\text{lcm}(f(x), y)) = v_p(\text{lcm}(f(x), f(y))$ for all primes, which implies the result. Hence, all functions of the form $f(x) = \text{lcm}(c, x)$ work, where $c$ is a constant.