Fairly similar to this Iran TST 2011 problem:https://artofproblemsolving.com/community/c6h406526p2270100

BarisKoyuncu wrote: Let $\mathbb{N}$ denote the set of all positive integers. Find all functions $f:\mathbb{N}\to\mathbb{N}$ such that $$x^2-y^2+2y(f(x)+f(y))$$is a square of an integer for all positive integers $x$ and $y$. Cool problem! Let \(P(x,y)\) denote the assertion of the given functional relation. We begin by proving a claim. Claim 1. \(p\mid f(kp)\) for all positive integers \(p\nmid k\) and a prime \(p\). Proof. \(P(kp,kp)\) tells us that \(4kpf(kp)\) is a perfect square, or \(kpf(kp)\) is a perfect square, implying that \(p\mid f(kp)\), as claimed. $\blacksquare$ For a fixed odd prime \(p\), let \(f(kp)=p\cdot g(k)\) for all \(k\nmid p\). Then, for any positive integers \(a\) and \(b\) such that \(a,b\nmid p\), \(P(ap,bp)\) implies that \[a^2-b^2+2b(g(a)+g(b))\]is a perfect square, and \(P(bp,ap)\) implies that \[b^2-a^2+2a(g(a)+g(b))\]is a perfect square. We would like to get rid of that \(g(a)+g(b)\) term, since it is common in both the expressions. For that, we need to subtract the two expressions by multiplying one expression by a perfect square to preserve the perfect square condition. So, if \(a=k^2\cdot b\), then subtracting the first expression by \(k^2\) times the second tells us that \((k^4-1)(k^2-1)b^2\) is a difference of two squares. Since \(b\) and \(k\) are free variables, letting \(b=1\) and \(k=2\) to reduce casework (note we can always make such a substitution since \(p\) is odd), we see that \(75\) is a difference of two squares. Since only \(3\) such pairs whose difference of squares is \(75\) exist, we conclude that \(f(p)=cp\) for infinitely many \(p\), where \(c\) is a constant. Now, let \(n\) be any positive integer. \(P(p,n)\) gives us that \[p^2-n^2+2n(cp+f(n))=(p+cn)^2+2nf(n)-(c^2+1)n^2\]is a perfect square. Now, varying over all primes \(p\) with \(f(p)=cp\), we see that \(2nf(n)-(c^2+1)n^2\) is a difference of two squares and the number of such pairs is infinite, implying that it is \(0\). Therefore, \(2nf(n)=(c^2+1)n^2\), implying that \(f(n)=\frac{c^2+1}{2}n\) for all positive integers \(n\). If we plug \(n=p\), where \(f(p)=cp\), we see that \(\frac{c^2+1}{2}=c\), implying that \(c=1\). Therefore, \(f(n)=n\) for all positive integers \(n\), which is indeed a solution. $\blacksquare$

rama1728 wrote: Claim 1. \(p\mid f(kp)\) for all positive integers \(k\) and a prime \(p\). Proof. \(P(kp,kp)\) tells us that \(4kpf(kp)\) is a perfect square, or \(kpf(kp)\) is a perfect square, implying that \(p\mid f(kp)\), as claimed. $\blacksquare$ What if $p|k$?.

Quote: For a fixed odd prime \(p\), let \(f(kp)=p\cdot g(k)\) for all \(k\nmid p\). you cant assume this since if it was true $ f(kpq)=pqg(k)=pf(qk)=qf(pk)$ and thus giving f a property that we don't know yet that is true

rama1728 wrote: \(p\mid f(kp)\) for all positive integers \(k\nmid p\) and a prime \(p\). Um..... this is wrong. What about $k = 2p$?

he meant $p$ does not divide $k$.

Let $P(x,y)$ be the main equation. $P(p,p) : 4pf(p)$ is perfect square so $p | f(p)$. $P(n^2p,n^2p) : 4n^2pf(n^2p)$ is perfect square so $p | f(n^2p)$. Let $X = \frac{f(p)}{p}$ and $Y = \frac{f(n^2p)}{p}$. Assume $n > 1$. $P(n^2p,p) : (n^4-1)p^2 + 2p^2(X+Y)$ is perfect square so $(n^4-1) + 2(X+Y)$ is perfect square as well. $P(p,n^2p) : -(n^4-1)p^2 + 2n^2p^2(X+Y)$ is perfect square so $-(n^4-1) + 2n^2(X+Y)$ is perfect square as well. Now we have $(n^2+1)(n^4-1)$ is difference of infinite couple of numbers and $(n^2+1)(n^4-1) > 0$ so By Pigeonhole principle for infinite prime numbers like $p$ we have $X+Y$ is constant or $f(p) = ap$. $P(p,x) : p^2 - x^2 + 2xf(x) + 2xap = (p+ax)^2 - x^2a^2 - x^2 + 2xf(x)$ so $- x^2a^2 - x^2 + 2xf(x)$ is difference of infinite couple of numbers so $- x^2a^2 - x^2 + 2xf(x) = 0 \implies f(x) = \frac{x(a^2+1)}{2}$. Let $p$ be prime number such that $f(p) = ap$ so $ap = \frac{p(a^2+1)}{2}$ or $2a = a^2 + 1 \implies a = 1$. so $f(x) = x$. we're Done.