Let ABCD be our rectangle, with AB = CD = 3 and BC = DA = 4, and let X, Y, Z, U be the four inner points of its sides AB, BC, CD, DA, respectively, such that XY = x, YZ = y, ZU = z and UX = u. Call AX = a, BY = b, CZ = c and DU = d. Then, BX = AB - AX = 3 - a, CY = BC - BY = 4 - b, DZ = CD - CZ = 3 - c and AU = DA - DU = 4 - d. The Pythagoras theorem, applied to the right-angled triangle XBY, yields $x^2=XY^2=BX^2+BY^2=\left(3-a\right)^2+b^2$. Similarly, $y^2=\left(4-b\right)^2+c^2$, $z^2=\left(3-c\right)^2+d^2$ and $u^2=\left(4-d\right)^2+a^2$. Hence, $x^2+y^2+z^2+u^2$ $=\left(\left(3-a\right)^2+b^2\right)+\left(\left(4-b\right)^2+c^2\right)+\left(\left(3-c\right)^2+d^2\right)+\left(\left(4-d\right)^2+a^2\right)$ $=\left(\left(3-a\right)^2+a^2\right)+\left(\left(4-b\right)^2+b^2\right)+\left(\left(3-c\right)^2+c^2\right)+\left(\left(4-d\right)^2+d^2\right)$. Now, since a = AX, and X is an inner point of the side AB of the rectangle, we have 0 < a < 3. Hence, AM-QM yields $\frac{\left(3-a\right)^2+a^2}{2}\geq\left(\frac{\left(3-a\right)+a}{2}\right)^2$ and thus $\left(3-a\right)^2+a^2 \geq 2\left(\frac{\left(3-a\right)+a}{2}\right)^2 = 2\left(\frac32\right)^2 = \frac92$. Similarly, $\left(4-b\right)^2+b^2 \geq 8$, $\left(3-c\right)^2+c^2 \geq \frac92$ and $\left(4-d\right)^2+d^2 \geq 8$, so that $x^2+y^2+z^2+u^2$ $=\left(\left(3-a\right)^2+a^2\right)+\left(\left(4-b\right)^2+b^2\right)+\left(\left(3-c\right)^2+c^2\right)+\left(\left(4-d\right)^2+d^2\right)$ $\geq \frac92+8+\frac92+8=25$. This proves the left part of the inequality $25 \leq x^2+y^2+z^2+u^2 \leq 50$. Remains to prove the right part. Since 0 < a < 3, we have $9-\left(\left(3-a\right)^2+a^2\right) = 2a\left(3-a\right) > 0$, so that $\left(3-a\right)^2+a^2 < 9$. Similarly, $\left(4-b\right)^2+b^2 < 16$, $\left(3-c\right)^2+c^2 < 9$ and $\left(4-d\right)^2+d^2 < 16$. Thus, $x^2+y^2+z^2+u^2$ $=\left(\left(3-a\right)^2+a^2\right)+\left(\left(4-b\right)^2+b^2\right)+\left(\left(3-c\right)^2+c^2\right)+\left(\left(4-d\right)^2+d^2\right)$ $< 9+16+9+16=50$. This proves even more than the right part of the inequality $25 \leq x^2+y^2+z^2+u^2 \leq 50$; this even proves that the right $\leq$ sign can be replaced by an < sign. (In fact, the problem proposer probably used the $\leq$ sign since he allowed the points X, Y, Z, U to coincide with the vertices of the rectangle; in the solution above, I don't allow this). This is in fact Darij's solution, re-posted.

This problem is a easy and nice exercise for the introduction a some simple inequalities in geometry. In generally, this problem can show thus: For the rectangle $ABCD\ (AB=CD=a,\ BC=AD=b)$ and the points $X\in [AB],\ Y\in [BC],\ Z\in [CD],\ T\in [DA]$ $(XY=x,\ YZ=y,\ ZT=z,\ TX=t)$ there is the following relation: $a^2+b^2\le x^2+y^2+z^2+t^2\le 2(a^2+b^2).$