We'll do it using complex numbers.. WLOG , $A = 3$ Let $B = z$ so that we have $|z| = 5$ We will now find $C$.. Let $C=x$ We will use the following result (which I suppose is a standard result) If $z_1 , z_2$ and $z_3$ form the vertices of an equilateral triangle , then we have : ${z_1}^2+{z_2}^2+{z_3}^2 = z_1z_2 + z_2z_3 + z_3z_1$ Hence , we get $9 + z^2 + x^2 = 3z + 3x + zx$ $x^2 - (3+z)x + (z^2-3z+9) = 0$ This gives , $x = \boxed{\frac{(1+\sqrt{3}i)z + 3(1-\sqrt{3}i)}{2}}$ OR $x = \boxed{\frac{(1-\sqrt{3}i)z + 3(1+\sqrt{3}i)}{2}}$ Using triangle inequality , that is $|z_1+z_2| \le |z_1| + |z_2|$ we get $|x| \le 8$ Hence , maximum value of length of $[OC]$ is $8$ I hope I haven't made any calculation errors..