Denote $B'C'=a' , C'A'=b'$ and $A'B'=c'.$ Perpendicular lines to $BC,CA,AB$ through $A',B',C'$ concur at the reflection of the incenter $I$ about the circumcenter $O.$ If $(I)$ touches $BC,CA,AB$ at $X,Y,Z,$ then by Euler theorem we have $\frac{[ \triangle A'B'C']}{[ \triangle ABC]}=\frac{[ \triangle XYZ]}{[ \triangle ABC]}= \frac{p(I,(O))}{4R^2}=\frac{2Rr}{4R^2}$ $\Longrightarrow [ \triangle A'B'C']=\frac{r}{2R} \cdot [ \triangle ABC]$ $[\triangle A'B'C']=\frac{r}{2R} \cdot [ \triangle ABC]=\frac{a'b'c'}{4R'} \Longrightarrow R'=\frac{a'b'c'}{[ \triangle ABC]} \cdot \frac{R}{2r} \ \ (1)$ By cosine law in $\triangle AB'C'$ we have $(a')^2=(s-b)^2+(s-c)^2-2(s-b)(s-c) \cos A$ $(a')^2=\frac{_1}{^2}(a^2+b^2+c^2)-bc-\frac{S_A}{bc}(bc-S_A)$ $(a')^2=\frac{_1}{^2}(a^2+b^2+c^2)-\frac{_1}{^2}(b^2+c^2-a^2)-\frac{S^2}{bc}$ $(a')^2=a^2-\frac{S^2}{bc}=a^2-bc\sin ^2 A=a(2R-h_a)\sin A$ By cyclic exchange of elements we obtain the expressions $(b')^2=b (2R-h_b) \sin B\ , \ (c')^2=c(2R-h_c) \sin C$ $a'b'c'=\sqrt{abc\sin A \sin B \sin C (2R-h_a)(2R-h_b)(2R-h_c)}$ $a'b'c'=[\triangle ABC] \sqrt{ \frac{2(2R-h_a)(2R-h_b)(2R-h_c)}{R}} \ \ (2) $ Combining the expressions $(1)$ and $(2)$ we get $R'=\frac{[\triangle ABC]}{[\triangle ABC]}\sqrt{ \frac{2(2R-h_a)(2R-h_b)(2R-h_c)}{R}} \cdot \frac{R}{2r}$ $R'=\frac{1}{2r}\sqrt{2R(2R-h_a)(2R-h_b)(2R-h_c)}$

See and the message LX from my blog, where is a proof similarly to luisgeometria's.