Let $AB=c, BC=a, CA=b$. Let the $A$-median intersect $BC$ at $H$ and the circumcircle of $\triangle ABC$ again at $X$. Firstly, we will prove that $AFED$ is cyclic if and only if $\frac{BF \cdot BA}{BE} + \frac{CD \cdot CA}{CE} = a$ (*). If $AFED$ is cyclic, let the circumcircle of $AFED$ intersect $BC$ again at $Y$. Then $BF \cdot BA = BE \cdot BY \implies BY = \frac{BF \cdot BA}{BE}$ and $CY \cdot CE = CD \cdot CA \implies CY = \frac{CD \cdot CA}{CE}$. Since $BY + CY = a$, $\frac{BF \cdot BA}{BE} + \frac{CD \cdot CA}{CE} = a$. Now assume that $\frac{BF \cdot BA}{BE} + \frac{CD \cdot CA}{CE} = a$. Let $Y$ be the point on $BC$ such that $AFEY$ is cyclic, so $BF \cdot BA = BE \cdot BY \implies BY = \frac{BF \cdot BA}{BE}$. Note that $\frac{BF \cdot BA}{BE} + \frac{CD \cdot CA}{CE} = a \implies BY = \frac{BF \cdot BA}{BE} < a$ so $Y$ lies on segment $BC$. Then $a = BY + CY \implies \frac{CD \cdot CA}{CE} = CY \implies CD \cdot CA = CE \cdot CY \implies ADYE$ is cyclic. Thus $AFEYD$ is cyclic so $AFED$ is cyclic. Thus (*) holds. $AM \perp CM \implies MD = \frac{b}{2}$, and $BM:MD = 1:2 \implies BD = \frac{3b}{4}$. By Apollonius’ Theorem, $BD = \frac{\sqrt{2a^2 + 2c^2 - b^2}}{2} \implies \frac{9b^2}{16} = \frac{2a^2 + 2c^2 - b^2}{4} \implies 9a^2 = 8a^2 + 8c^2 - 4b^2 \implies 8a^2 + 8c^2 = 13b^2$. By Menelaus’ Theorem, $\frac{AD}{AC} \cdot \frac{CE}{EB} \cdot \frac{BM}{MD} = 1 \implies \frac{CE}{EB} = 4 \implies BE = \frac{a}{5}, EC = \frac{4a}{5}$. Similarly, $BF = \frac{c}{5}, AF = \frac{4c}{5}$. Thus $AFED$ is cyclic if and only if $\frac{c^2}{a} + \frac{5b^2}{8a} = a \Leftrightarrow 8c^2 + 5b^2 = 8a^2 \Leftrightarrow 16c^2 + 5b^2 = 8a^2 + 8c^2 \Leftrightarrow 16c^2 + 5b^2 = 13b^2 \Leftrightarrow 16c^2 = 8b^2 \Leftrightarrow 2c^2 = b^2$. By Menelaus’ Theorem, $AH, EF$ meet on the circumcircle of $\triangle ABC$ (**) if and only if $E,F,X$ collinear if and only if $\frac{XH}{XA} \cdot \frac{AF}{BF} \cdot \frac{BE}{EF} = 1$. By Apollonius’ Theorem, $AH = \frac{\sqrt{2b^2 + 2c^2 - a^2}}{2}$. $AH \cdot HX = BH \cdot HC \implies HX = \frac{a^2}{2\sqrt{2b^2+2c^2 - a^2}} \implies \frac{XH}{XA} = \frac{XH}{AH + XH} = \frac{a^2}{2b^2+2c^2}$. So (**) $\Leftrightarrow \frac{a^2}{2b^2+2c^2} \cdot 4 \cdot \frac{\frac{a}{5}}{\frac{a}{2} - \frac{a}{5}} = 1 \Leftrightarrow \frac{a^2}{2b^2+2c^2} \cdot \frac{8}{3} = 1 \Leftrightarrow 8a^2 = 6b^2 + 6c^2 \Leftrightarrow 8a^2 + 8c^2 = 6b^2 + 14c^2 \Leftrightarrow 13b^2 = 6b^2 + 14c^2 \Leftrightarrow 14c^2 = 7b^2 \Leftrightarrow 2c^2 = b^2 \Leftrightarrow AFED$ cyclic.

Here's another solution, probably similar enough to above. Let $N$ be the midpoint of $BC$. What follows is a flurry of computational techniques. Firstly, by Menelaus, we can easily find that $\frac{BF}{FA} = \frac{BE}{EC} = \frac{1}{4}$. In particular, $EF || AC$. Let the side lengths of triangle $ABC$ be $a, b, c$, respectively. For $AM \perp MC$, we need that \[ AF^2 + EC^2 = EF^2 +AC^2 \] \[ \left(\frac{4}{5}c \right)^2 + \left(\frac{4}{5}a \right)^2 = \left(\frac{1}{5}b \right)^2 + b^2 \] \[ \boxed{8(a^2+c^2) = 13b^2} \] Since $AC || EF$, for $ADEF$ to be cyclic, we must have that $AF = ED$. We do this by the Law of Cosines. $\cos C = \frac{a^2+b^2-c^2}{2ab}$. Therefore, \[ ED^2 = CD^2+CE^2-2 \cdot CD \cdot CE \cdot \cos C = \left(\frac{b}{2} \right)^2 + \left(\frac{4}{5}a \right)^2 - 2\cdot \left(\frac{b}{2} \right) \cdot \left(\frac{4}{5}a \right) \cos C \] \[ = \frac{b^2}{4} + \frac{16}{25}a^2 - \frac{2}{5}(a^2+b^2-c^2) \] Since $AF^2 = \left(\frac{4}{5} c \right)^2 = \frac{16}{25}c^2$, we must have that \[ \frac{b^2}{4} + \frac{16}{25}a^2 - \frac{2}{5}(a^2+b^2-c^2) = \frac{16}{25}c^2 \implies \boxed{8(a^2-c^2) = 5b^2} \] The last condition is that $AN$ ($N$ is defined at the very top) and $EF$ intersect on the circumcircle. We do this with barycentric coordinates. If we set $ABC$ as our reference triangle in the normal way, obviously $E = (0, 4/5, 1/5), F = (1/5, 4/5, 0), N = (0, 1/2, 1/2)$. Therefore, all points on the line $AN$ are of the form $(1-2t, t, t)$, while all points on the line $EF$ are of the form $(1/5-t, 4/5, t)$. Intersecting these gives the common intersection $(-3/5, 4/5, 4/5)$. The circle equation for the circumcircle of $ABC$ is $a^2yz+b^2xz+c^2xy = 0$. For $(-3/5, 4/5, 4/5)$ to lie on that circle, we need that $\frac{16}{25}a^2 - \frac{12}{25}(b^2+c^2) = 0 \implies \boxed{4a^2 = 3(b^2+c^2)}$. It suffices to show that these 3 boxed equations imply each other. Adding and subtracting the first two equations gives that $a = \frac{3\sqrt{2}}{4}b, c = \frac{\sqrt{2}}{2}b$. Plugging these into the last equation, $4a^2 = 3(b^2+c^2) \implies \frac{9}{2}b^2 = 3\left(b^2+\frac{b^2}{2} \right)$, which is true.

First off $EF \parallel AC$. Let $a,b$ and $C$ the sides of the $\triangle ABC$. By Menelaus' theorem $BE=\frac{a}{5};EC=\frac{4a}{5}$, so too $BF=\frac{c}{5};FA=\frac{4c}{5}$ and $FE=\frac{b}{5}$ by date $AE \perp CF \Rightarrow b^2+\frac{b^2}{25}=\frac{16a^2}{25}+\frac{16c^2}{25} \Rightarrow 13b^2=8(a^2+c^2) \ \ ... (1)$ $X$ is midpoint of $BC \Rightarrow \frac{EX}{XC}=\frac{3}{5} \ \ EF \cap AX=P \Rightarrow XP=\frac{3AX}{5}$ $ABPC$ is cyclic $\Longleftrightarrow AX.XP=BX.XC \Longleftrightarrow \frac{3AX^2}{5}=\frac{a^2}{4}$ But $2AX^2+\frac{a^2}{2}=c^2+b^2$ $ABPC$ is cyclic $\Longleftrightarrow c^2+b^2=\frac{4a^2}{3}$ Using $(1)$ we got $ABPC$ is cyclic $\Longleftrightarrow 3c=2a$ Now $DE \cap AB=Y \Rightarrow (A,B,F,Y)=-1 \Rightarrow AY=\frac{4c}{3}$ $AFED$ is cyclic $\Longleftrightarrow \angle EDA= \angle A \Longleftrightarrow \frac{AD}{2AY}=cos (\angle A)=\frac{b^2+c^2-a^2}{2bc}$ $AFED$ is cyclic $\Longleftrightarrow \frac{3b}{16c}= \frac{b^2+c^2-a^2}{2bc} \Longleftrightarrow a^2-c^2=\frac{5b^2}{8}$ Using $(1)$ we got $AFED$ is cyclic $\Longleftrightarrow 3c=2a$ Done!

I wonder if there is any geometric proof for this problem?

Nice problem.Solution with barycentric coordinate. Let $Q=AN\cap FE,$ where $N$ is midpoint of $BC.$ We can find easily $M=(\frac{1}{6},\frac{2}{3},\frac{1}{6}).$ From $AM\perp CM$ we find $8(a^2+c^2)=13b^2.$ $(1)$ Also it is easy to find $F(\frac{1}{5},\frac{4}{5},0)$ and $E=(0,\frac{4}{5},\frac{1}{5}).$ Let's to find coordinate of $Q.$ From $AN\to y=z,$ from $FE\to 4x=-3y,$ where $z\not= 0.$ Then $ABQC$ is cyclic iff $0=a^2yz+b^2zx+c^2xy=z^2(\frac{7b^2-14c^2}{8}) \Longleftrightarrow b^2=2c^2,$ $(2)$ where to use $(1)$ Lets to find d the equation of $(AFE).$ From $A\to u=0,F\to v=\frac{c^2}{5},E\to w=\frac{4(a^2-c^2)}{5}.$ Then $AFDE$ is cyclic iff $-a^2yz-b^2zx-c^2xy+(x+y+z)(\frac{c^2y+4z(a^2-c^2)}{5})\Longleftrightarrow b^2=\frac{8(a^2-c^2)}{5}$ $\Longleftrightarrow b^2=2c^2,$ $(3)$ where to use $(1).$ From $(3)$ and $(2)$ we get contradiction

drmzjoseph wrote: $$\frac{AD}{2AY}=cos (\angle A)$$ why is this true?