Please post your solutions. This is just a solution template to write up your solutions in a nice way and formatted in LaTeX. But maybe your solution is so well written that this is not required finally. For more information and instructions regarding the ISL/ILL problems please look here: introduction for the IMO ShortList/LongList project and regardingsolutions

I think it's achieved for the regular pentagon, so it's $\phi=\frac{1+\sqrt 5}2$. First of all, let's notice that if it's not a convex pentagon, then we have a point inside a triangle, so that point is inside a triangle formed by the centroid and two vertices of the triangle, so the ratio would be at least $3>\phi$. This means that the pentagons which reach the minimal value are convex. Furthermore, it's easy to see that, since the pentagon is convex, the triangle with the minimal area is formed by three consecutive vertices, while the one with the largest area is formed by three vertices which are not consecutive. Assume our pentagon is $ABCDE$, and the triangle with minimal area is $BAE$. Since we can apply an affine transformation which maps $A,B,E$ to three consecutive vertices of a regular pentagon and this transformation preserves the ratios of areas, we may assume there is a regular pentagon $ABC'D'E$. Take $X$ s.t. $C'X\|BD',\ D'X\|EC'$. Since $S(C'AE)\le S(CAE),\ S(D'AB)\le S(DAB)$, and given the convexity of $ABCDE$, at least one of $C,D$ lies inside $C'XD'$ (just draw a picture to see why). Assume it's $D$. We can move $C,D$ along lines parallel to $BD$ s.t. the area of $BCD$ increases, and this time $C,D$ lie on the union of the two segments math=inline]$XC'$[/math, math=inline]$XD'$[/math. It's easy now to see that we can enlarge the area of $BDC$ even more by moving $C,D$ s.t. they reach positions $C',D'$, but this means that the initial area of $BCD$ was smaller that that of $BAE$, and this is a contradiction, so the minimal ratio is reached for regular pentagons. I know the argument is a bit murky, but it's not that hard to see why it works if you draw a picture.

grobber wrote: Furthermore, it's easy to see that, since the pentagon is convex, the triangle with the minimal area is formed by three consecutive vertices, while the one with the largest area is formed by three vertices which are not consecutive. While this is not necessarily true, the case in which the triangle with largest area consists of three consecutive vertices easily gives a ratio of $\ge 2$. BTW, another way to deal with this convex case is to affine transform the largest triangle into the golden triangle.

Indeed, if $ABC$ is the maximal triangle, then if $B'$ is selected so that $ABCB'$ is a parallelogram, then $D,E$ must lie within $AB'C$, which means that $[ADC]+[DEC]\le[ABC]$. No suppose $ABD$ is the maximal triangle. Take any line $g$ which passes through $B$ and intersects the interior of segment $AD$. Since $ABD$ is maximal, $E$ must lie within $ADB'$, where $ABDB'$ is a parallelogram. So the line through $E$ parallel to $g$ intersects the interior of $AD$ as well. Let $x,y$ be the distances from $B,E$ to $AD$ along an axis parallel to $g$, and let $a,b, y+z$ be the distances from $A,D,E$ to $BC$ along the same axis (positive distances). For the purposes of this problem, we can assume WLOG, $a \le b$. So $x,z \ge a$. If $\theta$ is the maximal area ratio, then $y/x = [AED]/[ABD] \ge \frac{1}{\theta}$. Also $(z+y)/a =[EBC]/[ABC] \le \theta$, which means that $1+ \frac{1}{\theta} \le \theta$, and we get the golden ratio solution, with equality in the case of the regular pentagon. While not really any shorter than the other solutions, I thought I'd post this solution because it does give a sense why the golden ratio is the answer.

Solution with coordinate

The answer is $\phi=\frac{1+\sqrt{5}}{2}$, achieved for example at a regular pentagon. Suppose for sake of contradiction that we had a set $S$ such that for any two triangle areas $\alpha$ and $\beta$, we have \[\alpha/\beta \in (1/\phi,\phi).\]We have the following claim. Claim: The convex hull of $S$ has exactly five points. Proof: First, suppose that it had only three points, say $ABC$, and $DE$ were inside triangle $ABC$. Then, one of $[DAB],[DBC],[DCA]$ is at most $\frac{1}{3}[ABC]$, so we can get a triangle area ratio of at least $3$, which is a contradiction. Now, suppose that it had four points, say $ABCD$ in that order, with $E$ inside the quadrilateral. One of $[EAB],[EBC],[ECD],[EDA]$ is at most $\frac{1}{4}[ABCD]$, and one of $[ABC]$ and $[CDA]$ is at least $\frac{1}{2}[ABCD]$, so we have a triangle area ratio of at least $2$, which is a contradiction. Thus, the convex hull must have at exactly five points. $\blacksquare$ Suppose the points form a convex pentagon $ABCDE=A_1A_2A_3A_4A_5$ in that order. Claim: The minimal area triangle is $\triangle A_iA_{i+1}A_{i+2}$ for some $i$ (indices mod $5$). Proof: Suppose that WLOG $\triangle ABD$ was instead of minimal area, so in particular, $[ABD]\le [ABC],[ABE]$. Let $AB$ be the $x$-axis, with $A$ to the right of $B$. The condition is then saying that the $y$-coordinate of $D$ is at most that of $C$ and $E$.

Note that $D$ has to be in the region bounded by the half plane with line $BC$ on the same side as $A$, the half plane with line $AE$ on the same side as $C$, and the half plane with line $CE$ on the opposite side as $A,B$. This is a convex region, and by cutting with the line $y=N$ for large enough $N$ so that it is above $D$, we get a convex polygon that $D$ must lie in. Thus, the point in this polygon with smallest $y$-coordinate must be one of its vertices, and not on $y=N$ (or potentially $BC\cap AE$, as that is further along ray $BC$ than $C$), so it must be $C$ or $E$. Thus, one of $C$ or $E$ has a lower $y$-coordinate than $D$, which is the desired contradiction. $\blacksquare$ WLOG, let $\triangle ABC$ be of minimal area. Affine transform the picture so that $AB=BC$ and $\angle ABC=3\pi/5$. Note that affine transformations preserve ratios of signed areas, so it preserves that $ABCDE$ is a convex pentagon in that order, that $\triangle ABC$ is of minimal area, and all the unsigned triangle area ratios still lie in $(1/\phi,\phi)$. So we have a convex pentagon $ABCDE$ in that order such that $AB=BC$ and $\angle ABC=3\pi/5$,$[ABC]$ is the minimal area triangle, and that all triangle area ratios are in $(1/\phi,\phi)$. We will derive a contradiction.

Indeed, complete $ABC$ to a regular pentagon $ABCD'E'$. We see that $[ABD],[ABE]\ge [ABC]$, so $D$ and $E$ lie on the opposite side of $CE'$ as $A,B$, and similarly $D,E$ lie on the opposite side of $AD'$ as $B,C$. Furthermore, $[ABD],[ABE]\le[ABD']$ (else we have an area ratio of $>\phi$), so they lie on the same side of $\ell_1$ as $A,B$, where $\ell_1$ is the line through $D'$ parallel to $AB$. Similarly, $D,E$ lie on the same side of $\ell_2$ as $A,B$ where $\ell_2$ is the line through $E'$ parallel to $BC$. Let $X=AD'\cap CE'$ and $Y=\ell_1\cap\ell_2$. All of this shows that $D$ and $E$ are in the region $XD'YE'$. For now, imagine that $D$ and $E$ vary over this region. We show that $[ADE]$ is maximized at when $\{D,E\}=\{D',E'\}$. Imagine fixing $D$, and consider a varying line parallel to $AD$. We see that $E$ must be on a line as far from $AD$ as possible, and since the direction of $AD$ is between that of $AD'$ and $AE'$, this implies that $E$ must be one of $D'$ or $E'$ (the direction is to eliminate the possibility of $X$ or $Y$). Now, holding $E$ fixed at that location, varying $D$ shows it must be at the other one of $D'$ or $E'$. Now, going back to the original locations of $D$ and $E$, this shows that \[[ADE]\le[AD'E'] = [ABC],\]so equality must hold (as $\triangle ABC$ is of minimal area), so $\{D,E\}=\{D',E'\}$, so $ABCDE$ is a convex pentagon. The ratio value is now $\phi\not\in(1/\phi,\phi)$, so we have our desired contradiction (finally!).

The source of this problem is the following article: MR0286678 Paul Erdős, Amram Meir, Vera T. Sós, Pál Turán: On some applications of graph theory. II. Studies in Pure Mathematics (Presented to Richard Rado) pp. 89–99 Academic Press, London, 1971.

Clearly, $M(S)/m(S)$ achieves $\tfrac{1+\sqrt{5}}{2}$ when the set $S$ are the vertices of a regular pentagon. Suppose that we can do better. If $S$ is concave, then there exists a point $P$ inside $S'$, the convex hull of $S$. Triangulate $S'$ and find that $P$ lies inside triangle $ABC$. Then, either $[APB],[BPC],$ or $[CPA]$ is at most one-third $[ABC]$ so $M(S)/m(S)>3$. If $S$ is convex, let the points be $A$, $B$, $C$, $D$, $E$ in counterclockwise order. Let $[ABC]$ be the smallest area out of the five areas $[ABC],[BCD],[CDA],[DEA],[EAB]$. Then, take the affine transform that takes $A$ to $(-1,0)$, $B$ to $(-1,-1)$ and $C$ to $(0,-1)$. Clearly, $[BCD]\ge 0.5$ implies that $D$ is on or above the $x$-axis. Similarly, $E$ must be on or to the right of the $y$-axis. Now, since convexity is conserved by affine transforms, if $D$ were to the left of the $y$-axis then $E$ would be to the left of the $y$-axos. This can only mean that $D$ and $E$ are both in the first quadrant. We have \[\frac{M(S)}{m(S)}<\frac{1+\sqrt5}{2}\implies [ACD],[ACE]<\frac{1+\sqrt5}{4}\]so the distance from $D$ and $E$ to $AC$ is less than $\tfrac{\sqrt{2}+\sqrt{10}}{4}$. That is, both lie below the line $\ell=x+y=\tfrac{\sqrt5-1}{2}$. Let $\ell$ intersect the $x$ and $y$ axes at $X$ and $Y$, respectively. We have $[CDE]<[CXY]<1$ from calculations, which is absurd.