I like this problem! The answer is yes. We are going to ignore the function $G(x)$ and use only the functions $F(x)$ and $H(x)$. Claim. We can write a function $h(x)$ on the board so that $|h(x)-x|<\frac{1}{2}$ for all $x\in[\sqrt[4]{12},10].$ Proof: There are multiple ways to prove this. One way is to use Bernstein polynomials: $F$ is injective over the interval $[\sqrt[4]{12},10]$ (as $x^2+\frac{12}{x^2}=y^2+\frac{12}{y^2}\iff (x^2-y^2)(12-x^2y^2)=0$) and all functions written on the board are of the form $P\circ F(x)$, with $P$ a real polynomial. Thus, the existence of a function $h=P\circ F$ with $|P\circ F(x)-x|<\frac{1}{2}$ on $[\sqrt[4]{12},10]$ is equivalent to the existence of a polynomial $P$ with $|P(x)-F^{-1}(x)|<\frac{1}{2}$ on $F^{-1}[\sqrt[4]{12},10]$. One can construct such a polynomial via Bernstein polynomials. Another way is to use heavier machinery: again, note that $F$ is injective on $[\sqrt[4]{12},10]$, hence it separates points on that compact metric space. Thus, one can use the Stone-Weierstrass Theorem to conclude. Returning to the original problem, we claim that the function $h$ produced by the above claim satisfies the problem's condition. Indeed, it does so on the interval $[\sqrt[4]{12},10]$ by construction. Since $F(x)=F\left(\frac{\sqrt{12}}{x}\right)$, so will $h(x)=h\left(\frac{\sqrt{12}}{x}\right)$. Thus, on the interval $[1,\sqrt[4]{12})$, noting that $\frac{\sqrt{12}}{x}\in[\sqrt[4]{12},10]$, we have $$|h(x)-x|\le \left|h(x)-\frac{\sqrt{12}}{x}\right|+\left|\frac{\sqrt{12}}{x}-x\right|=\left|h\left(\frac{\sqrt{12}}{x}\right)-\frac{\sqrt{12}}{x}\right|+\left|\frac{\sqrt{12}}{x}-x\right|<\frac{1}{2}+\left|\frac{\sqrt{12}}{x}-x\right|\le\frac{1}{2}+\sqrt{12}-1<3.$$

Doesn't just $h(x) = \frac{1}{10}F(x)$ work?

It seems like a weak statement, $h(x)=\frac1{10}\left(x^2+\frac{12}{x^2}\right)+1$ works.

The original problem is $\tfrac{1}{3}$ rather than $3$.

Thanks, I don't know what I was doing when I posted this, I've edited the question now

In that case, the answer is NO (and again, the $\frac{1}{3}$ is very weak but now in the opposite direction). Indeed, suppose we have $x,y \in [1,10]$ such that $x^2y^2=12$ and $x^2-y^2$ is an even integer. Then $F(x)=F(y)$ and $G(x)=G(y)$ and hence $h(x)=h(y)$ for any function $h$ we can write on the board. Hence one of $\vert h(x)-x\vert$ and $\vert h(y)-y\vert$ is at least $\frac{\vert x-y\vert}{2}$ by the triangle inequality. The rest is just a matter of computation: Let $x^2=y^2+10$ so that we need $y \in (1,2)$ with $y^2(y^2+10)=12$. Solving this quadratic in $y^2$ we get $y=\sqrt{\sqrt{37}-5}$ and $x=\sqrt{\sqrt{37}+5}$. So we get that at least one of $\vert h(x)-x\vert$ and $\vert h(y)-y\vert$ is at least $\frac{\vert x-y\vert}{2} \approx 1.144>\frac{1}{3}$.