We claim that this happens iff $n$ is squarefree or $n=4$. Let $n=p_1^{e_1}\dots p_k^{e_k}$. If $e_i \ge 2$ for some odd $p_i$, then note that $p_i^{e_i-1} \ge e_i+1$ (since $p_i \ge 3$) and $p_j^{e_j} \ge e_j+1$ for all the others, so $p_i^{p_i^{e_i-1}-1} \cdot \prod_{j \ne i} p_j^{p_j^{e_j}-1} \cdot q^{p_i-1}$ has $n$ divisors for an arbitrary prime $q$ different from the $p_i$ and is a multiple of $n$, so there are infinitely many such multiples. Similarly, if $p_1=2$ and $e_1 \ge 3$, then $2^{e_1-1} \ge e_1+1$ and we can use the same argument. Finally, if $n=4p_1\dots p_k$ for $k \ge 1$, then we can choose $2^{p_1-1} p_1 p_2^{p_2-1} \dots p_k^{p_k-1} q$ for any prime $q$ different from the $p_i$, and there are infinitely many such multiples. On the other hand, suppose that $n$ is squarefree with $k$ prime divisors. Then any number with $n$ divisors can have at most $k$ distinct prime divisors since each of them contributes a factor with at least one prime to the number of divisors. But then these primes must be the primes dividing $n$ and there are clearly only finitely many such combinations. Finally, the result for $n=4$ is trivial.