Let $n = \lfloor \frac{q}{p} \rfloor$, then $q = pn + r$ with $r \in \{1,2, \dots, p-1\}$. It suffices to show for each $r$, $pn+r$ divides $\sum_{k = 1}^{n} k^{p-1}$ for only finitely many $n$. Now write $\sum_{k = 1}^{n} k^{p-1} = \frac{f(n)}{m}$, where $f$ is an integer polynomial and $m$ is a positive integer. Since $\sum_{k = 1}^{n} k^{p-1} \sim \frac{n^p}{p}$ for large $n$, we know that $f$ has degree $p$ and the leading coefficient of $f$ equals $\frac{m}{p}$. By canceling common constants if necessary, we may assume $f$ is not the zero polynomial in $F_p[x]$. If there are infinitely many $n$ such that $pn+r$ divides $\frac{f(n)}{m}$, then $px + r$ divides $f(x)$ as polynomials in $Q[x]$. Since $p$ is a prime, $px+r$ is a primitive polynomial, so by Gauss' lemma $px + r$ divides $f(x)$ in $Z[x]$. In particular $p$ divides the leading coefficient of $f$, so $f$ has degree at most $p-1$ in $F_p[x]$. But from $\sum_{k = 1}^{n} k^{p-1} = \frac{f(n)}{m}$ and $p | m$ we have $p | f(n)$ for all $n$. This forces $f = 0$ in $F_p[x]$, contradiction!

It was also RMM Shortlist 2016 N2

Here's a solution given by my friend, Triangle_Center: We claim that $S_k^n:=1^k+2^k+\cdots+n^k$ can be expressed as $P_k(n)/(k+1)!$ where $P_k$ is a polynomial with the following properties: $P_k$ has integer coefficients, its degree is $k+1$ and its leading coefficient is $k!.$ $(*)$ We proceed by induction. Assume that the latter holds for $k=0,1,...,k-1$ (the base case is trivial). Then, note that \begin{align*}(n+1)^{k+1}-1&=\sum_{i=1}^{n}\big((i+1)^{k+1}-i^{k+1}\big)=\sum_{i=1}^{n}\sum_{j=0}^ki^j\binom{k+1}{j}=\sum_{j=0}^k\sum_{i=1}^{n}i^j\binom{k+1}{j} \\ &=\sum_{j=0}^{k}S_j^n\binom{k+1}{j} =S_k^n(k+1)+\sum_{j=0}^{k-1}\frac{P_j(n)}{(j+1)!}\binom{k+1}{j}=S_k^n(k+1)+\frac{Q(n)}{k!}\end{align*}for some polynomial $Q$ with integer coefficients and degree $k.$ But then, observe that \[S_k^n=\frac{(n+1)^{k+1}-1}{k+1}-\frac{Q(n)}{(k+1)!}=\frac{k!(n+1)^{k+1}-k!-Q(n)}{(k+1)!}\]and indeed, $P_k(n)=k!(n+1)^{k+1}-k!-Q(n)$ is a polynomial which satisfies the desired properties. Now, back to our problem, let $q=pa+b$ $($with $0\leq b\leq p-1).$ We can WLOG fix $b.$ We wish to show that for big enough $a$ \[q=pa+b\nmid \sum_{i=1}^{a}i^{p-1}:=S_{p-1}\underset{(*)}{=}\frac{P_{p-1}(a)}{p!}.\]Assume that $a>1.$ For the sake of brevity, we will denote $P_{p-1}$ by $P$ from now on. Then, observe the following: \[q\mid\frac{P(a)}{p!}\iff q\mid P(a)\iff q\mid P(a)\cdot p^p= P\bigg(\frac{q-b}{p}\bigg)\cdot p^p.\]Now, recall that $P$ has degree $p,$ so let $P(x)=x^pa_p+x^{p-1}a_{p-1}+\cdots+a_0.$ Then, by working in $\mathbb{Z}_q$ we have that\begin{align*}0&= P\bigg(\frac{q-b}{p}\bigg)\cdot p^p=p^p\sum_{i=0}^p\bigg(\frac{q-b}{p}\bigg)^ia_i=\sum_{i=0}^p(q-b)^i\cdot p^{p-i}\cdot a_i \\ &=\sum_{i=0}^p(-b)^i\cdot p^{p-i}\cdot a_i=C\text{ (an integer constant!)}\end{align*}Observe that $C\equiv a_p\bmod{p}$ and by our claim, the leading coefficient of $P,$ namely $a_p$ is equal to $(p-1)!.$ It follows that $C\not\equiv 0\bmod{p}$ so $C\neq 0.$ But if $a$ is big enough such that $q>|C|$ then since $C\neq 0$ we cannot have $0=C$ in $\mathbb{Z}_q,$ so $q$ cannot divide $P(a)/p!.$ So for a fixed $b,$ if $a$ is big enough, then it will not divide the given sum. Doing so for all $b$ (which are finitely many values), we can conclude that, overall, if $q$ is big enough, then it will not divide the given identity.

Let $P(x)$ be a polynomial of degree $p$ such that $P(x)=\sum\limits_{k=1}^x k^{p-1}$ for every singe integer $x$. This exists by noting $P(x)-P(x-1)=x^{p-1}$ is a polynomial. Since $P(x)$ can be written as $\sum_{k=0}^p a_k\binom xk$ for $a_k\in \mathbb{Z}$, it follows that $p!P(x)\in \mathbb{Z}[x]$ Let $a=\lfloor \frac qp \rfloor$. I claim for every $r\in \{1,\cdots,p-1\}$, there exists finitely many primes $q$ such that $q=ap+r$ and $q\mid P(a)$. Assume for contradiction that $q\mid P(\frac{q-r}{p})$ for infinitely many $r$. Then this implies $q\mid p^p\cdot p!P(\frac{-r}{p})$ for all primes $q$. In other words, $P(\frac{-r}{p})=0$. Observe that $[x^p]P(x)=\frac 1p$. However, if we consider $p!P(\frac{-r}{p})$, we will see that the leading term of this integral polynomial is $(p-1)!$. Therefore, $(p-1)! (-r/p)^p = -\sum\limits_{j=0}^{p-1} b_j(\frac{-r}{p})^j $ We can see $\nu_p(LHS)=-p$ while $\nu_p(RHS)\ge 1-p$, contradiction!

By finite differences, $\sum_{i=1}^n n^{p-1}$ is a degree $p$ polynomial with leading coefficient $\tfrac{1}{p}$. By Lagrange interpolation, can be written in the form $P(n)/p!$ for some integer polynomial $P$. Now, ignoring $q=p$, if $q$ divides the sum then it should divide $P(-\{q/p\})$. If infinitely many $q$ exist then $P(-k/p)$ should be zero for some $k \in \{1,\ldots,p-1\}$, but the leading coefficient of $P$ cannot be divisible by $p$ since $\nu_p(p!)=1$, which is a contradiction by rational root theorem. $\blacksquare$