Claim: Every prime in the sequence is the AM of the adjacent members. Assume $k \in n $, then, $$a_{k+2}-a_{k}=a_{k+1}+d$$$$a_{k+3}-a_{k+2}=a_{k+1}+d$$Subtracting them we get, $$2a_{k+2}=a_{k}+a_{k+3}$$To be continued...

Let $b_n=a_n+d$, we get that $b_{n+2}=b_{n+1}+b_n$ and $|b_n-d|$ is prime for all positive integer $n$. It's well-known that for any positive integer $m$, the sequence of pairs $(b_i,b_{i+1})\pmod{m}$ is periodic. So there exists infinitely many positive integer $n$ that $b_n\equiv b_1\pmod{|b_1-d|}\implies |b_1-d|\mid b_n-d$. Hence, there exists infinitely many $n$ that $b_n-d\in \{ |b_1-d|,-|b_1-d|\}$. Not hard to see that this is possible only when $b_n=0$ for all $n$, done.