a). Since $rank(A^2B)=rank(A)\geq rank(A^2)\geq rank(A^2B)$, we have $rank(A)=rank(A^2)$. Then we can show that $\mathbb{C}^n=\ker A\oplus Im A$. Choose a basis of $\ker A$ and $Im A$, we may assume that $A$ is of the form $\begin{pmatrix}0&0\\0&A_1\end{pmatrix}$ where $A_1$ is invertible. Suppose that $B=\begin{pmatrix}B_{11}&B_{12}\\B_{21}&B_{22}\end{pmatrix}$, then by $A^2B=A$, we get $B_{21}=0$ and $B_{22}=A_1^{-1}$. Then $AB-BA=\begin{pmatrix}0&-B_{12}A_1\\0&0\end{pmatrix}$, so $(AB-BA)^2=0$. b). Let $A=\begin{pmatrix}0&0\\0&I_k\end{pmatrix}$ and $B=\begin{pmatrix}0&B_{12}\\0&I_k\end{pmatrix}$ where $rank(B_{12})=k$. Then we have $A^2B=A$ and $rank(AB-BA)=rank\begin{pmatrix}0&-B_{12}\\0&0\end{pmatrix}=k$.