Consider any two rows or any two columns, and suppose their intersections form $\begin{bmatrix} a & b \\ c & d \end{bmatrix}$. Notice that if we do any operation, $a+d-b-c$ is invariant. When all entries are divisible by $n$, $a+d-b-c$ is also divisible by $n$. So $a+d-b-c$ is divisible by infinitely many integers and is thus 0. This applies for any pair of rows and columns. Now add an appropriate amount to each row so that every entry in the first column is 0. Then add an appropriate amount to each column so that every entry in the first row is 0. For the matrix entry $x$ in row $m > 1$, column $n > 1$, considering rows $1$ and $m$ and columns $1$ and $n$ gets the matrix $\begin{bmatrix} 0 & 0 \\ 0 & x \end{bmatrix}$. By the above paragraph, $x = 0$. So in fact the entire matrix is 0.

Here is what I did(similar) : Consider such an $m \times n$ matrix. Call its entries $a_{ij}$. Let $r_i$ be the total number added to the $i-$th row, and $c_j$ be the total number added to the $j-$th row so as to make all of its entries divisible by some really big $n$. Note that $a_{ij} - a_{i1} + a_{m1} - a_{mj} \equiv -(r_i+ c_j - r_i - c_1 + r_m + c_1 - r_m - c_j) \equiv 0 \mod n$; implying that for all $(i,j) \in [m]\times [n]$, we have $a_{ij} - a_{i1} + a_{m1} - a_{mj} = 0$. Now take the matrix,add $- a_{i1}$ to the $i-$th row.So the 1st column becomes zero. Then subtract the number required to make the last row zero. Now all the entries not in 1st column or last row is of the form $a_{ij} - a_{i1} + a_{m1} - a_{mj}$, which is equal to zero too and we are done.