Given,
A5=B5
and
A3B2=A2B3
Also,
A2−B2
is invertible.
From
A3B2=A2B3
we get
A2B2(A−B)=0
Now,
A5−B5=0
Factorizing,
(A−B)(A4+A3B+A2B2+AB3+B4)=0
Since powers of matrices commute here, write
A4+A3B+A2B2+AB3+B4
as
(A2+B2)(A2+AB+B2)−A2B2
Using
A2B2(A−B)=0,
we get
A2B2=0
Hence,
A5−B5=(A−B)(A2+B2)(A2+AB+B2)=0
But
A2−B2=(A−B)(A+B)
is invertible.
Therefore,
A−B
is invertible.
Hence from
(A−B)(A4+A3B+A2B2+AB3+B4)=0
we get
A4+A3B+A2B2+AB3+B4=0
Now multiply by
A+B
Using algebraic identity,
A5+B5=0
But given
A5=B5
Therefore,
2A5=0
A5=0
Hence,
B5=0
Thus both A and B are nilpotent matrices.
Therefore,
A3+B3
is also nilpotent.
Determinant of a nilpotent matrix is always zero.
Hence,
0