We are given A=[21−10] and S={m∈Z:Am2+Am=3I−A−6}.
The characteristic equation of A: det(A−λI)=(2−λ)(0−λ)−(−1)(1)=λ2−2λ+1=(λ−1)2=0.
By the Cayley-Hamilton theorem: (A−I)2=0, so A2−2A+I=0, meaning A2=2A−I.
Since (A−I)2=0, let N=A−I, so N2=0. Then A=I+N.
An=(I+N)n=I+nN=I+n(A−I)=(1−n)I+nA
Let us verify: A2=−I+2A=2A−I. This matches A2=2A−I. Good.
So An=(1−n)I+nA for all integers n.
A−6=(1−(−6))I+(−6)A=7I−6A
Am2+Am=3I−A−6
[(1−m2)I+m2A]+[(1−m)I+mA]=3I−(7I−6A)
[(1−m2)+(1−m)]I+[m2+m]A=−4I+6A
[2−m2−m]I+[m2+m]A=−4I+6A
Since I and A are linearly independent (as A=kI):
Coefficient of A: m2+m=6, so m2+m−6=0, giving (m+3)(m−2)=0, so m=−3 or m=2.
Coefficient of I: 2−m2−m=−4, so m2+m=6. This gives the same equation.
Both conditions are identical, so m∈{−3,2} and n(S)=2.
The answer is 2.