We need the coefficient of x256 in (1−x)101(x2+x+1)100.
Observe the factorisation 1−x3=(1−x)(x2+x+1), so (x2+x+1)100=(1−x)100(1−x3)100.
Therefore the expression becomes:
(1−x)101⋅(1−x)100(1−x3)100=(1−x)(1−x3)100.
We need the coefficient of x256 in (1−x)(1−x3)100, which equals
[coeff of x256 in (1−x3)100]−[coeff of x255 in (1−x3)100].
The general term in (1−x3)100 is (k100)(−1)kx3k, so only powers that are multiples of 3 appear.
Since 256 is not a multiple of 3, the coefficient of x256 in (1−x3)100 is 0.
Since 255=3×85, the coefficient of x255 in (1−x3)100 is (85100)(−1)85=−(85100)=−(15100).
Therefore the coefficient of x256 in the original expression is
0−(−(15100))=(15100).