Let us write x−1=h so that x→1 is equivalent to h→0. Re-expressing everything in terms of the new variable h gives
limh→0hsinh∫0h2tcost2dt.
We now approximate the numerator. For small arguments we may use the Maclaurin series cosz=1−2z2+O(z4). Putting z=t2 we obtain
cost2=1−2(t2)2+O(t8)=1−2t4+O(t8).
Hence
tcost2=t−2t5+O(t9).
We integrate this term by term from 0 to h2:
∫0h2tdt=[2t2]0h2=2(h2)2=2h4,
∫0h2−2t5dt=−21[6t6]0h2=−12(h2)6=−12h12,
and the next terms are of still higher order. Therefore
∫0h2tcost2dt=2h4+O(h12).
Now we study the denominator. The standard Maclaurin expansion for sine is
sinh=h−6h3+O(h5).
Hence
hsinh=h(h−6h3+O(h5))=h2−6h4+O(h6).
Putting the two approximations together, our limit becomes
limh→0h2−6h4+O(h6)2h4+O(h12).
First we divide numerator and denominator by h2:
=limh→01−6h2+O(h4)2h2+O(h10).
Next, using the algebraic identity 1−a1=1+a+O(a2) valid for small a, and taking a=6h2+O(h4), we have
1−6h2+O(h4)1=1+6h2+O(h4).
Multiplying, we get
(2h2+O(h10))(1+6h2+O(h4))=2h2+O(h4).
As h→0, the expression 2h2+O(h4) clearly tends to 0. Therefore the desired limit is 0.
Hence, the correct answer is Option 4.