For a system of three linear equations to possess infinitely many solutions, the coefficient matrix A A A must be singular (determinant 0 0 0 ) and the augmented matrix must have the same rank as A A A (that rank will then be 2 2 2 ).
Write the coefficient matrix and its determinant:
A = ∣ 2 λ 3 3 2 − 1 4 5 μ ∣ A=\begin{vmatrix}2 & \lambda & 3\\ 3 & 2 & -1\\ 4 & 5 & \mu\end{vmatrix} A = 2 3 4 λ 2 5 3 − 1 μ
Expanding along the first row,
det A = 2 ∣ 2 − 1 5 μ ∣ − λ ∣ 3 − 1 4 μ ∣ + 3 ∣ 3 2 4 5 ∣ = 2 ( 2 μ + 5 ) − λ ( 3 μ + 4 ) + 3 ( 15 − 8 ) = 4 μ + 10 − λ ( 3 μ + 4 ) + 21 = 4 μ + 31 − λ ( 3 μ + 4 ) . \begin{aligned}
\det A &= 2\begin{vmatrix}2 & -1\\ 5 & \mu\end{vmatrix}
-\lambda\begin{vmatrix}3 & -1\\ 4 & \mu\end{vmatrix}
+3\begin{vmatrix}3 & 2\\ 4 & 5\end{vmatrix}\\[4pt]
&= 2(2\mu+5)-\lambda(3\mu+4)+3(15-8)\\[4pt]
&= 4\mu+10-\lambda(3\mu+4)+21\\[4pt]
&= 4\mu+31-\lambda(3\mu+4).
\end{aligned} det A = 2 2 5 − 1 μ − λ 3 4 − 1 μ + 3 3 4 2 5 = 2 ( 2 μ + 5 ) − λ ( 3 μ + 4 ) + 3 ( 15 − 8 ) = 4 μ + 10 − λ ( 3 μ + 4 ) + 21 = 4 μ + 31 − λ ( 3 μ + 4 ) .
Setting det A = 0 \det A = 0 det A = 0 gives
λ ( 3 μ + 4 ) = 4 μ + 31 − ( 1 ) \lambda(3\mu+4)=4\mu+31 \quad -(1) λ ( 3 μ + 4 ) = 4 μ + 31 − ( 1 )
Because det A = 0 \det A = 0 det A = 0 , the three rows are linearly dependent. Assume the third row is a linear combination of the first two:
R 3 = p R 1 + q R 2 R_3 = pR_1 + qR_2 R 3 = p R 1 + q R 2
This yields four scalar equations (for the coefficients of x , y , z x,\,y,\,z x , y , z and the constants):
2 p + 3 q = 4 − ( 2 ) λ p + 2 q = 5 − ( 3 ) 3 p − q = μ − ( 4 ) 5 p + 7 q = 9 − ( 5 ) \begin{aligned}
2p+3q &= 4 \quad -(2)\\
\lambda p + 2q &= 5 \quad -(3)\\
3p - q &= \mu \quad -(4)\\
5p + 7q &= 9 \quad -(5)
\end{aligned} 2 p + 3 q λ p + 2 q 3 p − q 5 p + 7 q = 4 − ( 2 ) = 5 − ( 3 ) = μ − ( 4 ) = 9 − ( 5 )
Solve equations ( 2 ) (2) ( 2 ) and ( 3 ) (3) ( 3 ) for p , q p,\,q p , q .
From ( 2 ) : (2): ( 2 ) : p = 4 − 3 q 2 . p=\dfrac{4-3q}{2}. p = 2 4 − 3 q .
Insert this into ( 3 ) : (3): ( 3 ) :
5 = λ ( 4 − 3 q 2 ) + 2 q ⟹ ( − 3 λ + 4 ) q = 10 − 4 λ . 5 = \lambda\left(\dfrac{4-3q}{2}\right)+2q
\;\;\Longrightarrow\;\;
(-3\lambda+4)q = 10-4\lambda. 5 = λ ( 2 4 − 3 q ) + 2 q ⟹ ( − 3 λ + 4 ) q = 10 − 4 λ .
Set D = 4 − 3 λ D = 4-3\lambda D = 4 − 3 λ (note D ≠ 0 D\neq0 D = 0 for rank 2 2 2 ). Then
q = 10 − 4 λ D , p = − 7 D . q=\dfrac{10-4\lambda}{D},\qquad
p=\dfrac{-7}{D}. q = D 10 − 4 λ , p = D − 7 .
Check the constant equation ( 5 ) (5) ( 5 ) for consistency:
5 p + 7 q = 5 ( − 7 D ) + 7 ( 10 − 4 λ D ) = 35 − 28 λ D . 5p+7q \;=\; 5\!\left(\dfrac{-7}{D}\right)+7\!\left(\dfrac{10-4\lambda}{D}\right)
= \dfrac{35-28\lambda}{D}. 5 p + 7 q = 5 ( D − 7 ) + 7 ( D 10 − 4 λ ) = D 35 − 28 λ .
For infinite solutions this must equal 9 9 9 :
35 − 28 λ D = 9 ⟹ 35 − 28 λ = 9 ( 4 − 3 λ ) = 36 − 27 λ . \dfrac{35-28\lambda}{D}=9
\;\;\Longrightarrow\;\;
35-28\lambda = 9(4-3\lambda)
= 36-27\lambda. D 35 − 28 λ = 9 ⟹ 35 − 28 λ = 9 ( 4 − 3 λ ) = 36 − 27 λ .
Simplifying gives λ = − 1. \lambda = -1. λ = − 1.
Now substitute λ = − 1 \lambda=-1 λ = − 1 into ( 1 ) (1) ( 1 ) to find μ \mu μ :
( − 1 ) ( 3 μ + 4 ) = 4 μ + 31 ⟹ − 3 μ − 4 = 4 μ + 31 ⟹ 7 μ = − 35 ⟹ μ = − 5. (-1)(3\mu+4)=4\mu+31
\;\;\Longrightarrow\;\;
-3\mu-4 = 4\mu+31
\;\;\Longrightarrow\;\;
7\mu = -35
\;\;\Longrightarrow\;\;
\mu = -5. ( − 1 ) ( 3 μ + 4 ) = 4 μ + 31 ⟹ − 3 μ − 4 = 4 μ + 31 ⟹ 7 μ = − 35 ⟹ μ = − 5.
Finally,
( λ 2 + μ 2 ) = ( − 1 ) 2 + ( − 5 ) 2 = 1 + 25 = 26. (\lambda^2+\mu^2)=(-1)^2+(-5)^2 = 1+25 = 26. ( λ 2 + μ 2 ) = ( − 1 ) 2 + ( − 5 ) 2 = 1 + 25 = 26.
Hence the required value is 26 26 26 , corresponding to Option C.