We are asked to find that member of the family of planes which (i) passes through the line of intersection of the two given planes and (ii) is at the specified distance 2 21 \sqrt{\dfrac{2}{21}} 21 2 from the origin.
First we recall that the equation of the family of planes containing the intersection of the two planes
x − y − z − 1 = 0 and 2 x + y − 3 z + 4 = 0 x - y - z - 1 = 0 \quad\text{and}\quad 2x + y - 3z + 4 = 0 x − y − z − 1 = 0 and 2 x + y − 3 z + 4 = 0
is written by adding an arbitrary scalar multiple λ \lambda λ of the second equation to the first. Thus we have
( x − y − z − 1 ) + λ ( 2 x + y − 3 z + 4 ) = 0. \bigl(x - y - z - 1\bigr) + \lambda\bigl(2x + y - 3z + 4\bigr)=0. ( x − y − z − 1 ) + λ ( 2 x + y − 3 z + 4 ) = 0.
Simplifying term by term we obtain
( 1 + 2 λ ) x + ( − 1 + λ ) y + ( − 1 − 3 λ ) z + ( − 1 + 4 λ ) = 0 , or A x + B y + C z + D = 0 , \begin{aligned}
(1+2\lambda)\,x \;+\; (-1+\lambda)\,y \;+\; (-1-3\lambda)\,z \;+\; (-1+4\lambda) &= 0,\\[4pt]
\text{or}\qquad A x + B y + C z + D &= 0,
\end{aligned} ( 1 + 2 λ ) x + ( − 1 + λ ) y + ( − 1 − 3 λ ) z + ( − 1 + 4 λ ) or A x + B y + C z + D = 0 , = 0 ,
where
A = 1 + 2 λ A = 1+2\lambda A = 1 + 2 λ , B = − 1 + λ \qquad B = -1+\lambda B = − 1 + λ , C = − 1 − 3 λ \qquad C = -1-3\lambda C = − 1 − 3 λ , D = − 1 + 4 λ . \qquad D = -1+4\lambda. D = − 1 + 4 λ .
Next we use the distance formula. For a plane A x + B y + C z + D = 0 Ax + By + Cz + D = 0 A x + B y + C z + D = 0 , the perpendicular distance of the origin ( 0 , 0 , 0 ) (0,0,0) ( 0 , 0 , 0 ) from the plane is
Distance = ∣ D ∣ A 2 + B 2 + C 2 . \;=\; \frac{|D|}{\sqrt{A^{2}+B^{2}+C^{2}}}. = A 2 + B 2 + C 2 ∣ D ∣ .
We are told that this distance equals 2 21 \sqrt{\dfrac{2}{21}} 21 2 . Therefore
∣ D ∣ A 2 + B 2 + C 2 = 2 21 . \frac{|D|}{\sqrt{A^{2}+B^{2}+C^{2}}} \;=\; \sqrt{\dfrac{2}{21}}. A 2 + B 2 + C 2 ∣ D ∣ = 21 2 .
Because the distance is non-negative, we square both sides to remove the absolute value without sign ambiguity and obtain
D 2 A 2 + B 2 + C 2 = 2 21 , \frac{D^{2}}{A^{2}+B^{2}+C^{2}} \;=\; \frac{2}{21}, A 2 + B 2 + C 2 D 2 = 21 2 ,
⟹ 21 D 2 = 2 ( A 2 + B 2 + C 2 ) . \Longrightarrow\; 21D^{2} \;=\; 2\bigl(A^{2}+B^{2}+C^{2}\bigr). ⟹ 21 D 2 = 2 ( A 2 + B 2 + C 2 ) .
We now compute the needed squares.
A 2 = ( 1 + 2 λ ) 2 = 1 + 4 λ + 4 λ 2 , B 2 = ( − 1 + λ ) 2 = 1 − 2 λ + λ 2 , C 2 = ( − 1 − 3 λ ) 2 = 1 + 6 λ + 9 λ 2 . \begin{aligned}
A^{2} &= (1+2\lambda)^{2} = 1 + 4\lambda + 4\lambda^{2},\\[4pt]
B^{2} &= (-1+\lambda)^{2} = 1 - 2\lambda + \lambda^{2},\\[4pt]
C^{2} &= (-1-3\lambda)^{2} = 1 + 6\lambda + 9\lambda^{2}.
\end{aligned} A 2 B 2 C 2 = ( 1 + 2 λ ) 2 = 1 + 4 λ + 4 λ 2 , = ( − 1 + λ ) 2 = 1 − 2 λ + λ 2 , = ( − 1 − 3 λ ) 2 = 1 + 6 λ + 9 λ 2 .
Adding these three results term by term gives
A 2 + B 2 + C 2 = 3 + 8 λ + 14 λ 2 . A^{2}+B^{2}+C^{2} = 3 + 8\lambda + 14\lambda^{2}. A 2 + B 2 + C 2 = 3 + 8 λ + 14 λ 2 .
Next we square D = − 1 + 4 λ D = -1 + 4\lambda D = − 1 + 4 λ to get
D 2 = ( − 1 + 4 λ ) 2 = 1 − 8 λ + 16 λ 2 . D^{2} = (-1+4\lambda)^{2} = 1 - 8\lambda + 16\lambda^{2}. D 2 = ( − 1 + 4 λ ) 2 = 1 − 8 λ + 16 λ 2 .
Substituting A 2 + B 2 + C 2 A^{2}+B^{2}+C^{2} A 2 + B 2 + C 2 and D 2 D^{2} D 2 into the equation
21 D 2 = 2 ( A 2 + B 2 + C 2 ) 21D^{2} = 2(A^{2}+B^{2}+C^{2}) 21 D 2 = 2 ( A 2 + B 2 + C 2 ) we have
21 ( 1 − 8 λ + 16 λ 2 ) = 2 ( 3 + 8 λ + 14 λ 2 ) . 21\bigl(1 - 8\lambda + 16\lambda^{2}\bigr) \;=\; 2\bigl(3 + 8\lambda + 14\lambda^{2}\bigr). 21 ( 1 − 8 λ + 16 λ 2 ) = 2 ( 3 + 8 λ + 14 λ 2 ) .
Expanding both sides gives
21 − 168 λ + 336 λ 2 = 6 + 16 λ + 28 λ 2 . 21 - 168\lambda + 336\lambda^{2} \;=\; 6 + 16\lambda + 28\lambda^{2}. 21 − 168 λ + 336 λ 2 = 6 + 16 λ + 28 λ 2 .
Moving every term to the left we find
21 − 168 λ + 336 λ 2 − 6 − 16 λ − 28 λ 2 = 0 , 21 - 168\lambda + 336\lambda^{2} - 6 - 16\lambda - 28\lambda^{2} = 0, 21 − 168 λ + 336 λ 2 − 6 − 16 λ − 28 λ 2 = 0 ,
⟹ 15 − 184 λ + 308 λ 2 = 0. \Longrightarrow\; 15 - 184\lambda + 308\lambda^{2} = 0. ⟹ 15 − 184 λ + 308 λ 2 = 0.
This is a quadratic equation in λ \lambda λ :
308 λ 2 − 184 λ + 15 = 0. 308\lambda^{2} - 184\lambda + 15 = 0. 308 λ 2 − 184 λ + 15 = 0.
We compute its discriminant:
Δ = ( − 184 ) 2 − 4 ⋅ 308 ⋅ 15 = 33856 − 18480 = 15376. \Delta = (-184)^{2} - 4\cdot 308 \cdot 15 = 33856 - 18480 = 15376. Δ = ( − 184 ) 2 − 4 ⋅ 308 ⋅ 15 = 33856 − 18480 = 15376.
Because 15376 = 16 × 961 = ( 4 × 31 ) 2 , 15376 = 16 \times 961 = (4\times 31)^{2}, 15376 = 16 × 961 = ( 4 × 31 ) 2 , we have Δ = 124. \sqrt{\Delta} = 124. Δ = 124. Therefore
λ = 184 ± 124 2 ⋅ 308 = 184 ± 124 616 . \lambda \;=\; \frac{184 \pm 124}{2\cdot 308} = \frac{184 \pm 124}{616}. λ = 2 ⋅ 308 184 ± 124 = 616 184 ± 124 .
Hence the two possible values of λ \lambda λ are
λ 1 = 184 + 124 616 = 308 616 = 1 2 , λ 2 = 184 − 124 616 = 60 616 = 15 154 . \lambda_{1} = \frac{184 + 124}{616} = \frac{308}{616} = \frac{1}{2}, \qquad
\lambda_{2} = \frac{184 - 124}{616} = \frac{60}{616} = \frac{15}{154}. λ 1 = 616 184 + 124 = 616 308 = 2 1 , λ 2 = 616 184 − 124 = 616 60 = 154 15 .
We now write the equation of the plane for each value.
For λ = 1 2 \lambda = \dfrac12 λ = 2 1 :
A = 1 + 2 ( 1 2 ) = 1 + 1 = 2 , B = − 1 + 1 2 = − 1 2 , C = − 1 − 3 ( 1 2 ) = − 1 − 3 2 = − 5 2 , D = − 1 + 4 ( 1 2 ) = − 1 + 2 = 1. \begin{aligned}
A &= 1 + 2\!\left(\tfrac12\right) = 1 + 1 = 2,\\
B &= -1 + \tfrac12 = -\tfrac12,\\
C &= -1 - 3\!\left(\tfrac12\right) = -1 - \tfrac32 = -\tfrac52,\\
D &= -1 + 4\!\left(\tfrac12\right) = -1 + 2 = 1.
\end{aligned} A B C D = 1 + 2 ( 2 1 ) = 1 + 1 = 2 , = − 1 + 2 1 = − 2 1 , = − 1 − 3 ( 2 1 ) = − 1 − 2 3 = − 2 5 , = − 1 + 4 ( 2 1 ) = − 1 + 2 = 1.
The corresponding plane is therefore
2 x − 1 2 y − 5 2 z + 1 = 0. 2x - \tfrac12\,y - \tfrac52\,z + 1 = 0. 2 x − 2 1 y − 2 5 z + 1 = 0.
To clear the fractions we multiply the entire equation by 2:
4 x − y − 5 z + 2 = 0. 4x - y - 5z + 2 = 0. 4 x − y − 5 z + 2 = 0.
For λ = 15 154 \lambda = \dfrac{15}{154} λ = 154 15 we would obtain another plane, but that plane does not match any of the given options, whereas the plane obtained above exactly coincides with Option 4.
Thus the required plane is
4 x − y − 5 z + 2 = 0. 4x - y - 5z + 2 = 0. 4 x − y − 5 z + 2 = 0.
Hence, the correct answer is Option 4.