The general equation of a circle in the plane is x2+y2+2gx+2fy+c=0, where
center = (−g,−f) and radius r=g2+f2−c.
Because three non-collinear points determine a unique circle, first obtain the circle passing through the points (4,6), (−1,5) and (0,0).
Substituting (4,6) gives
16+36+8g+12f+c=0⇒52+8g+12f+c=0−(1)
Substituting (−1,5) gives
1+25−2g+10f+c=0⇒26−2g+10f+c=0−(2)
Substituting (0,0) gives
c=0−(3)
With c=0, equations (1) and (2) reduce to
52+8g+12f=0−(4)
26−2g+10f=0−(5)
Solve (4) and (5) simultaneously:
Multiply (5) by 4: 104−8g+40f=0.
Add to (4): (52+104)+(12f+40f)=0⇒156+52f=0,
so f=−3.
Insert f=−3 into (4):
52+8g+12(−3)=0⇒52+8g−36=0⇒8g=−16⇒g=−2.
Hence the circle through the three known points is
x2+y2−4x−6y=0.
Its center is (−g,−f)=(2,3) and the radius is
r=g2+f2−c=(−2)2+(−3)2−0=4+9=13,
so r2=13.
Now impose the condition that the fourth point (k,3k) also satisfies the circle equation:
k2+(3k)2−4k−6(3k)=0
⟹k2+9k2−4k−18k=0
⟹10k2−22k=0
⟹2k(5k−11)=0.
Because the four points are distinct, k=0; therefore
5k−11=0⇒k=511.
Finally, evaluate 10k+r2:
10k+r2=10(511)+13=22+13=35.
Hence 10k+r2=35, which corresponds to Option D.