\underline{Assignment 8}

8.1 Fundamental Matrix

We have the epipolar constraint:

m̃_{2}F m̃_{1} = 0 = ( \begin{matrix}u_{2} \\ v_{2} \\ 1 \end{matrix} )^{⊤} F ( \begin{matrix}u_{1} \\ v_{1} \\ 1 \end{matrix} ) .

Reformulating this, we get

0 = ( \begin{matrix}2û_{2} \\ 2v̂_{2} \\ 1 \end{matrix} )^{⊤} F ( \begin{matrix}2û_{1} \\ 2v̂_{1} \\ 1 \end{matrix} )
= (( \begin{matrix}2 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 1 \end{matrix} ) ( û_{2} ; v̂_{2} ; 1 ))^{⊤} F (( \begin{matrix}2 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 1 \end{matrix} ) ( û_{1} ; v̂_{1} ; 1 ))
= ( û_{2} ; v̂_{2} ; 1 )^{⊤} ( \begin{matrix}2 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 1 \end{matrix} )^{⊤} F ( \begin{matrix}2 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 1 \end{matrix} ) ( û_{1} ; v̂_{1} ; 1 )
= ( û_{2} ; v̂_{2} ; 1 )^{⊤} \underbrace{( \begin{matrix}4f_{11} & 4f_{12} & 2f_{13} \\ 4f_{21} & 4f_{22} & 2f_{23} \\ 2f_{31} & 2f_{32} & f_{33} \end{matrix} )}_{F̂} ( û_{1} ; v̂_{1} ; 1 )

We also know

F = A_{int,2}^{−⊤}[t]_{×}RA_{int,1}^{−1}

with

A_{int} = ( \begin{matrix}k_{u} & −k_{u} cot φ & u_{0} \\ 0 & k_{v}/ sin φ & v_{0} \\ 0 & 0 & 1 \end{matrix} ) .

\underline{Assignment 8} Remark: k_{u} and k_{v} are measured in \frac{pix}{m}. With k̂_{u} = \frac{1}{2}k_{u} and k̂_{v} = \frac{1}{2}k_{v}, we get

Â_{int} = ( \begin{matrix}k̂_{u} & −k̂_{u} cot φ & û_{0} \\ 0 & k̂_{v}/ sin φ & v̂_{0} \\ 0 & 0 & 1 \end{matrix} )
= ( \begin{matrix}\frac{1}{2}k_{u} & −\frac{1}{2}k_{u} cot φ & \frac{1}{2}u_{0} \\ 0 & \frac{1}{2}k_{v}/ sin φ & \frac{1}{2}v_{0} \\ 0 & 0 & 1 \end{matrix} )
= ( \begin{matrix}\frac{1}{2} & 0 & 0 \\ 0 & \frac{1}{2} & 0 \\ 0 & 0 & 1 \end{matrix} ) A_{int}
Â_{int}^{−1} = (( \begin{matrix}\frac{1}{2} & 0 & 0 \\ 0 & \frac{1}{2} & 0 \\ 0 & 0 & 1 \end{matrix} ) A_{int})^{−1}
= A_{int}^{−1} ( \begin{matrix}2 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 1 \end{matrix} )

8.2 Stereo Reconstruction

a. We have

A_{1}^{int} = A_{2}^{int} = ( \begin{matrix}1 & 0 & 10 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} )
(A_{1}^{int})^{−1} = (A_{2}^{int})^{−1} = ( \begin{matrix}1 & 0 & −10 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} )
A_{1}^{ext} = ( \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} )
A_{2}^{ext} = ( \begin{matrix}\frac{1}{\sqrt{2}} & −\frac{1}{\sqrt{2}} & 0 & −4 \\ \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} )

\underline{Assignment 8}

In general, it holds:

A = ( \begin{matrix}R^{3×3} & t^{3} \\ 0^{3^{⊤}} & 1 \end{matrix} )
A^{−1} = ( \begin{matrix}R^{−1} & R^{−1}t \\ 0^{3^{⊤}} & 1 \end{matrix} )

It follows from this fact

(A_{2}^{ext})^{−1} = ( \begin{matrix}\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} & 0 & \frac{4}{\sqrt{2}} \\ −\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} & 0 & −\frac{4}{\sqrt{2}} \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} )

b. The ”roadmap” looks as follows:

3-D world coordinates

↓  extrinsic parameters

3-D camera coordinates

↓  projection step

2-D image coordinates

↓  intrinsic parameters

2-D pixel coordinates

Given 2-D Euclidean coordinates

x_{1} = (10.5, 0.5)^{⊤}
x_{2} = (9.5, \frac{1}{\sqrt{2}})^{⊤}

we have to go to 2-D homogeneous coordinates

x̂_{1} = (10.5, 0.5, 1)^{⊤}
x̂_{2} = (9.5, \frac{1}{\sqrt{2}}, 1)^{⊤}

\underline{Assignment 8}

(A_{1}^{int})^{−1} x̂_{1} = ( \begin{matrix}1 & 0 & −10 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} ) ( \begin{matrix}10.5 \\ 0.5 \\ 1 \end{matrix} )

= ( \begin{matrix}0.5 \\ 0.5 \\ 1 \end{matrix} )

(A_{2}^{int})^{−1} x̂_{2} = ( \begin{matrix}1 & 0 & −10 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} ) ( 9.5 ; \frac{1}{\sqrt{2}} ; 1 )

= ( −0.5 ; \frac{1}{\sqrt{2}} ; 1 )

Let’s go to the 3-D camera coordinates (→ 3-D Euclidean coordinates):

X_{1} = λ_{1} ( \begin{matrix}0.5 \\ 0.5 \\ 1 \end{matrix} ) , λ_{1} ∈ ℝ  ← seen from first camera

X_{2} = λ_{2} ( −0.5 ; \frac{1}{\sqrt{2}} ; 1 ) , λ_{2} ∈ ℝ ← seen from the second camera

Remark: Do not intersect these two points! They live in different coordinate systems!

c. Let’s go to 3-D homogeneous coordinates:

X̂_{1} = (0.5λ_{1}, 0.5λ_{1}, λ_{1}, 1)^{⊤}
X̂_{2} = (−0.5λ_{2}, \frac{1}{\sqrt{2}}λ_{2}, λ_{2}, 1)^{⊤}

Now, we can go to 3-D world coordinates:

(A_{1}^{ext})^{−1} X̂_{1} = ( \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ) ( \begin{matrix}0.5λ_{1} \\ 0.5λ_{1} \\ 1 \end{matrix} )

= ( \begin{matrix}0.5λ_{1} \\ 0.5λ_{1} \\ λ_{1} \\ 1 \end{matrix} )

(A_{2}^{ext})^{−1} X̂_{2} = ( \begin{matrix}\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} & 0 & \frac{4}{\sqrt{2}} \\ −\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} & 0 & −\frac{4}{\sqrt{2}} \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ) ( \begin{matrix}−0.5λ_{2} \\ \frac{1}{\sqrt{2}}λ_{2} \\ λ_{2} \\ 1 \end{matrix} )

= ( \begin{matrix}\frac{2−\sqrt{2}}{4}λ_{2} + 2\sqrt{2} \\ \frac{2+\sqrt{2}}{4}λ_{2} − 2\sqrt{2} \\ λ_{2} \\ 1 \end{matrix} )

d. 3-D Euclidean is now

L_{1} = λ_{1} ( \begin{matrix}0.5 \\ 0.5 \\ 1 \end{matrix} ) , λ_{1} ∈ ℝ

L_{2} = λ_{2} ( \frac{2−\sqrt{2}}{4} ; \frac{2+\sqrt{2}}{4} ; 1 ) + ( \begin{matrix}2\sqrt{2} \\ −2\sqrt{2} \\ 0 \end{matrix} ) , λ_{2} ∈ ℝ

These two lines can now be intersected since they live in the same coordinate system:

λ_{1} ( \begin{matrix}0.5 \\ 0.5 \\ 1 \end{matrix} ) = λ_{2} ( \frac{2−\sqrt{2}}{4} ; \frac{2+\sqrt{2}}{4} ; 1 ) + ( \begin{matrix}2\sqrt{2} \\ −2\sqrt{2} \\ 0 \end{matrix} ) ← tells λ_{1} = λ_{2}

λ_{1} ( \frac{\sqrt{2}}{4} ; −\frac{\sqrt{2}}{4} ; 0 ) = ( \begin{matrix}2\sqrt{2} \\ −2\sqrt{2} \\ 0 \end{matrix} )
⇒ λ_{1} = λ_{2} = 8

M_{w} = (4, 4, 8)^{⊤}: The depth of this point is 8.