|
| [+QUICK
INFO+] |
|
| TEAM
MEMBERS : |
|
Kenny
Teng
Jeremy
Ng
Shirlene
Lim |
| ASSIGNMENT
: |
|
Write
a program that is able to read in a video stream of
any kind, generate the 3-D model of the objects in the
video feed and introduce a virtual object into the movie. |
| SPECIFICATIONS
: |
|
The
system will be written using Matlab. It should not take
a long time to generate the edited movie with the augmented
reality in it.
|
| RESULT: |
|
Got
an A in the class |
| |
|
|
| [+OVERVIEW+] |
|
This
flowchart shows the steps used in callibrating and
editing of the video stream to create the final movie
with augmented reality. The input data is an uncompressed
avi movie that has been taken with a digital camera.
|
 |
For
the first part of the Augmented Reality system, strong
interests points are detected on the first frame of
the movie sequence using a Harris detector. Once the
feature points are collected, they are tracked across
the whole frame sequence using a Lucas-Kanade motion
tracker. Here is an example of a set of points that
have been detected and tracked through 20 frames. The
red tracks on the images are the tracked points.
|
 |
| Using
the tracked points, the camera positions and locations
can be tracked too at the same time. A recursive method
is used to project the points from each frame into a
3-D space and continuously try to adjust the positions
of every single point so that they eventually lie at
a position where they give this 3-D impression. In this
sense, the matrices M for the motion of the camera,
and S for the 3-D shape that the tracked feature points
generate, can both be used to generate the 3-D reconstruction
of the hotel. This 3-D plot has been done in Matlab,
and I manually drew a red outline on the edge of the
points so that the shape can be easily seen. You can
clearly distinguish the 3-D shape of the holel. This
3-D reconstruction is NOT positioned at the exact 'real'
location of the object in euclidean coordinates of the
world. The positions of the tracked features are accurate
relative to each other only, to give this idea of a
3-D object. |
From
the 3-D reconstruction of the hotel, it is easy to
select the location where the augmented object is going to
be introduced. In this case, I decided to draw a symemtric
box, for the sake of simplicity and ease of computation. The position, shape
and size of the box, has been calculated using simple vector computation. Once,
the augmented object has been introduced on the 3-D
model, the inverse mapping that was used to create
the 3-D image is used to transfer the augmented object
on every single frame of the video sequence.
|
3-D
Reconstruction of tracked features |
| Below
is an example of 1 frame containing the augmented
object that has been mapped back from the 3-D model
to the corresponding frame, using the respective
mapping matrix associated with this frame. |
 |
 |
Once
the mapping process is done, the whole augmented movie
can be played. Below are couple of examples that have
been generated using the Matlab codes. On the left,
you can view the movie file used as the input, completely
uncallibrated and unedited. The right version is the
final one with the augmented object.
|
Unedited
Movie
Augmented Movie
 |
|
