|
| [+QUICK
INFO+] |
|
| TEAM
MEMBERS : |
|
Kenny
Teng |
| ASSIGNMENT
: |
|
Investigate if there
are any discriminating features in the AsymmetryFaces
when applied to wavelet transforms.
Verify if classification rates can be improved
by using those wavelet transforms of the AsymmetryFaces. |
| SPECIFICATIONS
: |
|
The
system has been prototyped on Matlab. |
| PUBLICATION : |
|
This research has been
published in Robotics
Institute Technical Reports 2006.
Download paper [pdf] |
| |
|
|
|
| [+OVERVIEW+] |
| Facial
attractiveness has always been closely related to
the approximate bilateral symmetry of human faces.
According to Thornhill et al, asymmetrical
faces are considered less attractive. Psychologists
and anthropologists have considered facial asymmetry
as a critical factor that can be used to evaluate
attractiveness and expressions, even though
most of it was done qualitatively using human observers.
For the first time, Liu et al have
defined a way to quantify facial asymmetry in frontal
human faces and showed that they contain discriminating
information for human identification under expression
variations.
Wavelet features may contain
richer and more discriminative information for expression
classification than spatial asymmetry features alone.
The theory of wavelet analysis has been well studied
and this is done by dividing the time-frequency
plane into regions with different resolutions. Wavelet
packets have proved to be very practical in applications
where time-frequency or space-frequency resolutions
are needed. Hennings et al have
successfully used wavelet transforms to significantly
improve identification and classification rates on
fingerprints. They showed that wavelet subspaces
contain features that are more pronounced for higher
accuracy in recognizing fingerprints.
This is why we believe that wavelet analysis can be
advantageous to classifiers applied to asymmetry faces.
Different from previous work, we want to investigate
whether asymmetry faces have consistent features that
are retained in the wavelet subspaces for better classification
of expressions. We will refer to those AsymmetryFaces
in wavelet domain as Wavelet
AsymmetryFaces. |
| [+FACIAL
ASYMMETRY+] |
 |
| The
above examples of faces are just to show you how different
each half of our faces are. The pictures on the left
are the original faces of the man and woman. The pictures
in the middle are the left half of each face and then
mirror-imaged to try to make an entire symmetrical
face. A perfectly symmetrical face can be more appealing
to many, as you can see in those pictures. As for the
pictures on the righh, the same thing was done but
this time using the right half of the face. Clearly,
the two halves of the face are not symmetrical at all
and this gives rise to basically 2 different persons. |
| [+QUANTIFICATION
OF ASYMMETRY+] |
We
are going to be using the same asymmetry faces defined
by Liu et
al in Facial
Asymmetry Quantification for Expression Invariant
Human Identification.
Below are examples of the D-face and S-face images
defined by Liu. The D-face is basically a difference
between an intensity image and its vertically inverted
image, while the S-face is a measure of the angle
between the edges of an intensity image and its vertically
inverted version. The results are two distinct images
that represent the level of asymmetry in an image. |
 |
| D-face represents
left-right relative intensity variation while S-face is
affected by the zero-crossings of the intensity
field. A high value on a D-face means
that the face is very asymmetrical. In contrast,
a symmetrical face will yield a high value of the S-face .
From the way those facial asymmetry faces are constructed,
the two halves of the D-face are opposite
to each other while those of the S-face are
symmetric. Therefore, only half of the D-face and
S-face are retained for processing, and six projections
can then be defined as in [7], namely D ,
D x , D y , S , S x , S y and
they are called AsymmetryFaces . |
Faces |
Description |
Features |
| D |
Top 60 Eigen vectors of D-face |
60 |
| Dx |
Column-mean of D-face on
X-axis |
64 |
| Dy |
Row-mean of D-face on
Y-axis |
128 |
| S |
Top 100 Eigen vectors of S-face |
100 |
| Sx |
Column-mean of S-face on
X-axis |
64 |
| Sy |
Row-mean of S-face on
Y-axis |
128 |
|
| [+WAVELET
TRANSFORMATIONS+] |
Wavelet
transformations are a method of representing signals
across space and frequency. The signal is divided
across several layers of division in space and frequency
and then analyzed. The goal is to determine which
space/frequency bands contain the most information
about an image's unique features, both the parts
that define an image as a particular type (fingerprint,
face, etc.) and those parts which aid in classification
between different images of the same type.
One type of discrete wavelet transform (DWT) is
the orthogonal DWT. The orthogonal DWT projects an
image onto a set of orthogonal column vectors to
break the image down into coarse and fine features. |
 |
 |
| A
typical two-level full wavelet decomposition is shown
above. Each of the subspaces is obtained by taking
the original input and filtering it with a combination
of high-pass and low-pass filters, designed to maximize
the amount of information obtained within each subspace.
This decomposition can be repeated for n-levels. The
image can later be reconstructed from these subspaces.
By removing subspaces that contain comparatively low
amounts of information from a reconstruction, we can
achieve an image that is nearly as good as the original
but takes less space to store. This can be useful if
we are storing a large number of similar images.
The figure below shows examples of D-face and S-face under
full wavelet decomposition of level 3. |
|
 |
| [+FISHERFACE
AS A CLASSIFIER+] |
There
are many classifiers out there that have proved to
be very effective for classification of images. One
of the most popular classifier used specifically
in face recognition is Fisher Linear Discriminant
Analysis (FLDA) or Fisherface [15]. The latter is
a popular tool for multi-class pattern recognition
as it takes advantage of the class scatter to make
classification more reliable. The way Fisherface
works is that it is a cascading of transformations
of Principal Component Analysis (PCA) and Linear
Discriminant Analysis (LDA). PCA is used for dimensionality
reduction on the input data and LDA is then performed
in the reduced dimensional space. LDA is the maximization
of the ratio of discriminant of the projected between-class
scatter matrix, Sb, to the determinant of the within-class
scatter matrix, Sw. |
| [+PROPOSED
WORK+] |
 |
One dataset is used to test the
proposed method, which is a subset of the Cohn-Kanade
AU-Coded Facial Expression Database.
Each face image from the original database is subjected to an affine transformation
using three points: left and right inner canthi and the philtrum. Each of those
normalized faces is then cropped to 128 x 128 pixels. A subset of the normalized
database is extracted to create the main test bed.
|
|
To
test the effectiveness of Wavelet AsymmetryFaces ,
the normalized data set of peak faces from the Cohn-Kanade
AU-Coded Facial Expression Video Sequence Database
is going to be used. The data set consists of 3 classes
(joy, disgust and anger) with 55 subjects each. The
experimental setup is as follows. The training set
consists of 30 randomly selected faces from each
class, therefore leaving 25 faces per class for testing
purposes. Each classification test will be computed
over 20 repetitions and statistical results are going
to be recorded. |
| [+EXPERIMENTAL
RESULTS+] |
Overall,
the average false negative rates (FNR) for all the
Wavelet AsymmetryFaces and FF+AF decrease
compared to those without wavelet transforms. The
Error Improvement Rate (EIR) defined as (%Error
spatial - %Error wavelet )/%Error spatial x
100% , of S faces (S, S x
and S y ) ranges between 6.1% (Sx for disgust) and
38.6% (S for disgust). However, the biggest change
can be seen on the D faces (D, D x and D y ) where
the improvement of the average FNR ranges between
49.6% (D x for anger) and 86.2% (D for anger).
Similarly, the EIR of average false positive rates
(FPR) show major improvements on the Wavelet AsymmetryFaces.
Again, the D faces have the biggest overall improvement
with a range between 42.9% (D y for anger) and 93.4%
(D for joy). As for S faces, the EIR is smaller between
3.3% (S x for disgust) and 55.9% (S for joy). These
results clearly show that Wavelet AsymmetryFaces
have discriminative features that can help at classifying
classes more accurately with reduced FNR and FRP.
The most significant improvements can be seen for
Joy, which reaches a maximum improvement of roughly
93%.
As for the standard deviation of both the results
for FNR and FPR, there are improvements mostly for
D faces, but S faces tend to show some negative results
in the Wavelet AsymmetryFaces. Figures 6(b) and 7(b)
show how there are both improvements and deterioration
in the standard deviation in the classification of
the 3 expressions. The worst case that happens is
with the standard deviation of FNR of S x face on
anger. The standard deviation of that Wavelet AsymmetryFace
gets worse by 66.4% as it increases from 9.6% to
15.9% in the classification of Sx Wavelet
AsymmetryFace. Again, this shows that wavelet packets
help to retrieve features mostly in the D faces and
that the S faces do not contain any more features
to extract even in the wavelet domains.
Finally, the overall classification rates of Wavelet
AsymmetryFaces compared to AsymmetryFaces are 91.3%
and 86.6% respectively. |
 |
 |
For verification
evaluation of our method, a receiver operating characteristics
(ROC) curve has been plotted to show the probabilities
of authentic versus the probabilities of impostors.
From the following graph, it is clear that our optimized
tree achieved better results (blue line) than the
control test (red line). The probability of having
an authentic verification with zero probability of
having an impostor is about 83% for wavelet-domain
method and 67% for the standard filters.
As for classification
results, the error rates have been plotted for every
single class as shown in figure 5. There is a clear
reduction in classification error rate for the wavelet-domain
filters (red) compared to the standard correlation
filters. For class 4 where the error rate for the control test reaches 70%, it
is still hard for even the wavelet-domain filters to classify efficiently, even
though they yielded a better rate of 65%. However, for classes like 13 and 17
where the standard method had error in the order of 35% and 10% respectively,
the wavelet-domain method achieves a zero error rate. For the overall classification
rate, our wavelet-domain method yields 94.2% accuracy while the standard method
achieves 90%. |
| [+CONCLUSIONS+] |
| Previous
work has already shown that AsymmetryFaces combined
with standard classifier (Fisherface) have proved
to be very effective at classifying expressions (joy,
anger and disgust). The most
important finding in this study is that Wavelet AsymmetryFaces
can further improve classification of expressions.
Wavelet AsymmetryFaces have discriminative features
that are more prominent in certain subspaces that
can enhance discrimination between the different
facial expressions.
In this work, we have successfully investigated
the implications of wavelet transforms on AsymmetryFaces.
We have demonstrated that (1) by applying wavelet
transforms on D-faces , a significant improvement
can be achieved; (2) certain subspaces
of a wavelet tree have even more discriminative features
compared to others, for instance higher frequency
band (LH and HL from Table 4) of the wavelet tree;
(3) the way S-faces are constructed, their
image-intensity domain is already the optimal space
with maximum discriminative features.
Wavelet transforms are definitely useful at extracting
features that can be used to improve classification
rates. They constitute an adaptive system that can
be optimized for a specific purpose, and in this
case classification of expression. Future work may
involve the development of a pruning algorithm that
can optimize the automatic selection of wavelet subspaces
that will only contribute positively to the overall
classification rate of expression. |
Find the paper
for this research below.
Paper |
|
