Synthesis of human facial expressions based on the distribution of elastic force applied by control points

Facial expressions play an essential role in delivering emotions. Thus facial expression synthesis gain interests in many fields such as computer vision and graphics. Facial actions are generated by contraction and relaxation of the muscles innervated by facial nerves. The combination of those muscle motions is numerous. therefore, facial expressions are often person specific. But in general, facial expressions can be divided into six groups: anger, disgust, fear, happiness, sadness, and surprise. Facial expression variations include both global facial feature motions (e.g. opening or closing of eyes or mouth) and local appearance deformations (e.g. facial wrinkles and furrows).
Ghent and McDonald introduced the Facial Expression Shape model and Facial Expression Texture Model respectively for the synthesizing global and local changes. Zhang et al. published an elastic model to balance the local and global warping. Then, they added suitable illumination details to the warped face image with muscle-distribution-based model.
The goal of facial expression synthesis is to create expressional face image of the subject with the availability of neutral face image of that subject.
This paper proposes a new method for synthesis of human facial expressions, in which an elastic force is defined to simulate the displacement of facial points in various emotional expressions. The basis of this force is the presence of control points with specific coordinates and directions on the face image. In other words, each control point applies an elastic force into the points of the face and moves them in a certain direction. The force applied to each point is inversely proportional to the distance between that point and the control point. For several control points, the force applied to the points of the face is the result of the forces associated with all control points. To synthesize a specific expression, the location of the control points and parameters of the force are adjusted to achieve an expression face. Face detail is extracted with laplacian pyramid and added to the synthesized image.
The proposed method was implemented on the KDEF and Cohn-Kanade (CK+) databases and the results were put on for comparison. Two happy and sad expressions were selected for synthesis. The proper location of the control points and elastic force parameters were determined on the neutral image of the target person based on the expressional images in the database. Then, the neutral image of the person was warped with the elastic forces. Facial expression details have been added with laplacian pyramid method to the warped image. Finally, the experimental results were compared with the photo-realistic and facial expression cloning methods which demonstrate the high visual quality and low computational complexity of the proposed method in synthesizing the face image.