OpenCV基于Landmark实现人脸交换

川剧是中国最知名的戏曲剧种之一，变脸是川剧表演的特技之一，在对象传承上有着严格的师门派别。有点扯远啦，回来！其实主要是我们今天要用OpenCV干的事情跟这个有点关系，OpenCV基于Landmark实现人脸关键点提取，对结果善加利用可以实现人脸交换，对特定对象施加变脸术。OpenCV开发者不学川剧也一样可以给各种人变脸，当然前提是会写代码，会做OpenCV。首先简单说一下原理与流程。

一：原理与流程

基本原理是利用OpenCV的级联检测器实现人脸检测然后基于Landmak的LBF模型实现人脸68个关键点提取，基于关键点数据实现三角剖分与维诺图计算，经过几何变换之后得到mask区域，再利用OpenCV无缝克隆算法相关API实现换脸。整个工作流程如下：

二：代码实现

1.创建Landmark关键点检测器

1. // 人脸检测与Landmark68个关键点检测

2. CascadeClassifier face_cascade;

3. face_cascade.load(cascade_name);

4. FacemarkLBF::Params params;

5. params.n_landmarks = 68; // 68个标注点

6. params.initShape_n = 10;

7. params.stages_n = 5; // 算法的5个强化步骤

8. params.tree_n = 6; // 模型中每个标注点结构树 数目

9. params.tree_depth = 5; // 决策树深度

10. // 创建LBF landmark 检测器

11. Ptr<FacemarkLBF> facemark = FacemarkLBF::create(params);

12. facemark->setFaceDetector((FN_FaceDetector)myDetector, &face_cascade);

13. facemark->loadModel(modelfile_name);

14. cout << "Loaded model" << endl;

2.Landmark关键点检测

1. //vector to store the faces detected in the image

2. vector<Rect> faces1, faces2;

3. vector< vector<Point2f> > shape1, shape2;

4. //Detect faces in the current image

5. float ratio1 = (float)img1.cols / (float)img1.rows;

6. float ratio2 = (float)img2.cols / (float)img2.rows;

7. resize(img1, img1, Size((int)(640 * ratio1), (int)(640 * ratio1)), 0, 0, INTER_LINEAR_EXACT);

8. resize(img2, img2, Size((int)(640 * ratio2), (int)(640 * ratio2)), 0, 0, INTER_LINEAR_EXACT);

9. Mat img1Warped = img2.clone();

10. facemark->getFaces(img1, faces1);

11. facemark->getFaces(img2, faces2);

12. //Initialise the shape of the faces

13. facemark->fit(img1, faces1, shape1);

14. facemark->fit(img2, faces2, shape2);

3.三角剖分变换

1. vector<Point2f> points1 = shape1[z];

2. vector<Point2f> points2 = shape2[z];

3. img1.convertTo(img1, CV_32F);

4. img1Warped.convertTo(img1Warped, CV_32F);

5. // Find convex hull

6. vector<Point2f> boundary_image1;

7. vector<Point2f> boundary_image2;

8. vector<int> index;

9. convexHull(Mat(points2), index, false, false);

10. for (size_t i = 0; i < index.size(); i++)

11. {

12.    boundary_image1.push_back(points1[index[i]]);

13.    boundary_image2.push_back(points2[index[i]]);

14. }

15. // Triangulation for points on the convex hull

16. vector< vector<int> > triangles;

17. Rect rect(0, 0, img1Warped.cols, img1Warped.rows);

18. divideIntoTriangles(rect, boundary_image2, triangles);

19. // Apply affine transformation to Delaunay triangles

20. for (size_t i = 0; i < triangles.size(); i++)

21. {

22.    vector<Point2f> triangle1, triangle2;

23.    // Get points for img1, img2 corresponding to the triangles

24.    for (int j = 0; j < 3; j++)

25.    {

26.        triangle1.push_back(boundary_image1[triangles[i][j]]);

27.        triangle2.push_back(boundary_image2[triangles[i][j]]);

28.    }

29.    warpTriangle(img1, img1Warped, triangle1, triangle2);

30. }

4.计算与模板生成

1. // 计算与生成模板

2. vector<Point> hull;

3. for (size_t i = 0; i < boundary_image2.size(); i++)

4. {

5.    Point pt((int)boundary_image2[i].x, (int)boundary_image2[i].y);

6.    hull.push_back(pt);

7. }

8. Mat mask = Mat::zeros(img2.rows, img2.cols, img2.depth());

9. fillConvexPoly(mask, &hull[0], (int)hull.size(), Scalar(255, 255, 255));

5.无缝克隆

1. //    无缝克隆

2. Rect r = boundingRect(boundary_image2);

3. Point center = (r.tl() + r.br()) / 2;

4. Mat output;

5. img1Warped.convertTo(img1Warped, CV_8UC3);

6. seamlessClone(img1Warped, img2, mask, center, output, NORMAL_CLONE);

7. imshow("Face_Swapped", output);

8. imwrite("D:/face_swap_demo.png", output);

三：运行效果

原图一

原图二

人脸交换结果

---

关注公众号，发送 “源码” 两字获取下载地址

---

更多相关阅读

合抱之木，生于毫末；

九层之台，起于累土；

千里之行，始于足下。

关注【OpenCV学堂】

长按或者扫码二维码即可关注