Preprocess the data

[DWCTOD]

hi ,thansks for your nice work!
Would you tell how to preprocess the data (Extract landmark and crop the image by dlib)?

I checked other issue about this , like #34
that using demo.py ,but I think this is just for single image , but for video ,how to process .
Waiting for your reply, thanks you again

[lelechen63]

hi ,thansks for your nice work!
Would you tell how to preprocess the data (Extract landmark and crop the image by dlib)?

I checked other issue about this , like #34
that using demo.py ,but I think this is just for single image , but for video ,how to process .
Waiting for your reply, thanks you again

Hi,

You can sacle the proprocess into videos by yourself. It is just read video into images and process them one by one. You can also add a smooth term when you extract the facial landmarks for videos to obtain smoother transctions between video frames.

[DWCTOD]

Thanks

[DWCTOD]

hi ,thansks for your nice work!
Would you tell how to preprocess the data (Extract landmark and crop the image by dlib)?
I checked other issue about this , like #34
that using demo.py ,but I think this is just for single image , but for video ,how to process .
Waiting for your reply, thanks you again

Hi,

You can sacle the proprocess into videos by yourself. It is just read video into images and process them one by one. You can also add a smooth term when you extract the facial landmarks for videos to obtain smoother transctions between video frames.

Hi,
You say that " You can also add a smooth term when you extract the facial landmarks for videos to obtain smoother transctions between video frames."

I found your new work use url 3D landmark, and use smooth process for window. (oned_smooth function here

I want to kown about what here "smooth term " mean ?
smooth term is for face windows or landmarks?

[DWCTOD]

hi ,thansks for your nice work!
Would you tell how to preprocess the data (Extract landmark and crop the image by dlib)?
I checked other issue about this , like #34
that using demo.py ,but I think this is just for single image , but for video ,how to process .
Waiting for your reply, thanks you again

Hi,

You can sacle the proprocess into videos by yourself. It is just read video into images and process them one by one. You can also add a smooth term when you extract the facial landmarks for videos to obtain smoother transctions between video frames.

Hi , The duration of the video is very long.It need to cut? (because LRW's videos is only 1~5s for every video)

[DWCTOD]

Hi,
I try to use custom dataset retrain AT net model, I found maybe the landmark is only use mouth area.

Becasue all attention highlight area is in mouth area. And if I retrain model using all face landmark , result will be bad .

Another image shows only mouth is moving .

[liyuanyaun]

Hi,
I try to use custom dataset retrain AT net model, I found maybe the landmark is only use mouth area.

Becasue all attention highlight area is in mouth area. And if I retrain model using all face landmark , result will be bad .

Another image shows only mouth is moving .

hi did you figure out how to preprocess the 2d landmark? and can you share how you deal with the data? @htzheng

[DWCTOD]

Hi,
I try to use custom dataset retrain AT net model, I found maybe the landmark is only use mouth area.
Becasue all attention highlight area is in mouth area. And if I retrain model using all face landmark , result will be bad .

Another image shows only mouth is moving .

hi did you figure out how to preprocess the 2d landmark? and can you share how you deal with the data? @htzheng

你好，这里分享一下我自己的处理思路
这里对单张图片处理的思路，对视频处理是类似的，将视频拆分成图片，最后再拼接起来，处理完成后，需要对做的dataloader 进行一定的修改。
首先作者在 demo.py 中有人脸对齐操作（第一步）的，直接照搬使用，对对齐后的图片，将除嘴唇区域以外的关键点设置为标准模板里面的关键，仅保留嘴唇区域的关键点是由对齐生成的（第二步）（就是numpy 数组的重新赋值而已）。
1、demo.py 人脸对齐操作
2、仅保留嘴唇区域的关键点是由对齐生成的，其他替换成模板的关键点
大概思路就是这样子
代码和数据集都不能给你，抱歉

[liyuanyaun]

Hi,
I try to use custom dataset retrain AT net model, I found maybe the landmark is only use mouth area.
Becasue all attention highlight area is in mouth area. And if I retrain model using all face landmark , result will be bad .

Another image shows only mouth is moving .

hi did you figure out how to preprocess the 2d landmark? and can you share how you deal with the data? @htzheng

你好，这里分享一下我自己的处理思路
这里对单张图片处理的思路，对视频处理是类似的，将视频拆分成图片，最后再拼接起来，处理完成后，需要对做的dataloader 进行一定的修改。
首先作者在 demo.py 中有人脸对齐操作（第一步）的，直接照搬使用，对对齐后的图片，将除嘴唇区域以外的关键点设置为标准模板里面的关键，仅保留嘴唇区域的关键点是由对齐生成的（第二步）（就是numpy 数组的重新赋值而已）。
1、demo.py 人脸对齐操作
2、仅保留嘴唇区域的关键点是由对齐生成的，其他替换成模板的关键点
大概思路就是这样子
代码和数据集都不能给你，抱歉

好的，非常感谢，但是还有一个疑问是，如果“嘴唇区域以外的关键点设置为标准模板里面的关键”这样操作后，网络最终也就是回归嘴部动作，下巴和头部姿态是无法学习到的，尤其是下巴的变化，这样会不会有不自然的情况出现？