It seems that in your code, you are not using the mask of the video but rather the mask of a specific frame ?