To enable this function, please go to Video Extensions > Video Archives > Enable Lightbox for Video Post choose On. Then, when you click to the Play icon of video in the Archive layouts (category, search pages), it will play the video in a Lightbox popup.
click to the Play icon of video
You can see the video is played in a Lightbox popup
NOTE: To enable lightbox for video player of video items in Smart Content Box or Post Slider shortcode, you need to switch videoplayer_lightbox parameter in the settings to ON.