Zoom on Linux has poor virtual background support (unless you have a solid backdrop), so instead route through GStreamer/Python and replace the background using a pre-trained semantic segmentation neural net.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Garrett Wilson 7ebd6d1edd arguments, interactively change some 8 months ago
files v2 8 months ago
.gitignore if not TF Lite then convert to it, latency flag 8 months ago
README.md arguments, interactively change some 8 months ago
main.py arguments, interactively change some 8 months ago
run.sh support Messenger, Hangouts, etc., better latency 8 months ago


GStreamer Background Replace

Zoom background replacement doesn't work well for me on Linux. Thus, let's use a neural net to segment the image and replace the background. And... then just hope it's fast enough.

Also, while we're at it, adjust the white balance, crop the too-wide-angle GoPro view, add a slight vignette, etc.


If you want to output this as a virtual/fake webcam on Linux, e.g. to use in Zoom, Hangouts, Messenger, etc. then install v4l2loopback, for example if on Arch with the AUR package v4l2loopback-dkms (though make sure you have linux-headers installed as well, or else dkms won't build it for your kernel).

Then, gstreamer 1.0 with good/bad plugins, TensorFlow, Numpy, SciPy, and abseil-py (absl-py).

On Arch:

sudo pacman -S linux-headers python-{scipy,numpy,tensorflow} gst-{python,plugins{good,bad}} absl-py
aur sync v4l2loopback-dkms

(Or, python-tensorflow-cuda, python-tensorflow-opt, ...)


Load the kernel module:

sudo modprobe v4l2loopback card_label="Virtual Webcam" video_nr=7 exclusive_caps=1

Run the script (set the appropriate device, v4l2 output device, etc.):

./main.py --input_device="/dev/video0" --output_device="/dev/video7"

Or, see run.sh for a full example.


Decent, though not that great:


Compare that with Zoom's internal replacement algorithm result:

Zoom Background