Member-only story
Soft edges on display sources in Open Broadcaster Software
Display sources in Open Broadcaster Software (OBS) create hard edges around itself. This is great for some use cases, but a softer edge creates a much more pleasant visual experience. Applying this effect in OBS requires the creation of a mask that determines the opacity of parts of the display source
Creating a mask
To create the masks, you will need access to photo editing software that can do a gradient fill. This mask could be made using Adobe Photoshop, Affinity Photo, Microsoft Paint, Paint .NET, or even Microsoft PowerPoint!
Firstly create a canvas that matches your input display aspect ratio. I have a 16:9 input, therefore, I will create a 1600px x 900px canvas. If your background is not white, then you will need to fill the canvas with solid white.
In this mask, solid white areas will be fully visible, and solid black areas will be fully transparent; any in-between greys will result in semi-transparent regions based on the pixel intensity. Take advantage of this by filling the edges of your mask with a black to white gradient. This will create a soft transition from your background to your display sources. Once you are happy with your mask, export it to a PNG file.