It's a quite complex topic, but yes, when dealing with non-royalty-free technologies you are in some cases required to pay royalties to the patent holders, depending on your usage. This usually applies when you are the distributor of the video/audio content, but it depends from technology to technology.
Patent pools are groups of companies that claim fees for the usage of a particular technology that includes something protected by patents. An example of a patent pool is the one managed by the popular MPEG LA, which is concerned about technologies developed by the MPEG group (like AVC and HEVC).
It's also important to understand that while royalty-free technologies don't require you to pay, it doesn't mean that they don't include patented technologies. The creator of the technology (Google in this case) is risking for the usage of those technologies. They're basically saying "don't worry, we'll take care for you".
Source: An analysis of WebM and its patent risk.
A perfect example of a royalty-free technology is the new AV1 coding standard, developed by an alliance called AOMedia. For the fact that this alliance is composed of pretty much every important technology company that exists out there, the AV1 codec doesn't require content distributors to pay royalties to them.
John Luther from JW Player says:
AV1 could be different because collectively the AOM membership owns a vast trove of intellectual property. These assets can be used to defend the technology against patent claims from interests who might not want “a next-generation video format that is…interoperable and open.”
In the same way, the existence of a patent pool doesn't mean that companies in the pool are going to ask for money. For example, the HEVC decoding standard is composed of multiple patent pools, but one of the pools has still to decide whether it's going to ask for fees, and how much it's going to ask. Additionally, not all the companies holding patents regarding HEVC are included in the pools. This is one of the main reasons why HEVC hasn't spread so much so far.