They actually care quite a lot. Google ranks you down if this occurs frequently through a metric called Cumulative Layout Shift. You can also be put into confirm click status if you do this egregiously, which basically means you have to click "yes" after clicking an ad before you go to the actual page. It has a huge negative effect on revenue.