Anonymous one-star reviews ruining my ratings

I've worked SO hard to serve my community through my education podcast over the past three and a half years. I've poured so much into it every single week, and we are at nearly 400,000 downloads and with a loyal following. However, I keep noticing an increase of one-star reviews - with absolutely no written reviews whatsoever. What used to be an easily 4.9 rating has since fallen to 4.8, and with the increase in these anonymous one-star reviews, I am afraid it's going to continue. I looked at other education podcasts in my field and am seeing a similar trend. Although the written ratings are usually 4 and 5 star (mine are all 5 star as well), it seems very odd that we would have this increasing number of one-star reviews hurting our ratings. There is no way to track where these reviews are coming from and no way for me to learn how I can improve my rating if there is a genuine problem with something I am doing on my show, which makes it really hard to continue to strive for excellence - or to keep my motivation up to continue to podcast when it feels a little like sabotage. Please help.