In the past decades, in the context of a changing ocean submitted to an increasing human activity, a progressive decrease in the frequencies (pitch) of blue whale vocalizations has been observed worldwide. Its causes, of natural or anthropogenic nature, are still unclear. Based on 7 years of continuous acoustic recordings at widespread sites in the southern Indian Ocean, we show that this observation stands for five populations of large whales. The frequency of selected units of vocalizations of fin, Antarctic, and pygmy blue whales has steadily decreased at a rate of a few tenths of hertz per year since 2002. In addition to this interannual frequency decrease, blue whale vocalizations display seasonal frequency shifts. We show that these intra‐annual shifts correlate with seasonal changes in the ambient noise near their call frequency. This ambient noise level, in turn, shows a strong correlation with the seasonal presence of icebergs, which are one of the main sources of oceanic noise in the Southern Hemisphere. Although cause‐and‐effect relationships are difficult to ascertain, wide‐ranging changes in the acoustic environment seem to have a strong impact on the vocal behavior of large baleen whales. Seasonal frequency shifts may be due to short‐term changes in the ambient noise, and the interannual frequency decline to long‐term changes in the acoustic properties of the ocean and/or in postwhaling changes in whale abundances.