Passive acoustic monitoring is widely used to study underwater soundscapes. This technique requires the manual annotation of large acoustic datasets which is a time-consuming task that necessitates training. To address this challenge, we propose leveraging citizen science by using a web-based platform for manual annotation of acoustic data. In this study, we explore the interest of involving anonymous individuals in annotating marine acoustic recordings in order to build training and test sets for deep learning methods. Here, we designed a convolutional neural network (CNN) to provide automatic detection of vocalizations emitted by pygmy blue whales. We evaluated the performance of the model and compared the obtained results with manual annotations from non-expert annotators and an expert annotator. The results indicated that while the model trained on expert annotations outperformed the others, the models trained on non-expert annotations also exhibited promising results, suggesting the potential for generalization. Moreover, we highlighted the need for large annotated datasets and the challenges associated with obtaining accurate annotations. Leveraging citizen science can provide valuable contributions to the annotation process and foster a better understanding of marine ecosystems and their response to human activities.