Wearable EEG (electroencephalogram) systems have demonstrated potential in epilepsy monitoring, sleep assessment, and determining cognitive workload to improve human decision-making. However, analyzing EEG signals is challenging due to their non-stationary nature and susceptibility to noise. In particular, achieving high accuracy in machine learning tasks requires large amounts of labeled data, which is difficult to obtain due to the time-consuming and labor-intensive nature of data labeling. To address these challenges, we propose a self-supervised learning (SSL) approach for cognitive workload classification using wavelet-based augmentations of EEG signals. First, two augmentations per channel are generated, and their wavelets are computed. The visual representations of these wavelets are then fed to the SSL pretext phase as contrastive pairs to pre- train the model. Finally, the pre-trained model is fine-tuned for workload classification using small amounts of labeled EEG data. Experimental results on the EEG During Mental Arithmetic Tasks (EEGMAT) dataset show that our method outperforms the state-of-the-art supervised models. Notably, our model achieves an accuracy of 99.5% with only 50% of the labeled data, demonstrating the effectiveness of our approach in scenarios with limited labeled data availability. Furthermore, the proposed approach achieves an accuracy of 98.6% in the leave-one-subject-out analysis.