Coping with stress is one of the most frequently cited reasons for chronic cannabis use. Therefore, it is hypothesized that cannabis users exhibit distinct physiological stress responses compared to non-users, and that these differences may be especially pronounced during moments of cannabis consumption. However, there is a scarcity of publicly available datasets that allow such hypotheses to be tested under real-world conditions. This paper introduces a dataset named CAN-STRESS, collected using Empatica E4 wristbands. The dataset includes multimodal physiological measurements (such as skin conductance, heart rate,and skin temperature) from 82 participants (39 cannabis users and 43 non-users) as they went about their daily routines. In addition to sensor data, participants provided self-reported survey responses that included perceived stress ratings and timestamps of key daily events such as cannabis use, physical activity, and sleep. To demonstrate the utility of the dataset for downstream applications, we present a preliminary machine learning task aimed at classifying cannabis users versus non-users based on physiological features. Our model achieves a classification accuracy of approximately 96% and an f1-score of around 98%. An analysis of feature importance using SHAP values revealed that electrodermal activity and heart rate metrics were the most influential predictors, consistent with their established roles in stress detection. We publicly release the CAN-STRESS dataset, which we believe serves as a reliable and rich resource for studying the physiological correlates of cannabis use and stress in naturalistic settings.