Anomalous patterns in subjective votes can bias thermal comfort models built using data-driven approaches. A stochastic-based two-step framework to detect outliers in subjective thermal comfort data is proposed to address this problem. The anomaly detection technique involves defining similar conditions using a k-Nearest Neighbor (KNN) method and then quantifying the dissimilarity of the occupants' votes from their peers under similar thermal conditions through a Multivariate Gaussian approach. This framework is used to detect outliers in the ASHRAE Global Thermal Comfort Database I & II. The resulting anomaly-free dataset produced more robust comfort models avoiding dubious predictions. The proposed method has been proven to effectively distinguish outliers from inter-individual variabilities in thermal demand. The proposed anomaly detection framework could easily be applied to other applications with different variables or subjective metrics. Such a tool holds great promise for use in the development of occupancy responsive controls for automated building HVAC systems.