SimPer: Simple Self-Supervised Learning of Periodic Targets

Yuzhe Yang1      Xin Liu2      Jiang Wu3      Silviu Borac3      Dina Katabi1      Ming-Zher Poh3      Daniel McDuff2,3
1 MIT CSAIL      2 University of Washington      3 Google


From human physiology to environmental evolution, important processes in nature often exhibit meaningful and strong periodic or quasi-periodic changes. Due to their inherent label scarcity, learning useful representations for periodic tasks with limited or no supervision is of great benefit. Yet, existing self-supervised learning (SSL) methods overlook the intrinsic periodicity in data, and fail to learn representations that capture periodic or frequency attributes. In this paper, we present SimPer, a simple contrastive SSL regime for learning periodic information in data. To exploit the periodic inductive bias, SimPer introduces customized augmentations, feature similarity measures, and a generalized contrastive loss for learning efficient and robust periodic representations. Extensive experiments on common real-world tasks in human behavior analysis, environmental sensing, and healthcare domains verify the superior performance of SimPer compared to state-of-the-art SSL methods, highlighting its intriguing properties including better data efficiency, robustness to spurious correlations, and generalization to distribution shifts.


SimPer: Simple Self-Supervised Learning of Periodic Targets
Yuzhe Yang, Xin Liu, Jiang Wu, Silviu Borac, Dina Katabi, Ming-Zher Poh, and Daniel McDuff
International Conference on Learning Representations (ICLR 2023)
Oral Presentation (top 1.6%)  /  Notable Top 5% Paper
[Paper]  •  [OpenReview]  •  [Code]  •  [Poster]  •  [Google AI Blog]  •  [BibTeX]


Code, Data, and Models

Code, Data, and Models


(1) Overview of the SimPer Framework

(2) Periodic Feature Similarity

(3) SimPer Achieves Better Data Efficiency

(4) SimPer Generalizes to Unseen Distribution Shifts

(5) SimPer is Robust to Spurious Correlations


  title={SimPer: Simple Self-Supervised Learning of Periodic Targets},
  author={Yang, Yuzhe and Liu, Xin and Wu, Jiang and Borac, Silviu and Katabi, Dina and Poh, Ming-Zher and McDuff, Daniel},
  booktitle={International Conference on Learning Representations},