Kidney segmentation is leveraged to define an effective proxy task for kidney segmentation via self-supervised learning.
Evaluation results on a publicly available dataset containing computed tomography (ct) scans of the abdominal region shows that a boost in performance and fast convergence can be had relative to a network trained conventionally from scratch.
In this book chapter, we discuss 10 basic security and privacy problems for the pre-trained encoders in self-supervised learning, including six confidentiality problems, three integrity problems, and one availability problem.
We hope our bookchapter will inspire future research on the security and privacy of self-supervised learning.
Self-supervised learning aims to learn good representations with unlabeled data.
In this work, instead of designing a new pseudo task for self-supervised learning, we develop a model compression method to compress an already learned, deep self-supervised model (teacher) to a smaller one(student).
In this paper, we provide an information-theoretic perspective on variation-invariance-covariance regularization (vicreg) for self-supervised learning and derive a generalization bound for vicreg, providing generalization guarantees for downstream supervised learning tasks.
Next, we relate the vicregobjective to mutual information maximization and use it to highlight the underlying assumptions of the objective.