self-supervised learning

Home

Discover

Bookmarks

Keep Up With Latest Trending Papers. Computer Science, AI and more.Subscribe

CC BY

Computation and Language

generative self-supervised learning based on phrase reconstruction

Generative or Contrastive? Phrase Reconstruction for Better Sentence Representation Learning

Current pre-trained language models actually take less attention on acquiring sentence-level representation during its self-supervised pre-training.

To alleviate such obvious inconveniences, we instead propose a novel generative self-supervised learning objective based on phrase reconstruction.

Computer Vision

Machine Learning

Relational Self-Supervised Learning

Self-supervised Learning (SSL) including the mainstream contrastive learning has achieved great success in learning visual representations without data annotations. However, most methods mainly focus

More...

CC BY-SA

Computer Vision

Kidney segmentation via self-supervised learning

Self-Supervised Learning for Segmentation

Kidney segmentation is leveraged to define an effective proxy task for kidney segmentation via self-supervised learning.

Evaluation results on a publicly available dataset containing computed tomography (ct) scans of the abdominal region shows that a boost in performance and fast convergence can be had relative to a network trained conventionally from scratch.

Machine Learning

Artificial Intelligence

Stats Machine Learning

Credal Self-Supervised Learning

Self-training is an effective approach to semi-supervised learning. The key idea is to let the learner itself iteratively generate "pseudo-supervision" for unlabeled instances based on its current hyp

More...

Machine Learning

Targeted Adversarial Self-Supervised Learning

Recently, unsupervised adversarial training (AT) has been extensively studied to attain robustness with the models trained upon unlabeled data. To this end, previous studies have applied existing supe

More...

CC BY

Machine Learning

Cryptography and Security

10 Security and Privacy Problems for Pre-trained encoders in Self-Supervised Learning

10 Security and Privacy Problems in Self-Supervised Learning

In this book chapter, we discuss 10 basic security and privacy problems for the pre-trained encoders in self-supervised learning, including six confidentiality problems, three integrity problems, and one availability problem.

We hope our bookchapter will inspire future research on the security and privacy of self-supervised learning.

Computer Vision

Machine Learning

Quantum Physics

Quantum Self-Supervised Learning

The popularisation of neural networks has seen incredible advances in pattern recognition, driven by the supervised learning of human annotations. However, this approach is unsustainable in relation t

More...

CC BY

Machine Learning

Artificial Intelligence

Stats Machine Learning

Ensembling for Improved Self-Supervised Learning

Weighted Ensemble Self-Supervised Learning

Ensemble methods have proven to be a powerful technique for boosting modelperformance, uncertainty estimation, and robustness in supervised learning.

Recent advances in self-supervised learning (ssl) enable leveraging large unlabeled corpora for state-of-the-art few-shot and supervised learning performance.

Computer Vision

Machine Learning

Artificial Intelligence

Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration

Our work reveals a structured shortcoming of the existing mainstream self-supervised learning methods. Whereas self-supervised learning frameworks usually take the prevailing perfect instance level in

More...

CC BY

Computer Vision

Mean-Shift Image Learning

Mean Shift for Self-Supervised Learning

We introduce a simple mean-shift algorithm that learns representations by grouping images together withoutcontrasting between them or adopting much of prior on the structure of the image clusters.

Our model achieves 72.4% on imagenet linear evaluation with resnet50 at 200epochs outperforming byol.

Machine Learning

Co-learning: Learning from Noisy Labels with Self-supervision

Noisy labels, resulting from mistakes in manual labeling or webly data collecting for supervised learning, can cause neural networks to overfit the misleading information and degrade the generalizatio

More...

Computer Vision

Artificial Intelligence

TriBYOL: Triplet BYOL for Self-Supervised Representation Learning

This paper proposes a novel self-supervised learning method for learning better representations with small batch sizes. Many self-supervised learning methods based on certain forms of the siamese netw

More...

Bookmarked byNicolás

Computer Vision

Machine Learning

SiT: Self-supervised vIsion Transformer

Self-supervised learning methods are gaining increasing traction in computer vision due to their recent success in reducing the gap with supervised learning. In natural language processing (NLP) self-

More...

Machine Learning

Artificial Intelligence

Automated Self-Supervised Learning for Graphs

Graph self-supervised learning has gained increasing attention due to its capacity to learn expressive node representations. Many pretext tasks, or loss functions have been designed from distinct pers

More...

Computer Vision

Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey

Large-scale labeled data are generally required to train deep neural networks in order to obtain better performance in visual feature learning from images or videos for computer vision applications. T

More...

CC BY

Computer Vision

Machine Learning

A Model Compression Method for Self-Supervised Learning

CompRess: Self-Supervised Learning by Compressing Representations

Self-supervised learning aims to learn good representations with unlabeled data.

In this work, instead of designing a new pseudo task for self-supervised learning, we develop a model compression method to compress an already learned, deep self-supervised model (teacher) to a smaller one(student).

Computer Vision

Self-Supervised Learning Across Domains

Human adaptability relies crucially on learning and merging knowledge from both supervised and unsupervised tasks: the parents point out few important concepts, but then the children fill in the gaps

More...

CC BY

Computer Vision

Knowledge Distilling of Self-Supervised Vision Transformer

Attention Distillation: self-supervised vision transformer students need more guidance

Knowledge distillation of self-supervised vision transformers (vit-sskd) is an important research topic to increase their performance on memory and compute devices.

We show that directly distilling information from the crucial attention mechanism from teacher to student can significantly narrow the performance gap between both.

CC BY

Information Theory

Artificial Intelligence

Information-Theoretic Variance-In-Co Regularization for Self-Supervised Learning

An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularization

In this paper, we provide an information-theoretic perspective on variation-invariance-covariance regularization (vicreg) for self-supervised learning and derive a generalization bound for vicreg, providing generalization guarantees for downstream supervised learning tasks.

Next, we relate the vicregobjective to mutual information maximization and use it to highlight the underlying assumptions of the objective.

Home

Discover

Bookmarks

◐ Latest Activity

◐ Recommended Follows