Ablation Studies in Artificial Neural Networks
Ablation Studies in Artificial Neural Networks
We performed two ablation studies in two fundamentally different artificial neural networks (anns) to investigate their innerrepresentations of two well-known benchmark datasets from the computer vision domain.We found that features distinct to the local and global structure of the data are selectively represented in specific parts of the network and that some of these representations are redundant, awarding the network a certain robustness to structural damages.We further determined the importance of specific parts of the network for the classification task solely based on the weight structure of single units.Finally, we examined the ability of damaged networks to recover from the consequences of ablations by means of recovery training.We argue that ablations studies are a feasible method to investigate knowledge representations in anns and are especially helpful to examine a networks robustness to structural damages, a feature of anns that will become increasingly important for future safety-critical applications.
