Does the Curse of Dimensionality Apply to Unsupervised SHM? Investigating the Trade-Off Between Loss of Information and Generalizability to Unseen Structural Conditions

Abstract

The curse of dimensionality (CD) brings difficulties in pattern recognition problems, such as those found in structural health monitoring (SHM). Dimensionality reduction techniques (DR) make data more manageable by reducing noise and noninformative portions. There exists a trade-off between CD and the loss of information due to the application of DR. Even though in supervised SHM, DR techniques are shown to be effective, for unsupervised SHM, the trade-off must be assessed due to the unknown data population of novel classes. This study assesses the trade-off concerning a novel method working with a raw frequency-domain feature, the fast Fourier transform (FFT). Different DR techniques are applied to the initial FFT-based feature to assess the trade-off, and detection results are compared. The results indicate that the loss of information can have detrimental effects, such as lowering the detection accuracy by 60% for the autoencoder-based DR. The accuracy reduction is present for all different DR techniques applied in the study; however, regularization lessens the accuracy decrements. This phenomenon indicates the assumption that novelties show themselves in less-vary portions of the baseline condition to be not true.