The IRMA Community
Newsletters
Research IRM
Click a keyword to search titles using our InfoSci-OnDemand powered search:
|
Metamorphic Testing of Image Classification and Consistency Analysis Using Clustering
Abstract
Testing deep learning systems requires expensive labeled data. In recent years, researchers began to leverage metamorphic testing to address this issue. However, metamorphic relations on image data remain poorly understood. To gain a deeper understanding of these metamorphic relations, we survey common image operations modeling covariate shift, manually classify and categorize the underlying metamorphic relations, and conduct experiments to validate our classifications. In our experiments, we train three popular convolutional neural network architectures on an image classification task. Next, we apply metamorphic operations on input test images and measure the change in classification accuracy and cross-entropy loss. A hierarchical clustering algorithm cluster these results and plots a dendrogram. We compare the groups from manual classification and the clusters from the algorithm to provide key insights. We find that Affine and Noise relations are consistent. Furthermore, we recommend metamorphic relationships to save time and better test deep learning systems in the future.
Related Content
|
Yasasi Abeysinghe, Bhanuka Mahanama, Gavindya Jayawardena, Yasith Jayawardana, Mohan Sunkara, Andrew T. Duchowski, Vikas Ashok, Sampath Jayarathna.
© 2024.
20 pages.
|
|
Chengxuan Huang, Evan Brock, Dalei Wu, Yu Liang.
© 2023.
23 pages.
|
|
Duleep Rathgamage Don, Jonathan Boardman, Sudhashree Sayenju, Ramazan Aygun, Yifan Zhang, Bill Franks, Sereres Johnston, George Lee, Dan Sullivan, Girish Modgil.
© 2023.
17 pages.
|
|
Wei-An Teng, Su-Ling Yeh, Homer H. Chen.
© 2023.
17 pages.
|
|
Hemanth Gudaparthi, Prudhviraj Naidu, Nan Niu.
© 2022.
20 pages.
|
|
Anchen Sun, Yudong Tao, Mei-Ling Shyu, Angela Blizzard, William Andrew Rothenberg, Dainelys Garcia, Jason F. Jent.
© 2022.
19 pages.
|
|
Suvojit Acharjee, Sheli Sinha Chaudhuri.
© 2022.
16 pages.
|
|
|