Please use this identifier to cite or link to this item:
http://bura.brunel.ac.uk/handle/2438/24415
Title: | Class Density and Dataset Quality in High-Dimensional, Unstructured Data |
Authors: | Byerly, A Kalganova, T |
Keywords: | class density;dataset quality;completeness;data reduction;dynamic data reduction |
Issue Date: | 8-Feb-2022 |
Publisher: | Cornell University |
Citation: | Byerly, A. and Kalganova, T. (2022) 'Class Density and Dataset Quality in High-Dimensional, Unstructured Data', arXiv:2202.03856v1 [cs.LG], pp. 1-13. doi: 10.48550/arXiv.2202.03856. |
Abstract: | Copyright © 2022 The Authors. We provide a definition for class density that can be used to measure the aggregate similarity of the samples within each of the classes in a high-dimensional, unstructured dataset. We then put forth several candidate methods for calculating class density and analyze the correlation between the values each method produces with the corresponding individual class test accuracies achieved on a trained model. Additionally, we propose a definition for dataset quality for high-dimensional, unstructured data and show that those datasets that met a certain quality threshold (experimentally demonstrated to be > 10 for the datasets studied) were candidates for eliding redundant data based on the individual class densities. |
URI: | https://bura.brunel.ac.uk/handle/2438/24415 |
Other Identifiers: | arXiv:2202.03856v1 |
Appears in Collections: | Dept of Electronic and Electrical Engineering Research Papers |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
Preprint.pdf | 154.68 kB | Adobe PDF | View/Open |
Items in BURA are protected by copyright, with all rights reserved, unless otherwise indicated.