International Journal of Innovative Research in Computer and Communication Engineering
ISSN Approved Journal | Impact factor: 8.771 | ESTD: 2013 | Follows UGC CARE Journal Norms and Guidelines
| Monthly, Peer-Reviewed, Refereed, Scholarly, Multidisciplinary and Open Access Journal | High Impact Factor 8.771 (Calculated by Google Scholar and Semantic Scholar | AI-Powered Research Tool | Indexing in all Major Database & Metadata, Citation Generator | Digital Object Identifier (DOI) |
| TITLE | Autonomous Data Quality Assessment Frameworks: A Review of Methods, Tools, and Open Research Challenges |
|---|---|
| ABSTRACT | With the increasing reliance on data for decision making, analytics and machine learning to generate actionable insights from data, the quality of the underlying data has become a matter of great importance. Evaluation of data quality has so far been a subjective and ad-hoc process, with few objective and systematic approaches to assessing it. In this article, we review and evaluate the cutting-edge methods and tools used for autonomous and score-based data quality assessment. Specifically, we discuss approaches that allow to combine objective assessments of individual data quality dimensions into a composite score and focus on rule-based data profiling and validation, modern data observability and anomaly detection methods. Following an overview of the theoretical foundations associated with each individual data quality dimension, we analyze and evaluate the five leading commercial products, covering both rule-based data validation/cleaning and data observability/anomaly detection methods in terms of their abilities in data profiling, data validation, data cleaning, data scoring, data workflow automation and transparency. We find that while all the five products reviewed have strengths in specific areas of the data pipeline, none of them provide truly end-to-end, autonomous and transparent quality assessment of data that would produce a composite score. We also find that while all commercial products reviewed have rule sets enabling them to perform data validation and data cleaning, these rules typically require labor-intensive configuration by data quality experts. We conclude that commercial data quality products identified lack several key features that would render them fit for purpose in terms of enabling high data quality. The most notable shortcomings are the absence of standardized and explainable approaches to composite data quality scoring and limited support for high-velocity and real-time data and domain-specific weighting of data quality dimensions. A unifying framework allowing to combine different data quality dimensions into a composite, automated and explainable score would greatly benefit the practice of data quality management. |
| AUTHOR | NIKHIL R, YOGESHWARAN K, PROF. CHANDAN HEGDE Student, Department of Master of Computer Applications, Surana College (Autonomous), Bengaluru, India Assistant Professor, Department of Master of Computer Applications, Surana College (Autonomous), Bengaluru, India |
| VOLUME | 187 |
| DOI | DOI: 10.15680/IJIRCCE.2026.1408013 |
| pdf/13_Autonomous Data Quality Assessment Frameworks A Review of Methods, Tools, and Open Research Challenges.pdf | |
| KEYWORDS | |
| References | [1]. Al-Toq, Razan, and Abdulaziz Almaslukh. "DQMAF—Data Quality Modeling and Assessment Framework." Information 16.10 (2025): 911. [2]. Miller, Russell, et al. "An Overview of Current and New Data Quality Dimensions under a Common Framework." (2024). [3]. Gong, Youdi, et al. "A survey on dataset quality in machine learning." Information and Software Technology 162 (2023): 107268. [4]. Alexakis, Theodoros, et al. "Evaluating Data Quality: Comparative Insights on Standards, Methodologies, and Modern Software Tools." Electronics 14.15 (2025): 3038. [5]. Ramisetty, Gopinath. "Real-Time Data Integrity Nexus: Autonomous Quality Assurance Framework." Journal of Computer Science and Technology Studies 7.10 (2025): 665-671. [6]. Podobnikar, Tomaž. "Bridging Perceived and Actual Data Quality: Automating the Framework for Governance Reliability." Geosciences 15.4 (2025): 117. [7]. Bernardi, Filipe Andrade, et al. "Data quality in health research: integrative literature review." Journal of medical Internet research 25 (2023): e41446. [8]. Liu, Mingyu, et al. "A survey on autonomous driving datasets: Statistics, annotation quality, and a future outlook." IEEE Transactions on Intelligent Vehicles (2024). |
