In previous posts, ISO standards relating to the governance, management and quality of AI (I) and (II), we present the standards that address topics relating to the governance and management of AI. On this occasion, we will discuss the standards [1] relating to the quality of AI systems, starting with the standards relating to data.
Data lifecycle
ISO/IEC 8183 Information technology — Artificial intelligence — Data life cycle framework
The aim of this standard is to provide a high-level, practical and general framework applicable to any organisation that processes data for AI, regardless of whether it uses machine learning-based approaches or other AI technologies. To this end, it regulates the activities associated with data processing throughout the entire lifecycle of the AI system, from initial conception to its final decommissioning; it applies to organisations of any size or sector that acquire, create, develop, deploy or maintain data intended for use in AI systems.
By structuring the data flow, ISO/IEC 8183 serves as a cornerstone for meeting the data governance requirements set out in the ISO/IEC 42001 standard (AI Management System) and international regulations on ethical and legal compliance.
Data quality
The aim of this standard, the first in the ISO/IEC 5259 family of standards, is to establish a common language and a unified reference framework for describing, measuring and managing the quality of data used in the development and operation of AI and ML (Machine Learning) systems. It recognises that data quality is the determining factor for the performance, fairness and security of models.
The main aim of this standard is to replace subjective assessments of ‘good or bad data’ with standardised and reproducible quality measures and indicators. It applies to both structured data (tables, relational databases) and unstructured data (text, images, audio and video), which are commonly used in the training of machine learning (ML) models. The standard builds on previous data quality standards such as ISO/IEC 25012, whilst adapting them to the specific needs of AI.
The main objective of this standard is to ensure that data quality for AI/ML is not an isolated technical endeavour, but rather a continuous, systematic and governed process at an institutional level. It adopts the philosophy of traditional ISO management systems (such as the PDCA cycle: Plan, Do, Check, Act) and applies it specifically to the governance of analytical data, mitigating operational, ethical and legal risks before the data is fed into algorithms.
The standard sets out a series of institutional obligations for organisations that develop or deploy AI: a Data Quality Policy for AI, clear roles and responsibilities, and the allocation of resources and competences.
The primary objective of this standard is to define a reference framework of standard processes for AI-oriented data engineering; it describes the technical activities, operational tasks, inputs and outputs that technical teams must carry out to ensure that data remains fit for purpose throughout its entire processing cycle. It establishes a logical engineering sequence applicable to both small machine learning projects and large cloud-based data architectures.
The main aim of this standard is to ensure that the quality of AI data is treated as a strategic corporate asset and a responsibility of senior management, rather than merely as a technical issue for the engineering department.
It is based on global IT governance principles (such as the ISO/IEC 38500 series) and adapts them to the specific risks associated with AI, such as the legal, financial and reputational consequences of training models using biased, out-of-date or dubious data.
The aim of this technical report is to provide guidelines and best practices for translating complex numerical data quality metrics (as defined in Part 2) into comprehensible visual representations; serving as a guide for engineering and design. It applies to the visualisation of data quality at any stage of the lifecycle (as per ISO/IEC 8183), covering both structured and unstructured data (images, text, audio).
The main objective of the standard is to provide an international framework and technical guidelines for assessing, measuring and managing the quality of outputs generated by generative AI applications. It aims to ensure that organisations have standardised methodologies in place to determine whether the text, images, programming code, audio or video produced by a model (such as an LLM or a diffusion model) are safe, reliable and suitable for commercial or operational use.
The aim of this technical report is to provide an overview and conceptual guidance on how to create and use data profiles specifically designed for advanced analytics and machine learning (ML) projects.
In the context of AI, a data profile is a set of structured metadata that comprehensively describes the characteristics, limitations, origin and quality of a dataset. The standard aims to standardise these profiles so that any developer, auditor or regulator can immediately understand whether a dataset is suitable and safe for training a specific model.
Other standards
There are specific standards and technical reports for Big Data which may also be useful for AI systems:
- ISO/IEC 20546 Information technology — Big data — Overview and vocabulary
- ISO/IEC TR 20547-1 Information technology — Big data reference architecture Part 1: Framework and application process
- ISO/IEC TR 20547-2 Information technology — Big data reference architecture Part 2: Use cases and derived requirements
- ISO/IEC 20547-3 Information technology — Big data reference architecture Part 3: Reference architecture
- ISO/IEC 20547-4: Information technology — Big data reference architecture Part 4: Security and privacy
- ISO/IEC TR 20547-5 Information technology — Big data reference architecture Part 5: Standards roadmap
- ISO/IEC 24668 Information technology — Artificial intelligence — Process management framework for big data analytics
- ISO/IEC DIS 27045.2 Information technology — Big data security and privacy — Guidelines for managing big data risks
[1] The summaries of the content presented in this post have been provided directly by GEMINI and reviewed by I2SC.