As we pointed out, a functional, high-quality AI is the essential foundation for an ethical and legal AI. In this post, we summarise the two main standards for creating, evaluating and certifying both the software processes for developing AI systems (ISO/IEC 5338) and the AI software product itself (ISO/IEC 25029).
Artificial Intelligence Engineering
In the 2010s, the development of AI systems was already gaining momentum, particularly those based on machine learning/deep learning (ML/DL) models, which required engineering solutions to ensure the quality of AI software in production and its deployment in industry, highlighting the need for AI Engineering.
In fact, since the early 2000s, various proposals have been put forward life cycle models for AI (CRISP-DM, CRISP-ML(Q), TDSP, ADLC, etc.) as well as adaptations of agile and DevOps methods for AI and MLOps.
More recently, this field has undergone rapid transformation with the incorporation of AI itself to develop software, using methods such as Vibe Engineering or SDD (Spec Driven Development), as well as to specify, develop and test more quickly and automatically; or even with the incorporation of agents in the creation and deployment of software.
In any case, as reality invariably confirms, even as technology advances, the principles remain valid; it is therefore essential to have a set of appropriate processes in place for software development and evolution.
Quality of AI software processes
The ISO/IEC 5338 standard defines a total of 33 processes, which amend those in the ISO/IEC 12207 standard and introduce some new ones. Based on the model MMIS and the previous standard, the MMSIA model has been defined, which is a maturity model for Artificial Intelligence software processes in which the following processes are defined:
With this model, companies that develop AI software can assess the quality of their processes and gradually improve them, as well as obtain certifications based on the ISO/IEC 33000, as many organisations with “traditional” software already have.
Quality of the AI software product
Furthermore, it is important to ensure the quality of the AI software itself (as a product). To this end, based on the ISO/IEC 25000 family of standards, the standard ISO/IEC 25059 has been developed, which sets out the Product Quality characteristics that any AI system must meet.
With regard to the ISO/IEC 25010 standard, the definitions have been amended to bring them into line with AI systems, and changes have been made to several sub-characteristics, such as Interaction Capability, Reliability, Safety, and Functional Suitability. By using this standard, and through assessment by an accredited laboratory (AQCLab is the only one in Spain authorised to do so), AI software producers can certify their products, as has been done for years with “traditional” software products.
Obviously, if the AI software uses a machine learning system, then we will also need to assess (and eventually certify) the quality of the data used for its training, validation and testing.