Guidebook for Trustworthy AI | AI Ethics Practice | AI Ethics Communication Channel

Guidebook for Development of Trustworthy AI 2023 General Sector

CONTENTS Checklist Checklist for using the guide 6 PART 1 Overview 11 1. Background and purpose of publishing the guide 12 2. Artificial intelligence reliability trend 13 3. Process of preparing the guide 17 4. Target audience 24 5. How to use the guide 26 PART 2 Requirements and verification items 27 1. Life cycle management 32 2. Data collection and processing 52 3. Artificial intelligence model development 68 4. System implementation 83 5. Operation and Monitoring 95 PART 3 Appendix 99 1. Abbreviation Table 100 2. Terminology Table 101 3. Stakeholders by Requirement 122 4. Stakeholder Definition 123 5. References 124 6. Search 128

Guidebook for Trustworthy AI(General) - Guidebook for Trustworthy AI(Sector)

Level 1 Guidebook for Trustworthy
AI(General)
Level 2 Guidebook for Trustworthy AI(Sector)

1. Guidebook for Trustworthy AI(General)

Providing 15 development requirements and 67 verification items to be used as reference materials for ensuring trustworthiness in the process of developing AI products and services

Ensuring high relevance with key requirements of AI Ethics Standards^*, Reflecting International Requirements for Ensuring AI trustworthiness^**

* Applying the 10 key requirements of “AI Ethics Standards” and concluded with 'respect for diversity', 'liability', 'safety', and 'transparency' as the essential requirements and assessment questions from a technical perspective.

** Deriving and concretizing technical requirements based on policies, recommendations, and standards announced by standardization bodies, technical organizations, international entities, and major nations for ensuring AI trustworthiness.

Requirements/diversity Respect/Responsibility/Safety/Transparency. 01. development and Execution of Risk Management Plans for Artificial Intelligence Systems:Responsibility,Transparency. 02. Establishment of an Artificial Intelligence Governance Framework:Diversity Respect,Responsibility,Safety,Transparency. 03. Formulation of Reliability Testing Plans for Artificial Intelligence Systems:Safety,Transparency. 04. Detailed Information Provision for Data Utilization:Responsibility,Transparency 05. Anomaly Data Checking for Data Robustness Assurance:Safety. 06. Bias Removal from Collected and Processed Training Data:Diversity Respect,Responsibility,Transparency. 07. Ensuring Security and Compatibility of Open Source Libraries:Responsibility,Safety. 08. Bias Mitigation in Artificial Intelligence Models:Diversity Respect. 09. Establishment of Defense Measures Against Attacks on Artificial Intelligence Models:Safety. 10. Provision of Specifications for Artificial Intelligence Models and Explanations for Inference Results:Responsibility,Transparency. 11. Bias Removal during the Implementation of Artificial Intelligence Systems:Diversity. 12. Implementation of Safety Modes and Establishment of Procedures for Problem Notification in Artificial Intelligence Systems:Responsibility,Safety,Transparency. 13. Enhancement of User Understanding for Explanations of Artificial Intelligence System Behavior:Transparency. 14. Ensuring Traceability and Change History of Artificial Intelligence Systems:Responsibility,Transparency. 15. Providing Descriptions for the Scope of Service Delivery and Interaction Targets:Responsibility,Transparency.

2. Guidebook for Trustworthy AI(Sector)

Providing sector-specific specialized use cases based on requirements and assessment questions of the 'Guidebook for Trustworthy AI(General)' to enhance practical use

Currently('23), 'Guidebook for Trustworthy AI' for the 'Public & Social,' 'Autonomous Driving,' and 'Medical' sectors has been provided. In addition, 'Generative AI,' 'Smart Security,' and 'Hiring' sectors-specific 'Guidebook for Trustworthy' AI is going to be provided in the first half of next year ('24)

Guidebook for Trustworthy AI(Sector) Coverage

Medical

'Machine Learning-enabled Medical Devices (MLMD) inclduing disease diagnosis/prediction and medical imaging software, etc.
Autonomous Driving

Autonomous driving system including autonomous driving algorithm (e.g. judgment/control system based on cognitive results)
Public ＆ Social

Government to Customer (G2C) and Business to Government (B2G) AI services
Generative AI

AI services that provide content(e.g. texts and images) based on AI generation technologies
Smart Security

CCTV video contents-based smart security AI services (e.g. crime detection)
Hiring

User-face-to-face hiring services based on AI technologies

Recommend selecting appropriate sector-specific requirements and assessment questions considering the characteristics of AI services during AI trustworthiness assurance activities

Structure of the Guidebook for Trustworthy AI Development

① 안전성 ② 요구사항 05 데이터 강건성 확보를 위한 이상 abnormal 데이터 점검 ③ 대표 행위자 데이터 과학자 협력 대상 데이터 공급자 인공지능 모델 개발자 ④ 인공지능 모델의 학습에 활용되는 데이터는 이상값, 중독 및 회피 등에 영향을 받지 않아야 하며, 이의 점검 및 방어 기법의 적용을 통해 강건성을 확보한다. ⑤ 05-1 이상 데이터의 식별 및 정상 여부를 점검하였는가? Yes No N/A ⑥ 이상 데이터란 학습용 데이터를 구성하는 데이터셋의 수집 및 가공 과정에서 발생할 수 있는 다양한 오류 error 와 일반적인 데이터의 범위에서 크게 벗어난 데이터 이상값 outlier 을 포괄한다. 학습 데이터의 수집 및 가공 과정에서 발생하는 이상 데이터는 데이터상의 노이즈, 학습 데이터 내의 편향, 잘못된 라벨링, 라벨링 누락 등 다양한 원인에 의해 생길 수 있으며 이를 해결하지 않으면 인공지능 모델의 성능 및 강건성 확보가 어렵다. ① Core Attributes Core Attributes: Listing ethical attributes that can be achieved that can be activated through requirements. ② Requirements Specifying technical requirements for ensuring AI reliability at different stages of the lifecycle. ③ Key Actors Identifying representatives who will carry out trust assurance activities for each requirement. ④ Requirement Explanation Providing explanations, including the necessity and technical details of the requirements, to enhance understanding. ⑤ Detailed Requirements Enumerating specific sub-items necessary to meet each requirement. ⑥ Detailed Requirement Explanation Offering explanations for the necessity and technical aspects of the sub-items to facilitate comprehension. ⑦ 05-1b 학습 데이터 이상값 식별 기법을 적용하였는가? Yes No N/A ⑧ 데이터 전처리 과정에서 중요한 활동 중 하나는 데이터 이상값을 식별하고 이를 제거하는 것이다. 데이터 누락과는 달리 데이터 이상값의 경우에는 데이터값이 이미 정해져 있지만, 전체 데이터셋을 기준으로 정상 범주를 벗어난 값이므로 단순 탐색만으로 발견하기 쉽지 않다. 데이터 이상값을 식별하는 방법에는 주로 데이터 전체에 대해 통계적 기법을 적용하여 전체 데이터셋을 고려하였을 때 차별화되는 데이터 포인트를 찾아내는 방법 등이 있으며, 이와 관련 대표적인 기법은 Z-점수, 사분위수 범위 등이다. ⑨ 데이터 이상값 식별 기법 예시 이상값 식별 기법 분류 설명 Z-점수 가장 간단한 통계적 측정 방법으로, Z-점수를 주어진 데이터셋의 분포 평균과 표준편차를 이용하여 관찰된 데이터 포인트가 전체 데이터로부터 얼마나 멀리 떨어져 있는지를 수치화한다. 사분위수 중앙값(Q2)으로 데이터를 두 부분으로 나누고, 다시 왼쪽 중앙값(Q1)과 오른쪽 중앙값(Q3)으로 나누어 총 4개의 범위를 정하여 사분위수 범위(Q3-Q1)를 구해 해당 범위를 벗어나면 이상값으로 판별한다. ⑦ Self-Assessment Items Self-Assessment Items for Meeting Detailed Requirements: Identifying self-assessment criteria necessary to fulfill the detailed requirements. ⑧ Self-Assessment Item Explanation Technical Requirements for Ensuring AI Reliability at Each Lifecycle Stage: Specifying the technical requirements for ensuring AI reliability in various stages of the lifecycle. ⑨ References Examples, Cases, and Reference Information Related to Detailed Requirements: Providing illustrative examples, case studies, and additional reference materials related to the detailed requirements.