Traditional AI vs GenAI: Amplified Risks and Challenges in AI Governance Explained

Demystifying the AI journey is always the goal. This article by Martijn Wiertz includes a deep dive explanation of how foundation models are different from conventional AI models and how that affects AI governance. Below are the key takeaways.

MLOps Differs from FMOps

Understanding the operational differences between traditional machine learning (MLOps) and foundation model operations (FMOps) is critical for governance:

• Foundation models are trained for multiple tasks, while conventional models are trained for a specific task.
• Foundation models learn general representations, while conventional models learn task-specific patterns.
• Foundation models are trained on large general datasets, while conventional models are trained on specific data.
• Users can interact with foundation models through prompt engineering, prompt tuning, and fine-tuning.
• Foundation models offer additional parameters like temperature, top-K, repetition penalty, and minimum token count.
• Foundation models support tasks like summarization, entity extraction, and generative tasks.
• Evaluation of foundation models requires different metrics like ROUGE and BLEU.

Traditional AI vs GenAI: Amplified Risks

Risks Associated with Input

• Traditional risks: Legal restrictions on data usage
• Amplified risks: Copyright and other IP issues with the content
• New risks: Vulnerabilities to new types of adversarial attacks such as prompt injection

Risks Associated with Output

• Traditional risks: Performance disparity across individuals or groups
• Amplified risks: Challenges in explaining why output was generated
• New risks: Hallucination, or false content generation

Challenges

• Traditional risks: Documenting data and model details, purpose, potential uses, and harms
• Amplified risks: Increased carbon emission due to high energy requirement to train and operate
• New risks: Homogenizing of culture and thoughts

Foundation Model Selection Questions

When evaluating a foundation model candidate, ask these types of questions about that model:

1. What data sources are used to train the model?
2. How is the training data governed and processed?
3. How are socio-technical harms and risks mitigated?
4. What are the energy consumption and carbon emissions, for training and for inferencing?
5. What are the terms and conditions for using this model (e.g., will your data be used in retraining of the model)?
6. What are the costs of using this model?
7. How is our data secured when we use this model?
8. What intellectual property indemnification does the model creator provide?
9. Which tasks does the model support, and at what quality?
10. Which of the supported tasks will we allow?

FM Use Case Governance Workflow

A structured governance workflow for foundation model use cases should include:

1. Model approval
2. Use case approval
3. Model selection
4. Model fine tuning
5. Prompt development
6. Evaluation and monitoring
7. Change requests