MEET OUR TRAINING PARTNER 

COURSE DESCRIPTION

Lecture & hands-on lab work!

Learning Objectives

Upon completion of this course, you will be able to:

• Understand the role of machine learning and AI in financial services
• Identify model risk management challenges with machine learning models
• Validate machine learning models: Quantifying risk, best practices and templates
• Translate regulatory guidance
• Apply lessons learned from practical case studies with sample code

Join Sri for an optional Guided Exercise, August 18 - 25, 2020 
Participants will go through a guided exercise to perform model validation on a chosen machine learning model of their choice. Guidance will be provided in scoping and implementing the project. You will then have the opportunity to demonstrate your findings and receive feedback. This is a great opportunity to apply what you have learned in a test environment and receive feedback to ensure your understanding. Subsequently, this will allow you to immediately apply new skills to your role/position.

How It Works

This course is delivered by QuantUniversity. Shortly before the course start date, you will be provided with login credentials and a link to the course lecture and labs. Each Tuesday a lesson is launched. Although this is a self-paced course, we highly recommend you attend weekly classes and participate in labs during the week scheduled to get the most out of the course and support from the faculty. 

• Drivers of Model Risk in the age of data science and AI
• Machine Learning vs Traditional quant models: How has the world changed?
• A tour of Machine Learning and AI methods
• Supervised vs Unsupervised Learning (Regression, Neural Networks, XGBoost, PCA, Clustering)
• Deep Learning & Reinforcement Learning (Keras, Tensorflow, PyTorch)
• Automatic Machine Learning (Data Robot, H20.ai etc.) & Machine Learning APIs (Google,
  Comprehend, Watson)
• ML on the cloud vs On-prem

• Models redefined: Data, Modeling environment, Modeling tools, Modeling process
• The Decalogue: Ten key aspects to factor when developing your model risk management framework when integrating Machine Learning models:
  1. Models redefined: It’s not just input, process and output
  2. Governing the Machine Learning process
  3. Model Verification and Validation for Machine Learning Models
  4. Performance Metrics and Evaluation criteria
  5. Model Inventory and tracking

• The Decalogue: Ten things to think about when developing your model governance framework when integrating Machine Learning models (cont’d): 
  6. Integrating Data Governance and Model Governance
  7. Development Models vs Production Models
  8. Fairness, Reproducibility, Auditability, Explainability, Interpretability & Bias
      a. How do we objectively measure these?
      b. Review of the Apple-Goldman Sachs credit card debacle
  9. Machine Learning options and considerations
       a. AutoML (Data Robot, H20.ai, etc.), ML as a service (Google, Comprehend, Watson) and home-cooked custom models
  10. ML and Governance: Roles and Responsibilities redefined

• Challenges and best practices for pragmatic model management within the enterprise
• Working with open source projects
• Working with vendor models and machine learning APIs
• Quantifying model risk for machine learning models
• Model risk management for deep-learning models
• Validation criteria and best practices
• Templates for Model Validation for machine learning models
• Use of Synthetic datasets

 Validating a Credit Risk Machine Learning Model:

• A case study illustrating a model validation of a credit risk model involving machine learning
• Working with Regression, Neural Networks, and Random Forest models
• Development models vs Production models
  • Sample templates and worksheets will be provided
• Roadmap for the MRM team to upskill and keep abreast of changes in the AI and ML landscape
• Training, education, and expectation setting
• Future outlook: Regulation, Sandboxes, Frameworks
• Review of recent regulatory efforts
• How should companies proactively plan for changes and the future?

Participants will demonstrate their findings to the class and obtain feedback from instructors and industry participants.

• Model Risk Professionals, Model Validators, Regulators and Financial Professionals new to data-driven methodologies
• Quantitative Analysts, Investment Professionals, machine learning enthusiasts interested in understanding model risk and governance aspects in Fintech
• Insurance and financial organizations