Model Risk Management for Machine Learning Models

Model Risk Management for Machine Learning Models
QuantUniversity is back with a NEW COURSE! Join our expert, Sri Krishnamurthy, as he brings clarity on some of the model risk management and validation challenges with data science and machine learning models in the enterprise.  The core course is five weeks. Add-on a Guided Exercise to enhance your learning experience.  PRMIA Special Pricing: Receive a $100 discount at check-out use code:  MRMPRMIA
 

Register Now
Note: You will leave PRMIA.org and be taken to our training partner's registration site. 

Agenda

Course Experts

CRL Credits


Presented By:
Sri Krishnamurthy, CFA, CAP
Founder, QuantUniversity.com


Lesson Length:
90-minute lessons & labs
5 lessons/one per week


Time: Self-paced


Course Dates:
July 14 - August 11, 2020
Optional Guided Exercise:
August 18 - 25, 2020
Instructor Access concludes August 31, 2020



PRMIA Special Pricing:
Receive a $100 discount at check-out use code:  MRMPRMIA

About This Course
 

MEET OUR TRAINING PARTNER 

This course is being delivered by a PRMIA Training PartnerQuantUniversity. All registrations, payments, and course operations will be managed exclusively by QuantUniversity. When registering for the course, you will leave PRMIA.org and use QuantUniversity's registration system, proprietary learning management system, and www.qu.academy for labs. See QuantUniversity course policy below.


COURSE DESCRIPTION

Lecture & hands-on lab work!
The financial industry has been adopting AI and machine learning at a rapid pace. Alternative datasets including text analytics, cloud computing, algorithmic trading are game changers for many firms exploring novel modeling methods to augment their traditional investment and decision workflows. As more and more open-source technologies penetrate enterprises, quants and data scientists have a plethora of choices for building, testing and scaling models. While there is significant enthusiasm, model risk professionals and risk managers are concerned about the onslaught of new technologies, programming languages, and data sets that are entering the enterprise.  With little formal guidance from regulators on how to validate models and quantify model risk, organizations are developing their own home-cooked methods to address model risk management challenges.

In this course, we aim to bring clarity on some of the model risk management and validation challenges with data science and machine learning models in the enterprise. We will discuss key drivers of model risk in today’s environment and how the scope of model risk management is changing. We will introduce key concepts and discuss aspects to be considered when developing a model risk management framework incorporating data science techniques and machine learning methodologies in a pragmatic way.

Learning Objectives
Upon completion of this course, you will be able to:

• Understand the role of machine learning and AI in financial services
• Identify model risk management challenges with machine learning models
• Validate machine learning models: Quantifying risk, best practices and templates
• Translate regulatory guidance
• Apply lessons learned from practical case studies with sample code

Join Sri for an optional Guided Exercise, August 18 - 25, 2020 
Participants will go through a guided exercise to perform model validation on a chosen machine learning model of their choice. Guidance will be provided in scoping and implementing the project. You will then have the opportunity to demonstrate your findings and receive feedback. This is a great opportunity to apply what you have learned in a test environment and receive feedback to ensure your understanding. Subsequently, this will allow you to immediately apply new skills to your role/position.

How It Works

This course is delivered by QuantUniversity. Shortly before the course start date, you will be provided with login credentials and a link to the course lecture and labs. Each Tuesday a lesson is launched. Although this is a self-paced course, we highly recommend you attend weekly classes and participate in labs during the week scheduled to get the most out of the course and support from the faculty. 

 
 
Agenda
 Lesson/Week   Topic
 Lesson 1
 July 14, 2020   

  Machine Learning and AI: A Model Risk Perspective
  • Drivers of Model Risk in the age of data science and AI
  • Machine Learning vs Traditional quant models: How has the world changed?
  • A tour of Machine Learning and AI methods
  • Supervised vs Unsupervised Learning (Regression, Neural Networks, XGBoost, PCA, Clustering)
  • Deep Learning & Reinforcement Learning (Keras, Tensorflow, PyTorch)
  • Automatic Machine Learning (Data Robot, H20.ai etc.) & Machine Learning APIs (Google,
    Comprehend, Watson)
  • ML on the cloud vs On-prem
 Lesson 2 
 July 21, 2020

  Model Risk Management for Machine Learning Models - Part 1
  • Models redefined: Data, Modeling environment, Modeling tools, Modeling process
  • The Decalogue: Ten key aspects to factor when developing your model risk management framework when integrating Machine Learning models:
    1. Models redefined: It’s not just input, process and output
    2. Governing the Machine Learning process
    3. Model Verification and Validation for Machine Learning Models
    4. Performance Metrics and Evaluation criteria
    5. Model Inventory and tracking
 Lesson 3 
 July 28, 2020

  Model Risk Management for Machine Learning Models – Part 2
  • The Decalogue: Ten things to think about when developing your model governance framework when integrating Machine Learning models (cont’d): 
    6. Integrating Data Governance and Model Governance
    7. Development Models vs Production Models
    8. Fairness, Reproducibility, Auditability, Explainability, Interpretability & Bias
        a. How do we objectively measure these?
        b. Review of the Apple-Goldman Sachs credit card debacle
    9. Machine Learning options and considerations
         a. AutoML (Data Robot, H20.ai, etc.), ML as a service (Google, Comprehend, Watson) and home-cooked custom models
    10. ML and Governance: Roles and Responsibilities redefined
 Lesson 4 
 August 4, 2020
   Pragmatic Model Risk Management for AI/ML models
  • Challenges and best practices for pragmatic model management within the enterprise
  • Working with open source projects
  • Working with vendor models and machine learning APIs
  • Quantifying model risk for machine learning models
  • Model risk management for deep-learning models
  • Validation criteria and best practices
  • Templates for Model Validation for machine learning models
  • Use of Synthetic datasets
 Lesson 5
 August 11, 2020
   Hands-on Case study

 Validating a Credit Risk Machine Learning Model:

  • A case study illustrating a model validation of a credit risk model involving machine learning
  • Working with Regression, Neural Networks, and Random Forest models
  • Development models vs Production models
    • Sample templates and worksheets will be provided
  • Roadmap for the MRM team to upskill and keep abreast of changes in the AI and ML landscape
  • Training, education, and expectation setting
  • Future outlook: Regulation, Sandboxes, Frameworks
  • Review of recent regulatory efforts
  • How should companies proactively plan for changes and the future?
(Optional) 
Lesson 6
August 18, 2020
  Guided Exercise, Part 1: Scope and Implement
Put your newly learned skills to practice while being mentored through the process. Participants will go through a guided exercise to perform model validation on a machine learning model of their choice. Guidance will be provided in scoping and implementing the project.      
       
(Optional) 
Lesson 7
August 25, 2020
  Guided Exercise, Part 2: Demonstrate Your Skills

Participants will demonstrate their findings to the class and obtain feedback from instructors and industry participants.

 
 
 


Who Should Attend
  • Model Risk Professionals, Model Validators, Regulators and Financial Professionals new to data-driven methodologies
  • Quantitative Analysts, Investment Professionals, machine learning enthusiasts interested in understanding model risk and governance aspects in Fintech
  • Insurance and financial organizations

About Our Experts

  
  Sri Krishnamurthy, CFA, CAP is the founder of QuantUniversity.com, a data and Quantitative Analysis Company, and the creator of the Analytics Certificate program and Fintech Certificate program. Sri has more than 15 years of experience in analytics, quantitative analysis, statistical modeling and designing large-scale applications. Prior to starting QuantUniversity, Sri worked at Citigroup, Endeca, MathWorks, and with more than 25 customers in the financial services and energy industries. He has trained more than 1,000 students in quantitative methods, analytics, and big data in the industry and at Babson College, Northeastern University, and Hult International Business School. Sri earned an MS in Computer Systems Engineering, an MS in Computer Science, both from Northeastern University, and an MBA with a focus on Investments from Babson College.

 
QuantUniversity  is a quantitative analytics and machine learning advisory company based in Boston, Massachusetts. QuantUniversity runs various data science and machine learning workshops in Boston, New York, Chicago, San Francisco and online.  


Continued Risk Learning Credits: 9
(+ 3.5 for optional exercise)

PRMIA Continued Risk Learning (CRL) programs provide you with the opportunity to formally recognize your professional development, documenting your evolution as a risk professional. Employers can see that you are not static, making you a highly valued, dynamic, and desirable employee. The CRL program is open to all Contributing, Sustaining, and Risk Leader members, providing a convenient and easily accessible way to submit, manage, track and document your activities online through the PRMIA CRL Center. To request CRL credits, please email learning@prmia.org.

Registration
Exclusive PRMIA NETWORK Discount: PRMIA has partnered with QuantUniversity to offer a $100 discount for registrations to this course. To avail this discount, use code: MRMPRMIA when registering.
 Membership Type: 5-week Core Course w/Optional Exercise

 Use discount code ==>>  MRMPRMIA
 All PRMIA Members and Network Members $549  $699
 General Public Rate $649 $799
     

If this is your first time accessing the PRMIA website you will need to create a short user profile to register. Save on registration by becoming a member.

 

Register Now

 Note: You will leave PRMIA.org and be taken to our training partner's registration site. 
Need support?  Contact QuantUniversity at info@qusandbox.com

QUANTUNIVERSITY REGISTRATION POLICIES

This course is being delivered by a PRMIA Training Partner, QuantUniversity. All registrations, payments, and course operations will be managed exclusively by QuantUniversity. When registering for the course, you will leave PRMIA.org and use QuantUniversity's registration system, proprietary learning management system, and www.qu.academy for labs. Questions and requests should be directed to QuantUniversity at info@qusandbox.com.

Cancellations received up to one week from the course start date (July 7, 2020) will receive a full refund less a $100 processing fee.  After this date, refunds or credits will not be issued; the registrant forfeits full payment. This cancellation policy supersedes PRMIA's posted cancellation policy.  In the event sufficient registrations are not received, PRMIA and/or QuantUniversity reserve the right to cancel or reschedule this course one week prior to the start date. Full refunds will be issued if cancelled by the training partner. 

 
When
7/14/2020 - 8/25/2020
Where
Virtual Course
 

Sign In to Register for Event