About This Course
The use of AI and machine learning in finance has grown significantly in the last few years. As more and more AI and ML applications are being deployed in enterprises, concerns are growing about the increased complexity of models, the growing ecosystem of untested frameworks and products, potential for AI accidents, model and reputation risk.  As the debate about explainability, fairness, bias, and privacy grows, there is increased attention to understanding how the models work and whether the models are designed and  thoroughly tested to address potential issues.  The growth of data-driven applications have changed the financial industry. AI and ML models have accelerated business transformation, reduced turn-around times and have enabled applications that weren’t feasible just a few years ago. Institutions have ramped up the adoption of ML models and are seeing significant benefits through the growing portfolios of ML based decision making models. While the interest is huge, the challenges of comprehensively testing and evaluating ML models remain. AI accidents and the risks associated with algorithmic decision making is challenging enterprises to innovate and adopt risk management techniques factoring the new realities! In this QuantUniversity Course, we will discuss the key aspects of risks in ML models and discuss key techniques in stress, scenario testing and evaluation of machine learning models. Through examples and case studies, we will discuss the state-of-the-art in testing and evaluation of ML-based models and how to comprehensively address risk when developing, deploying and monitoring ML applications. By the end of the course, participants will have a clear idea of the challenges, best practices and pragmatic tools that can be used to address risks in machine learning models Hands-on examples and case studies through QuSandbox will be provided to reinforce concepts.
Agenda
Lesson		Topic
Lesson 1		Introduction to Machine Learning, AI & Risk • Machine Learning In Finance: A tour of key methods used In Machine Learning • Define risks in ML models • Concept drift, data drift, model drift • Stress, scenario testing and evaluation • Key metrics • The role of algorithmic audits for ML models
Lesson 2		Stress Testing and Scenario Generation • How are AI/ML models different from traditional models? • Scenario stress testing • Reverse stress testing  o Identifying and assessing tail risk scenarios  o Scenario generation  o Role of synthetic data and data augmentation • The ML life cycle and risks
Lesson 3		Metrics and Evaluation for Risk in ML Models • Metrics for quantifying risks In ML models • Working with sensitive data • Detecting data leakage • Quantifying risk and metrics for ML models • Monitoring and retuning/retraining • ML risk reporting • Case Study: A Dashboard for Measuring and Evaluating Risk In ML Models
Lesson 4		Anomalies and Outliers • Detecting and addressing anomalies • Explainability and outlier analysis • Methods for generating and testing for anomalies • Checks for plausibility • Data techniques and ensemble methods to address anomalies • Case Study: Anomaly Detection in Time-series Datasets Using GANS
Lesson 5		Model Validation of Machine Learning Models • Verification vs Validation of ML models • Benchmarking ML models • Challenger models • Backup models • Issues when adopting ML models:   o Model selection challenges   o Interpretability and explainability • Case Study: Validating a ML Model for Credit Risk
Lesson 6		Frontier Topics & Wrap-up • Operationalizing Evaluation Of Risk In ML Models.     o Real-time and near-real time risk evaluation      o Architecture choices for scaling risk calculations     o Issues with integrating traditional and ML models     o Governance mechanisms to address risk in ML models     o Algorithm auditing and issues of bias and fairness     o Adversarial attacks, sensitive data and unknown risks • Frontier Topics     o Deep learning and other ML innovations     o Technologies and trends to look out for

About Our Experts
		Sri Krishnamurthy is the founder of QuantUniversity, a data and Quantitative Analysis Company and the creator of the Analytics Certificate program and Fintech Certificate program. Sri has more than two decades of experience in analytics, quantitative analysis, statistical modeling and designing large-scale applications. Prior to starting QuantUniversity, Sri has worked at Citigroup, Endeca, MathWorks and with more than 25 customers in the financial services and energy industries. He has trained more than 1000 students in quantitative methods, analytics and big data in the industry and at Babson College, Northeastern University and Hult International Business School. Sri earned an MS in Computer Systems Engineering and another MS in Computer Science, both from Northeastern University and an MBA with a focus on Investments from Babson College.
		About Our Training Partner: QuantUniversity is a quantitative analytics advisory focusing on the intersection of Data science, Machine learning and Quantitative Finance. We take a practitioner’s approach to working with pragmatic applications of frontier topics to real-world financial and energy problems. QuantUniversity advises various companies in Quant Finance application development, validation and in algorithmic auditing. We also run data science and machine learning workshops in the United States and online in its Explore-Experience-Excel series through QuAcademy. QuantUniversity is pioneering the next generation platform for Algorithmic auditing that supports anonymization, model escrow and tracking, synthetic data generation and experimentation through the QuSandbox.

PRMIA Continued Risk Learning (CRL) programs provide you with the opportunity to formally recognize your professional development, documenting your evolution as a risk professional. Employers can see that you are not static, making you a highly valued, dynamic, and desirable employee. The CRL program is open to all Contributing, Sustaining, and Risk Leader members, providing a convenient and easily accessible way to submit, manage, track and document your activities online through the PRMIA CRL Center. To request CRL credits, please email [email protected].

If this is your first time accessing the PRMIA website you will need to create a short user profile to register. Save on registration by becoming a member.

Register Now