AI-300T00: Operationalize Machine Learning and Generative AI Solutions

Module 1: Experiment with Azure Machine Learning

• Introduction
• Preprocess data and configure featurization
• Run an automated machine learning experiment
• Evaluate and compare models
• Configure MLflow for model tracking in notebooks
• Train and track models in notebooks
• Evaluate models with the Responsible AI dashboard
• Exercise – Find the best classification model with Azure Machine Learning
• Module assessment

Module 2: Perform hyperparameter tuning with Azure Machine Learning

• Introduction
• Define a search space
• Configure a sampling method
• Configure early termination
• Use a sweep job for hyperparameter tuning
• Exercise – Run a sweep job
• Module assessment

Module 3: Run pipelines in Azure Machine Learning

• Introduction
• Create components
• Create a pipeline
• Run a pipeline job
• Exercise – Run a pipeline job
• Module assessment

Module 4: Trigger Azure Machine Learning jobs with GitHub Actions

• Introduction
• Understand the business problem
• Explore the solution architecture
• Use GitHub Actions for model training
• Exercise
• Module assessment
• Module 5: Trigger GitHub Actions with feature-based development
• Introduction
• Understand the business problem
• Explore the solution architecture
• Trigger a workflow
• Exercise
• Module assessment

Module 5: Work with environments in GitHub Actions

• Introduction
• Understand the business problem
• Explore the solution architecture
• Set up environments
• Exercise
• Module assessment

Module 6: Deploy a model with GitHub Actions

• Introduction
• Understand the business problem
• Explore the solution architecture
• Model deployment
• Exercise
• Module assessment

Module 7: Plan and prepare a GenAIOps solution

• Introduction
• Explore use cases for GenAIOps
• Select the right generative AI model
• Understand the development lifecycle of a language model application
• Explore available tools and frameworks to implement GenAIOps
• Exercise – Compare language models from the model catalog
• Module assessment

Module 8: Manage prompts for agents in Microsoft Foundry with GitHub

• Introduction
• Apply version control to prompts
• Understand Microsoft Foundry agents and prompt versioning
• Organize prompts in GitHub repositories
• Develop safe prompt deployment workflows
• Exercise – Develop prompt and agent versions
• Knowledge check

Module 9: Evaluate and optimize AI agents through structured experiments

• Introduction
• Design evaluation experiments
• Apply Git-based workflows to optimization experiments
• Apply evaluation rubrics for consistent scoring
• Exercise – Evaluate and compare AI agent versions
• Knowledge check

Module 10: Automate AI evaluations with Microsoft Foundry and GitHub Actions

• Introduction
• Understand why automated evaluations matter
• Align evaluators with human criteria
• Create evaluation datasets
• Implement batch evaluations with Python
• Integrate evaluations into GitHub Actions
• Exercise – Set up automated evaluations
• Knowledge check

Module 11: Monitor your generative AI application

• Introduction
• Why do you need to monitor?
• Understand key metrics to monitor
• Explore how to monitor with Microsoft Azure
• Integrate monitoring into your app
• Interpret monitoring results
• Exercise – Enable monitoring for a generative AI application
• Knowledge check

Module 12: Analyze and debug your generative AI app with tracing

• Introduction
• Why do you need to use tracing?
• Identify what to trace in generative AI applications
• Implement tracing in generative AI applications
• Debug complex workflows with advanced tracing patterns
• Make informed decisions with trace data analysis
• Exercise – Enable tracing for a generative AI application
• Knowledge check