Live and Recorded Tests

MCP Tools

IMPORTANT: ALWAYS use the azure-sdk-mcp:azsdk_package_run_tests MCP tool to run tests. NEVER run test commands directly in the terminal (e.g., pytest, dotnet test, mvn test, npm test, go test). The MCP tool handles test mode configuration, environment setup, and automatic asset pushing in record mode.

Prerequisites

• azure-sdk-mcp server must be running
• Azure PowerShell module (Az) must be installed. If Get-AzContext fails with a command-not-found error, instruct the user to install it with Install-Module Az -Scope CurrentUser -Force.
• For live/record modes: an Azure subscription with permissions to create resources
  • Use the Azure SDK Test Resources - TME (id: 4d042dc6-fe17-4698-a23f-ec6a8d1e98f4) subscription if not already authenticated and no other subscription is specified.

Steps

1. Identify service directory — Determine the service directory for the package under test (e.g., keyvault, storage). This is required by the test resource deployment script.
2. Check for existing deployment — Look for a .env file at either the service directory level (next to test-resources.bicep, e.g. sdk/storage/.env) or at the package level (e.g. sdk/storage/storage-blob/.env). If one exists, inform the user that a previous deployment appears to be available and ask whether to reuse the existing deployment or redeploy test resources. If reusing, skip to step 7.
3. Verify Azure context — Run Get-AzContext to check for an active Azure PowerShell session. If the command fails because the Az module is not installed, instruct the user to run Install-Module Az -Scope CurrentUser -Force first. If no context exists, guide the user through Connect-AzAccount -TenantId [TME tenant ID]. If Connect-AzAccount appears to hang and no browser window opens for authentication, tell the user to copy and paste the command into a new PowerShell window. Confirm the correct subscription is selected.
4. Confirm deployment — Even if no existing .env file is found, confirm with the user before proceeding to deploy test resources. Deployment creates Azure resources that may incur costs.
5. Deploy test resources — Run eng/common/TestResources/New-TestResources.ps1 with the service directory, tenant ID, subscription ID, and any user-provided parameters. Use the TME tenant ID and subscription ID if one has not already been provided. See deployment parameters for details. The script outputs environment variables needed for live/record test runs.
6. Save environment — Capture the environment variables output by the deployment script. If the script writes a .env file, note its path. Otherwise, collect the environment variables from the script output. In azure-sdk-for-net, if the script outputs a file like test-resources.bicep.env instead of a .env file, move on to step 7 and call the MCP tool WITHOUT passing an env filepath. The test framework will automatically find and handle this special file type.
7. Run tests — Call the azure-sdk-mcp:azsdk_package_run_tests MCP tool (do NOT run test commands directly in the terminal). Provide the appropriate test mode (record, live, or playback) and the path to the .env file containing test environment variables. When tests run in record mode and all tests pass, the tool automatically pushes recorded test assets to the assets repo.
8. Clean up — Ask the user whether to clean up test resources. If yes, run eng/common/TestResources/Remove-TestResources.ps1. If no, inform the user that resources remain deployed for subsequent test runs. See cleanup details.

Examples

• "Deploy test resources and run live tests for this package"
• "Run the tests in record mode using the live test deployment specified in .env"
• "Clean up my test resources for keyvault"
• "Run tests in record mode against my existing deployment"
• "Set up live test resources for storage and run all tests"

Troubleshooting

• No Azure context: Run Connect-AzAccount and select the target subscription with Set-AzContext -SubscriptionId <id>. If the browser window for authentication fails to open and the command appears to hang, tell the user to copy and paste the Connect-AzAccount command into a new PowerShell window and run it there instead.
• Az module not installed: If Get-AzContext fails with a command-not-found error, the Azure PowerShell module is not installed. Instruct the user to run Install-Module Az -Scope CurrentUser -Force and then retry.
• Deployment fails with auth error: Verify that the signed-in account has Contributor or Owner role on the target subscription. Check Get-AzContext output.
• Deployment fails with resource conflict: A previous deployment may still exist. Try running Remove-TestResources.ps1 first, or use a different BaseName.
• Tests fail in record mode: Check that all environment variables from the deployment are being passed correctly. Verify the .env file path is correct.
• Assets push fails after recording: Ensure the assets repo is configured and accessible. Check git authentication.