зеркало из https://github.com/microsoft/lisa.git
Juhlee/contents update (#40)
* revision in how-to-use.md file * revision in how-to-use.md file * Removed testing lines * Update TODO * fixed the url
This commit is contained in:
Ju-Hyoung Lee
GitHub
Родитель
d034de81f0
Коммит
d94e1cfd1a
|
|
@ -5,51 +5,24 @@
|
|||
## Objective
|
||||
|
||||
This document instructs for partners to create customized menu of Jenkins pipeline and run either/both Microsoft-provided tests or/and customized test cases in Jenkins. This instruction has 2 major parts; creating menu from xml files and executing tests.
|
||||
|
||||
## Prepare a VHD for tests.
|
||||
|
||||
1. Start PowerShell with the Run As Administrator option.
|
||||
2. In powershell, Goto Automation Folder.
|
||||
3. Run following command.
|
||||
.\AzureAutomationManager.ps1 -xmlConfigFile .\Azure_ICA_all.xml -cycleName autosetup -Distro YourDistroName -runtests
|
||||
|
||||
This command will install "Minimum Required Packages" and will capture the VHD which can be used to run further tests like, Network tests & VNET tests.
|
||||
|
||||
List of minimum packages in VHD
|
||||
iperf
|
||||
mysql
|
||||
gcc
|
||||
bind
|
||||
bind-utils
|
||||
bind9
|
||||
python
|
||||
python-argparse
|
||||
python-crypto
|
||||
python-paramiko
|
||||
libstdc++6
|
||||
psmisc
|
||||
nfs-utils
|
||||
nfs-common
|
||||
tcpdump
|
||||
|
||||
4.Once you get the prepared VHD Name, create a new element "Distro" in XML file and give prepared VHD name in Distro element.
|
||||
Creating menu may be optional in your test development. If you keep the same menu structure and add new tests into existing menu, you will only update existing XML files in ./XML/TestCases/
|
||||
If you add new test menu, you will need to re-build Jenkins menu, and then can run new tests.
|
||||
|
||||
## Develop tests in GitHub
|
||||
|
||||
`Source: https://github.com/LIS/LISAv2`
|
||||
|
||||
1. XML folder: it has pre-defined global configuration and account information. It also has Region information in xml files. This folder also has two sub folders; TestCases and VMConfigurations. VMConfigurations has the list of xml files for each test case. TestCases folder has the list of xml files for each test category. The master branch is owned by Microsoft, and actively manage the PR in LISAv2. New test case development and/or new menu development must be approved by Microsoft.
|
||||
2. TestScripts folder has the number of test scripts defined in TestCases, and separated by OS type.
|
||||
3. Tools folder has binary files required for test execution.
|
||||
4. This repo will provide Microsoft-provided test cases as well as Partner-developed test cases.
|
||||
a. Microsoft will share the test development plan and its log with partners. If this is the case, you can skip the next paragraph to ‘Verify a published image on Azure’.
|
||||
b. Partners’ developed test cases should be followed below steps.
|
||||
i. Sync up the local master branch from remote master branch in the GitHub project, if new work branch is in the LISAv2. Otherwise, you can folk the LISAv2 repo to your own GitHub account.
|
||||
1. ./XML folder: it has pre-defined global configuration and account information as well as Region information. This folder also has two sub folders; 'TestCases' and 'VMConfigurations'. 'VMConfigurations' has the list of xml files for each test case, which require the list of VM configuration information. 'TestCases' folder has the list of xml files for each test category. The master branch is owned by Microsoft, and actively manage changes in LISAv2. New test case development and/or new menu development must be approved by Microsoft.
|
||||
2. 'Testscripts' folder has the number of test scripts used in TestCases, and separated by OS type.
|
||||
3. 'Tools' folder has binary files required for test execution.
|
||||
4. This repo will provide Microsoft-provided test cases as well as capability of Partner-developed test cases.
|
||||
a. Microsoft will share the test development plan and its log with partners. If you run Microsoft-provided tests, you can skip the next paragraph to ‘Verify a published image on Azure’.
|
||||
b. Partner-developed test cases should be followed below steps.
|
||||
i. Sync up the local 'master' branch from remote 'master' branch in the GitHub project, if new work branch is in the LISAv2. Otherwise, you can folk the LISAv2 repo to your own GitHub account.
|
||||
ii. Branch out for work and pull down to your local system.
|
||||
iii. Once change is ready to review, create a PR from LISAv2 in your account to LIS account. Or, new working branch to master in LISAv2 repo.
|
||||
iv. Add ‘LisaSupport@microsoft.com’ to ‘Reviewers’.
|
||||
v. Once it is approved, then you can merge the PR to master branch.
|
||||
vi. In this case, you will need to rebuild menu by ‘<Partner name>-Refresh-Test-Selection-Menus’
|
||||
iv. Add ‘LisaSupport@microsoft.com’ to ‘Reviewers’, or send email to 'lisasupport@microsoft.com'.
|
||||
v. Once it is approved, you can merge the PR to master branch. In this case, you will need to rebuild menu by ‘<Partner name>-Refresh-Test-Selection-Menus’.
|
||||
|
||||
## Verify a published image on Azure
|
||||
|
||||
|
|
@ -58,9 +31,9 @@
|
|||
1. Sign in to Jenkins page with the assigned user name & password
|
||||
2. Browse to '<Partner name>-Refresh-Test-Selection-Menu', if you would like to apply new menu or test cases before test execution.
|
||||
3. Click 'Build with Parameters' in left panel menu
|
||||
a. Enter git repo ULR and branch name for menu xml file. Recommend keeping the default Repo URL and branch name.
|
||||
a. Enter the git repo ULR and branch name of new menu xml file. Recommend keeping the default Repo URL and branch name.
|
||||
b. Click 'Build' button.
|
||||
c. Click the rotation icon of running job and verify 'SUCCESS' inside 'Console Output'. You have new menu/test cases in Jenkins.
|
||||
c. Click the rotation icon of running job in 'Build History' and verify "SUCCESS" inside 'Console Output'. You have new menu/test cases in Jenkins.
|
||||
4. Browse to '<Partner name>-Launch-Tests' and submit the job for test execution.
|
||||
5. Click 'Build with Parameters' in left panel menu
|
||||
a. Select 'ImageSource' or navigate to 'CustomVHD'. If you have external source URL, you can enter it in 'CustomVHDURL' text box.
|
||||
|
|
@ -69,17 +42,19 @@
|
|||
i. Supported platform: Azure, etc.
|
||||
ii. Available Category: BVT, Community, Functional, Performance, Smoke.
|
||||
iii. Available Tags: boot, bvt, disk, ext4, gpu, hv_netvsc, etc.
|
||||
6. Enter the email address for report notification.
|
||||
6. Enter partner's email address for report notification.
|
||||
|
||||
`API/cmdline Instruction`
|
||||
|
||||
A single script executes the test launch/execute with pre-defined parameters.
|
||||
TODO: TBD
|
||||
`‘Script name and its parameters’`
|
||||
$TriggerTestPipelineRemotely 'parameters'
|
||||
Example,
|
||||
TODO: TBD
|
||||
`‘Script name’ ‘parameters’`
|
||||
A single script executes the test launch/execute with pre-defined parameters. It also offers a parameter file mode, which all parameters are set before execution. The test script loads all paramenters from the file. The parameter file is located at ./Utilities/TestParameters.sh
|
||||
$./LaunchTestPipelineRemotely.sh -JenkinsUser "<Azure user account for ApiToken>" -ApiToken "<your access token>" -FtpUsername "<FTP user account>" -FtpPassword "<FTP user account password>" [-ImageSource "<Publisher Offer Sku Version>" | -CustomVHD "<path of vhd file>" | -CustomVHDURL "<URL of custom vhd file>"] -Kernel "[default|custom|linuxnext]" [-CustomKernelFile "<ONLY IF you set Kernel=custom>" | -CustomKernelURL "<ONLY IF you set Kernel=custom>"] -GitUrlForAutomation "https://github.com/LIS/LISAv2.git" -GitBranchForAutomation "master" [-TestByTestname "" | -TestByCategorisedTestname "" | -TestByCategory "" | -TestByTag ""] -Email "<Partner email for test result report>" -TestPipeline "<Partner pipeline name>" -LinuxUsername "<Linux guest OS user name>" -LinuxPassword "<Linux guest OS user password>" [-WaitForResult "yes"]
|
||||
|
||||
$./LaunchTestPipelineRemotely.sh -ParametersFile "<parameter definition file>"
|
||||
|
||||
`‘Script name and its parameters’ example`
|
||||
$./LaunchTestPipelineRemotely.sh -JenkinsUser "microsoft" -ApiToken "123451234512345dlkwekl2kfo" -FtpUsername "ftpuser" -FtpPassword "ftppassword! [-ImageSource "linux-next_1.2" | -CustomVHD "/path/to/local/vhd/vhdx/vhd.xz" | -CustomVHDURL "http://downloadable/link/to/your/file.vhd/vhdx/vhd.xz"] -Kernel "default" -GitUrlForAutomation "https://github.com/LIS/LISAv2.git" -GitBranchForAutomation "master" -TestByTestname "Azure>>VERIFY-DEPLOYMENT-PROVISION>>eastasia,Azure>>VERIFY-HOSTNAME>>westeurope" -TestByCategorisedTestname "Azure>>Smoke>>default>>VERIFY-DEPLOYMENT-PROVISION>>northeurope,Azure>>Functional>>SRIOV>>VERIFY-SRIOV-LSPCI>>southcentralus" -TestByCategory "Azure>>Functional>>SRIOV>>eastus,Azure>>Community>>LTP>>westeurope" -TestByTag "Azure>>boot>>northcentralus,Azure>>wala>>westeurope,Azure>>gpu>>eastus" -Email "lisasupport@microsoft.com" -TestPipeline "Microsoft-Test-Execution-Pipeline" -LinuxUsername "linuxuser" -LinuxPassword "linuxpassword?"
|
||||
|
||||
$./LaunchTestPipelineRemotely.sh -ParametersFile "TestParameters.sh"
|
||||
|
||||
## GlobalConfigurations.xml in XML folder
|
||||
|
||||
|
|
@ -95,29 +70,28 @@
|
|||
|
||||
## XML files in XML/TestCases folder
|
||||
|
||||
This location has the list of XML files for test cases. Each XML file names after category for each maintenance / sharing.
|
||||
This location has the list of XML files for test cases. Each XML file names after category for each maintenance / sharing.
|
||||
1. BVT.xml: BVT (Build Validation Test) test cases
|
||||
2. CommunityTests.xml: Tests from Open Source Community.
|
||||
3. FunctionalTests.xml: Feature tests for SR-IOV, GPU, DPDK, etc.
|
||||
4. Other.xml: If any does not fall into existing Category, add to here.
|
||||
5. PerformanceTests.xml: Performance test cases
|
||||
6. RegressionTests.xml: Add any tests for regression cycle.
|
||||
6. RegressionTests.xml: Add any tests for regression cycle.
|
||||
7. SmokeTests.xml: It will run before BVT test runs.
|
||||
8. StressTests.xml: Under development. Network traffic and stroage IO testing under heavy CPU and Memory stress.
|
||||
|
||||
Here is the format inside of TestCases.xml file. TODO: Revise the definition, and required field or not.
|
||||
<testName></testName>: Represent unique Test Case name
|
||||
<testScript></testScript>
|
||||
<PowershellScript></PowershellScript>: Actual PS script file name.
|
||||
<files></files>
|
||||
<setupType></setupType>: The name represents VM's size and its definition in TestsConfigurations.xml file, VMConfigurations folder.
|
||||
<TestType></TestType>
|
||||
<TestFeature></TestFeature>
|
||||
<Platform></Platform>: Supported platform names. Azure, HyperV, etc.
|
||||
<Category></Category>: Available Test Category
|
||||
<Area></Area>: Test Area
|
||||
<Tags></Tags>: Tag information seperated by comma
|
||||
<TestID></TestID>: Unique Test ID used in Jenkins.
|
||||
[Req] Required
|
||||
[Opt] Optional
|
||||
<testName></testName>: Represent unique Test Case name [Req]
|
||||
<testScript></testScript>: test script file name [Opt]
|
||||
<PowershellScript></PowershellScript>: Actual launch PS script file name. [Req]
|
||||
<files></files>: If test requires data files, add the file names here [Opt]
|
||||
<setupType></setupType>: The name represents VM definition in <Category name>TestsConfigurations xml file, VMConfigurations folder. [Req]
|
||||
<Platform></Platform>: Supported platform names. Azure, HyperV, etc. [Req]
|
||||
<Category></Category>: Available Test Category [Req]
|
||||
<Area></Area>: Test Area [Req]
|
||||
<Tags></Tags>: Tag information seperated by comma [Opt]
|
||||
|
||||
## TestsConfigurations.xml in XML/VMConfigurations
|
||||
|
||||
|
|
|
|||
40
README.md
40
README.md
|
|
@ -4,7 +4,7 @@ Automation platform for Linux images testing on Microsoft Azure and Hyper-V
|
|||
|
||||
## Overview
|
||||
|
||||
LISAv2 (Linux Integrated Service Automation) is the One-stop automation solution for Linux images/kernel testing on Microsoft Azure and Hyper-V. LISA-v2 supports both Microsoft Azure and Hyper-V automation, and they use PowerShell, BASH and python scripts. Tests for feature, performance, stress and regression about new Linux Operating Systems and kernels. The test suite provides Build Verification Tests (BVTs), Azure VNET Tests and Network tests.
|
||||
LISAv2 (Linux Integrated Service Automation) is the One-stop automation solution for Linux images/kernel testing on Microsoft Azure and Hyper-V. LISA-v2 supports both Microsoft Azure and Hyper-V automation, and they use PowerShell, BASH and python scripts. It includes feature, performance, stress and regression tests about new Linux Operating Systems and Kernels. The test suite provides Build Verification Tests (BVTs), Azure VNET Tests and Network tests also.
|
||||
|
||||
### Prerequisite
|
||||
|
||||
|
|
@ -28,20 +28,18 @@ LISAv2 (Linux Integrated Service Automation) is the One-stop automation solution
|
|||
|
||||
### Download Latest Azure PowerShell
|
||||
|
||||
1. Download Web Platform Installer from [here](http://go.microsoft.com/fwlink/p/?linkid=320376&clcid=0x409)
|
||||
2. Start Web Platform Installer and select Azure PowerShell and proceed for Azure PowerShell Installation.
|
||||
1. Download Web Platform Installer from [here](http://go.microsoft.com/fwlink/p/?linkid=320376&clcid=0x409)
|
||||
2. Start Web Platform Installer and select Azure PowerShell (Recommend 6.0.0 or above) and proceed for Azure PowerShell Installation.
|
||||
|
||||
### Authenticate Your Machine with Your Azure Subscription
|
||||
|
||||
There are two ways to authenticate your machine with your subscription.
|
||||
|
||||
1. Azure AD method
|
||||
|
||||
This creates a 12 Hours temporary session in PowerShell, in that session, you are allowed to run Windows Azure Cmdlets to control / use your subscription. After 12 hours you will be asked to enter username and password of your subscription. This may create problems long running automations, hence we use certificate method.
|
||||
This creates a 12 Hours temporary session in PowerShell, in that session, you are allowed to run Windows Azure Cmdlets to control / use your subscription. After 12 hours you will be asked to enter username and password of your subscription. This may create problems long running automations, hence we use service principal method.
|
||||
|
||||
2. Certificate Method.
|
||||
2. Service Principal method
|
||||
|
||||
To learn more about how to configure your PowerShell with your subscription, please visit [here](http://azure.microsoft.com/en-us/documentation/articles/powershell-install-configure/#Connect).
|
||||
Refer to this URL [here](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal)
|
||||
|
||||
### Update GlobalConfigurations.xml file
|
||||
|
||||
|
|
@ -70,7 +68,7 @@ There are two ways to authenticate your machine with your subscription.
|
|||
|
||||
```
|
||||
|
||||
2. Save file.
|
||||
2. Save files.
|
||||
|
||||
### Prepare VHD to work in Azure
|
||||
|
||||
|
|
@ -88,7 +86,7 @@ Please follow the steps mentioned at [here](https://docs.microsoft.com/en-us/azu
|
|||
|
||||
`Applicable if you are using already uploaded VHD / Platform Image to run automation.`
|
||||
|
||||
To run automation code successfully, you need have following packages installed in your Linux VHD.
|
||||
To run automation code successfully, below are the required packages in your Linux VHD.
|
||||
|
||||
1. iperf
|
||||
2. mysql-server
|
||||
|
|
@ -109,26 +107,6 @@ To run automation code successfully, you need have following packages installed
|
|||
17. nfs-common
|
||||
18. tcpdump
|
||||
|
||||
### Create SSH Key Pair
|
||||
|
||||
`PublicKey.cer <20> PrivateKey.ppk`
|
||||
|
||||
A Linux Virtual machine login can be done with Password authentication or SSH key pair authentication. You must create a Public Key and Private key to run automation successfully. To learn more about how to create SSH key pair, please visit [here](http://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-use-ssh-key/).
|
||||
|
||||
After creating Public Key (.cer) and putty compatible private key (.ppk), you must put it in your `automation_root_folder\ssh\` folder and mention their names in Azure XML file.
|
||||
|
||||
### VNET Preparation
|
||||
|
||||
`Required for executing Virtual Network Tests`
|
||||
|
||||
#### Create a Virtual Network in Azure
|
||||
|
||||
A virtual network should be created and connected to Customer Network before running VNET test cases. To learn about how to create a virtual network on Azure, please visit [here](https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-howto-site-to-site-resource-manager-portal).
|
||||
|
||||
#### Create A customer site using RRAS
|
||||
|
||||
Apart from Virtual Network in Azure, you also need a network (composed of Subnets and DNS server) to work as Customer Network. If you don<6F>t have separate network to run VNET, you can create a virtual customer network using RRAS. To learn more, please visit [here](https://social.msdn.microsoft.com/Forums/en-US/b7d15a76-37b3-4307-98e3-d9efef5767b8/azure-site-to-site-vpn-routing?forum=WAVirtualMachinesVirtualNetwork).
|
||||
|
||||
### How to Start Automation
|
||||
|
||||
Before starting Automation, make sure that you have completed steps in chapter [Prepare Your Machine for Automation Cycle](#prepare)
|
||||
|
|
@ -151,7 +129,7 @@ Before starting Automation, make sure that you have completed steps in chapter [
|
|||
|
||||
#### Command to Start any of the Automation Cycle
|
||||
|
||||
.\AzureAutomationManager.ps1 -xmlConfigFile .\Azure_ICA_ALL.xml -runtests -email <20>Distro <DistroName> -cycleName <TestCycleToExecute> -UseAzureResourceManager
|
||||
.\RunTests.ps1 -TestPlatform "Azure" -TestLocation "<Region location>" -RGIdentifier "<Identifier of the resource group>" [-ARMImageName "<publisher offer SKU version>" | -OsVHD "<VHD from storage account>" ] [[-TestCategory "<Test Catogry from Jenkins pipeline>" | -TestArea "<Test Area from Jenkins pipeline>"]* | -TestTag "<A Tag from Jenkins pipeline>" | -TestNames "<Test cases separated by comma>"]
|
||||
|
||||
#### More Information
|
||||
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче