зеркало из https://github.com/microsoft/lisa.git
rewrap touched documents
This commit is contained in:
Chi Song
Chi Song
Родитель
f5944a6e15
Коммит
96703c2c2a
55
README.md
55
README.md
|
|
@ -1,24 +1,48 @@
|
|||
# Linux Integration Services Automation (LISA)
|
||||
|
||||
[](https://github.com/microsoft/lisa/actions?query=workflow%3A%22CI+Workflow+for+LISAv3%22+event%3Apush+branch%3Amain)
|
||||
[](https://github.com/microsoft/lisa/blob/main/LICENSE)
|
||||
[](https://github.com/microsoft/lisa/actions?query=workflow%3A%22CI+Workflow+for+LISAv3%22+event%3Apush+branch%3Amain)
|
||||
[](https://github.com/microsoft/lisa/blob/main/LICENSE)
|
||||
|
||||
**Linux Integration Services Automation (LISA)** is designed to be an end-to-end solution for verifying Linux kernels and distributions quality on Microsoft virtualization technologies. It can be used on other quality validation, and virtualization technologies as well.
|
||||
**Linux Integration Services Automation (LISA)** is designed to be an end-to-end
|
||||
solution for verifying Linux kernels and distributions quality on Microsoft
|
||||
virtualization technologies. It can be used on other quality validation, and
|
||||
virtualization technologies as well.
|
||||
|
||||
LISA is composed by two parts,
|
||||
|
||||
* A powerful framework to perform tests on Azure, Hyper-V, and other virtualization platforms.
|
||||
* A comprehensive test suite to verify the integration between Linux kernels/distributions and Microsoft virtualization platforms.
|
||||
* A powerful framework to perform tests on Azure, Hyper-V, and other
|
||||
virtualization platforms.
|
||||
* A comprehensive test suite to verify the integration between Linux
|
||||
kernels/distributions and Microsoft virtualization platforms.
|
||||
|
||||
## Why LISA
|
||||
|
||||
There are a lot of classic tools and tests, which focus on the quality of Linux kernels or distributions. They are important to ensure the quality of kernels and distributions. The integration validation on virtualization platforms is a little different with classic Linux testing. It covers diverse types of resources with manageable cost. So that, it needs to plan resources creation and deletion automatically.
|
||||
There are a lot of classic tools and tests, which focus on the quality of Linux
|
||||
kernels or distributions. They are important to ensure the quality of kernels
|
||||
and distributions. The integration validation on virtualization platforms is a
|
||||
little different with classic Linux testing. It covers diverse types of
|
||||
resources with manageable cost. So that, it needs to plan resources creation and
|
||||
deletion automatically.
|
||||
|
||||
LISA focuses on validating the integration of Linux kernels/distributions and virtualization platforms. It needs more interactive with virtualization platforms to run tests for different purposes, like test different capabilities, hardware, and so on.
|
||||
LISA focuses on validating the integration of Linux kernels/distributions and
|
||||
virtualization platforms. It needs more interactive with virtualization
|
||||
platforms to run tests for different purposes, like test different capabilities,
|
||||
hardware, and so on.
|
||||
|
||||
* **End-to-end**: LISA defines several sets of test suites to validate Linux kernels and distributions in Microsoft Azure, Hyper-V, etc. The test suites can help find integration issues easily.
|
||||
* **Ease-to-use**: The complexity and diversity of Linux kernels/distributions are wrapped in different components of LISA. When running LISA, it doesn't need to know details. Developers can focus on validation logic, when creating new tests.
|
||||
* **Extensibility**: LISA is extendable in many components to support various scenarios, including virtualization platforms, commands, Linux distributions, community test suites, etc. LISA supports to validate Microsoft virtualization platforms natively, but also can be extended to other cloud or on-premises platforms.
|
||||
* **End-to-end**: LISA defines several sets of test suites to validate Linux
|
||||
kernels and distributions in Microsoft Azure, Hyper-V, etc. The test suites
|
||||
can help find integration issues easily.
|
||||
* **Ease-to-use**: The complexity and diversity of Linux kernels/distributions
|
||||
are wrapped in different components of LISA. When running LISA, it doesn't
|
||||
need to know details. Developers can focus on validation logic, when creating
|
||||
new tests.
|
||||
* **Extensibility**: LISA is extendable in many components to support various
|
||||
scenarios, including virtualization platforms, commands, Linux distributions,
|
||||
community test suites, etc. LISA supports to validate Microsoft virtualization
|
||||
platforms natively, but also can be extended to other cloud or on-premises
|
||||
platforms.
|
||||
|
||||
## Documents
|
||||
|
||||
|
|
@ -33,13 +57,18 @@ LISA focuses on validating the integration of Linux kernels/distributions and vi
|
|||
|
||||
## Contribute
|
||||
|
||||
You are very welcome to contribute. Please follow [the contribution document](docs/contributing.md) for details.
|
||||
You are very welcome to contribute. Please follow [the contribution
|
||||
document](docs/contributing.md) for details.
|
||||
|
||||
## History and road map
|
||||
|
||||
The previous LISA called LISAv2, which is in [master branch](https://github.com/microsoft/lisa/tree/master). The previous LISA can be used standalone or called from the current LISA. Learn more from [how to run LISAv2 test cases](docs/run_legacy.md).
|
||||
The previous LISA called LISAv2, which is in [master
|
||||
branch](https://github.com/microsoft/lisa/tree/master). The previous LISA can be
|
||||
used standalone or called from the current LISA. Learn more from [how to run
|
||||
LISAv2 test cases](docs/run_legacy.md).
|
||||
|
||||
LISA is in active developing, and a lot of exciting features are implementing. We're listening your [feedback](https://github.com/microsoft/lisa/issues/new).
|
||||
LISA is in active developing, and a lot of exciting features are implementing.
|
||||
We're listening your [feedback](https://github.com/microsoft/lisa/issues/new).
|
||||
|
||||
## License
|
||||
|
||||
|
|
|
|||
|
|
@ -12,32 +12,46 @@
|
|||
- [Install Poetry in Windows](#install-poetry-in-windows)
|
||||
- [FAQ and Troubleshooting](#faq-and-troubleshooting)
|
||||
|
||||
LISA can be installed on any computer without having to be installed on each tested node. It can be installed on tested nodes to perform local testing. It can connect to another tested node using SSH. The following are examples showing where LISA is installed.
|
||||
LISA can be installed on any computer without having to be installed on each
|
||||
tested node. It can be installed on tested nodes to perform local testing. It
|
||||
can connect to another tested node using SSH. The following are examples showing
|
||||
where LISA is installed.
|
||||
|
||||
|
||||
|
||||
LISA supports to run on Windows and Linux. Follow below steps to install LISA from source code.
|
||||
LISA supports to run on Windows and Linux. Follow below steps to install LISA
|
||||
from source code.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
LISA needs to be installed on a computer that can access the computer to be tested. For tested computers, LISA is not required to be installed. The computer under test can be a local computer.
|
||||
LISA needs to be installed on a computer that can access the computer to be
|
||||
tested. For tested computers, LISA is not required to be installed. The computer
|
||||
under test can be a local computer.
|
||||
|
||||
- Can access the tested platform, like Azure, Hyper-V, or else. It recommends having good bandwidth and low network latency.
|
||||
- Can access the tested platform, like Azure, Hyper-V, or else. It recommends
|
||||
having good bandwidth and low network latency.
|
||||
- At least 2 CPU cores and 4GB memory.
|
||||
|
||||
## Install Python
|
||||
|
||||
Lisa is tested on [Python 3.8 64 bits](https://www.python.org/). If there are Python installed already, please make sure effective Python's version is 3.8 64-bit or above.
|
||||
Lisa is tested on [Python 3.8 64 bits](https://www.python.org/). If there are
|
||||
Python installed already, please make sure effective Python's version is 3.8
|
||||
64-bit or above.
|
||||
|
||||
LISA is developed and tested with Python 3.8 (64 bit). The latest version of Python 3.8 is recommended. If LISA is not compatible with higher Python version, [file an issue](https://github.com/microsoft/lisa/issues/new) to us.
|
||||
LISA is developed and tested with Python 3.8 (64 bit). The latest version of
|
||||
Python 3.8 is recommended. If LISA is not compatible with higher Python version,
|
||||
[file an issue](https://github.com/microsoft/lisa/issues/new) to us.
|
||||
|
||||
### Windows
|
||||
|
||||
Navigate to [Python releases for Windows](https://www.python.org/downloads/windows/). Download and install *Windows installer (64-bit)* from latest Python 3.8 64-bits or higher version.
|
||||
Navigate to [Python releases for
|
||||
Windows](https://www.python.org/downloads/windows/). Download and install
|
||||
*Windows installer (64-bit)* from latest Python 3.8 64-bits or higher version.
|
||||
|
||||
### Linux
|
||||
|
||||
For some Linux distributions, you can install latest Python 3.8 (64-bit) by its guidance, or build from source code. Below is how to install in Ubuntu.
|
||||
For some Linux distributions, you can install latest Python 3.8 (64-bit) by its
|
||||
guidance, or build from source code. Below is how to install in Ubuntu.
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
|
|
@ -48,19 +62,23 @@ sudo apt install python3.8 python3.8-dev -y
|
|||
|
||||
## Install dependencies
|
||||
|
||||
Since LISA is installed from source, `git` is necessary. And some packages need to be built, so the build tools are also needed.
|
||||
Since LISA is installed from source, `git` is necessary. And some packages need
|
||||
to be built, so the build tools are also needed.
|
||||
|
||||
In Linux, it needs `git`, `gcc`, and other Azure dependencies. Below is depended packages on Ubuntu.
|
||||
In Linux, it needs `git`, `gcc`, and other Azure dependencies. Below is depended
|
||||
packages on Ubuntu.
|
||||
|
||||
```bash
|
||||
sudo apt install git gcc libgirepository1.0-dev libcairo2-dev virtualenv -y
|
||||
```
|
||||
|
||||
In Windows, you need to install [git](https://git-scm.com/downloads) and [Visual C++ redistributable package](https://aka.ms/vs/16/release/vc_redist.x64.exe)
|
||||
In Windows, you need to install [git](https://git-scm.com/downloads) and [Visual
|
||||
C++ redistributable package](https://aka.ms/vs/16/release/vc_redist.x64.exe)
|
||||
|
||||
## Clone code
|
||||
|
||||
Open a terminal window, and enter the folder, which uses to put LISA code. If you want to use the latest version, checkout the main branch.
|
||||
Open a terminal window, and enter the folder, which uses to put LISA code. If
|
||||
you want to use the latest version, checkout the main branch.
|
||||
|
||||
```sh
|
||||
git clone https://github.com/microsoft/lisa.git
|
||||
|
|
@ -69,9 +87,11 @@ cd lisa
|
|||
|
||||
## Install Poetry and Python dependencies
|
||||
|
||||
Poetry is used to manage Python dependencies of LISA. Execute corresponding script to install Poetry.
|
||||
Poetry is used to manage Python dependencies of LISA. Execute corresponding
|
||||
script to install Poetry.
|
||||
|
||||
Note, it's important to enter LISA's folder to run below command, since Poetry manages dependencies by the working folder.
|
||||
Note, it's important to enter LISA's folder to run below command, since Poetry
|
||||
manages dependencies by the working folder.
|
||||
|
||||
### Install Poetry in Linux
|
||||
|
||||
|
|
|
|||
|
|
@ -8,37 +8,66 @@
|
|||
- [Test tier](#test-tier)
|
||||
- [Test cases specification](#test-cases-specification)
|
||||
|
||||
The test suite in LISA is called Microsoft tests, which are provided by Microsoft Linux System Group. This document introduces how Microsoft tests were defined, categorized, and how to have the appropriate coverage.
|
||||
The test suite in LISA is called Microsoft tests, which are provided by
|
||||
Microsoft Linux System Group. This document introduces how Microsoft tests were
|
||||
defined, categorized, and how to have the appropriate coverage.
|
||||
|
||||
## How to run Microsoft tests
|
||||
|
||||
### Quick start
|
||||
|
||||
Microsoft tests are organized under the folder `microsoft/runbook`. The root folder contains runbooks for azure, ready, and local. Learn more from [how to run LISA tests](run.md) to run different tiers on an image or existing environment.
|
||||
Microsoft tests are organized under the folder `microsoft/runbook`. The root
|
||||
folder contains runbooks for azure, ready, and local. Learn more from [how to
|
||||
run LISA tests](run.md) to run different tiers on an image or existing
|
||||
environment.
|
||||
|
||||
### Advanced
|
||||
|
||||
If you want to verify on specified conditions, like to select some VM size in azure, or select test cases by names, learn more from [runbook reference](runbook.md).
|
||||
If you want to verify on specified conditions, like to select some VM size in
|
||||
azure, or select test cases by names, learn more from [runbook
|
||||
reference](runbook.md).
|
||||
|
||||
## Test priority
|
||||
|
||||
The priority of each test case is determined by the impact if it's failed. The smaller number means the higher priority. For example, if a high-priority test case fails, it means the operating system cannot start. If a lower-priority test case fails, it may mean that a function does not work.
|
||||
The priority of each test case is determined by the impact if it's failed. The
|
||||
smaller number means the higher priority. For example, if a high-priority test
|
||||
case fails, it means the operating system cannot start. If a lower-priority test
|
||||
case fails, it may mean that a function does not work.
|
||||
|
||||
Note that when multiple test cases fail, we should first check the failure of high-priority test cases to speed up the analysis.
|
||||
Note that when multiple test cases fail, we should first check the failure of
|
||||
high-priority test cases to speed up the analysis.
|
||||
|
||||
- **P0**. The system fails/hangs on start/restart using default settings.
|
||||
- **P1**. The system fails/hangs on start/restart using popular configurations, for example, add firewall rules, install some popular packages. There is data loss with popular configurations. The system cannot be connected via network with default settings. The system performance drops significantly, like SRIOV doesn't work as expected; only one CPU core works on multiple core machine; an important feature doesn't work with default settings; or the system can be used with limited functionality.
|
||||
- **P2**. The system fails/hangs on start/restart using unpopular configurations. Data loss with unpopular configurations. The system cannot be connected with popular configurations. The system performance drops obviously. An important feature doesn't work with popular configurations.
|
||||
- **P1**. The system fails/hangs on start/restart using popular configurations,
|
||||
for example, add firewall rules, install some popular packages. There is data
|
||||
loss with popular configurations. The system cannot be connected via network
|
||||
with default settings. The system performance drops significantly, like SRIOV
|
||||
doesn't work as expected; only one CPU core works on multiple core machine; an
|
||||
important feature doesn't work with default settings; or the system can be
|
||||
used with limited functionality.
|
||||
- **P2**. The system fails/hangs on start/restart using unpopular
|
||||
configurations. Data loss with unpopular configurations. The system cannot be
|
||||
connected with popular configurations. The system performance drops obviously.
|
||||
An important feature doesn't work with popular configurations.
|
||||
- **P3**. A feature doesn't work with unpopular configurations with low impact.
|
||||
- **P4**. The system has obvious but not serious problems on long-haul, stress or performance test scenarios.
|
||||
- **P4**. The system has obvious but not serious problems on long-haul, stress
|
||||
or performance test scenarios.
|
||||
|
||||
Please Note that the above examples do not cover all situations and are for reference. For example, in a cloud environment, one host version may cause problems of some Linux virtual machines. The impact is affected by the percentage the problematic version also.
|
||||
Please Note that the above examples do not cover all situations and are for
|
||||
reference. For example, in a cloud environment, one host version may cause
|
||||
problems of some Linux virtual machines. The impact is affected by the
|
||||
percentage the problematic version also.
|
||||
|
||||
## Test tier
|
||||
|
||||
Ideally, all tests should be run to maximize the coverage. But the time and resource are limited, and the risks need to be minimized based on the limitations. In LISA, Microsoft tests are organized into several tiers to have the appropriate coverage using limited resource.
|
||||
Ideally, all tests should be run to maximize the coverage. But the time and
|
||||
resource are limited, and the risks need to be minimized based on the
|
||||
limitations. In LISA, Microsoft tests are organized into several tiers to have
|
||||
the appropriate coverage using limited resource.
|
||||
|
||||
Test tiers can be T0, T1, T2, T3, T4. It maps to priorities of test cases. For example, T0 means all P0 test cases are selected in a test run. T2 means all P0, P1, P2 test cases are selected in a test run.
|
||||
Test tiers can be T0, T1, T2, T3, T4. It maps to priorities of test cases. For
|
||||
example, T0 means all P0 test cases are selected in a test run. T2 means all P0,
|
||||
P1, P2 test cases are selected in a test run.
|
||||
|
||||
| name | test priorities | time restriction | resource restriction | automation requirement |
|
||||
| ---- | ------------------ | ---------------- | --------------------------------------- | ------------------------------------------------------------ |
|
||||
|
|
@ -52,4 +81,5 @@ Test tiers can be T0, T1, T2, T3, T4. It maps to priorities of test cases. For e
|
|||
|
||||
TODO: add spec of LISA test cases.
|
||||
|
||||
Learn more on not migrated [legacy LISAv2 tests](https://github.com/microsoft/lisa/blob/master/Documents/LISAv2-TestCase-Statistics.md).
|
||||
Learn more on not migrated [legacy LISAv2
|
||||
tests](https://github.com/microsoft/lisa/blob/master/Documents/LISAv2-TestCase-Statistics.md).
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче