microsoft
/
lisa
зеркало из https://github.com/microsoft/lisa.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Initial version of documents 

Complete first version README.

Some documents are in very draft stage. They will be filled one by one.
This commit is contained in:
Chi Song 2021-03-03 19:06:21 +08:00
Родитель 1b24ce5c76
Коммит 40e108cd72
13 изменённых файлов: 721 добавлений и 219 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

9
CODE_OF_CONDUCT.md
Просмотреть файл

@ -1,9 +0,0 @@
# Microsoft Open Source Code of Conduct
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
Resources:
- [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/)
- [Microsoft Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/)
- Contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with questions or concerns

173
README.md
Просмотреть файл

@ -1,163 +1,42 @@
# Linux Integration Services Automation 3.0 (LISAv3) # Linux Integration Services Automation (LISA)
[![CI Workflow for LISAv3](https://github.com/LIS/LISAv2/workflows/CI%20Workflow%20for%20LISAv3/badge.svg?branch=main)](https://github.com/LIS/LISAv2/actions?query=workflow%3A%22CI+Workflow+for+LISAv3%22+event%3Apush+branch%3Amain) [![CI Workflow](https://github.com/microsoft/lisa/workflows/CI%20Workflow/badge.svg?branch=main)](https://github.com/microsoft/lisa/actions?query=workflow%3A%22CI+Workflow+for+LISAv3%22+event%3Apush+branch%3Amain)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![GitHub license](https://img.shields.io/github/license/microsoft/lisa)](https://github.com/microsoft/lisa/blob/main/LICENSE)
[![GitHub license](https://img.shields.io/github/license/LIS/LISAv2)](https://github.com/LIS/LISAv2/blob/main/LICENSE-2.0.txt)
LISA is a Linux test automation framework with built-in test cases to verify the quality of **Linux Integration Services Automation (LISA)** is designed to be an end-to-end solution to validate Linux kernels and distributions quality on virtualization platforms of Microsoft and others. It can be used on more tests, or support other cloud platforms as well.
Linux distributions on multiple platforms (such as Azure, On-prem HyperV, and Linux bare metal).
This version (v3) is a brand new implementation in Python 3, supports running on Windows and Linux distros. * **End-to-end**: LISA defines several set of test suites to validate Linux kernels and distributions in Microsoft Azure, and Hyper-V. The test suites can help find integration issues easily.
* **Ease-to-use**: The complexity and diversity of testing or Linux distributions is wrapped in different components of LISA. When running LISA, it doesn't need to know much on details.
* **Extensibility**: LISA is extendable in many components to support various scenarios, including virtualization platforms, commands, Linux distributions, community test suites, etc. LISA supports to validate Microsoft virtualization platforms natively, but also is able to extend to support other cloud or on-premises platforms.
## Getting Started ## Why LISA
### Install Python 3 There are a lot of tools, which focus on Linux kernels or distributions quality. They are very important to guarantee quality of kernels and distributions. The integration validation on virtualization platforms is a little different with classic Linux testing. It needs to cover different types of resources with manageable cost. It needs to create/delete resources with a good plan automatically.
Install [Python 3.8](https://www.python.org/) from Linux distribution’s package LISA focuses on validating the integration of Linux kernels/distributions and virtualization platforms. It needs more interactive with virtualization platforms to run tests for different purposes, like test different capacity, hardware, and so on.
repositories, or [python.org](https://www.python.org/). We also need `gcc` and
the `libpython` headers for the `usjon` package.
On Ubuntu 18.04 and 20.04: ## Documents
```bash * [Key concepts](docs/concepts.md)
sudo apt install git python3.8 libpython3.8-dev python3-distutils python3-apt gcc * [Install LISA](docs/install.md)
``` * [Run tests with LISA](docs/run.md)
* [Command line reference](docs/command_line.md)
* [Runbook reference](docs/runbook.md)
* [Microsoft test suites](https://github.com/microsoft/lisa/blob/master/Documents/LISAv2-TestCase-Statistics.md)
* [How to write tests](docs/write_case.md)
* [How to extend and customize LISA](docs/extension.md)
* [How to run previous version LISA (aka LISAv2)](docs/run_legacy.md)
NOTE: If you are using WSL, installing Poetry on both Windows and Linux may ## Contribute
cause both platforms’ versions of Poetry to be on your path, as Windows binaries
are mapped into WSL’s `PATH`. This means that the Linux `poetry` binary _must_
appear in your `PATH` before the Windows version, or this error will appear:
> `/usr/bin/env: ‘python\r’: No such file or directory` You are very welcome to contribute. Please follow [the contribution document](docs/contributing.md) for details.
### Install Poetry ## History and road map
On Linux (or WSL): The previous LISA called LISAv2, which is in [master branch](https://github.com/microsoft/lisa/tree/master). The previous LISA can be used standalone or called from current LISA. Refer to [how to run LISAv2 test cases](docs/run_legacy.md) for details.
```bash LISA is in active developing, and a lot of exciting features are implementing. We're listening your feedback.
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3 - --preview --version 1.1.4
source $HOME/.poetry/env
```
On Windows (in PowerShell):
```powershell
(Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python - --preview --version 1.1.4
# the path can be added to system, so it applies to every terminal.
$env:PATH += ";$env:USERPROFILE\.poetry\bin"
```
If you already have Poetry installed, you can update it like:
```bash
poetry self update 1.1.4
```
### Install Python packages
Then use Poetry to install LISAv3's Python package dependencies:
```bash
git clone -b main https://github.com/LIS/LISAv2.git
cd LISAv2
poetry install
```
To obtain the path of the Poetry virtual environment setup for LISA (where the
isolated Python installation and packages are located), run:
```bash
poetry env list --full-path
```
### Run LISAv3
Run LISAv3 using Poetry’s environment:
```bash
poetry run python lisa/main.py
```
### Make
We now also have a GNU Makefile that automates some tasks. Try:
```bash
# Install Python packages
make setup
# Run LISAv3
make run
# Run unit tests
make test
# Run syntactic, semantic, formatting and type checkers
make check
# Generate coverage report (slow, reruns LISAv3 and tests)
make coverage
# Print current Python virtualenv
make venv
```
### Editor Setup
Install and enable [ShellCheck](https://github.com/koalaman/shellcheck) to find
bash errors locally.
#### Visual Studio Code
First, click the Python version in the bottom left, then enter the path emitted
by the command above. This will point Code to the Poetry virtual environment.
Make sure below settings are in root level of `.vscode/settings.json`
```json
{
"python.analysis.typeCheckingMode": "strict",
"python.formatting.provider": "black",
"python.linting.enabled": true,
"python.linting.flake8Enabled": true,
"python.linting.mypyEnabled": true,
"python.linting.pylintEnabled": false,
"editor.formatOnSave": true,
"python.linting.mypyArgs": [
"--strict",
"--namespace-packages",
"--show-column-numbers"
],
"python.sortImports.path": "isort",
"python.analysis.useLibraryCodeForTypes": false,
"python.analysis.autoImportCompletions": false,
"files.eol": "\n",
}
```
#### Emacs
Use the [pyvenv](https://github.com/jorgenschaefer/pyvenv) package:
```emacs-lisp
(use-package pyvenv
:ensure t
:hook (python-mode . pyvenv-tracking-mode))
```
Then run `M-x add-dir-local-variable RET python-mode RET pyvenv-activate RET
<path/to/virtualenv>` where the value is the path given by the command above.
This will create a `.dir-locals.el` file which looks like this:
```emacs-lisp
;;; Directory Local Variables
;;; For more information see (info "(emacs) Directory Variables")
((python-mode . ((pyvenv-activate . "~/.cache/pypoetry/virtualenvs/lisa-s7Q404Ij-py3.8"))))
```
## Contributing
## License ## License
The entire codebase is under [MIT license](LICENSE) The entire codebase is under [MIT license](LICENSE).

26
docs/command_line.md Normal file
Просмотреть файл

@ -0,0 +1,26 @@
# command line reference
- [common arguments](#common-arguments)
- [-d, --debug](#-d---debug)
- [-r, --runbook](#-r---runbook)
- [-h, --help](#-h---help)
- [-v, --variable](#-v---variable)
- [run](#run)
- [check](#check)
- [list](#list)
## common arguments
### -d, --debug
### -r, --runbook
### -h, --help
### -v, --variable
## run
## check
## list

20
docs/concepts.md Normal file
Просмотреть файл

@ -0,0 +1,20 @@
# Concepts
- [Runbook](#runbook)
- [Environment and Node](#environment-and-node)
- [Tools and Scripts](#tools-and-scripts)
- [Platform](#platform)
- [Feature](#feature)
- [Requirement and Capability](#requirement-and-capability)
## Runbook
## Environment and Node
## Tools and Scripts
## Platform
## Feature
## Requirement and Capability

42
docs/contributing.md Normal file
Просмотреть файл

@ -0,0 +1,42 @@
# Contributing
You are very welcome to contribute in any way. There are serval ways to guide the contribution. They have no intention of restricting you, they will only increase efficiency.
- [Provide feedback or ask a question](#provide-feedback-or-ask-a-question)
- [File a bug](#file-a-bug)
- [Contribute more](#contribute-more)
- [Contribute fixing or extension](#contribute-fixing-or-extension)
- [Contribute feature](#contribute-feature)
- [Code of Conduct](#code-of-conduct)
## Provide feedback or ask a question
You are welcome to provide feedback by [filing an issue](https://github.com/microsoft/lisa/issues/new).
When there is any question, please check below FAQs first. If there is no satisfying answer, don't hesitate to [file an issue](https://github.com/microsoft/lisa/issues/new).
## File a bug
Before file a bug, please try to search keywords on existing bugs. If you cannot find any similar bugs, feel free to [file a new one](https://github.com/microsoft/lisa/issues/new).
## Contribute more
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
### Contribute fixing or extension
If you think there is a bug, and want to fix it. Feel free to submit a pull request. We will review it proactively. Once it's signed off, we will merge the change.
If there is any extension is needed by others, submit a pull request.
Learn more on [development guideline](./development.md).
### Contribute feature
Please file an issue to describe why the feature is needed, and how it will be implemented. Once we agree on that, follow [Contribute fixing or extension](#contribute-fixing-or-extension) to implement it.
## Code of Conduct
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.

177
CONTRIBUTING.md → docs/development.md
Просмотреть файл

@ -1,8 +1,78 @@
# Contributing Guidelines # Development Guideline
This document describes the existing developer tooling we have in place (and what to This document describes the existing developer tooling we have in place (and what to
expect of it), as well as our design and development philosophy. expect of it), as well as our design and development philosophy.
- [Editor Setup](#editor-setup)
- [Visual Studio Code](#visual-studio-code)
- [Emacs](#emacs)
- [other tools](#other-tools)
- [Create a pull request](#create-a-pull-request)
- [Naming Conventions](#naming-conventions)
- [Automated Tooling](#automated-tooling)
- [Metadata](#metadata)
- [Package Dependencies](#package-dependencies)
- [Developer Dependencies](#developer-dependencies)
- [Type Annotations](#type-annotations)
- [Runbook schema](#runbook-schema)
- [Committing Guidelines](#committing-guidelines)
- [Design Patterns](#design-patterns)
- [Future Sections](#future-sections)
## Editor Setup
### Visual Studio Code
First, click the Python version in the bottom left, then enter the path emitted by the command above. This will point Code to the Poetry virtual environment.
Make sure below settings are in root level of `.vscode/settings.json`
```json
{
"python.analysis.typeCheckingMode": "strict",
"python.formatting.provider": "black",
"python.linting.enabled": true,
"python.linting.flake8Enabled": true,
"python.linting.mypyEnabled": true,
"python.linting.pylintEnabled": false,
"editor.formatOnSave": true,
"python.linting.mypyArgs": [
"--strict",
"--namespace-packages",
"--show-column-numbers"
],
"python.sortImports.path": "isort",
"python.analysis.useLibraryCodeForTypes": false,
"python.analysis.autoImportCompletions": false,
"files.eol": "\n",
}
```
### Emacs
Use the [pyvenv](https://github.com/jorgenschaefer/pyvenv) package:
```emacs-lisp
(use-package pyvenv
:ensure t
:hook (python-mode . pyvenv-tracking-mode))
```
Then run `M-x add-dir-local-variable RET python-mode RET pyvenv-activate RET <path/to/virtualenv>` where the value is the path given by the command above. This will create a `.dir-locals.el` file which looks like this:
```emacs-lisp
;;; Directory Local Variables
;;; For more information see (info "(emacs) Directory Variables")
((python-mode . ((pyvenv-activate . "~/.cache/pypoetry/virtualenvs/lisa-s7Q404Ij-py3.8"))))
```
### other tools
- Install and enable [ShellCheck](https://github.com/koalaman/shellcheck) to find bash errors locally.
## Create a pull request
## Naming Conventions ## Naming Conventions
Naming conventions are not automatically enforced, so please read the [naming Naming conventions are not automatically enforced, so please read the [naming
@ -10,23 +80,16 @@ conventions](https://www.python.org/dev/peps/pep-0008/#naming-conventions)
section of PEP 8, which describes what each of the different styles means. A section of PEP 8, which describes what each of the different styles means. A
short summary of the most important parts: short summary of the most important parts:
* Modules (and hence files) should have short, all-lowercase names. - Modules (and hence files) should have short, all-lowercase names.
* Class (and exception) names should normally use the `CapWords` convention - Class (and exception) names should normally use the `CapWords` convention (also known as `CamelCase`).
(also known as `CamelCase`). - Function and variable names should be lowercase, with words separated by underscores as necessary to improve readability (also known as `snake_case`).
* Function and variable names should be lowercase, with words separated by - To avoid collisions with the standard library, an underscore can be appended, such as `id_`.
underscores as necessary to improve readability (also known as `snake_case`). - Always use `self` for the first argument to instance methods.
* To avoid collisions with the standard library, an underscore can be appended, - Always use `cls` for the first argument to class methods.
such as `id_`. - One leading underscore like `_data` is for non-public methods and instance variables. And it can be used by sub-classes. If it won't be used in sub-classes, use like `__data`.
* Always use `self` for the first argument to instance methods. - If there is a pair of `get_x` and `set_x` methods, they should instead be a proper property, which is easy to do with the built-in `@property` decorator.
* Always use `cls` for the first argument to class methods. - Constants should be `CAPITALIZED_SNAKE_CASE`.
* One leading underscore like `_data` is for non-public methods and instance - When importing a function, try to avoid renaming it with `import as` because it introduces cognitive overhead to track yet another name.
variables. And it can be used by sub-classes. If it won't be used in
sub-classes, use like `__data`.
* If there is a pair of `get_x` and `set_x` methods, they should instead be a
proper property, which is easy to do with the built-in `@property` decorator.
* Constants should be `CAPITALIZED_SNAKE_CASE`.
* When importing a function, try to avoid renaming it with `import as` because
it introduces cognitive overhead to track yet another name.
When in doubt, adhere to existing conventions, or check the style guide. When in doubt, adhere to existing conventions, or check the style guide.
@ -81,24 +144,24 @@ on Windows, you will want to setup your own, perhaps using [Conda][].
[pyenv]: https://github.com/pyenv/pyenv [pyenv]: https://github.com/pyenv/pyenv
[Conda]: https://docs.conda.io/en/latest/ [Conda]: https://docs.conda.io/en/latest/
* python: We pinned Python to version 3.8 so everyone uses the same version. - python: We pinned Python to version 3.8 so everyone uses the same version.
* psutil: TODO @squirrelsc will document - psutil: TODO @squirrelsc will document
* pyyaml: TODO @squirrelsc will document - pyyaml: TODO @squirrelsc will document
* retry: TODO @squirrelsc will document - retry: TODO @squirrelsc will document
* paramiko: TODO @squirrelsc will document - paramiko: TODO @squirrelsc will document
* spurplus: TODO @squirrelsc will document - spurplus: TODO @squirrelsc will document
* dataclasses-json: TODO @squirrelsc will document (brings in `usjon` which - dataclasses-json: TODO @squirrelsc will document (brings in `usjon` which
requires `gcc` and `libpython`) requires `gcc` and `libpython`)
* portalocker: TODO @squirrelsc will document - portalocker: TODO @squirrelsc will document
* azure-*: TODO @squirrelsc will document - azure-*: TODO @squirrelsc will document
### Developer Dependencies ### Developer Dependencies
@ -107,29 +170,29 @@ add --dev <package_name>` records our _developer_ packages. These are not
necessary for LISAv3 to execute, but are used by developers to automatically necessary for LISAv3 to execute, but are used by developers to automatically
adhere to our coding standards. adhere to our coding standards.
* [Black](https://github.com/psf/black), the opinionated code formatter which - [Black](https://github.com/psf/black), the opinionated code formatter which
settles all debates as to how our Python files should be formatted. It follows settles all debates as to how our Python files should be formatted. It follows
[PEP 8](https://www.python.org/dev/peps/pep-0008/), the official Python style [PEP 8](https://www.python.org/dev/peps/pep-0008/), the official Python style
guide, and where ambiguous makes the decision for us. guide, and where ambiguous makes the decision for us.
* [Flake8](https://flake8.pycqa.org/en/latest/) (and integrations), the semantic - [Flake8](https://flake8.pycqa.org/en/latest/) (and integrations), the semantic
analyzer, used to coordinate most of the other tools. analyzer, used to coordinate most of the other tools.
* [isort](https://timothycrosley.github.io/isort/), the `import` sorter, which - [isort](https://timothycrosley.github.io/isort/), the `import` sorter, which
automatically splits imports into the expected, alphabetized sections. automatically splits imports into the expected, alphabetized sections.
* [mypy](http://mypy-lang.org/), the static type checker, which coupled with - [mypy](http://mypy-lang.org/), the static type checker, which coupled with
type annotations allows us to avoid the pitfalls of Python being a dynamically type annotations allows us to avoid the pitfalls of Python being a dynamically
typed language. typed language.
* [python-language-server](https://github.com/palantir/python-language-server) - [python-language-server](https://github.com/palantir/python-language-server)
(and integrations), the de facto LSP server. While Microsoft is developing (and integrations), the de facto LSP server. While Microsoft is developing
their own LSP servers, they do not integrate with the existing ecosystem of their own LSP servers, they do not integrate with the existing ecosystem of
tools, and their latest tool, Pyright, simply does not support tools, and their latest tool, Pyright, simply does not support
`pyproject.toml`. Since pyls is used far more widely, and supports every `pyproject.toml`. Since pyls is used far more widely, and supports every
editor, we use it. editor, we use it.
* [rope](https://github.com/python-rope/rope), to provide completions and - [rope](https://github.com/python-rope/rope), to provide completions and
renaming support to pyls. renaming support to pyls.
With these packages installed and a correctly setup editor (see the readme and With these packages installed and a correctly setup editor (see the readme and
@ -204,12 +267,12 @@ example](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html):
> >
> Further paragraphs come after blank lines. > Further paragraphs come after blank lines.
> >
> * Bullet points are okay, too > - Bullet points are okay, too
> >
> * Typically a hyphen or asterisk is used for the bullet, followed by a > - Typically a hyphen or asterisk is used for the bullet, followed by a
> single space, with blank lines in between, but conventions vary here > single space, with blank lines in between, but conventions vary here
> >
> * Use a hanging indent > - Use a hanging indent
You should also feel free to use Markdown in the commit messages, as our project You should also feel free to use Markdown in the commit messages, as our project
is hosted on GitHub which renders it (and Markdown is human readable). is hosted on GitHub which renders it (and Markdown is human readable).
@ -240,15 +303,15 @@ carefully create a well-designed and maintainable framework.
Several popular patterns that actually _do not_ work well in Python are: Several popular patterns that actually _do not_ work well in Python are:
* [The Abstract Factory Pattern](https://python-patterns.guide/gang-of-four/abstract-factory/) - [The Abstract Factory Pattern](https://python-patterns.guide/gang-of-four/abstract-factory/)
* [The Factory Method Pattern](https://python-patterns.guide/gang-of-four/factory-method/) - [The Factory Method Pattern](https://python-patterns.guide/gang-of-four/factory-method/)
* [The Prototype Pattern](https://python-patterns.guide/gang-of-four/prototype/) - [The Prototype Pattern](https://python-patterns.guide/gang-of-four/prototype/)
* [The Singleton Pattern](https://python-patterns.guide/gang-of-four/singleton/) - [The Singleton Pattern](https://python-patterns.guide/gang-of-four/singleton/)
Conversely, patterns that are a natural fit to Python include: Conversely, patterns that are a natural fit to Python include:
* [The Composite Pattern](https://python-patterns.guide/gang-of-four/composite/) - [The Composite Pattern](https://python-patterns.guide/gang-of-four/composite/)
* [The Iterator Pattern](https://python-patterns.guide/gang-of-four/iterator/) - [The Iterator Pattern](https://python-patterns.guide/gang-of-four/iterator/)
(caution: it is actually better to implement these with `yield`!) (caution: it is actually better to implement these with `yield`!)
Finally, a high-level guide to all things Python is [The Hitchhiker’s Guide to Finally, a high-level guide to all things Python is [The Hitchhiker’s Guide to
@ -257,28 +320,16 @@ Python world. If you make it through even some of these guides, you will be well
on your way to being a “Pythonista” (a Python developer) writing “Pythonic” on your way to being a “Pythonista” (a Python developer) writing “Pythonic”
(canonically correct Python) code left and right. (canonically correct Python) code left and right.
### Async IO
With Python 3.4, the Async IO pattern found in languages such as C# and Go is
available through the keywords `async` and `await`, along with the Python module
`asyncio`. Please read [Async IO in Python: A Complete
Walkthrough](https://realpython.com/async-io-python/) to understand at a high
level how asynchronous programming works. As of Python 3.7, One major “gotcha”
is that `asyncio.run(...)` should be used [exactly once in
`main`](https://docs.python.org/3/library/asyncio-task.html), it starts the
event loop. Everything else should be a coroutine or task which the event loop
schedules.
## Future Sections ## Future Sections
Just a collection of reminders for the author to expand on later. Just a collection of reminders for the author to expand on later.
* [unittest](https://docs.python.org/3/library/unittest.html) - [unittest](https://docs.python.org/3/library/unittest.html)
* [doctest](https://docs.python.org/3/library/doctest.html) - [doctest](https://docs.python.org/3/library/doctest.html)
* [subprocess](https://pymotw.com/3/subprocess/index.html) - [subprocess](https://pymotw.com/3/subprocess/index.html)
* [GitHub Actions](https://github.com/LIS/LISAv2/actions) - [GitHub Actions](https://github.com/LIS/LISAv2/actions)
* [ShellCheck](https://www.shellcheck.net/) - [ShellCheck](https://www.shellcheck.net/)
* [Governance](https://opensource.guide/leadership-and-governance/) - [Governance](https://opensource.guide/leadership-and-governance/)
* [Maintenance Cost](https://web.archive.org/web/20120313070806/http://users.jyu.fi/~koskinen/smcosts.htm) - [Maintenance Cost](https://web.archive.org/web/20120313070806/http://users.jyu.fi/~koskinen/smcosts.htm)
* Parallelism and multi-plexing - Parallelism and multi-plexing
* Versioned inputs and outputs - Versioned inputs and outputs

11
docs/extension.md Normal file
Просмотреть файл

@ -0,0 +1,11 @@
# How to write extensions in LISA
## Platform
## Feature
## Notifier
## Tool
## Linux Distribution

Двоичные данные
docs/img/smoke_test_result.png Normal file
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом"

После

Ширина:  |  Высота:  |  Размер: 68 KiB

95
docs/install.md Normal file
Просмотреть файл

@ -0,0 +1,95 @@
# Install LISA
- [Prerequisites](#prerequisites)
- [Install Python](#install-python)
- [Install dependencies](#install-dependencies)
- [Clone code](#clone-code)
- [Install Poetry and Python dependencies](#install-poetry-and-python-dependencies)
- [Install Poetry in Linux](#install-poetry-in-linux)
- [Install Poetry in Windows](#install-poetry-in-windows)
- [FAQ and Troubleshooting](#faq-and-troubleshooting)
- [How to use LISA in WSL](#how-to-use-lisa-in-wsl)
- [Cannot find package after run `poetry install`](#cannot-find-package-after-run-poetry-install)
- [Poetry related questions](#poetry-related-questions)
- [Other issues](#other-issues)
LISA supports to run on both Windows and Linux. Follow below steps to install LISA from source code.
## Prerequisites
LISA needs to be installed on a computer, which
* Can access the tested platform, like Azure, Hyper-V, or else. It recommends to have good bandwidth and low network latency.
* At least 2 CPU cores and 4GB memory.
## Install Python
LISA is developed in Python, and tested with Python 3.8 (64 bit). The latest version of Python 3.8 is recommended, but feel free to file an issue, if LISA is not compatible with higher Python version.
Install latest [Python 3.8 64 bits](https://www.python.org/). If there are Python installed already, it needs to make sure effective Python's version is above 3.8 and 64 bits.
## Install dependencies
Some Python packages need to be built from source, so it needs build tools installed.
In Linux, it needs `git` and `gcc`. Below is depended packages on Ubuntu.
```bash
sudo apt install git gcc
```
In Windows, you need to install [git](https://git-scm.com/downloads) and [Visual C++ redistributable package](https://aka.ms/vs/16/release/vc_redist.x64.exe)
## Clone code
Open a terminal window, and enter the folder, which to download lisa code.
```sh
git clone https://github.com/microsoft/lisa.git
cd lisa
```
## Install Poetry and Python dependencies
Poetry is used to manage Python dependencies of LISA. Execute corresponding script to install Poetry.
Note, it's important to enter LISA's folder to run below command, since Poetry manages dependencies by working folders.
### Install Poetry in Linux
```bash
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3 -
source $HOME/.poetry/env
poetry install
```
### Install Poetry in Windows
Open a PowerShell command prompt and execute
```powershell
(Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python -
# the path can be added to system, so it applies to every terminal.
$env:PATH += ";$env:USERPROFILE\.poetry\bin"
poetry install
```
## FAQ and Troubleshooting
### How to use LISA in WSL
If you are using WSL, installing Poetry on both Windows and WSL may cause both platforms' versions of Poetry to be on your path, as Windows binaries are mapped into `PATH` of WSL. This means that the WSL `poetry` binary _must_ appear in your `PATH` before the Windows version, or this error will appear:
> `/usr/bin/env: ‘python\r’: No such file or directory`
### Cannot find package after run `poetry install`
Poetry is case sensitive. When in windows, make sure the cases of path is consistent every time.
### Poetry related questions
Poetry is very useful to manage dependencies of Python. It's a virtual environment, not a complete environment like Conda. So make sure the right version of Python is effective. Know more about Poetry on [installation](https://python-poetry.org/docs/#installation) or [commands](https://python-poetry.org/docs/cli/).
### Other issues
Please check [known issues](https://github.com/microsoft/lisa/issues), or [file an issue](https://github.com/microsoft/lisa/issues/new) if it doesn't exists.

183
docs/run.md Normal file
Просмотреть файл

@ -0,0 +1,183 @@
# Run LISA
- [Quick start](#quick-start)
- [Run locally](#run-locally)
- [Run in Azure](#run-in-azure)
- [Run in Ready VMs](#run-in-ready-vms)
- [Run Microsoft tests](#run-microsoft-tests)
- [Test priority and tier](#test-priority-and-tier)
- [Run legacy LISAv2 test cases](#run-legacy-lisav2-test-cases)
- [FAQ and Troubleshooting](#faq-and-troubleshooting)
## Quick start
In Linux, define an alias to simplify the command. If you want the alias effective every time, add into `.bashrc`
```bash
alias lisa="./lisa.sh"
```
### Run locally
If not arguments specified, LISA runs test cases on local machine. Those cases won't modify local environment, and they are helpful to validate LISA installation.
```bash
lisa
```
After run, the terminal output is like below. If there is any error, check troubleshooting part of [installation](install.md).
The log shows 5 test cases are scheduled, 2 passed, and 3 skipped. For each skipped case, there is a reason, why it's skipped. There is a html test report, and its link shows at end of log.
```text
2021-03-04 06:32:30.904 INFO LISA.RootRunner ________________________________________
2021-03-04 06:32:30.904 INFO LISA.RootRunner HelloWorld.hello: PASSED
2021-03-04 06:32:30.905 INFO LISA.RootRunner HelloWorld.bye: PASSED
2021-03-04 06:32:30.905 INFO LISA.RootRunner MultipleNodesDemo.os_info: SKIPPED no available environment: ['no enough nodes, requirement: 2, capability: 1. ']
2021-03-04 06:32:30.906 INFO LISA.RootRunner MultipleNodesDemo.perf_network_tcp_ipv4_throughput_ntttcp_synthetic_singleconnection: SKIPPED no available environment: ['no enough nodes, requirement: 2, capability: 1. ']
2021-03-04 06:32:30.906 INFO LISA.RootRunner WithScript.script: SKIPPED no available environment: ["os_type: requirements excludes ['Windows']"]
2021-03-04 06:32:30.907 INFO LISA.RootRunner test result summary
2021-03-04 06:32:30.907 INFO LISA.RootRunner TOTAL : 5
2021-03-04 06:32:30.908 INFO LISA.RootRunner NOTRUN : 0
2021-03-04 06:32:30.908 INFO LISA.RootRunner RUNNING : 0
2021-03-04 06:32:30.908 INFO LISA.RootRunner FAILED : 0
2021-03-04 06:32:30.908 INFO LISA.RootRunner PASSED : 2
2021-03-04 06:32:30.910 INFO LISA.RootRunner SKIPPED : 3
2021-03-04 06:32:30.913 INFO LISA.notifier[Html] report: D:\code\LISAv2\runtime\runs\20210304\20210304-063226-496\lisa.html
2021-03-04 06:32:30.923 INFO LISA completed in 4.428 sec
```
### Run in Azure
Below commands shows how to run Microsoft tier 0 tests on an Azure gallery image.
1. Sign in to Azure
Make sure [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli) or [Azure PowerShell](https://docs.microsoft.com/en-us/powershell/azure/install-az-ps) installed. It needs to sign in to run LISA on current machine. LISA supports other Azure credential mechanisms also, refer to [runbook reference](runbook.md).
Here uses Azure CLI as the example. Open a terminal.
```bash
az login
```
2. Find subscription id
If you have multiple Azure subscriptions. Use below command to set default subscription. LISA will create resource groups in it. To prevent missing use, the subscription id is a required parameter to run LISA.
Run below command to retrieve subscription information.
```bash
az account show --subscription "<Subscription Name to run LISA>"
```
You will see output like below, find `<subscription id>` for next steps.
```json
{
"environmentName": "AzureCloud",
"homeTenantId": "<tenant id>",
"id": "<subscription id>",
"isDefault": true,
"managedByTenants": [],
"name": "<subscription name>",
"state": "Enabled",
"tenantId": "<tenant id>",
"user": {
"name": "<user account>",
"type": "user"
}
}
```
3. Prepare ssh key pair
LISA connects to VMs by SSH, and a public/private key pair is necessary for authentication. If you don't have a key pair, follow this step to create one.
Run below command, and follow prompt to create a key pair. Most Linux distributions have SSH client installed. On windows, follow [install OpenSSH](https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse) instruction.
Note, don't use passphrase to generate the key. LISA doesn't support passphrase.
```bash
ssh-keygen
```
Find `<private key file>` for next steps. If you want to move key files, keep public/private key files are in same folder, and keep the same base file name.
```bash
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in <private key file>.
Your public key has been saved in <public key file>.
The key fingerprint is:
SHA256:OIzc1yE7joL2Bzy8!gS0j8eGK7bYaH1FmF3sDuMeSj8 username@server@LOCAL-HOSTNAME
The key's randomart image is:
+--[ED25519 256]--+
| . |
| o |
| . + + . |
| o B * = . |
| o= B S . |
| .=B O o |
| + =+% o |
| *oo.O.E |
|+.o+=o. . |
+----[SHA256]-----+
```
4. Run LISA
Use above `<subscription id>` and `<private key file>` to replace in below command.
It may take several minutes to complete.
```bash
lisa -r ./microsoft/runbook/azure.yml -v subscription_id:<subscription id> -v "admin_private_key_file:<private key file>"
```
5. Verify test results
After test completed, the html report is like below. The tier 0 includes a smoke test case, run on an Ubuntu image.
![image](img/smoke_test_result.png)
It doesn't need to create new VMs each time. And the test Linux distribution can be other gallery image or a VHD. Learn more from [runbook reference](runbook.md).
### Run in Ready VMs
The tests can run in an existing environment in lab or cloud. LISA calls the platform as `ready`.
1. Get IP address which can be accessed by LISA.
2. Prepare the SSH public/private key pair like steps in Azure.
3. Run LISA in existing VM.
Fill in related arguments and run. Note, since the existing VM doesn't support serial log, so smoke test cannot run in ready environment.
```bash
lisa -r ./microsoft/runbook/ready.yml -v public_address:<public address> -v "user_name:<user name>" -v "admin_private_key_file:<private key file>"
```
The existing VMs can be multiple to support networking testing, and it also support authenticate by password. Learn more from [runbook reference](runbook.md).
## Run Microsoft tests
LISA is an end-to-end solution to validate Linux in Azure, Hyper-V or other Microsoft virtualization platform. In quick start, it runs tier 0 Microsoft test cases in Azure. It tests an image can boot up and reboot without kernel panic. Refer to [test priority and tier](#test-priority-and-tier) know more about how test tiers defined.
If you want to validate more scenarios, run with `<tier id>` from 1 to 4. To run tier 1 to 4, it needs legacy LISAv2 tests. The LISA must be run in a Windows computer, and prepare a secret file. Learn more from [how to run legacy test cases](run_legacy.md).
```bash
lisa -r ./microsoft/runbook/azure.yml -v subscription_id:<subscription id> -v "admin_private_key_file:<private key file>" -v tier:<tier id>
```
Learn more from [command-line arguments](command_line.md) and the [runbook reference](runbook.md).
## Test priority and tier
## Run legacy LISAv2 test cases
Learn more from [how to run legacy LISAv2 tests](run_legacy.md).
## FAQ and Troubleshooting

26
docs/run_legacy.md Normal file
Просмотреть файл

@ -0,0 +1,26 @@
# How to run Legacy LISAv2 tests
[LISAv2](https://github.com/microsoft/lisa) brings incredible value to Linux quality on Microsoft virtualization platforms. To keep and increase the value of LISA, we decide to have more innovation and continue invest on LISA. So we start the current version of LISA.
During the transition time, we keep to validate Linux distributions. We cannot and don't want to stop to wait the exciting current LISA. The two versions will be co-existing for a while, under current LISA experience.
With this document, you will know how to run LISAv2 tests for validation in the current LISA.
The current LISA can clone any repo of LISAv2 to run, and parse LISAv2 log to generate test results with new format.
## Preparation
LISAv2 should run in the latest Windows 10 client 64 bits, or Windows Server 2019 editions.
Follow [LISAv2 document](https://github.com/microsoft/lisa/blob/master/README.md) to understand prerequisites, and prepare secret files.
Note, you don't need to run git clone, the current LISA will clone LISAv2 when running.
## Limitations
1. Test in parallel of LISAv2 doesn't support with the current LISA together. The current LISA will implement test matrix to replace current test in parallel in LISAv2. But there is no plan to compatible with LISAv2.
2. The LISAv2 results has low chance to be missed. There is rare race condition on accessing conflict of LISAv2 test log. If it happens on key logs which relates to test result parsing, it may cause the status of results are not shown correctly.
## road map
We're migrating LISAv2 test cases to current LISA by test case priority. We will keep t0 to tx runbooks update to date. When test cases migrated, they will be included in current LISA, and remove from LISAv2. It will be transparent to you.

115
docs/runbook.md Normal file
Просмотреть файл

@ -0,0 +1,115 @@
# Runbook Reference
- [How-to](#how-to)
- [Configure Azure deployment](#configure-azure-deployment)
- [Select and set test cases](#select-and-set-test-cases)
- [Use variable and secrets](#use-variable-and-secrets)
- [Use partial runbook](#use-partial-runbook)
- [Use extensions](#use-extensions)
- [Reference](#reference)
- [name](#name)
- [test_project](#test_project)
- [test_pass](#test_pass)
- [tags](#tags)
- [parent](#parent)
- [path](#path)
- [strategy](#strategy)
- [extension](#extension)
- [variable](#variable)
- [is_secret](#is_secret)
- [file](#file)
- [name](#name-1)
- [value](#value)
- [artifact](#artifact)
- [name](#name-2)
- [locations](#locations)
- [type](#type)
- [path](#path-1)
- [notifier](#notifier)
- [console](#console)
- [html](#html)
- [environment](#environment)
- [name](#name-3)
- [topology](#topology)
- [nodes](#nodes)
- [nodes_requirement](#nodes_requirement)
- [type](#type-1)
- [platform](#platform)
- [testcase](#testcase)
## How-to
### Configure Azure deployment
### Select and set test cases
### Use variable and secrets
### Use partial runbook
### Use extensions
## Reference
### name
### test_project
### test_pass
### tags
### parent
#### path
#### strategy
Not implemented
### extension
### variable
#### is_secret
#### file
#### name
### value
### artifact
#### name
#### locations
##### type
##### path
### notifier
#### console
#### html
### environment
#### name
#### topology
#### nodes
#### nodes_requirement
##### type
### platform
### testcase

63
docs/write_case.md Normal file
Просмотреть файл

@ -0,0 +1,63 @@
# How to write test cases
- [Troubleshooting excellence](#troubleshooting-excellence)
- [Document excellence](#document-excellence)
- [Test suite and test case](#test-suite-and-test-case)
- [Metadata](#metadata)
- [Test suite metadata](#test-suite-metadata)
- [Test case metadata](#test-case-metadata)
- [Requirement](#requirement)
- [Use tool](#use-tool)
- [Use Assertions](#use-assertions)
- [Use feature](#use-feature)
- [Debug test case](#debug-test-case)
## Troubleshooting excellence
Tuning when writing
One line summary
Test case log
Other log files
Repro
## Document excellence
Metadata
Code comments
Git comments
Logging
Error message
Meaningful Assertion
Write code like test logic instead of test action
## Test suite and test case
## Metadata
### Test suite metadata
### Test case metadata
## Requirement
## Use tool
## Use Assertions
## Use feature
## Debug test case
in ready environment