зеркало из https://github.com/microsoft/lisa.git
Initial version of documents
Complete first version README. Some documents are in very draft stage. They will be filled one by one.
This commit is contained in:
Chi Song
Родитель
1b24ce5c76
Коммит
40e108cd72
|
|
@ -1,9 +0,0 @@
|
|||
# Microsoft Open Source Code of Conduct
|
||||
|
||||
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
|
||||
|
||||
Resources:
|
||||
|
||||
- [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/)
|
||||
- [Microsoft Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/)
|
||||
- Contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with questions or concerns
|
||||
173
README.md
173
README.md
|
|
@ -1,163 +1,42 @@
|
|||
# Linux Integration Services Automation 3.0 (LISAv3)
|
||||
# Linux Integration Services Automation (LISA)
|
||||
|
||||
[](https://github.com/LIS/LISAv2/actions?query=workflow%3A%22CI+Workflow+for+LISAv3%22+event%3Apush+branch%3Amain)
|
||||
[](https://github.com/psf/black)
|
||||
[](https://github.com/LIS/LISAv2/blob/main/LICENSE-2.0.txt)
|
||||
[](https://github.com/microsoft/lisa/actions?query=workflow%3A%22CI+Workflow+for+LISAv3%22+event%3Apush+branch%3Amain)
|
||||
[](https://github.com/microsoft/lisa/blob/main/LICENSE)
|
||||
|
||||
LISA is a Linux test automation framework with built-in test cases to verify the quality of
|
||||
Linux distributions on multiple platforms (such as Azure, On-prem HyperV, and Linux bare metal).
|
||||
**Linux Integration Services Automation (LISA)** is designed to be an end-to-end solution to validate Linux kernels and distributions quality on virtualization platforms of Microsoft and others. It can be used on more tests, or support other cloud platforms as well.
|
||||
|
||||
This version (v3) is a brand new implementation in Python 3, supports running on Windows and Linux distros.
|
||||
* **End-to-end**: LISA defines several set of test suites to validate Linux kernels and distributions in Microsoft Azure, and Hyper-V. The test suites can help find integration issues easily.
|
||||
* **Ease-to-use**: The complexity and diversity of testing or Linux distributions is wrapped in different components of LISA. When running LISA, it doesn't need to know much on details.
|
||||
* **Extensibility**: LISA is extendable in many components to support various scenarios, including virtualization platforms, commands, Linux distributions, community test suites, etc. LISA supports to validate Microsoft virtualization platforms natively, but also is able to extend to support other cloud or on-premises platforms.
|
||||
|
||||
## Getting Started
|
||||
## Why LISA
|
||||
|
||||
### Install Python 3
|
||||
There are a lot of tools, which focus on Linux kernels or distributions quality. They are very important to guarantee quality of kernels and distributions. The integration validation on virtualization platforms is a little different with classic Linux testing. It needs to cover different types of resources with manageable cost. It needs to create/delete resources with a good plan automatically.
|
||||
|
||||
Install [Python 3.8](https://www.python.org/) from Linux distribution’s package
|
||||
repositories, or [python.org](https://www.python.org/). We also need `gcc` and
|
||||
the `libpython` headers for the `usjon` package.
|
||||
LISA focuses on validating the integration of Linux kernels/distributions and virtualization platforms. It needs more interactive with virtualization platforms to run tests for different purposes, like test different capacity, hardware, and so on.
|
||||
|
||||
On Ubuntu 18.04 and 20.04:
|
||||
## Documents
|
||||
|
||||
```bash
|
||||
sudo apt install git python3.8 libpython3.8-dev python3-distutils python3-apt gcc
|
||||
```
|
||||
* [Key concepts](docs/concepts.md)
|
||||
* [Install LISA](docs/install.md)
|
||||
* [Run tests with LISA](docs/run.md)
|
||||
* [Command line reference](docs/command_line.md)
|
||||
* [Runbook reference](docs/runbook.md)
|
||||
* [Microsoft test suites](https://github.com/microsoft/lisa/blob/master/Documents/LISAv2-TestCase-Statistics.md)
|
||||
* [How to write tests](docs/write_case.md)
|
||||
* [How to extend and customize LISA](docs/extension.md)
|
||||
* [How to run previous version LISA (aka LISAv2)](docs/run_legacy.md)
|
||||
|
||||
NOTE: If you are using WSL, installing Poetry on both Windows and Linux may
|
||||
cause both platforms’ versions of Poetry to be on your path, as Windows binaries
|
||||
are mapped into WSL’s `PATH`. This means that the Linux `poetry` binary _must_
|
||||
appear in your `PATH` before the Windows version, or this error will appear:
|
||||
## Contribute
|
||||
|
||||
> `/usr/bin/env: ‘python\r’: No such file or directory`
|
||||
You are very welcome to contribute. Please follow [the contribution document](docs/contributing.md) for details.
|
||||
|
||||
### Install Poetry
|
||||
## History and road map
|
||||
|
||||
On Linux (or WSL):
|
||||
The previous LISA called LISAv2, which is in [master branch](https://github.com/microsoft/lisa/tree/master). The previous LISA can be used standalone or called from current LISA. Refer to [how to run LISAv2 test cases](docs/run_legacy.md) for details.
|
||||
|
||||
```bash
|
||||
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3 - --preview --version 1.1.4
|
||||
source $HOME/.poetry/env
|
||||
```
|
||||
|
||||
On Windows (in PowerShell):
|
||||
|
||||
```powershell
|
||||
(Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python - --preview --version 1.1.4
|
||||
# the path can be added to system, so it applies to every terminal.
|
||||
$env:PATH += ";$env:USERPROFILE\.poetry\bin"
|
||||
```
|
||||
|
||||
If you already have Poetry installed, you can update it like:
|
||||
|
||||
```bash
|
||||
poetry self update 1.1.4
|
||||
```
|
||||
|
||||
### Install Python packages
|
||||
|
||||
Then use Poetry to install LISAv3's Python package dependencies:
|
||||
|
||||
```bash
|
||||
git clone -b main https://github.com/LIS/LISAv2.git
|
||||
cd LISAv2
|
||||
poetry install
|
||||
```
|
||||
|
||||
To obtain the path of the Poetry virtual environment setup for LISA (where the
|
||||
isolated Python installation and packages are located), run:
|
||||
|
||||
```bash
|
||||
poetry env list --full-path
|
||||
```
|
||||
|
||||
### Run LISAv3
|
||||
|
||||
Run LISAv3 using Poetry’s environment:
|
||||
|
||||
```bash
|
||||
poetry run python lisa/main.py
|
||||
```
|
||||
|
||||
### Make
|
||||
|
||||
We now also have a GNU Makefile that automates some tasks. Try:
|
||||
|
||||
```bash
|
||||
# Install Python packages
|
||||
make setup
|
||||
|
||||
# Run LISAv3
|
||||
make run
|
||||
|
||||
# Run unit tests
|
||||
make test
|
||||
|
||||
# Run syntactic, semantic, formatting and type checkers
|
||||
make check
|
||||
|
||||
# Generate coverage report (slow, reruns LISAv3 and tests)
|
||||
make coverage
|
||||
|
||||
# Print current Python virtualenv
|
||||
make venv
|
||||
```
|
||||
|
||||
### Editor Setup
|
||||
|
||||
Install and enable [ShellCheck](https://github.com/koalaman/shellcheck) to find
|
||||
bash errors locally.
|
||||
|
||||
#### Visual Studio Code
|
||||
|
||||
First, click the Python version in the bottom left, then enter the path emitted
|
||||
by the command above. This will point Code to the Poetry virtual environment.
|
||||
|
||||
Make sure below settings are in root level of `.vscode/settings.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"python.analysis.typeCheckingMode": "strict",
|
||||
"python.formatting.provider": "black",
|
||||
"python.linting.enabled": true,
|
||||
"python.linting.flake8Enabled": true,
|
||||
"python.linting.mypyEnabled": true,
|
||||
"python.linting.pylintEnabled": false,
|
||||
"editor.formatOnSave": true,
|
||||
"python.linting.mypyArgs": [
|
||||
"--strict",
|
||||
"--namespace-packages",
|
||||
"--show-column-numbers"
|
||||
],
|
||||
"python.sortImports.path": "isort",
|
||||
"python.analysis.useLibraryCodeForTypes": false,
|
||||
"python.analysis.autoImportCompletions": false,
|
||||
"files.eol": "\n",
|
||||
}
|
||||
```
|
||||
|
||||
#### Emacs
|
||||
|
||||
Use the [pyvenv](https://github.com/jorgenschaefer/pyvenv) package:
|
||||
|
||||
```emacs-lisp
|
||||
(use-package pyvenv
|
||||
:ensure t
|
||||
:hook (python-mode . pyvenv-tracking-mode))
|
||||
```
|
||||
|
||||
Then run `M-x add-dir-local-variable RET python-mode RET pyvenv-activate RET
|
||||
<path/to/virtualenv>` where the value is the path given by the command above.
|
||||
This will create a `.dir-locals.el` file which looks like this:
|
||||
|
||||
```emacs-lisp
|
||||
;;; Directory Local Variables
|
||||
;;; For more information see (info "(emacs) Directory Variables")
|
||||
|
||||
((python-mode . ((pyvenv-activate . "~/.cache/pypoetry/virtualenvs/lisa-s7Q404Ij-py3.8"))))
|
||||
```
|
||||
|
||||
## Contributing
|
||||
LISA is in active developing, and a lot of exciting features are implementing. We're listening your feedback.
|
||||
|
||||
## License
|
||||
|
||||
The entire codebase is under [MIT license](LICENSE)
|
||||
The entire codebase is under [MIT license](LICENSE).
|
||||
|
|
|
|||
|
|
@ -0,0 +1,26 @@
|
|||
# command line reference
|
||||
|
||||
- [common arguments](#common-arguments)
|
||||
- [-d, --debug](#-d---debug)
|
||||
- [-r, --runbook](#-r---runbook)
|
||||
- [-h, --help](#-h---help)
|
||||
- [-v, --variable](#-v---variable)
|
||||
- [run](#run)
|
||||
- [check](#check)
|
||||
- [list](#list)
|
||||
|
||||
## common arguments
|
||||
|
||||
### -d, --debug
|
||||
|
||||
### -r, --runbook
|
||||
|
||||
### -h, --help
|
||||
|
||||
### -v, --variable
|
||||
|
||||
## run
|
||||
|
||||
## check
|
||||
|
||||
## list
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
# Concepts
|
||||
|
||||
- [Runbook](#runbook)
|
||||
- [Environment and Node](#environment-and-node)
|
||||
- [Tools and Scripts](#tools-and-scripts)
|
||||
- [Platform](#platform)
|
||||
- [Feature](#feature)
|
||||
- [Requirement and Capability](#requirement-and-capability)
|
||||
|
||||
## Runbook
|
||||
|
||||
## Environment and Node
|
||||
|
||||
## Tools and Scripts
|
||||
|
||||
## Platform
|
||||
|
||||
## Feature
|
||||
|
||||
## Requirement and Capability
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
# Contributing
|
||||
|
||||
You are very welcome to contribute in any way. There are serval ways to guide the contribution. They have no intention of restricting you, they will only increase efficiency.
|
||||
|
||||
- [Provide feedback or ask a question](#provide-feedback-or-ask-a-question)
|
||||
- [File a bug](#file-a-bug)
|
||||
- [Contribute more](#contribute-more)
|
||||
- [Contribute fixing or extension](#contribute-fixing-or-extension)
|
||||
- [Contribute feature](#contribute-feature)
|
||||
- [Code of Conduct](#code-of-conduct)
|
||||
|
||||
## Provide feedback or ask a question
|
||||
|
||||
You are welcome to provide feedback by [filing an issue](https://github.com/microsoft/lisa/issues/new).
|
||||
|
||||
When there is any question, please check below FAQs first. If there is no satisfying answer, don't hesitate to [file an issue](https://github.com/microsoft/lisa/issues/new).
|
||||
|
||||
## File a bug
|
||||
|
||||
Before file a bug, please try to search keywords on existing bugs. If you cannot find any similar bugs, feel free to [file a new one](https://github.com/microsoft/lisa/issues/new).
|
||||
|
||||
## Contribute more
|
||||
|
||||
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
|
||||
|
||||
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
|
||||
|
||||
### Contribute fixing or extension
|
||||
|
||||
If you think there is a bug, and want to fix it. Feel free to submit a pull request. We will review it proactively. Once it's signed off, we will merge the change.
|
||||
|
||||
If there is any extension is needed by others, submit a pull request.
|
||||
|
||||
Learn more on [development guideline](./development.md).
|
||||
|
||||
### Contribute feature
|
||||
|
||||
Please file an issue to describe why the feature is needed, and how it will be implemented. Once we agree on that, follow [Contribute fixing or extension](#contribute-fixing-or-extension) to implement it.
|
||||
|
||||
## Code of Conduct
|
||||
|
||||
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
|
||||
|
|
@ -1,8 +1,78 @@
|
|||
# Contributing Guidelines
|
||||
# Development Guideline
|
||||
|
||||
This document describes the existing developer tooling we have in place (and what to
|
||||
expect of it), as well as our design and development philosophy.
|
||||
|
||||
- [Editor Setup](#editor-setup)
|
||||
- [Visual Studio Code](#visual-studio-code)
|
||||
- [Emacs](#emacs)
|
||||
- [other tools](#other-tools)
|
||||
- [Create a pull request](#create-a-pull-request)
|
||||
- [Naming Conventions](#naming-conventions)
|
||||
- [Automated Tooling](#automated-tooling)
|
||||
- [Metadata](#metadata)
|
||||
- [Package Dependencies](#package-dependencies)
|
||||
- [Developer Dependencies](#developer-dependencies)
|
||||
- [Type Annotations](#type-annotations)
|
||||
- [Runbook schema](#runbook-schema)
|
||||
- [Committing Guidelines](#committing-guidelines)
|
||||
- [Design Patterns](#design-patterns)
|
||||
- [Future Sections](#future-sections)
|
||||
|
||||
## Editor Setup
|
||||
|
||||
### Visual Studio Code
|
||||
|
||||
First, click the Python version in the bottom left, then enter the path emitted by the command above. This will point Code to the Poetry virtual environment.
|
||||
|
||||
Make sure below settings are in root level of `.vscode/settings.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"python.analysis.typeCheckingMode": "strict",
|
||||
"python.formatting.provider": "black",
|
||||
"python.linting.enabled": true,
|
||||
"python.linting.flake8Enabled": true,
|
||||
"python.linting.mypyEnabled": true,
|
||||
"python.linting.pylintEnabled": false,
|
||||
"editor.formatOnSave": true,
|
||||
"python.linting.mypyArgs": [
|
||||
"--strict",
|
||||
"--namespace-packages",
|
||||
"--show-column-numbers"
|
||||
],
|
||||
"python.sortImports.path": "isort",
|
||||
"python.analysis.useLibraryCodeForTypes": false,
|
||||
"python.analysis.autoImportCompletions": false,
|
||||
"files.eol": "\n",
|
||||
}
|
||||
```
|
||||
|
||||
### Emacs
|
||||
|
||||
Use the [pyvenv](https://github.com/jorgenschaefer/pyvenv) package:
|
||||
|
||||
```emacs-lisp
|
||||
(use-package pyvenv
|
||||
:ensure t
|
||||
:hook (python-mode . pyvenv-tracking-mode))
|
||||
```
|
||||
|
||||
Then run `M-x add-dir-local-variable RET python-mode RET pyvenv-activate RET <path/to/virtualenv>` where the value is the path given by the command above. This will create a `.dir-locals.el` file which looks like this:
|
||||
|
||||
```emacs-lisp
|
||||
;;; Directory Local Variables
|
||||
;;; For more information see (info "(emacs) Directory Variables")
|
||||
|
||||
((python-mode . ((pyvenv-activate . "~/.cache/pypoetry/virtualenvs/lisa-s7Q404Ij-py3.8"))))
|
||||
```
|
||||
|
||||
### other tools
|
||||
|
||||
- Install and enable [ShellCheck](https://github.com/koalaman/shellcheck) to find bash errors locally.
|
||||
|
||||
## Create a pull request
|
||||
|
||||
## Naming Conventions
|
||||
|
||||
Naming conventions are not automatically enforced, so please read the [naming
|
||||
|
|
@ -10,23 +80,16 @@ conventions](https://www.python.org/dev/peps/pep-0008/#naming-conventions)
|
|||
section of PEP 8, which describes what each of the different styles means. A
|
||||
short summary of the most important parts:
|
||||
|
||||
* Modules (and hence files) should have short, all-lowercase names.
|
||||
* Class (and exception) names should normally use the `CapWords` convention
|
||||
(also known as `CamelCase`).
|
||||
* Function and variable names should be lowercase, with words separated by
|
||||
underscores as necessary to improve readability (also known as `snake_case`).
|
||||
* To avoid collisions with the standard library, an underscore can be appended,
|
||||
such as `id_`.
|
||||
* Always use `self` for the first argument to instance methods.
|
||||
* Always use `cls` for the first argument to class methods.
|
||||
* One leading underscore like `_data` is for non-public methods and instance
|
||||
variables. And it can be used by sub-classes. If it won't be used in
|
||||
sub-classes, use like `__data`.
|
||||
* If there is a pair of `get_x` and `set_x` methods, they should instead be a
|
||||
proper property, which is easy to do with the built-in `@property` decorator.
|
||||
* Constants should be `CAPITALIZED_SNAKE_CASE`.
|
||||
* When importing a function, try to avoid renaming it with `import as` because
|
||||
it introduces cognitive overhead to track yet another name.
|
||||
- Modules (and hence files) should have short, all-lowercase names.
|
||||
- Class (and exception) names should normally use the `CapWords` convention (also known as `CamelCase`).
|
||||
- Function and variable names should be lowercase, with words separated by underscores as necessary to improve readability (also known as `snake_case`).
|
||||
- To avoid collisions with the standard library, an underscore can be appended, such as `id_`.
|
||||
- Always use `self` for the first argument to instance methods.
|
||||
- Always use `cls` for the first argument to class methods.
|
||||
- One leading underscore like `_data` is for non-public methods and instance variables. And it can be used by sub-classes. If it won't be used in sub-classes, use like `__data`.
|
||||
- If there is a pair of `get_x` and `set_x` methods, they should instead be a proper property, which is easy to do with the built-in `@property` decorator.
|
||||
- Constants should be `CAPITALIZED_SNAKE_CASE`.
|
||||
- When importing a function, try to avoid renaming it with `import as` because it introduces cognitive overhead to track yet another name.
|
||||
|
||||
When in doubt, adhere to existing conventions, or check the style guide.
|
||||
|
||||
|
|
@ -81,24 +144,24 @@ on Windows, you will want to setup your own, perhaps using [Conda][].
|
|||
[pyenv]: https://github.com/pyenv/pyenv
|
||||
[Conda]: https://docs.conda.io/en/latest/
|
||||
|
||||
* python: We pinned Python to version 3.8 so everyone uses the same version.
|
||||
- python: We pinned Python to version 3.8 so everyone uses the same version.
|
||||
|
||||
* psutil: TODO @squirrelsc will document
|
||||
- psutil: TODO @squirrelsc will document
|
||||
|
||||
* pyyaml: TODO @squirrelsc will document
|
||||
- pyyaml: TODO @squirrelsc will document
|
||||
|
||||
* retry: TODO @squirrelsc will document
|
||||
- retry: TODO @squirrelsc will document
|
||||
|
||||
* paramiko: TODO @squirrelsc will document
|
||||
- paramiko: TODO @squirrelsc will document
|
||||
|
||||
* spurplus: TODO @squirrelsc will document
|
||||
- spurplus: TODO @squirrelsc will document
|
||||
|
||||
* dataclasses-json: TODO @squirrelsc will document (brings in `usjon` which
|
||||
- dataclasses-json: TODO @squirrelsc will document (brings in `usjon` which
|
||||
requires `gcc` and `libpython`)
|
||||
|
||||
* portalocker: TODO @squirrelsc will document
|
||||
- portalocker: TODO @squirrelsc will document
|
||||
|
||||
* azure-*: TODO @squirrelsc will document
|
||||
- azure-*: TODO @squirrelsc will document
|
||||
|
||||
### Developer Dependencies
|
||||
|
||||
|
|
@ -107,29 +170,29 @@ add --dev <package_name>` records our _developer_ packages. These are not
|
|||
necessary for LISAv3 to execute, but are used by developers to automatically
|
||||
adhere to our coding standards.
|
||||
|
||||
* [Black](https://github.com/psf/black), the opinionated code formatter which
|
||||
- [Black](https://github.com/psf/black), the opinionated code formatter which
|
||||
settles all debates as to how our Python files should be formatted. It follows
|
||||
[PEP 8](https://www.python.org/dev/peps/pep-0008/), the official Python style
|
||||
guide, and where ambiguous makes the decision for us.
|
||||
|
||||
* [Flake8](https://flake8.pycqa.org/en/latest/) (and integrations), the semantic
|
||||
- [Flake8](https://flake8.pycqa.org/en/latest/) (and integrations), the semantic
|
||||
analyzer, used to coordinate most of the other tools.
|
||||
|
||||
* [isort](https://timothycrosley.github.io/isort/), the `import` sorter, which
|
||||
- [isort](https://timothycrosley.github.io/isort/), the `import` sorter, which
|
||||
automatically splits imports into the expected, alphabetized sections.
|
||||
|
||||
* [mypy](http://mypy-lang.org/), the static type checker, which coupled with
|
||||
- [mypy](http://mypy-lang.org/), the static type checker, which coupled with
|
||||
type annotations allows us to avoid the pitfalls of Python being a dynamically
|
||||
typed language.
|
||||
|
||||
* [python-language-server](https://github.com/palantir/python-language-server)
|
||||
- [python-language-server](https://github.com/palantir/python-language-server)
|
||||
(and integrations), the de facto LSP server. While Microsoft is developing
|
||||
their own LSP servers, they do not integrate with the existing ecosystem of
|
||||
tools, and their latest tool, Pyright, simply does not support
|
||||
`pyproject.toml`. Since pyls is used far more widely, and supports every
|
||||
editor, we use it.
|
||||
|
||||
* [rope](https://github.com/python-rope/rope), to provide completions and
|
||||
- [rope](https://github.com/python-rope/rope), to provide completions and
|
||||
renaming support to pyls.
|
||||
|
||||
With these packages installed and a correctly setup editor (see the readme and
|
||||
|
|
@ -204,12 +267,12 @@ example](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html):
|
|||
>
|
||||
> Further paragraphs come after blank lines.
|
||||
>
|
||||
> * Bullet points are okay, too
|
||||
> - Bullet points are okay, too
|
||||
>
|
||||
> * Typically a hyphen or asterisk is used for the bullet, followed by a
|
||||
> - Typically a hyphen or asterisk is used for the bullet, followed by a
|
||||
> single space, with blank lines in between, but conventions vary here
|
||||
>
|
||||
> * Use a hanging indent
|
||||
> - Use a hanging indent
|
||||
|
||||
You should also feel free to use Markdown in the commit messages, as our project
|
||||
is hosted on GitHub which renders it (and Markdown is human readable).
|
||||
|
|
@ -240,15 +303,15 @@ carefully create a well-designed and maintainable framework.
|
|||
|
||||
Several popular patterns that actually _do not_ work well in Python are:
|
||||
|
||||
* [The Abstract Factory Pattern](https://python-patterns.guide/gang-of-four/abstract-factory/)
|
||||
* [The Factory Method Pattern](https://python-patterns.guide/gang-of-four/factory-method/)
|
||||
* [The Prototype Pattern](https://python-patterns.guide/gang-of-four/prototype/)
|
||||
* [The Singleton Pattern](https://python-patterns.guide/gang-of-four/singleton/)
|
||||
- [The Abstract Factory Pattern](https://python-patterns.guide/gang-of-four/abstract-factory/)
|
||||
- [The Factory Method Pattern](https://python-patterns.guide/gang-of-four/factory-method/)
|
||||
- [The Prototype Pattern](https://python-patterns.guide/gang-of-four/prototype/)
|
||||
- [The Singleton Pattern](https://python-patterns.guide/gang-of-four/singleton/)
|
||||
|
||||
Conversely, patterns that are a natural fit to Python include:
|
||||
|
||||
* [The Composite Pattern](https://python-patterns.guide/gang-of-four/composite/)
|
||||
* [The Iterator Pattern](https://python-patterns.guide/gang-of-four/iterator/)
|
||||
- [The Composite Pattern](https://python-patterns.guide/gang-of-four/composite/)
|
||||
- [The Iterator Pattern](https://python-patterns.guide/gang-of-four/iterator/)
|
||||
(caution: it is actually better to implement these with `yield`!)
|
||||
|
||||
Finally, a high-level guide to all things Python is [The Hitchhiker’s Guide to
|
||||
|
|
@ -257,28 +320,16 @@ Python world. If you make it through even some of these guides, you will be well
|
|||
on your way to being a “Pythonista” (a Python developer) writing “Pythonic”
|
||||
(canonically correct Python) code left and right.
|
||||
|
||||
### Async IO
|
||||
|
||||
With Python 3.4, the Async IO pattern found in languages such as C# and Go is
|
||||
available through the keywords `async` and `await`, along with the Python module
|
||||
`asyncio`. Please read [Async IO in Python: A Complete
|
||||
Walkthrough](https://realpython.com/async-io-python/) to understand at a high
|
||||
level how asynchronous programming works. As of Python 3.7, One major “gotcha”
|
||||
is that `asyncio.run(...)` should be used [exactly once in
|
||||
`main`](https://docs.python.org/3/library/asyncio-task.html), it starts the
|
||||
event loop. Everything else should be a coroutine or task which the event loop
|
||||
schedules.
|
||||
|
||||
## Future Sections
|
||||
|
||||
Just a collection of reminders for the author to expand on later.
|
||||
|
||||
* [unittest](https://docs.python.org/3/library/unittest.html)
|
||||
* [doctest](https://docs.python.org/3/library/doctest.html)
|
||||
* [subprocess](https://pymotw.com/3/subprocess/index.html)
|
||||
* [GitHub Actions](https://github.com/LIS/LISAv2/actions)
|
||||
* [ShellCheck](https://www.shellcheck.net/)
|
||||
* [Governance](https://opensource.guide/leadership-and-governance/)
|
||||
* [Maintenance Cost](https://web.archive.org/web/20120313070806/http://users.jyu.fi/~koskinen/smcosts.htm)
|
||||
* Parallelism and multi-plexing
|
||||
* Versioned inputs and outputs
|
||||
- [unittest](https://docs.python.org/3/library/unittest.html)
|
||||
- [doctest](https://docs.python.org/3/library/doctest.html)
|
||||
- [subprocess](https://pymotw.com/3/subprocess/index.html)
|
||||
- [GitHub Actions](https://github.com/LIS/LISAv2/actions)
|
||||
- [ShellCheck](https://www.shellcheck.net/)
|
||||
- [Governance](https://opensource.guide/leadership-and-governance/)
|
||||
- [Maintenance Cost](https://web.archive.org/web/20120313070806/http://users.jyu.fi/~koskinen/smcosts.htm)
|
||||
- Parallelism and multi-plexing
|
||||
- Versioned inputs and outputs
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
# How to write extensions in LISA
|
||||
|
||||
## Platform
|
||||
|
||||
## Feature
|
||||
|
||||
## Notifier
|
||||
|
||||
## Tool
|
||||
|
||||
## Linux Distribution
|
||||
Двоичный файл не отображается.
|
После Ширина: | Высота: | Размер: 68 KiB |
|
|
@ -0,0 +1,95 @@
|
|||
# Install LISA
|
||||
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Install Python](#install-python)
|
||||
- [Install dependencies](#install-dependencies)
|
||||
- [Clone code](#clone-code)
|
||||
- [Install Poetry and Python dependencies](#install-poetry-and-python-dependencies)
|
||||
- [Install Poetry in Linux](#install-poetry-in-linux)
|
||||
- [Install Poetry in Windows](#install-poetry-in-windows)
|
||||
- [FAQ and Troubleshooting](#faq-and-troubleshooting)
|
||||
- [How to use LISA in WSL](#how-to-use-lisa-in-wsl)
|
||||
- [Cannot find package after run `poetry install`](#cannot-find-package-after-run-poetry-install)
|
||||
- [Poetry related questions](#poetry-related-questions)
|
||||
- [Other issues](#other-issues)
|
||||
|
||||
LISA supports to run on both Windows and Linux. Follow below steps to install LISA from source code.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
LISA needs to be installed on a computer, which
|
||||
|
||||
* Can access the tested platform, like Azure, Hyper-V, or else. It recommends to have good bandwidth and low network latency.
|
||||
* At least 2 CPU cores and 4GB memory.
|
||||
|
||||
## Install Python
|
||||
|
||||
LISA is developed in Python, and tested with Python 3.8 (64 bit). The latest version of Python 3.8 is recommended, but feel free to file an issue, if LISA is not compatible with higher Python version.
|
||||
|
||||
Install latest [Python 3.8 64 bits](https://www.python.org/). If there are Python installed already, it needs to make sure effective Python's version is above 3.8 and 64 bits.
|
||||
|
||||
## Install dependencies
|
||||
|
||||
Some Python packages need to be built from source, so it needs build tools installed.
|
||||
|
||||
In Linux, it needs `git` and `gcc`. Below is depended packages on Ubuntu.
|
||||
|
||||
```bash
|
||||
sudo apt install git gcc
|
||||
```
|
||||
|
||||
In Windows, you need to install [git](https://git-scm.com/downloads) and [Visual C++ redistributable package](https://aka.ms/vs/16/release/vc_redist.x64.exe)
|
||||
|
||||
## Clone code
|
||||
|
||||
Open a terminal window, and enter the folder, which to download lisa code.
|
||||
|
||||
```sh
|
||||
git clone https://github.com/microsoft/lisa.git
|
||||
cd lisa
|
||||
```
|
||||
|
||||
## Install Poetry and Python dependencies
|
||||
|
||||
Poetry is used to manage Python dependencies of LISA. Execute corresponding script to install Poetry.
|
||||
|
||||
Note, it's important to enter LISA's folder to run below command, since Poetry manages dependencies by working folders.
|
||||
|
||||
### Install Poetry in Linux
|
||||
|
||||
```bash
|
||||
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3 -
|
||||
source $HOME/.poetry/env
|
||||
poetry install
|
||||
```
|
||||
|
||||
### Install Poetry in Windows
|
||||
|
||||
Open a PowerShell command prompt and execute
|
||||
|
||||
```powershell
|
||||
(Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python -
|
||||
# the path can be added to system, so it applies to every terminal.
|
||||
$env:PATH += ";$env:USERPROFILE\.poetry\bin"
|
||||
poetry install
|
||||
```
|
||||
|
||||
## FAQ and Troubleshooting
|
||||
|
||||
### How to use LISA in WSL
|
||||
|
||||
If you are using WSL, installing Poetry on both Windows and WSL may cause both platforms' versions of Poetry to be on your path, as Windows binaries are mapped into `PATH` of WSL. This means that the WSL `poetry` binary _must_ appear in your `PATH` before the Windows version, or this error will appear:
|
||||
|
||||
> `/usr/bin/env: ‘python\r’: No such file or directory`
|
||||
|
||||
### Cannot find package after run `poetry install`
|
||||
|
||||
Poetry is case sensitive. When in windows, make sure the cases of path is consistent every time.
|
||||
|
||||
### Poetry related questions
|
||||
|
||||
Poetry is very useful to manage dependencies of Python. It's a virtual environment, not a complete environment like Conda. So make sure the right version of Python is effective. Know more about Poetry on [installation](https://python-poetry.org/docs/#installation) or [commands](https://python-poetry.org/docs/cli/).
|
||||
|
||||
### Other issues
|
||||
|
||||
Please check [known issues](https://github.com/microsoft/lisa/issues), or [file an issue](https://github.com/microsoft/lisa/issues/new) if it doesn't exists.
|
||||
|
|
@ -0,0 +1,183 @@
|
|||
# Run LISA
|
||||
|
||||
- [Quick start](#quick-start)
|
||||
- [Run locally](#run-locally)
|
||||
- [Run in Azure](#run-in-azure)
|
||||
- [Run in Ready VMs](#run-in-ready-vms)
|
||||
- [Run Microsoft tests](#run-microsoft-tests)
|
||||
- [Test priority and tier](#test-priority-and-tier)
|
||||
- [Run legacy LISAv2 test cases](#run-legacy-lisav2-test-cases)
|
||||
- [FAQ and Troubleshooting](#faq-and-troubleshooting)
|
||||
|
||||
## Quick start
|
||||
|
||||
In Linux, define an alias to simplify the command. If you want the alias effective every time, add into `.bashrc`
|
||||
|
||||
```bash
|
||||
alias lisa="./lisa.sh"
|
||||
```
|
||||
|
||||
### Run locally
|
||||
|
||||
If not arguments specified, LISA runs test cases on local machine. Those cases won't modify local environment, and they are helpful to validate LISA installation.
|
||||
|
||||
```bash
|
||||
lisa
|
||||
```
|
||||
|
||||
After run, the terminal output is like below. If there is any error, check troubleshooting part of [installation](install.md).
|
||||
|
||||
The log shows 5 test cases are scheduled, 2 passed, and 3 skipped. For each skipped case, there is a reason, why it's skipped. There is a html test report, and its link shows at end of log.
|
||||
|
||||
```text
|
||||
2021-03-04 06:32:30.904 INFO LISA.RootRunner ________________________________________
|
||||
2021-03-04 06:32:30.904 INFO LISA.RootRunner HelloWorld.hello: PASSED
|
||||
2021-03-04 06:32:30.905 INFO LISA.RootRunner HelloWorld.bye: PASSED
|
||||
2021-03-04 06:32:30.905 INFO LISA.RootRunner MultipleNodesDemo.os_info: SKIPPED no available environment: ['no enough nodes, requirement: 2, capability: 1. ']
|
||||
2021-03-04 06:32:30.906 INFO LISA.RootRunner MultipleNodesDemo.perf_network_tcp_ipv4_throughput_ntttcp_synthetic_singleconnection: SKIPPED no available environment: ['no enough nodes, requirement: 2, capability: 1. ']
|
||||
2021-03-04 06:32:30.906 INFO LISA.RootRunner WithScript.script: SKIPPED no available environment: ["os_type: requirements excludes ['Windows']"]
|
||||
2021-03-04 06:32:30.907 INFO LISA.RootRunner test result summary
|
||||
2021-03-04 06:32:30.907 INFO LISA.RootRunner TOTAL : 5
|
||||
2021-03-04 06:32:30.908 INFO LISA.RootRunner NOTRUN : 0
|
||||
2021-03-04 06:32:30.908 INFO LISA.RootRunner RUNNING : 0
|
||||
2021-03-04 06:32:30.908 INFO LISA.RootRunner FAILED : 0
|
||||
2021-03-04 06:32:30.908 INFO LISA.RootRunner PASSED : 2
|
||||
2021-03-04 06:32:30.910 INFO LISA.RootRunner SKIPPED : 3
|
||||
2021-03-04 06:32:30.913 INFO LISA.notifier[Html] report: D:\code\LISAv2\runtime\runs\20210304\20210304-063226-496\lisa.html
|
||||
2021-03-04 06:32:30.923 INFO LISA completed in 4.428 sec
|
||||
```
|
||||
|
||||
### Run in Azure
|
||||
|
||||
Below commands shows how to run Microsoft tier 0 tests on an Azure gallery image.
|
||||
|
||||
1. Sign in to Azure
|
||||
|
||||
Make sure [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli) or [Azure PowerShell](https://docs.microsoft.com/en-us/powershell/azure/install-az-ps) installed. It needs to sign in to run LISA on current machine. LISA supports other Azure credential mechanisms also, refer to [runbook reference](runbook.md).
|
||||
|
||||
Here uses Azure CLI as the example. Open a terminal.
|
||||
|
||||
```bash
|
||||
az login
|
||||
```
|
||||
|
||||
2. Find subscription id
|
||||
|
||||
If you have multiple Azure subscriptions. Use below command to set default subscription. LISA will create resource groups in it. To prevent missing use, the subscription id is a required parameter to run LISA.
|
||||
|
||||
Run below command to retrieve subscription information.
|
||||
|
||||
```bash
|
||||
az account show --subscription "<Subscription Name to run LISA>"
|
||||
```
|
||||
|
||||
You will see output like below, find `<subscription id>` for next steps.
|
||||
|
||||
```json
|
||||
{
|
||||
"environmentName": "AzureCloud",
|
||||
"homeTenantId": "<tenant id>",
|
||||
"id": "<subscription id>",
|
||||
"isDefault": true,
|
||||
"managedByTenants": [],
|
||||
"name": "<subscription name>",
|
||||
"state": "Enabled",
|
||||
"tenantId": "<tenant id>",
|
||||
"user": {
|
||||
"name": "<user account>",
|
||||
"type": "user"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. Prepare ssh key pair
|
||||
|
||||
LISA connects to VMs by SSH, and a public/private key pair is necessary for authentication. If you don't have a key pair, follow this step to create one.
|
||||
|
||||
Run below command, and follow prompt to create a key pair. Most Linux distributions have SSH client installed. On windows, follow [install OpenSSH](https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse) instruction.
|
||||
|
||||
Note, don't use passphrase to generate the key. LISA doesn't support passphrase.
|
||||
|
||||
```bash
|
||||
ssh-keygen
|
||||
```
|
||||
|
||||
Find `<private key file>` for next steps. If you want to move key files, keep public/private key files are in same folder, and keep the same base file name.
|
||||
|
||||
```bash
|
||||
Enter passphrase (empty for no passphrase):
|
||||
Enter same passphrase again:
|
||||
Your identification has been saved in <private key file>.
|
||||
Your public key has been saved in <public key file>.
|
||||
The key fingerprint is:
|
||||
SHA256:OIzc1yE7joL2Bzy8!gS0j8eGK7bYaH1FmF3sDuMeSj8 username@server@LOCAL-HOSTNAME
|
||||
|
||||
The key's randomart image is:
|
||||
+--[ED25519 256]--+
|
||||
| . |
|
||||
| o |
|
||||
| . + + . |
|
||||
| o B * = . |
|
||||
| o= B S . |
|
||||
| .=B O o |
|
||||
| + =+% o |
|
||||
| *oo.O.E |
|
||||
|+.o+=o. . |
|
||||
+----[SHA256]-----+
|
||||
```
|
||||
|
||||
4. Run LISA
|
||||
|
||||
Use above `<subscription id>` and `<private key file>` to replace in below command.
|
||||
|
||||
It may take several minutes to complete.
|
||||
|
||||
```bash
|
||||
lisa -r ./microsoft/runbook/azure.yml -v subscription_id:<subscription id> -v "admin_private_key_file:<private key file>"
|
||||
```
|
||||
|
||||
5. Verify test results
|
||||
|
||||
After test completed, the html report is like below. The tier 0 includes a smoke test case, run on an Ubuntu image.
|
||||
|
||||
|
||||
|
||||
It doesn't need to create new VMs each time. And the test Linux distribution can be other gallery image or a VHD. Learn more from [runbook reference](runbook.md).
|
||||
|
||||
### Run in Ready VMs
|
||||
|
||||
The tests can run in an existing environment in lab or cloud. LISA calls the platform as `ready`.
|
||||
|
||||
1. Get IP address which can be accessed by LISA.
|
||||
|
||||
2. Prepare the SSH public/private key pair like steps in Azure.
|
||||
|
||||
3. Run LISA in existing VM.
|
||||
|
||||
Fill in related arguments and run. Note, since the existing VM doesn't support serial log, so smoke test cannot run in ready environment.
|
||||
|
||||
```bash
|
||||
lisa -r ./microsoft/runbook/ready.yml -v public_address:<public address> -v "user_name:<user name>" -v "admin_private_key_file:<private key file>"
|
||||
```
|
||||
|
||||
The existing VMs can be multiple to support networking testing, and it also support authenticate by password. Learn more from [runbook reference](runbook.md).
|
||||
|
||||
## Run Microsoft tests
|
||||
|
||||
LISA is an end-to-end solution to validate Linux in Azure, Hyper-V or other Microsoft virtualization platform. In quick start, it runs tier 0 Microsoft test cases in Azure. It tests an image can boot up and reboot without kernel panic. Refer to [test priority and tier](#test-priority-and-tier) know more about how test tiers defined.
|
||||
|
||||
If you want to validate more scenarios, run with `<tier id>` from 1 to 4. To run tier 1 to 4, it needs legacy LISAv2 tests. The LISA must be run in a Windows computer, and prepare a secret file. Learn more from [how to run legacy test cases](run_legacy.md).
|
||||
|
||||
```bash
|
||||
lisa -r ./microsoft/runbook/azure.yml -v subscription_id:<subscription id> -v "admin_private_key_file:<private key file>" -v tier:<tier id>
|
||||
```
|
||||
|
||||
Learn more from [command-line arguments](command_line.md) and the [runbook reference](runbook.md).
|
||||
|
||||
## Test priority and tier
|
||||
|
||||
## Run legacy LISAv2 test cases
|
||||
|
||||
Learn more from [how to run legacy LISAv2 tests](run_legacy.md).
|
||||
|
||||
## FAQ and Troubleshooting
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
# How to run Legacy LISAv2 tests
|
||||
|
||||
[LISAv2](https://github.com/microsoft/lisa) brings incredible value to Linux quality on Microsoft virtualization platforms. To keep and increase the value of LISA, we decide to have more innovation and continue invest on LISA. So we start the current version of LISA.
|
||||
|
||||
During the transition time, we keep to validate Linux distributions. We cannot and don't want to stop to wait the exciting current LISA. The two versions will be co-existing for a while, under current LISA experience.
|
||||
|
||||
With this document, you will know how to run LISAv2 tests for validation in the current LISA.
|
||||
|
||||
The current LISA can clone any repo of LISAv2 to run, and parse LISAv2 log to generate test results with new format.
|
||||
|
||||
## Preparation
|
||||
|
||||
LISAv2 should run in the latest Windows 10 client 64 bits, or Windows Server 2019 editions.
|
||||
|
||||
Follow [LISAv2 document](https://github.com/microsoft/lisa/blob/master/README.md) to understand prerequisites, and prepare secret files.
|
||||
|
||||
Note, you don't need to run git clone, the current LISA will clone LISAv2 when running.
|
||||
|
||||
## Limitations
|
||||
|
||||
1. Test in parallel of LISAv2 doesn't support with the current LISA together. The current LISA will implement test matrix to replace current test in parallel in LISAv2. But there is no plan to compatible with LISAv2.
|
||||
2. The LISAv2 results has low chance to be missed. There is rare race condition on accessing conflict of LISAv2 test log. If it happens on key logs which relates to test result parsing, it may cause the status of results are not shown correctly.
|
||||
|
||||
## road map
|
||||
|
||||
We're migrating LISAv2 test cases to current LISA by test case priority. We will keep t0 to tx runbooks update to date. When test cases migrated, they will be included in current LISA, and remove from LISAv2. It will be transparent to you.
|
||||
|
|
@ -0,0 +1,115 @@
|
|||
# Runbook Reference
|
||||
|
||||
- [How-to](#how-to)
|
||||
- [Configure Azure deployment](#configure-azure-deployment)
|
||||
- [Select and set test cases](#select-and-set-test-cases)
|
||||
- [Use variable and secrets](#use-variable-and-secrets)
|
||||
- [Use partial runbook](#use-partial-runbook)
|
||||
- [Use extensions](#use-extensions)
|
||||
- [Reference](#reference)
|
||||
- [name](#name)
|
||||
- [test_project](#test_project)
|
||||
- [test_pass](#test_pass)
|
||||
- [tags](#tags)
|
||||
- [parent](#parent)
|
||||
- [path](#path)
|
||||
- [strategy](#strategy)
|
||||
- [extension](#extension)
|
||||
- [variable](#variable)
|
||||
- [is_secret](#is_secret)
|
||||
- [file](#file)
|
||||
- [name](#name-1)
|
||||
- [value](#value)
|
||||
- [artifact](#artifact)
|
||||
- [name](#name-2)
|
||||
- [locations](#locations)
|
||||
- [type](#type)
|
||||
- [path](#path-1)
|
||||
- [notifier](#notifier)
|
||||
- [console](#console)
|
||||
- [html](#html)
|
||||
- [environment](#environment)
|
||||
- [name](#name-3)
|
||||
- [topology](#topology)
|
||||
- [nodes](#nodes)
|
||||
- [nodes_requirement](#nodes_requirement)
|
||||
- [type](#type-1)
|
||||
- [platform](#platform)
|
||||
- [testcase](#testcase)
|
||||
|
||||
## How-to
|
||||
|
||||
### Configure Azure deployment
|
||||
|
||||
### Select and set test cases
|
||||
|
||||
### Use variable and secrets
|
||||
|
||||
### Use partial runbook
|
||||
|
||||
|
||||
### Use extensions
|
||||
|
||||
## Reference
|
||||
|
||||
### name
|
||||
|
||||
### test_project
|
||||
|
||||
### test_pass
|
||||
|
||||
### tags
|
||||
|
||||
### parent
|
||||
|
||||
#### path
|
||||
|
||||
#### strategy
|
||||
|
||||
Not implemented
|
||||
|
||||
### extension
|
||||
|
||||
### variable
|
||||
|
||||
#### is_secret
|
||||
|
||||
#### file
|
||||
|
||||
#### name
|
||||
|
||||
### value
|
||||
|
||||
### artifact
|
||||
|
||||
#### name
|
||||
|
||||
#### locations
|
||||
|
||||
##### type
|
||||
|
||||
##### path
|
||||
|
||||
|
||||
### notifier
|
||||
|
||||
#### console
|
||||
|
||||
#### html
|
||||
|
||||
|
||||
### environment
|
||||
|
||||
#### name
|
||||
|
||||
#### topology
|
||||
|
||||
#### nodes
|
||||
|
||||
#### nodes_requirement
|
||||
|
||||
##### type
|
||||
|
||||
### platform
|
||||
|
||||
### testcase
|
||||
|
|
@ -0,0 +1,63 @@
|
|||
# How to write test cases
|
||||
|
||||
- [Troubleshooting excellence](#troubleshooting-excellence)
|
||||
- [Document excellence](#document-excellence)
|
||||
- [Test suite and test case](#test-suite-and-test-case)
|
||||
- [Metadata](#metadata)
|
||||
- [Test suite metadata](#test-suite-metadata)
|
||||
- [Test case metadata](#test-case-metadata)
|
||||
- [Requirement](#requirement)
|
||||
- [Use tool](#use-tool)
|
||||
- [Use Assertions](#use-assertions)
|
||||
- [Use feature](#use-feature)
|
||||
- [Debug test case](#debug-test-case)
|
||||
|
||||
|
||||
## Troubleshooting excellence
|
||||
|
||||
Tuning when writing
|
||||
|
||||
One line summary
|
||||
|
||||
Test case log
|
||||
|
||||
Other log files
|
||||
|
||||
Repro
|
||||
|
||||
## Document excellence
|
||||
|
||||
Metadata
|
||||
|
||||
Code comments
|
||||
|
||||
Git comments
|
||||
|
||||
Logging
|
||||
|
||||
Error message
|
||||
|
||||
Meaningful Assertion
|
||||
|
||||
Write code like test logic instead of test action
|
||||
|
||||
|
||||
## Test suite and test case
|
||||
|
||||
## Metadata
|
||||
|
||||
### Test suite metadata
|
||||
|
||||
### Test case metadata
|
||||
|
||||
## Requirement
|
||||
|
||||
## Use tool
|
||||
|
||||
## Use Assertions
|
||||
|
||||
## Use feature
|
||||
|
||||
## Debug test case
|
||||
|
||||
in ready environment
|
||||
Загрузка…
Ссылка в новой задаче