if there is predefined environment in runbook, it doesn't allow to
create environment automatically. For the usage scenarios, if there is
predefined environment, it's confusing to have generated environments.
This change force to use predefined environment for all test cases.
Improved UTs, add checks on case names
With this change, the code structure is more clear. The node is extendable
now, but it's not recommended to extend node with new schema. Because
the node is the isolate components, it should be the abstract SUT, so
that LISA features can run well with it. If the node schema is extended,
it means a node must be defined in runbook, not be managed by platform.
In this situation, the resource of node is fixed, not be able to reused
in other environments dynamically.
Co-authored-by: Gustavo Lima Chaves <gustavo.chaves@microsoft.com>
Many commands get same result when it reruns. So enable cached result by
default to save run time. Update withscript to force_rerun to demo for
how to use it too.
This version has more power on type assertions, and we can get rid of
a bunch of "# type: ignore" entries. My theory is that they point a
newer release of Typeshed (https://github.com/python/typeshed), so
that we have more type annotations available.
Real mistakes have been caught with this update, BTW.
Add environment, node and other args to before/after_suite/case. To
prevent error on inconsistent method signature, it has to keep kwargs
style. Another way is to expand every argument, it's more ugly than
current.
1. pass environment and node as test case methods. It's more like
functional programming and more friendly.
2. add Node and Environment to package level types, since it's used by
test code writer frequently.
3. update test cases to use new pattern.
The camelCase is recommended in yaml, but it's different with Python
style. The trouble is when searching in yaml, it's hard to find Python
schema, and versus. So change to use snake_case, for easier searching.
Since arm doesn't support nested loop, so use nested template to
workaround. Also add field and logic to support it.
other changes,
1. adjust priority of demo test cases, so easy to select by priority.
2. Remove AzureArmParameterNode, reuse AzureNodeSchema as their schema
is almost the same.
3. some performance tuning like adjust loop priority, reuse caches.
4. remove some unused code.
5. improve log messages.
6. other minor improvements.
1. update OS to support Ubuntu and CentOS
2. add logic to detect OS.
3. support package installation
4. add gcc, make tools
5. support git, ntttcp install
6. other minor improvements
1. add work flow to create env from cases.
2. add capability on environment and node.
3. add EnvironmentSpace for requirement and capability.
4. add prepare_environment method to match capability and requirements
5. use delay node schema nodes replace ugly package instrument.
6. move some test only code to test file.
7. add create factory to support windows tools.
8. support test case settings like retry, times, and so on.
9. other minor improvements.
1. add environments, nodes, platforms classes like a collection.
2. add tools to use a tool like [ToolType], and move related code to
executable.
3. other minor improvements.
Python file is a module, which is not used to put only one thing.
All related things should be put in same module.
platform.py causes error in uuid.py, so change name to platform_.py
1. add statistics and test result for test case progress
2. inherit from unittest to get all assert methods
3. add perf_timer for profiling, and improve current profiling
4. add document for tool.py
5. add init.py for mypy checking
6. other minor improvements
* Implement custom script
1. Implement custom script. As it's same type with different data, so it's different with tools. CustomScriptBuilder to create an instance of script on Node.
2. rename method and variables to follow pep8 guidaline.
3. rename [Suite/Case]Metadata to [TestSuite/TestCase]Metadata
4. create LisaException to all internal exceptions
5. other minor improvements
Chi Song
Divyansh
Divyansh
Chi Song
Chi Song
Gustavo Lima Chaves
LiliDeng
Chi Song