зеркало из https://github.com/Azure/aztk.git
Fix: release v0.8.0 (#600)
* update changelog and version * update changelog * add deprecation version and tests * fix tests * update changelog with deprecations * update changelog
This commit is contained in:
Родитель
4b2acc8491
Коммит
34b25855d5
31
CHANGELOG.md
31
CHANGELOG.md
|
@ -2,18 +2,11 @@
|
||||||
|
|
||||||
## 0.8.0 (2018-06-07)
|
## 0.8.0 (2018-06-07)
|
||||||
|
|
||||||
**Bug Fixes**
|
**Deprecated Features**
|
||||||
* --size-low-pri being ignored (#593) ([fa3ac0e](https://github.com/jafreck/aztk/commit/fa3ac0e)), closes [#593](https://github.com/jafreck/aztk/issues/593)
|
* ClusterConfiguration fields vm_count and vm_count_low_pri have been renamed to size and size_low_priority
|
||||||
* fix typos (#595) ([7d7a814](https://github.com/jafreck/aztk/commit/7d7a814)), closes [#595](https://github.com/jafreck/aztk/issues/595)
|
* command line flag `--size-low-pri` for `aztk spark cluster create` has been replaced with `--size-low-priority`
|
||||||
* getting started script reuse aad application (#569) ([3d16cf3](https://github.com/jafreck/aztk/commit/3d16cf3)), closes [#569](https://github.com/jafreck/aztk/issues/569)
|
* `default` secrets.yaml block has been deprecated, place all child parameters directly at the root
|
||||||
* models v2 deserialization (#584) ([1eeff23](https://github.com/jafreck/aztk/commit/1eeff23)), closes [#584](https://github.com/jafreck/aztk/issues/584)
|
* Spark version 1.6 has been deprecated
|
||||||
* optimize start task (#582) ([e5e529a](https://github.com/jafreck/aztk/commit/e5e529a)), closes [#582](https://github.com/jafreck/aztk/issues/582)
|
|
||||||
* remove deprecated vm_count call (#586) ([dbde8bc](https://github.com/jafreck/aztk/commit/dbde8bc)), closes [#586](https://github.com/jafreck/aztk/issues/586)
|
|
||||||
* Remove old spark-defaults.conf jars (#567) ([8b8cd62](https://github.com/jafreck/aztk/commit/8b8cd62)), closes [#567](https://github.com/jafreck/aztk/issues/567)
|
|
||||||
* set logger to stdout (#588) ([3f0c8f9](https://github.com/jafreck/aztk/commit/3f0c8f9)), closes [#588](https://github.com/jafreck/aztk/issues/588)
|
|
||||||
* switch create user to pool wide (#574) ([49a890a](https://github.com/jafreck/aztk/commit/49a890a)), closes [#574](https://github.com/jafreck/aztk/issues/574)
|
|
||||||
* switch from pycryptodome to pycryptodomex (#564) ([19dde42](https://github.com/jafreck/aztk/commit/19dde42)), closes [#564](https://github.com/jafreck/aztk/issues/564)
|
|
||||||
* allow cluster config to be printed when no username has been set (#597) ([1cc71c7](https://github.com/jafreck/aztk/commit/1cc71c7)), closes [#597](https://github.com/jafreck/aztk/issues/597)
|
|
||||||
|
|
||||||
**Added Features**
|
**Added Features**
|
||||||
* add cluster list quiet flag, ability to compose with delete (#581) ([88d0419](https://github.com/jafreck/aztk/commit/88d0419)), closes [#581](https://github.com/jafreck/aztk/issues/581)
|
* add cluster list quiet flag, ability to compose with delete (#581) ([88d0419](https://github.com/jafreck/aztk/commit/88d0419)), closes [#581](https://github.com/jafreck/aztk/issues/581)
|
||||||
|
@ -28,6 +21,20 @@
|
||||||
* Conda, Apt-Get and Pip Install Plugins (#594) ([fbf1bab](https://github.com/jafreck/aztk/commit/fbf1bab)), closes [#594](https://github.com/jafreck/aztk/issues/594)
|
* Conda, Apt-Get and Pip Install Plugins (#594) ([fbf1bab](https://github.com/jafreck/aztk/commit/fbf1bab)), closes [#594](https://github.com/jafreck/aztk/issues/594)
|
||||||
* Warnings show stacktrace on verbose (#587) ([b9a863b](https://github.com/jafreck/aztk/commit/b9a863b)), closes [#587](https://github.com/jafreck/aztk/issues/587)
|
* Warnings show stacktrace on verbose (#587) ([b9a863b](https://github.com/jafreck/aztk/commit/b9a863b)), closes [#587](https://github.com/jafreck/aztk/issues/587)
|
||||||
|
|
||||||
|
**Bug Fixes**
|
||||||
|
* add toolkit to sdk docs and example ([d688c9c](https://github.com/jafreck/aztk/commit/d688c9c))
|
||||||
|
* --size-low-pri being ignored (#593) ([fa3ac0e](https://github.com/jafreck/aztk/commit/fa3ac0e)), closes [#593](https://github.com/jafreck/aztk/issues/593)
|
||||||
|
* fix typos (#595) ([7d7a814](https://github.com/jafreck/aztk/commit/7d7a814)), closes [#595](https://github.com/jafreck/aztk/issues/595)
|
||||||
|
* getting started script reuse aad application (#569) ([3d16cf3](https://github.com/jafreck/aztk/commit/3d16cf3)), closes [#569](https://github.com/jafreck/aztk/issues/569)
|
||||||
|
* models v2 deserialization (#584) ([1eeff23](https://github.com/jafreck/aztk/commit/1eeff23)), closes [#584](https://github.com/jafreck/aztk/issues/584)
|
||||||
|
* optimize start task (#582) ([e5e529a](https://github.com/jafreck/aztk/commit/e5e529a)), closes [#582](https://github.com/jafreck/aztk/issues/582)
|
||||||
|
* remove deprecated vm_count call (#586) ([dbde8bc](https://github.com/jafreck/aztk/commit/dbde8bc)), closes [#586](https://github.com/jafreck/aztk/issues/586)
|
||||||
|
* Remove old spark-defaults.conf jars (#567) ([8b8cd62](https://github.com/jafreck/aztk/commit/8b8cd62)), closes [#567](https://github.com/jafreck/aztk/issues/567)
|
||||||
|
* set logger to stdout (#588) ([3f0c8f9](https://github.com/jafreck/aztk/commit/3f0c8f9)), closes [#588](https://github.com/jafreck/aztk/issues/588)
|
||||||
|
* switch create user to pool wide (#574) ([49a890a](https://github.com/jafreck/aztk/commit/49a890a)), closes [#574](https://github.com/jafreck/aztk/issues/574)
|
||||||
|
* switch from pycryptodome to pycryptodomex (#564) ([19dde42](https://github.com/jafreck/aztk/commit/19dde42)), closes [#564](https://github.com/jafreck/aztk/issues/564)
|
||||||
|
* allow cluster config to be printed when no username has been set (#597) ([1cc71c7](https://github.com/jafreck/aztk/commit/1cc71c7)), closes [#597](https://github.com/jafreck/aztk/issues/597)
|
||||||
|
|
||||||
|
|
||||||
## 0.7.1 (2018-05-11)
|
## 0.7.1 (2018-05-11)
|
||||||
|
|
||||||
|
|
|
@ -41,7 +41,7 @@ class ClusterConfiguration(Model):
|
||||||
|
|
||||||
def __init__(self, *args, **kwargs):
|
def __init__(self, *args, **kwargs):
|
||||||
if 'vm_count' in kwargs:
|
if 'vm_count' in kwargs:
|
||||||
deprecate("vm_count is deprecated for ClusterConfiguration.", "Please use size instead.")
|
deprecate("0.9.0", "vm_count is deprecated for ClusterConfiguration.", "Please use size instead.")
|
||||||
kwargs['size'] = kwargs.pop('vm_count')
|
kwargs['size'] = kwargs.pop('vm_count')
|
||||||
|
|
||||||
if 'vm_low_pri_count' in kwargs:
|
if 'vm_low_pri_count' in kwargs:
|
||||||
|
@ -51,22 +51,22 @@ class ClusterConfiguration(Model):
|
||||||
super().__init__(*args, **kwargs)
|
super().__init__(*args, **kwargs)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
@deprecated()
|
@deprecated("0.9.0")
|
||||||
def vm_count(self):
|
def vm_count(self):
|
||||||
return self.size
|
return self.size
|
||||||
|
|
||||||
@vm_count.setter
|
@vm_count.setter
|
||||||
@deprecated()
|
@deprecated("0.9.0")
|
||||||
def vm_count(self, value):
|
def vm_count(self, value):
|
||||||
self.size = value
|
self.size = value
|
||||||
|
|
||||||
@property
|
@property
|
||||||
@deprecated()
|
@deprecated("0.9.0")
|
||||||
def vm_low_pri_count(self):
|
def vm_low_pri_count(self):
|
||||||
return self.size_low_priority
|
return self.size_low_priority
|
||||||
|
|
||||||
@vm_low_pri_count.setter
|
@vm_low_pri_count.setter
|
||||||
@deprecated()
|
@deprecated("0.9.0")
|
||||||
def vm_low_pri_count(self, value):
|
def vm_low_pri_count(self, value):
|
||||||
self.size_low_priority = value
|
self.size_low_priority = value
|
||||||
|
|
||||||
|
@ -101,7 +101,7 @@ class ClusterConfiguration(Model):
|
||||||
)
|
)
|
||||||
|
|
||||||
if self.custom_scripts:
|
if self.custom_scripts:
|
||||||
deprecate("Custom scripts are DEPRECATED.", "Use plugins instead. See https://aztk.readthedocs.io/en/v0.7.0/15-plugins.html.")
|
deprecate("0.9.0", "Custom scripts are DEPRECATED.", "Use plugins instead. See https://aztk.readthedocs.io/en/v0.7.0/15-plugins.html.")
|
||||||
|
|
||||||
if self.scheduling_target == SchedulingTarget.Dedicated and self.size == 0:
|
if self.scheduling_target == SchedulingTarget.Dedicated and self.size == 0:
|
||||||
raise error.InvalidModelError("Scheduling target cannot be Dedicated if dedicated vm size is 0")
|
raise error.InvalidModelError("Scheduling target cannot be Dedicated if dedicated vm size is 0")
|
||||||
|
|
|
@ -55,7 +55,7 @@ class Toolkit(Model):
|
||||||
raise InvalidModelError("Toolkit '{0}' with version '{1}' is not available. Use one of: {2}".format(
|
raise InvalidModelError("Toolkit '{0}' with version '{1}' is not available. Use one of: {2}".format(
|
||||||
self.software, self.version, toolkit_def.versions))
|
self.software, self.version, toolkit_def.versions))
|
||||||
if self.version == "1.6":
|
if self.version == "1.6":
|
||||||
deprecate("Spark version 1.6 is being deprecated for Aztk.", "Please use 2.1 and above.")
|
deprecate("0.9.0", "Spark version 1.6 is being deprecated for Aztk.", "Please use 2.1 and above.")
|
||||||
|
|
||||||
if self.environment:
|
if self.environment:
|
||||||
if self.environment not in toolkit_def.environments:
|
if self.environment not in toolkit_def.environments:
|
||||||
|
|
|
@ -1,34 +1,34 @@
|
||||||
import warnings
|
import warnings
|
||||||
import functools
|
import functools
|
||||||
import inspect
|
import inspect
|
||||||
import aztk.version as version
|
|
||||||
|
|
||||||
def deprecated(reason: str = None):
|
def deprecated(version: str, advice: str = None):
|
||||||
"""
|
"""
|
||||||
This is a decorator which can be used to mark functions
|
This is a decorator which can be used to mark functions
|
||||||
as deprecated. It will result in a warning being emitted
|
as deprecated. It will result in a warning being emitted
|
||||||
when the function is used.
|
when the function is used.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
reason (str): Reason to why this class or function is being deprecated
|
version (str): The version in which the deprecated functionality will be removed
|
||||||
|
advice (str): Sentence explaining alternatives to the deprecated functionality.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def decorator(func):
|
def decorator(func):
|
||||||
if inspect.isclass(func):
|
if inspect.isclass(func):
|
||||||
msg = "Call to deprecated class {name} ({reason})."
|
msg = "Call to deprecated class {name}."
|
||||||
else:
|
else:
|
||||||
msg = "Call to deprecated function {name} ({reason})."
|
msg = "Call to deprecated function {name}."
|
||||||
|
|
||||||
@functools.wraps(func)
|
@functools.wraps(func)
|
||||||
def new_func(*args, **kwargs):
|
def new_func(*args, **kwargs):
|
||||||
deprecate(msg.format(name=func.__name__, reason=reason))
|
deprecate(version=version, message=msg.format(name=func.__name__, advice=advice), advice=advice)
|
||||||
return func(*args, **kwargs)
|
return func(*args, **kwargs)
|
||||||
return new_func
|
return new_func
|
||||||
|
|
||||||
return decorator
|
return decorator
|
||||||
|
|
||||||
|
|
||||||
def deprecate(message: str, advice: str = ""):
|
def deprecate(version: str, message: str, advice: str = ""):
|
||||||
"""
|
"""
|
||||||
Print a deprecation warning.
|
Print a deprecation warning.
|
||||||
|
|
||||||
|
@ -37,18 +37,8 @@ def deprecate(message: str, advice: str = ""):
|
||||||
advice (str): Sentence explaining alternatives to the deprecated functionality.
|
advice (str): Sentence explaining alternatives to the deprecated functionality.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
deprecated_version = _get_deprecated_version()
|
|
||||||
warnings.simplefilter('always', DeprecationWarning) # turn off filter
|
warnings.simplefilter('always', DeprecationWarning) # turn off filter
|
||||||
warnings.warn("{0} It will be removed in Aztk version {1}. {2}".format(message, deprecated_version, advice),
|
warnings.warn("{0} It will be removed in Aztk version {1}. {2}".format(message, version, advice),
|
||||||
category=DeprecationWarning,
|
category=DeprecationWarning,
|
||||||
stacklevel=2)
|
stacklevel=2)
|
||||||
warnings.simplefilter('default', DeprecationWarning) # reset filter
|
warnings.simplefilter('default', DeprecationWarning) # reset filter
|
||||||
|
|
||||||
|
|
||||||
def _get_deprecated_version():
|
|
||||||
"""
|
|
||||||
Returns the next version where the deprecated functionality will be removed
|
|
||||||
"""
|
|
||||||
if version.major == 0:
|
|
||||||
return "0.{minor}.0".format(minor=version.minor + 1)
|
|
||||||
return "{major}.0.0".format(major=version.major + 1)
|
|
||||||
|
|
|
@ -46,7 +46,7 @@ def _load_config_file(path: str):
|
||||||
|
|
||||||
def _merge_secrets_dict(secrets: SecretsConfiguration, secrets_config):
|
def _merge_secrets_dict(secrets: SecretsConfiguration, secrets_config):
|
||||||
if 'default' in secrets_config:
|
if 'default' in secrets_config:
|
||||||
deprecate("default key in secrets.yaml is deprecated.", "Place all child parameters directly at the root")
|
deprecate("0.9.0", "default key in secrets.yaml is deprecated.", "Place all child parameters directly at the root")
|
||||||
secrets_config = dict(**secrets_config, **secrets_config.pop('default'))
|
secrets_config = dict(**secrets_config, **secrets_config.pop('default'))
|
||||||
|
|
||||||
other = SecretsConfiguration.from_dict(secrets_config)
|
other = SecretsConfiguration.from_dict(secrets_config)
|
||||||
|
|
|
@ -44,7 +44,7 @@ def execute(args: typing.NamedTuple):
|
||||||
file_config, wait = config.read_cluster_config()
|
file_config, wait = config.read_cluster_config()
|
||||||
cluster_conf.merge(file_config)
|
cluster_conf.merge(file_config)
|
||||||
if args.size_low_pri is not None:
|
if args.size_low_pri is not None:
|
||||||
deprecate("--size-low-pri has been deprecated.", "Please use --size-low-priority.")
|
deprecate("0.9.0", "--size-low-pri has been deprecated.", "Please use --size-low-priority.")
|
||||||
args.size_low_priority = args.size_low_pri
|
args.size_low_priority = args.size_low_pri
|
||||||
|
|
||||||
cluster_conf.merge(ClusterConfiguration(
|
cluster_conf.merge(ClusterConfiguration(
|
||||||
|
|
|
@ -85,8 +85,8 @@ def test_scheduling_target_dedicated_with_no_dedicated_nodes_raise_error():
|
||||||
cluster_id="abc",
|
cluster_id="abc",
|
||||||
scheduling_target=SchedulingTarget.Dedicated,
|
scheduling_target=SchedulingTarget.Dedicated,
|
||||||
vm_size="standard_a2",
|
vm_size="standard_a2",
|
||||||
vm_count=0,
|
size=0,
|
||||||
vm_low_pri_count=2,
|
size_low_priority=2,
|
||||||
toolkit=Toolkit(software="spark", version="1.6.3"),
|
toolkit=Toolkit(software="spark", version="1.6.3"),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
|
@ -0,0 +1,80 @@
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
import aztk.utils.deprecation
|
||||||
|
from aztk.models import Model, fields
|
||||||
|
from aztk.utils.deprecation import deprecate, deprecated
|
||||||
|
|
||||||
|
|
||||||
|
def test_deprecated_function():
|
||||||
|
@deprecated(version="0.0.0")
|
||||||
|
def deprecated_function():
|
||||||
|
pass
|
||||||
|
|
||||||
|
with pytest.warns(DeprecationWarning):
|
||||||
|
deprecated_function()
|
||||||
|
|
||||||
|
|
||||||
|
def test_deprecated_class():
|
||||||
|
@deprecated(version="0.0.0")
|
||||||
|
class DeprecatedClass:
|
||||||
|
pass
|
||||||
|
|
||||||
|
with pytest.warns(DeprecationWarning):
|
||||||
|
DeprecatedClass()
|
||||||
|
|
||||||
|
|
||||||
|
def test_deprecated_field():
|
||||||
|
class DummyClass(Model):
|
||||||
|
non_deprecated_field = fields.Boolean()
|
||||||
|
|
||||||
|
def __init__(self, *args, **kwargs):
|
||||||
|
if 'deprecated_field' in kwargs:
|
||||||
|
deprecate("0.9.0", "deprecated_field is deprecated for DummyClass.", "Please use non_deprecated_field instead.")
|
||||||
|
kwargs['non_deprecated_field'] = kwargs.pop('deprecated_field')
|
||||||
|
|
||||||
|
super().__init__(*args, **kwargs)
|
||||||
|
|
||||||
|
@property
|
||||||
|
@deprecated("0.0.0")
|
||||||
|
def deprecated_field(self):
|
||||||
|
return self.non_deprecated_field
|
||||||
|
|
||||||
|
@deprecated_field.setter
|
||||||
|
@deprecated("0.0.0")
|
||||||
|
def deprecated_field(self, value):
|
||||||
|
self.non_deprecated_field = value
|
||||||
|
|
||||||
|
with pytest.warns(DeprecationWarning) as deprecation_warning:
|
||||||
|
dummy_class = DummyClass(deprecated_field=True)
|
||||||
|
assert dummy_class.deprecated_field is True
|
||||||
|
|
||||||
|
assert dummy_class.non_deprecated_field is True
|
||||||
|
assert "deprecated_field is deprecated for DummyClass." in str(deprecation_warning[0].message)
|
||||||
|
assert "Please use non_deprecated_field instead." in str(deprecation_warning[0].message)
|
||||||
|
|
||||||
|
|
||||||
|
def test_deprecate_version():
|
||||||
|
with pytest.warns(DeprecationWarning) as deprecation_warning:
|
||||||
|
deprecate("0.0.0", "message")
|
||||||
|
|
||||||
|
assert "0.0.0" in str(deprecation_warning[0].message)
|
||||||
|
|
||||||
|
with pytest.warns(DeprecationWarning) as deprecation_warning:
|
||||||
|
deprecate("0.1.0", "message")
|
||||||
|
|
||||||
|
assert "0.1.0" in str(deprecation_warning[0].message)
|
||||||
|
|
||||||
|
|
||||||
|
def test_deprecate_message():
|
||||||
|
with pytest.warns(DeprecationWarning) as deprecation_warning:
|
||||||
|
deprecate("0.0.0", message="test message")
|
||||||
|
|
||||||
|
assert "0.0.0" in str(deprecation_warning[0].message)
|
||||||
|
assert "test message" in str(deprecation_warning[0].message)
|
||||||
|
|
||||||
|
|
||||||
|
def test_deprecate_advice():
|
||||||
|
with pytest.warns(DeprecationWarning) as deprecation_warning:
|
||||||
|
deprecate("0.0.0", message="test message", advice="use this instead")
|
||||||
|
|
||||||
|
assert "use this instead" in str(deprecation_warning[0].message)
|
Загрузка…
Ссылка в новой задаче