Azure
/
azure-iot-sdk-python
зеркало из
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

[V3] Initial Client APIs (factory methods + shutdown) (#1106) 

* Implemented all factory methods for initial Client API layer
* Implemented `.shutdown()` for Client API layer
* Ported `connection_string.py` for V3
* All references to "methods" now refer to "direct methods"
* Added a custom type hint for FunctionOrCoroutine
* Removed `sastoken_ttl`, `connection_retry_interval`, and `gateway_hostname` kwargs from factory methods
* Renamed `connection_retry` factory method kwarg to `auto_reconnect` for clarity
* Added formal support for X509 connection strings
* Updated migration guide with details regarding these changes
* Split the migration guide to differentiate between IoTHub clients and Provisioning client
This commit is contained in:
Carter Tinney 2023-03-05 01:24:15 -08:00 коммит произвёл GitHub
Родитель d7384ff8b8
Коммит 68966a6257
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
15 изменённых файлов: 4144 добавлений и 349 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

198
migration_guide.md
Просмотреть файл

@ -1,198 +0,0 @@
# IoTHub Python SDK Migration Guide
This guide details how to update existing code that uses an `azure-iot-device` V2 release to use a V3 release instead. While the APIs remain mostly the same, there are a few differences you may need to account for in your application, as we have removed some of the implicit behaviors present in V2 in order to provide a more reliable and consistent user experience.
## Connecting to IoTHub
One of the primary changes in V3 is the removal of automatic connections when invoking other APIs on the `IoTHubDeviceClient` and `IoTHubModuleClient`. You must now make an explicit manual connection before sending or receiving any data.
### V2
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string("<Your Connection String>")
client.send_message("some message")
```
### V3
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string("<Your Connection String>")
client.connect()
client.send_message("some message")
```
Note that many people using V2 may already have been doing manual connects, as for some time, this has been our recommended practice.
Note also that this change does *not* affect automatic reconnection attempts in the case of network failure. Once the manual connect has been successful, the client will (under default settings) still attempt to retain that connected state as it did in V2.
## Receiving data from IoTHub
Similarly to the above, there is an additional explicit step you must now make when trying to receive data. In addition to setting your handler, you must explicitly start/stop receiving. Note also that the above step of manually connecting must also be done before starting to receive data.
Furthermore, note that the content of the message is now referred to by the 'payload' attribute on the message, rather than the 'data' attribute (see "Message" section below)
### V2
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string("<Your Connection String>")
# define behavior for receiving a message
def message_handler(message):
print("the data in the message received was ")
print(message.data)
print("custom properties are")
print(message.custom_properties)
# set the message handler on the client
client.on_message_received = message_handler
```
### V3
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string("<Your Connection String>")
# define behavior for receiving a message
def message_handler(message):
print("the payload of the message received was ")
print(message.payload)
print("custom properties are")
print(message.custom_properties)
# set the message handler on the client
client.on_message_received = message_handler
# connect and start receiving messages
client.connect()
client.start_message_receive()
```
Note that this must be done not just for receiving messages, but receiving any data. Consult the chart below to see which APIs you will need for the type of data you are receiving.
| Data Type | Handler name | Start Receive API | Stop Receive API |
|---------------------------------|----------------------------------------------|--------------------------------------------------|-------------------------------------------------|
| Messages | `.on_message_received` | `.start_message_receive()` | `.stop_message_receive()` |
| Method Requests | `.on_method_request_received` | `.start_method_request_receive()` | `.stop_method_request_receive()` |
| Twin Desired Properties Patches | `.on_twin_desired_properties_patch_received` | `.start_twin_desired_properties_patch_receive()` | `.stop_twin_desired_properties_patch_receive()` |
Finally, it should be clarified that the following receive APIs that were deprecated in V2 have been fully removed in V3:
* `.receive_message()`
* `.receive_message_on_input()`
* `.receive_method_request()`
* `.receive_twin_desired_properties_patch()`
All receives should now be done using the handlers in the table above.
## Message object - IoTHubDeviceClient/IoTHubModuleClient
Some changes have been made to the `Message` object used for sending and receiving data.
* The `.data` attribute is now called `.payload` for consistency with other objects in the API
* The `message_id` parameter is no longer part of the constructor arguments. It should be manually added as an attribute, just like all other attributes
* The payload of a received Message is now a unicode string value instead of a bytestring value.
It will be decoded according to the content encoding property sent along with the message.
### V2
```python
from azure.iot.device import Message
payload = "this is a payload"
message_id = "1234"
m = Message(data=payload, message_id=message_id)
assert m.data == payload
assert m.message_id = message_id
```
### V3
```python
from azure.iot.device import Message
payload = "this is a payload"
message_id = "1234"
m = Message(payload=payload)
m.message_id = message_id
assert m.payload == payload
```
## Modified Client Options - IoTHubDeviceClient/IoTHubModuleClient
Some keyword arguments provided at client creation have changed or been removed
| V2 | V3 | Explanation |
|-----------------------------|-------------|----------------------------------------|
| `auto_connect` | **REMOVED** | Initial manual connection now required |
| `ensure_desired_properties` | **REMOVED** | No more implicit twin updates |
## Shutting down - IoTHubDeviceClient/IoTHubModuleClient
While using the `.shutdown()` method when you are completely finished with an instance of the client has been a highly recommended practice for some time, some early versions of V2 did not require it. As of V3, in order to ensure a graceful exit, you must make an explicit shutdown.
### V2
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string("<Your Connection String>")
# ...
#<do things>
# ...
```
### V3
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string("<Your Connection String>")
# ...
#<do things>
# ...
client.shutdown()
```
## Shutting down - ProvisioningDeviceClient
As with the IoTHub clients mentioned above, the Provisioning clients now also require shutdown. This was implicit in V2, but now it must be explicit and manual to ensure graceful exit.
### V2
```python
from azure.iot.device import ProvisioningDeviceClient
client = ProvisioningDeviceClient.create_from_symmetric_key(
provisioning_host="<Your provisioning host>",
registration_id="<Your registration id>",
id_scope="<Your id scope>",
symmetric_key="<Your symmetric key">,
)
registration_result = client.register()
# Shutdown is implicit upon successful registration
```
### V3
```python
from azure.iot.device import ProvisioningDeviceClient
client = ProvisioningDeviceClient.create_from_symmetric_key(
provisioning_host="<Your provisioning host>",
registration_id="<Your registration id>",
id_scope="<Your id scope>",
symmetric_key="<Your symmetric key">,
)
registration_result = client.register()
# Manual shutdown for graceful exit
client.shutdown()
```

359
migration_guide_iothub.md Normal file
Просмотреть файл

@ -0,0 +1,359 @@
# Azure IoT Device SDK for Python Migration Guide - IoTHubDeviceClient and IoTHubModuleClient
This guide details how to update existing code that uses an `azure-iot-device` V2 release to use a V3 release instead. While the APIs remain mostly the same, there are several differences you will need to account for in your application, as some APIs have changed, and we have removed some of the implicit behaviors present in V2 in order to provide a more reliable and consistent user experience.
Note that this guide mostly refers to the `IoTHubDeviceClient`, although it's contents apply equally to the `IoTHubModuleClient`.
For changes to the `ProvisioningDeviceClient` please refer to `migration_guide_provisioning.md` in this same directory.
## Connecting to IoTHub
One of the primary changes in V3 is the removal of automatic connections when invoking other APIs on the `IoTHubDeviceClient` and `IoTHubModuleClient`. You must now make an explicit manual connection before sending or receiving any data.
### V2
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string("<Your Connection String>")
client.send_message("some message")
```
### V3
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string("<Your Connection String>")
client.connect()
client.send_message("some message")
```
Note that many people using V2 may already have been doing manual connects, as for some time, this has been our recommended practice.
Note also that this change does *not* affect automatic reconnection attempts in the case of network failure. Once the manual connect has been successful, the client will (under default settings) still attempt to retain that connected state as it did in V2.
## Receiving data from IoTHub
Similarly to the above, there is an additional explicit step you must now make when trying to receive data. In addition to setting your handler, you must explicitly start/stop receiving. Note also that the above step of manually connecting must also be done before starting to receive data.
Furthermore, note that the content of the message is now referred to by the 'payload' attribute on the message, rather than the 'data' attribute (see "Message" section below)
### V2
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string("<Your Connection String>")
# define behavior for receiving a message
def message_handler(message):
print("the data in the message received was ")
print(message.data)
print("custom properties are")
print(message.custom_properties)
# set the message handler on the client
client.on_message_received = message_handler
```
### V3
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string("<Your Connection String>")
# define behavior for receiving a message
def message_handler(message):
print("the payload of the message received was ")
print(message.payload)
print("custom properties are")
print(message.custom_properties)
# set the message handler on the client
client.on_message_received = message_handler
# connect and start receiving messages
client.connect()
client.start_message_receive()
```
Note that this must be done not just for receiving messages, but receiving any data. Consult the chart below to see which APIs you will need for the type of data you are receiving.
| Data Type | Handler name | Start Receive API | Stop Receive API |
|---------------------------------|----------------------------------------------|--------------------------------------------------|-------------------------------------------------|
| Messages | `.on_message_received` | `.start_message_receive()` | `.stop_message_receive()` |
| Method Requests | `.on_method_request_received` | `.start_direct_method_request_receive()` | `.stop_direct_method_request_receive()` |
| Twin Desired Properties Patches | `.on_twin_desired_properties_patch_received` | `.start_twin_desired_properties_patch_receive()` | `.stop_twin_desired_properties_patch_receive()` |
Finally, it should be clarified that the following receive APIs that were deprecated in V2 have been fully removed in V3:
* `.receive_message()`
* `.receive_message_on_input()`
* `.receive_method_request()`
* `.receive_twin_desired_properties_patch()`
All receives should now be done using the handlers in the table above.
## Direct Methods
For clarity, all references to direct methods are now explicit about being "direct methods", rather than the more generic (and overloaded) "method". As such, the following methods and objects have all had a name change:
* `.invoke_method()` -> `.invoke_direct_method()`
* `MethodRequest` -> `DirectMethodRequest`
* `MethodResponse` -> `DirectMethodResponse`
## Message object
Some changes have been made to the `Message` object used for sending and receiving data.
* The `.data` attribute is now called `.payload` for consistency with other objects in the API
* The `message_id` parameter is no longer part of the constructor arguments. It should be manually added as an attribute, just like all other attributes
* The payload of a received Message is now a unicode string value instead of a bytestring value.
It will be decoded according to the content encoding property sent along with the message.
### V2
```python
from azure.iot.device import Message
payload = "this is a payload"
message_id = "1234"
m = Message(data=payload, message_id=message_id)
assert m.data == payload
assert m.message_id = message_id
```
### V3
```python
from azure.iot.device import Message
payload = "this is a payload"
message_id = "1234"
m = Message(payload=payload)
m.message_id = message_id
assert m.payload == payload
```
## Shutting down
While using the `.shutdown()` method when you are completely finished with an instance of the client has been a highly recommended practice for some time, some early versions of V2 did not require it. As of V3, in order to ensure a graceful exit, you must make an explicit shutdown.
### V2
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string("<Your Connection String>")
# ...
#<do things>
# ...
```
### V3
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string("<Your Connection String>")
# ...
#<do things>
# ...
client.shutdown()
```
## Symmetric Key Authentication
Creating a client that uses a symmetric key to authenticate is now done via the new `.create()` factory method instead of `.create_from_symmetric_key()`
### V2
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_symmetric_key(
symmetric_key="<Your Symmetric Key>",
hostname="<Your Hostname>",
device_id="<Your Device ID>"
)
```
### V3
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create(
symmetric_key="<Your Symmetric Key>",
hostname="<Your Hostname>",
device_id="<Your Device ID>"
)
```
## Custom SAS Token Authentication
There have been significant changes surrounding this style of authentication - it was rather complex in V2, and we have tried to simplify it for V3. It now also uses the new `.create()` method rather than `.create_from_sastoken()`. With this new style of providing a custom token via callback, you no longer
will have to manually update the SAS token via the `.on_new_sastoken_required` handler, and as such,
the handler no longer exists.
### V2
```python
from azure.iot.device import IoTHubDeviceClient
def get_new_sastoken():
sastoken = # Do something here to create/retrieve a token
return sastoken
sastoken = get_new_sastoken()
client = IoTHubDeviceClient.create_from_sastoken(sastoken)
def sastoken_update_handler():
print("Updating SAS Token...")
sastoken = get_new_sastoken()
client.update_sastoken(sastoken)
print("SAS Token updated")
client.on_new_sastoken_required = sastoken_update_handler
```
### V3
```python
from azure.iot.device import IoTHubDeviceClient
def get_new_sastoken():
sastoken = # Do something here to create/retrieve a token
return sastoken
client = IoTHubDeviceClient.create(
hostname="<Your Hostname>",
device_id="<Your Device ID>",
sastoken_fn=get_new_sastoken,
)
```
## X509 Authentication
Using X509 authentication is now provided via the new `ssl_context` keyword for the `.create()` method, rather than having it's own `.create_from_x509_certificate()` method. This is to allow additional flexibility for customers who wish for more control over their TLS/SSL authorization. See "TLS/SSL customization" below for more information.
### V2
```python
from azure.iot.device import IoTHubDeviceClient, X509
x509 = X509(
cert_file="<Your X509 Cert File Path>",
key_file="<Your X509 Key File>",
pass_phrase="<Your X509 Pass Phrase>",
)
client = IoTHubDeviceClient.create_from_x509_certificate(
hostname="<Your IoTHub Hostname>",
device_id="<Your Device ID>",
x509=x509,
)
```
### V3
```python
from azure.iot.device import IoTHubDeviceClient
import ssl
ssl_context = ssl.SSLContext.create_default_context()
ssl_context.load_cert_chain(
certfile="<Your X509 Cert File Path>",
keyfile="<Your X509 Key File>",
password="<Your X509 Pass Phrase>",
)
client = IoTHubDeviceClient.create(
hostname="<Your IoTHub Hostname>",
device_id="<Your Device ID>",
ssl_context=ssl_context,
)
```
Note that SSLContexts can be used with the `.create_from_connection_string()` factory method as well, so V3 now fully supports X509 connection strings.
### V3
```python
from azure.iot.device import IoTHubDeviceClient
import ssl
ssl_context = ssl.SSLContext.create_default_context()
ssl_context.load_cert_chain(
certfile="<Your X509 Cert File Path>",
keyfile="<Your X509 Key File>",
password="<Your X509 Pass Phrase>",
)
client = IoTHubDeviceClient.create_from_connection_string(
"<Your X509 Connection String>",
ssl_context=ssl_context,
)
```
## TLS/SSL Customization
To allow users more flexibility, we have added the ability to inject an `SSLContext` object into the client via the optional `ssl_context` keyword argument to factory methods in order to customize the TLS/SSL encryption and authentication. As a result, some features previously handled via client APIs are now expected to have been directly set on the injected `SSLContext`.
By moving to a model that allows `SSLContext` injection we not only bring our client in line with standard practices, but we also allow for users to modify any aspect of their `SSLContext`, not just the ones we previously supported via API.
### **Server Verification Certificates (CA certs)**
### V2
```python
from azure.iot.device import IoTHubDeviceClient
certfile = open("<Your CA Certificate File Path>")
root_ca_cert = certfile.read()
client = IoTHubDeviceClient.create_from_connection_string(
"<Your Connection String>",
server_verification_cert=root_ca_cert
)
```
### V3
```python
from azure.iot.device import IoTHubDeviceClient
import ssl
ssl_context = ssl.SSLContext.create_default_context(
cafile="<Your CA Certificate File Path>",
)
client = IoTHubDeviceClient.create_from_connection_string(
"<Your Connection String>",
ssl_context=ssl_context,
)
```
### **Cipher Suites**
### V2
```python
from azure.iot.device import IoTHubDeviceClient
client = IoTHubDeviceClient.create_from_connection_string(
"<Your Connection String>",
cipher="<Your Cipher>"
)
```
### V3
```python
from azure.iot.device import IoTHubDeviceClient
import ssl
ssl_context = ssl.SSLContext.create_default_context()
ssl_context.set_ciphers("<Your Cipher>")
client = IoTHubDeviceClient.create_from_connection_string(
"<Your Connection String>",
ssl_context=ssl_context,
)
```
## Modified Client Options
Some keyword arguments provided at client creation have changed or been removed
| V2 | V3 | Explanation |
|-----------------------------|------------------|----------------------------------------------------------|
| `connection_retry` | `auto_reconnect` | Improved clarity |
| `connection_retry_interval` | **REMOVED** | Automatic reconnect no longer uses a static interval |
| `auto_connect` | **REMOVED** | Initial manual connection now required |
| `ensure_desired_properties` | **REMOVED** | No more implicit twin updates |
| `sastoken_ttl` | **REMOVED** | Unnecessary, but open to re-adding if a use case emerges |
| `gateway_hostname` | **REMOVED** | Supported via `hostname` parameter |
| `server_verification_cert` | **REMOVED** | Supported via SSL injection |
| `cipher` | **REMOVED** | Supported via SSL injection |

42
migration_guide_provisioning.md Normal file
Просмотреть файл

@ -0,0 +1,42 @@
# Azure IoT Device SDK for Python Migration Guide - ProvisioningDeviceClient
This guide details how to update existing code that uses an `azure-iot-device` V2 release to use a V3 release instead. While the APIs remain mostly the same, there are several differences you will need to account for in your application, as changes have been made in order to provide a more reliable and consistent user experience.
Note that this guide is a work in progress.
## Shutting down - ProvisioningDeviceClient
As with the IoTHub clients mentioned above, the Provisioning clients now also require shutdown. This was implicit in V2, but now it must be explicit and manual to ensure graceful exit.
### V2
```python
from azure.iot.device import ProvisioningDeviceClient
client = ProvisioningDeviceClient.create_from_symmetric_key(
provisioning_host="<Your provisioning host>",
registration_id="<Your registration id>",
id_scope="<Your id scope>",
symmetric_key="<Your symmetric key">,
)
registration_result = client.register()
# Shutdown is implicit upon successful registration
```
### V3
```python
from azure.iot.device import ProvisioningDeviceClient
client = ProvisioningDeviceClient.create_from_symmetric_key(
provisioning_host="<Your provisioning host>",
registration_id="<Your registration id>",
id_scope="<Your id scope>",
symmetric_key="<Your symmetric key">,
)
registration_result = client.register()
# Manual shutdown for graceful exit
client.shutdown()
```

154
v3_async_wip/tests/test_connection_string.py Normal file
Просмотреть файл

@ -0,0 +1,154 @@
# -------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
# license information.
# --------------------------------------------------------------------------
import pytest
import logging
from v3_async_wip.connection_string import ConnectionString
logging.basicConfig(level=logging.DEBUG)
# TODO: eliminate refernces to service connection string
@pytest.mark.describe("ConnectionString")
class TestConnectionString(object):
@pytest.mark.it("Instantiates from a given connection string")
@pytest.mark.parametrize(
"input_string",
[
pytest.param(
"HostName=my.host.name;SharedAccessKeyName=mykeyname;SharedAccessKey=Zm9vYmFy",
id="Service connection string",
),
pytest.param(
"HostName=my.host.name;DeviceId=my-device;SharedAccessKey=Zm9vYmFy",
id="Device connection string",
),
pytest.param(
"HostName=my.host.name;DeviceId=my-device;SharedAccessKey=Zm9vYmFy;GatewayHostName=mygateway",
id="Device connection string w/ gatewayhostname",
),
pytest.param(
"HostName=my.host.name;DeviceId=my-device;x509=True",
id="Device connection string w/ X509",
),
pytest.param(
"HostName=my.host.name;DeviceId=my-device;ModuleId=my-module;SharedAccessKey=Zm9vYmFy",
id="Module connection string",
),
pytest.param(
"HostName=my.host.name;DeviceId=my-device;ModuleId=my-module;SharedAccessKey=Zm9vYmFy;GatewayHostName=mygateway",
id="Module connection string w/ gatewayhostname",
),
pytest.param(
"HostName=my.host.name;DeviceId=my-device;ModuleId=my-module;x509=True",
id="Module connection string w/ X509",
),
],
)
def test_instantiates_correctly_from_string(self, input_string):
cs = ConnectionString(input_string)
assert isinstance(cs, ConnectionString)
@pytest.mark.it("Raises ValueError on invalid string input during instantiation")
@pytest.mark.parametrize(
"input_string",
[
pytest.param("", id="Empty string"),
pytest.param("garbage", id="Not a connection string"),
pytest.param("HostName=my.host.name", id="Incomplete connection string"),
pytest.param(
"InvalidKey=my.host.name;SharedAccessKeyName=mykeyname;SharedAccessKey=Zm9vYmFy",
id="Invalid key",
),
pytest.param(
"HostName=my.host.name;HostName=my.host.name;SharedAccessKey=mykeyname;SharedAccessKey=Zm9vYmFy",
id="Duplicate key",
),
pytest.param(
"HostName=my.host.name;DeviceId=my-device;ModuleId=my-module;SharedAccessKey=mykeyname;x509=true",
id="Mixed authentication scheme",
),
],
)
def test_raises_value_error_on_invalid_input(self, input_string):
with pytest.raises(ValueError):
ConnectionString(input_string)
@pytest.mark.it("Raises TypeError on non-string input during instantiation")
@pytest.mark.parametrize(
"input_val",
[
pytest.param(2123, id="Integer"),
pytest.param(23.098, id="Float"),
pytest.param(b"bytes", id="Bytes"),
pytest.param(object(), id="Complex object"),
pytest.param(["a", "b"], id="List"),
pytest.param({"a": "b"}, id="Dictionary"),
],
)
def test_raises_type_error_on_non_string_input(self, input_val):
with pytest.raises(TypeError):
ConnectionString(input_val)
@pytest.mark.it("Uses the input connection string as a string representation")
def test_string_representation_of_object_is_the_input_string(self):
string = "HostName=my.host.name;SharedAccessKeyName=mykeyname;SharedAccessKey=Zm9vYmFy"
cs = ConnectionString(string)
assert str(cs) == string
@pytest.mark.it("Supports indexing syntax to return the stored value for a given key")
def test_indexing_key_returns_corresponding_value(self):
cs = ConnectionString(
"HostName=my.host.name;SharedAccessKeyName=mykeyname;SharedAccessKey=Zm9vYmFy"
)
assert cs["HostName"] == "my.host.name"
assert cs["SharedAccessKeyName"] == "mykeyname"
assert cs["SharedAccessKey"] == "Zm9vYmFy"
@pytest.mark.it("Raises KeyError if indexing on a key not contained in the ConnectionString")
def test_indexing_key_raises_key_error_if_key_not_in_string(self):
with pytest.raises(KeyError):
cs = ConnectionString(
"HostName=my.host.name;SharedAccessKeyName=mykeyname;SharedAccessKey=Zm9vYmFy"
)
cs["SharedAccessSignature"]
@pytest.mark.it(
"Supports the 'in' operator for validating if a key is contained in the ConnectionString"
)
def test_item_in_string(self):
cs = ConnectionString(
"HostName=my.host.name;SharedAccessKeyName=mykeyname;SharedAccessKey=Zm9vYmFy"
)
assert "SharedAccessKey" in cs
assert "SharedAccessKeyName" in cs
assert "HostName" in cs
assert "FakeKeyNotInTheString" not in cs
@pytest.mark.describe("ConnectionString - .get()")
class TestConnectionStringGet(object):
@pytest.mark.it("Returns the stored value for a given key")
def test_calling_get_with_key_returns_corresponding_value(self):
cs = ConnectionString(
"HostName=my.host.name;SharedAccessKeyName=mykeyname;SharedAccessKey=Zm9vYmFy"
)
assert cs.get("HostName") == "my.host.name"
@pytest.mark.it("Returns None if the given key is invalid")
def test_calling_get_with_invalid_key_and_no_default_value_returns_none(self):
cs = ConnectionString(
"HostName=my.host.name;SharedAccessKeyName=mykeyname;SharedAccessKey=Zm9vYmFy"
)
assert cs.get("invalidkey") is None
@pytest.mark.it("Returns an optionally provided default value if the given key is invalid")
def test_calling_get_with_invalid_key_and_a_default_value_returns_default_value(self):
cs = ConnectionString(
"HostName=my.host.name;SharedAccessKeyName=mykeyname;SharedAccessKey=Zm9vYmFy"
)
assert cs.get("invalidkey", "defaultval") == "defaultval"

2779
v3_async_wip/tests/test_iothub_client.py Normal file
Просмотреть файл

Разница между файлами не показана из-за своего большого размера Загрузить разницу

101
v3_async_wip/tests/test_iothub_http_client.py
Просмотреть файл

@ -57,7 +57,7 @@ def mock_sastoken_provider(mocker, sastoken):
return provider return provider
@pytest.fixture @pytest.fixture(autouse=True)
def mock_session(mocker): def mock_session(mocker):
mock_session = mocker.MagicMock(spec=aiohttp.ClientSession) mock_session = mocker.MagicMock(spec=aiohttp.ClientSession)
# Mock out POST and it's response # Mock out POST and it's response
@ -114,9 +114,8 @@ class TestIoTHubHTTPClientInstantiation:
# This means that you must do graceful exit by shutting down the client at the end of all tests # This means that you must do graceful exit by shutting down the client at the end of all tests
# and you may need to do a manual mock of the underlying HTTP client where appropriate. # and you may need to do a manual mock of the underlying HTTP client where appropriate.
configurations = [ configurations = [
pytest.param(FAKE_DEVICE_ID, None, False, id="Device Configuration"), pytest.param(FAKE_DEVICE_ID, None, id="Device Configuration"),
pytest.param(FAKE_DEVICE_ID, FAKE_MODULE_ID, False, id="Module Configuration"), pytest.param(FAKE_DEVICE_ID, FAKE_MODULE_ID, id="Module Configuration"),
pytest.param(FAKE_DEVICE_ID, FAKE_MODULE_ID, True, id="Edge Module Configuration"),
] ]
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
@ -127,11 +126,10 @@ class TestIoTHubHTTPClientInstantiation:
@pytest.mark.it( @pytest.mark.it(
"Stores the `device_id` and `module_id` values from the IoTHubClientConfig as attributes" "Stores the `device_id` and `module_id` values from the IoTHubClientConfig as attributes"
) )
@pytest.mark.parametrize("device_id, module_id, is_edge_module", configurations) @pytest.mark.parametrize("device_id, module_id", configurations)
async def test_simple_ids(self, client_config, device_id, module_id, is_edge_module): async def test_simple_ids(self, client_config, device_id, module_id):
client_config.device_id = device_id client_config.device_id = device_id
client_config.module_id = module_id client_config.module_id = module_id
client_config.is_edge_module = is_edge_module
client = IoTHubHTTPClient(client_config) client = IoTHubHTTPClient(client_config)
assert client._device_id == device_id assert client._device_id == device_id
@ -140,12 +138,11 @@ class TestIoTHubHTTPClientInstantiation:
await client.shutdown() await client.shutdown()
@pytest.mark.it( @pytest.mark.it(
"Derives the `edge_module_id` from the `device_id` and `module_id` if the IoTHubClientConfig indicates use of an Edge Module" "Derives the `edge_module_id` from the `device_id` and `module_id` if the IoTHubClientConfig contains a `module_id`"
) )
async def test_edge_module_id(self, client_config): async def test_edge_module_id(self, client_config):
client_config.device_id = FAKE_DEVICE_ID client_config.device_id = FAKE_DEVICE_ID
client_config.module_id = FAKE_MODULE_ID client_config.module_id = FAKE_MODULE_ID
client_config.is_edge_module = True
expected_edge_module_id = "{device_id}/{module_id}".format( expected_edge_module_id = "{device_id}/{module_id}".format(
device_id=FAKE_DEVICE_ID, module_id=FAKE_MODULE_ID device_id=FAKE_DEVICE_ID, module_id=FAKE_MODULE_ID
) )
@ -155,18 +152,12 @@ class TestIoTHubHTTPClientInstantiation:
await client.shutdown() await client.shutdown()
@pytest.mark.it("Sets the `edge_module_id` to None if not using an Edge Module") # NOTE: It would be nice if we could only do this for Edge modules, but there's no way to
@pytest.mark.parametrize( # indicate a Module is Edge vs non-Edge
"device_id, module_id", @pytest.mark.it("Sets the `edge_module_id` to None if not using a Module")
[ async def test_no_edge_module_id(self, client_config):
pytest.param(FAKE_DEVICE_ID, None, id="Device Configuration"), client_config.device_id = FAKE_DEVICE_ID
pytest.param(FAKE_DEVICE_ID, FAKE_MODULE_ID, id="Non-Edge Module Configuration"), client_config.module_id = None
],
)
async def test_no_edge_module_id(self, client_config, device_id, module_id):
client_config.device_id = device_id
client_config.module_id = module_id
client_config.is_edge_module = False
client = IoTHubHTTPClient(client_config) client = IoTHubHTTPClient(client_config)
assert client._edge_module_id is None assert client._edge_module_id is None
@ -176,7 +167,7 @@ class TestIoTHubHTTPClientInstantiation:
@pytest.mark.it( @pytest.mark.it(
"Constructs the `user_agent_string` by concatenating the base IoTHub user agent with the `product_info` from the IoTHubClientConfig" "Constructs the `user_agent_string` by concatenating the base IoTHub user agent with the `product_info` from the IoTHubClientConfig"
) )
@pytest.mark.parametrize("device_id, module_id, is_edge_module", configurations) @pytest.mark.parametrize("device_id, module_id", configurations)
@pytest.mark.parametrize( @pytest.mark.parametrize(
"product_info", "product_info",
[ [
@ -188,12 +179,9 @@ class TestIoTHubHTTPClientInstantiation:
), ),
], ],
) )
async def test_user_agent( async def test_user_agent(self, client_config, device_id, module_id, product_info):
self, client_config, device_id, module_id, is_edge_module, product_info
):
client_config.device_id = device_id client_config.device_id = device_id
client_config.module_id = module_id client_config.module_id = module_id
client_config.is_edge_module = is_edge_module
client_config.product_info = product_info client_config.product_info = product_info
expected_user_agent = user_agent.get_iothub_user_agent() + product_info expected_user_agent = user_agent.get_iothub_user_agent() + product_info
@ -203,7 +191,7 @@ class TestIoTHubHTTPClientInstantiation:
await client.shutdown() await client.shutdown()
@pytest.mark.it("Does not URL encode the user agent string") @pytest.mark.it("Does not URL encode the user agent string")
@pytest.mark.parametrize("device_id, module_id, is_edge_module", configurations) @pytest.mark.parametrize("device_id, module_id", configurations)
@pytest.mark.parametrize( @pytest.mark.parametrize(
"product_info", "product_info",
[ [
@ -215,12 +203,11 @@ class TestIoTHubHTTPClientInstantiation:
], ],
) )
async def test_user_agent_no_url_encoding( async def test_user_agent_no_url_encoding(
self, client_config, device_id, module_id, is_edge_module, product_info self, client_config, device_id, module_id, product_info
): ):
# NOTE: The user agent DOES eventually get url encoded, just not here, and not yet # NOTE: The user agent DOES eventually get url encoded, just not here, and not yet
client_config.device_id = device_id client_config.device_id = device_id
client_config.module_id = module_id client_config.module_id = module_id
client_config.is_edge_module = is_edge_module
client_config.product_info = product_info client_config.product_info = product_info
expected_user_agent = user_agent.get_iothub_user_agent() + product_info expected_user_agent = user_agent.get_iothub_user_agent() + product_info
url_encoded_expected_user_agent = urllib.parse.quote_plus(expected_user_agent) url_encoded_expected_user_agent = urllib.parse.quote_plus(expected_user_agent)
@ -231,23 +218,13 @@ class TestIoTHubHTTPClientInstantiation:
await client.shutdown() await client.shutdown()
#
#
# TODO: hostname / gateway hostname test once we know whats going on there
#
#
@pytest.mark.it( @pytest.mark.it(
"Creates a aiohttp ClientSession configured for accessing a URL based on the hostname with a timeout of 10 seconds" "Creates a aiohttp ClientSession configured for accessing a URL based on the IoTHubClientConfig's `hostname`, with a timeout of 10 seconds"
) )
@pytest.mark.parametrize("device_id, module_id, is_edge_module", configurations) @pytest.mark.parametrize("device_id, module_id", configurations)
async def test_client_session( async def test_client_session(self, mocker, client_config, device_id, module_id):
self, mocker, client_config, device_id, module_id, is_edge_module
):
# TODO: this test needs to be altered when hostname/gateway hostname logic is worked out
client_config.device_id = device_id client_config.device_id = device_id
client_config.module_id = module_id client_config.module_id = module_id
client_config.is_edge_module = is_edge_module
spy_session_init = mocker.spy(aiohttp, "ClientSession") spy_session_init = mocker.spy(aiohttp, "ClientSession")
expected_base_url = "https://" + client_config.hostname expected_base_url = "https://" + client_config.hostname
@ -266,11 +243,10 @@ class TestIoTHubHTTPClientInstantiation:
await client.shutdown() await client.shutdown()
@pytest.mark.it("Stores the `ssl_context` from the IoTHubClientConfig as an attribute") @pytest.mark.it("Stores the `ssl_context` from the IoTHubClientConfig as an attribute")
@pytest.mark.parametrize("device_id, module_id, is_edge_module", configurations) @pytest.mark.parametrize("device_id, module_id", configurations)
async def test_ssl_context(self, client_config, device_id, module_id, is_edge_module): async def test_ssl_context(self, client_config, device_id, module_id):
client_config.device_id = device_id client_config.device_id = device_id
client_config.module_id = module_id client_config.module_id = module_id
client_config.is_edge_module = is_edge_module
assert client_config.ssl_context is not None assert client_config.ssl_context is not None
client = IoTHubHTTPClient(client_config) client = IoTHubHTTPClient(client_config)
@ -279,7 +255,7 @@ class TestIoTHubHTTPClientInstantiation:
await client.shutdown() await client.shutdown()
@pytest.mark.it("Stores the `sastoken_provider` from the IoTHubClientConfig as an attribute") @pytest.mark.it("Stores the `sastoken_provider` from the IoTHubClientConfig as an attribute")
@pytest.mark.parametrize("device_id, module_id, is_edge_module", configurations) @pytest.mark.parametrize("device_id, module_id", configurations)
@pytest.mark.parametrize( @pytest.mark.parametrize(
"sastoken_provider", "sastoken_provider",
[ [
@ -287,12 +263,9 @@ class TestIoTHubHTTPClientInstantiation:
pytest.param(None, id="No SasTokenProvider present"), pytest.param(None, id="No SasTokenProvider present"),
], ],
) )
async def test_sastoken_provider( async def test_sastoken_provider(self, client_config, device_id, module_id, sastoken_provider):
self, client_config, device_id, module_id, is_edge_module, sastoken_provider
):
client_config.device_id = device_id client_config.device_id = device_id
client_config.module_id = module_id client_config.module_id = module_id
client_config.is_edge_module = is_edge_module
client_config.sastoken_provider = sastoken_provider client_config.sastoken_provider = sastoken_provider
client = IoTHubHTTPClient(client_config) client = IoTHubHTTPClient(client_config)
@ -374,16 +347,14 @@ class TestIoTHubHTTPClientInvokeDirectMethod:
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
def modify_client_config(self, client_config): def modify_client_config(self, client_config):
"""Modify the client config to always be an Edge Module""" """Modify the client config to always be an Edge Module"""
# TODO: likely need to modify once hostname/gateway hostname is ironed out
client_config.device_id = FAKE_DEVICE_ID client_config.device_id = FAKE_DEVICE_ID
client_config.module_id = FAKE_MODULE_ID client_config.module_id = FAKE_MODULE_ID
client_config.is_edge_module = True
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
def modify_post_response(self, client): def modify_post_response(self, client):
fake_method_response = { fake_method_response = {
"status": 200, "status": 200,
"payload": "fake payload", "payload": {"fake": "payload"},
} }
mock_response = client._session.post.return_value.__aenter__.return_value mock_response = client._session.post.return_value.__aenter__.return_value
mock_response.json.return_value = fake_method_response mock_response.json.return_value = fake_method_response
@ -392,7 +363,7 @@ class TestIoTHubHTTPClientInvokeDirectMethod:
def method_params(self): def method_params(self):
return { return {
"methodName": "fake method", "methodName": "fake method",
"payload": "fake payload", "payload": {"fake": "payload"},
"connectTimeoutInSeconds": 47, "connectTimeoutInSeconds": 47,
"responseTimeoutInSeconds": 42, "responseTimeoutInSeconds": 42,
} }
@ -578,19 +549,11 @@ class TestIoTHubHTTPClientInvokeDirectMethod:
device_id=target_device_id, module_id=target_module_id, method_params=method_params device_id=target_device_id, module_id=target_module_id, method_params=method_params
) )
@pytest.mark.it("Raises IoTHubClientError if not configured as an Edge Module") # NOTE: It'd be really great if we could reject non-Edge modules, but we can't.
@pytest.mark.parametrize( @pytest.mark.it("Raises IoTHubClientError if not configured as a Module")
"module_id",
[
pytest.param(None, id="Device Configuration"),
pytest.param(FAKE_MODULE_ID, id="Non-Edge Module Configuration"),
],
)
@pytest.mark.parametrize("target_device_id, target_module_id", targets) @pytest.mark.parametrize("target_device_id, target_module_id", targets)
async def test_not_edge( async def test_not_edge(self, client, target_device_id, target_module_id, method_params):
self, client, module_id, target_device_id, target_module_id, method_params client._module_id = None
):
client._module_id = module_id
client._edge_module_id = None client._edge_module_id = None
with pytest.raises(IoTHubClientError): with pytest.raises(IoTHubClientError):
@ -678,10 +641,9 @@ class TestIoTHubHTTPClientInvokeDirectMethod:
class TestIoTHubHTTPClientGetStorageInfoForBlob: class TestIoTHubHTTPClientGetStorageInfoForBlob:
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
def modify_client_config(self, client_config): def modify_client_config(self, client_config):
"""Modify the client config to always be an Device""" """Modify the client config to always be a Device"""
client_config.device_id = FAKE_DEVICE_ID client_config.device_id = FAKE_DEVICE_ID
client_config.module_id = None client_config.module_id = None
client_config.is_edge_module = False
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
def modify_post_response(self, client): def modify_post_response(self, client):
@ -885,10 +847,9 @@ class TestIoTHubHTTPClientGetStorageInfoForBlob:
class TestIoTHubHTTPClientNotifyBlobUploadStatus: class TestIoTHubHTTPClientNotifyBlobUploadStatus:
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
def modify_client_config(self, client_config): def modify_client_config(self, client_config):
"""Modify the client config to always be an Device""" """Modify the client config to always be a Device"""
client_config.device_id = FAKE_DEVICE_ID client_config.device_id = FAKE_DEVICE_ID
client_config.module_id = None client_config.module_id = None
client_config.is_edge_module = False
@pytest.fixture(params=["Notify Upload Success", "Notify Upload Failure"]) @pytest.fixture(params=["Notify Upload Success", "Notify Upload Failure"])
def kwargs(self, request): def kwargs(self, request):

34
v3_async_wip/tests/test_iothub_mqtt_client.py
Просмотреть файл

@ -31,7 +31,6 @@ FAKE_MODULE_ID = "fake_module_id"
FAKE_DEVICE_CLIENT_ID = "fake_device_id" FAKE_DEVICE_CLIENT_ID = "fake_device_id"
FAKE_MODULE_CLIENT_ID = "fake_device_id/fake_module_id" FAKE_MODULE_CLIENT_ID = "fake_device_id/fake_module_id"
FAKE_HOSTNAME = "fake.hostname" FAKE_HOSTNAME = "fake.hostname"
FAKE_GATEWAY_HOSTNAME = "fake.gateway.hostname"
FAKE_SIGNATURE = "ajsc8nLKacIjGsYyB4iYDFCZaRMmmDrUuY5lncYDYPI=" FAKE_SIGNATURE = "ajsc8nLKacIjGsYyB4iYDFCZaRMmmDrUuY5lncYDYPI="
FAKE_EXPIRY = str(int(time.time()) + 3600) FAKE_EXPIRY = str(int(time.time()) + 3600)
FAKE_URI = "fake/resource/location" FAKE_URI = "fake/resource/location"
@ -176,13 +175,6 @@ class TestIoTHubMQTTClientInstantiation:
), ),
], ],
) )
@pytest.mark.parametrize(
"hostname, gateway_hostname",
[
pytest.param(FAKE_HOSTNAME, None, id="No Gateway Hostname"),
pytest.param(FAKE_HOSTNAME, FAKE_GATEWAY_HOSTNAME, id="Gateway Hostname"),
],
)
@pytest.mark.parametrize( @pytest.mark.parametrize(
"product_info", "product_info",
[ [
@ -205,14 +197,10 @@ class TestIoTHubMQTTClientInstantiation:
device_id, device_id,
module_id, module_id,
client_id, client_id,
hostname,
gateway_hostname,
product_info, product_info,
): ):
client_config.device_id = device_id client_config.device_id = device_id
client_config.module_id = module_id client_config.module_id = module_id
client_config.hostname = hostname
client_config.gateway_hostname = gateway_hostname
client_config.product_info = product_info client_config.product_info = product_info
ua = user_agent.get_iothub_user_agent() ua = user_agent.get_iothub_user_agent()
@ -226,7 +214,7 @@ class TestIoTHubMQTTClientInstantiation:
# Determine expected username based on config # Determine expected username based on config
if product_info.startswith(constant.DIGITAL_TWIN_PREFIX): if product_info.startswith(constant.DIGITAL_TWIN_PREFIX):
expected_username = "{hostname}/{client_id}/?api-version={api_version}&DeviceClientType={user_agent}&{digital_twin_prefix}={custom_product_info}".format( expected_username = "{hostname}/{client_id}/?api-version={api_version}&DeviceClientType={user_agent}&{digital_twin_prefix}={custom_product_info}".format(
hostname=hostname, hostname=client_config.hostname,
client_id=client_id, client_id=client_id,
api_version=constant.IOTHUB_API_VERSION, api_version=constant.IOTHUB_API_VERSION,
user_agent=url_encoded_user_agent, user_agent=url_encoded_user_agent,
@ -235,13 +223,12 @@ class TestIoTHubMQTTClientInstantiation:
) )
else: else:
expected_username = "{hostname}/{client_id}/?api-version={api_version}&DeviceClientType={user_agent}{custom_product_info}".format( expected_username = "{hostname}/{client_id}/?api-version={api_version}&DeviceClientType={user_agent}{custom_product_info}".format(
hostname=hostname, hostname=client_config.hostname,
client_id=client_id, client_id=client_id,
api_version=constant.IOTHUB_API_VERSION, api_version=constant.IOTHUB_API_VERSION,
user_agent=url_encoded_user_agent, user_agent=url_encoded_user_agent,
custom_product_info=url_encoded_product_info, custom_product_info=url_encoded_product_info,
) )
# NOTE: Regarding the above, no matter if we have a gateway hostname set or not, it is the hostname that is always used.
client = IoTHubMQTTClient(client_config) client = IoTHubMQTTClient(client_config)
# The expected username was derived # The expected username was derived
@ -284,15 +271,6 @@ class TestIoTHubMQTTClientInstantiation:
), ),
], ],
) )
@pytest.mark.parametrize(
"hostname, gateway_hostname, expected_hostname",
[
pytest.param(FAKE_HOSTNAME, None, FAKE_HOSTNAME, id="No Gateway Hostname"),
pytest.param(
FAKE_HOSTNAME, FAKE_GATEWAY_HOSTNAME, FAKE_GATEWAY_HOSTNAME, id="Gateway Hostname"
),
],
)
@pytest.mark.parametrize( @pytest.mark.parametrize(
"websockets, expected_transport, expected_port, expected_ws_path", "websockets, expected_transport, expected_port, expected_ws_path",
[ [
@ -307,9 +285,6 @@ class TestIoTHubMQTTClientInstantiation:
device_id, device_id,
module_id, module_id,
expected_client_id, expected_client_id,
hostname,
gateway_hostname,
expected_hostname,
websockets, websockets,
expected_transport, expected_transport,
expected_port, expected_port,
@ -318,8 +293,6 @@ class TestIoTHubMQTTClientInstantiation:
# Configure the client_config based on params # Configure the client_config based on params
client_config.device_id = device_id client_config.device_id = device_id
client_config.module_id = module_id client_config.module_id = module_id
client_config.hostname = hostname
client_config.gateway_hostname = gateway_hostname
client_config.websockets = websockets client_config.websockets = websockets
# Patch the MQTTClient constructor # Patch the MQTTClient constructor
@ -333,7 +306,7 @@ class TestIoTHubMQTTClientInstantiation:
assert mock_constructor.call_count == 1 assert mock_constructor.call_count == 1
assert mock_constructor.call_args == mocker.call( assert mock_constructor.call_args == mocker.call(
client_id=expected_client_id, client_id=expected_client_id,
hostname=expected_hostname, hostname=client_config.hostname,
port=expected_port, port=expected_port,
transport=expected_transport, transport=expected_transport,
keep_alive=client_config.keep_alive, keep_alive=client_config.keep_alive,
@ -780,7 +753,6 @@ class TestIoTHubMQTTClientShutdown:
# correctness, lest we have to repeat all .disconnect() tests here. # correctness, lest we have to repeat all .disconnect() tests here.
original_disconnect = client.disconnect original_disconnect = client.disconnect
client.disconnect = mocker.AsyncMock(side_effect=exception) client.disconnect = mocker.AsyncMock(side_effect=exception)
client.disconnect.side_effect = exception
assert not client._keep_credentials_fresh_bg_task.done() assert not client._keep_credentials_fresh_bg_task.done()
assert not client._process_twin_responses_bg_task.done() assert not client._process_twin_responses_bg_task.done()

6
v3_async_wip/v3_async_wip/config.py
Просмотреть файл

@ -71,7 +71,6 @@ class ClientConfig:
*, *,
ssl_context: ssl.SSLContext, ssl_context: ssl.SSLContext,
hostname: str, hostname: str,
gateway_hostname: Optional[str] = None,
sastoken_provider: Optional[SasTokenProvider] = None, sastoken_provider: Optional[SasTokenProvider] = None,
proxy_options: Optional[ProxyOptions] = None, proxy_options: Optional[ProxyOptions] = None,
keep_alive: int = 60, keep_alive: int = 60,
@ -81,7 +80,6 @@ class ClientConfig:
"""Initializer for ClientConfig """Initializer for ClientConfig
:param str hostname: The hostname being connected to :param str hostname: The hostname being connected to
:param str gateway_hostname: The gateway hostname optionally being used
:param sastoken_provider: Object that can provide SasTokens :param sastoken_provider: Object that can provide SasTokens
:type sastoken_provider: :class:`SasTokenProvider` :type sastoken_provider: :class:`SasTokenProvider`
:param proxy_options: Details of proxy configuration :param proxy_options: Details of proxy configuration
@ -97,7 +95,6 @@ class ClientConfig:
""" """
# Network # Network
self.hostname = hostname self.hostname = hostname
self.gateway_hostname = gateway_hostname
self.proxy_options = proxy_options self.proxy_options = proxy_options
# Auth # Auth
@ -116,7 +113,6 @@ class IoTHubClientConfig(ClientConfig):
*, *,
device_id: str, device_id: str,
module_id: Optional[str] = None, module_id: Optional[str] = None,
is_edge_module: bool = False,
product_info: str = "", product_info: str = "",
**kwargs: Any, **kwargs: Any,
) -> None: ) -> None:
@ -125,14 +121,12 @@ class IoTHubClientConfig(ClientConfig):
:param str device_id: The device identity being used with the IoTHub :param str device_id: The device identity being used with the IoTHub
:param str module_id: The module identity being used with the IoTHub :param str module_id: The module identity being used with the IoTHub
:param bool is_edge_module: Boolean indicating whether or not using an Edge Module
:param str product_info: A custom identification string. :param str product_info: A custom identification string.
Additional parameters found in the docstring of the parent class Additional parameters found in the docstring of the parent class
""" """
self.device_id = device_id self.device_id = device_id
self.module_id = module_id self.module_id = module_id
self.is_edge_module = is_edge_module
self.product_info = product_info self.product_info = product_info
super().__init__(**kwargs) super().__init__(**kwargs)

110
v3_async_wip/v3_async_wip/connection_string.py Normal file
Просмотреть файл

@ -0,0 +1,110 @@
# -------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
# license information.
# --------------------------------------------------------------------------
"""This module contains tools for working with Connection Strings"""
__all__ = ["ConnectionString"]
CS_DELIMITER = ";"
CS_VAL_SEPARATOR = "="
HOST_NAME = "HostName"
SHARED_ACCESS_KEY_NAME = "SharedAccessKeyName"
SHARED_ACCESS_KEY = "SharedAccessKey"
SHARED_ACCESS_SIGNATURE = "SharedAccessSignature"
DEVICE_ID = "DeviceId"
MODULE_ID = "ModuleId"
GATEWAY_HOST_NAME = "GatewayHostName"
X509 = "x509"
_valid_keys = [
HOST_NAME,
SHARED_ACCESS_KEY_NAME,
SHARED_ACCESS_KEY,
SHARED_ACCESS_SIGNATURE,
DEVICE_ID,
MODULE_ID,
GATEWAY_HOST_NAME,
X509,
]
# TODO: does this module need revision for V3?
class ConnectionString(object):
"""Key/value mappings for connection details.
Uses the same syntax as dictionary
"""
def __init__(self, connection_string):
"""Initializer for ConnectionString
:param str connection_string: String with connection details provided by Azure
:raises: ValueError if provided connection_string is invalid
"""
self._dict = _parse_connection_string(connection_string)
self._strrep = connection_string
def __contains__(self, item):
return item in self._dict
def __getitem__(self, key):
return self._dict[key]
def __repr__(self):
return self._strrep
def get(self, key, default=None):
"""Return the value for key if key is in the dictionary, else default
:param str key: The key to retrieve a value for
:param str default: The default value returned if a key is not found
:returns: The value for the given key
"""
try:
return self._dict[key]
except KeyError:
return default
def _parse_connection_string(connection_string):
"""Return a dictionary of values contained in a given connection string"""
try:
cs_args = connection_string.split(CS_DELIMITER)
except (AttributeError, TypeError):
raise TypeError("Connection String must be of type str")
try:
d = dict(arg.split(CS_VAL_SEPARATOR, 1) for arg in cs_args)
except ValueError:
# This occurs in an extreme edge case where a dictionary cannot be formed because there
# is only 1 token after the split (dict requires two in order to make a key/value pair)
raise ValueError("Invalid Connection String - Unable to parse")
if len(cs_args) != len(d):
# various errors related to incorrect parsing - duplicate args, bad syntax, etc.
raise ValueError("Invalid Connection String - Unable to parse")
if not all(key in _valid_keys for key in d.keys()):
raise ValueError("Invalid Connection String - Invalid Key")
_validate_keys(d)
return d
def _validate_keys(d):
"""Raise ValueError if incorrect combination of keys in dict d"""
host_name = d.get(HOST_NAME)
shared_access_key_name = d.get(SHARED_ACCESS_KEY_NAME)
shared_access_key = d.get(SHARED_ACCESS_KEY)
device_id = d.get(DEVICE_ID)
x509 = d.get(X509)
if shared_access_key and x509 and x509.lower() == "true":
raise ValueError("Invalid Connection String - Mixed authentication scheme")
# This logic could be expanded to return the category of ConnectionString
if host_name and device_id and (shared_access_key or x509):
pass
elif host_name and shared_access_key and shared_access_key_name:
pass
else:
raise ValueError("Invalid Connection String - Incomplete")

19
v3_async_wip/v3_async_wip/custom_typing.py
Просмотреть файл

@ -3,9 +3,14 @@
# Licensed under the MIT License. See License.txt in the project root for # Licensed under the MIT License. See License.txt in the project root for
# license information. # license information.
# -------------------------------------------------------------------------- # --------------------------------------------------------------------------
from typing import Union, Dict, List, Tuple, Callable, Awaitable, TypeVar
from typing_extensions import TypedDict, ParamSpec
_P = ParamSpec("_P")
_R = TypeVar("_R")
FunctionOrCoroutine = Union[Callable[_P, _R], Callable[_P, Awaitable[_R]]]
from typing import Union, Dict, List, Tuple
from typing_extensions import TypedDict
# typing does not support recursion, so we must use forward references here (PEP484) # typing does not support recursion, so we must use forward references here (PEP484)
JSONSerializable = Union[ JSONSerializable = Union[
@ -25,14 +30,18 @@ Twin = Dict[str, Dict[str, JSONSerializable]]
TwinPatch = Dict[str, JSONSerializable] TwinPatch = Dict[str, JSONSerializable]
# TODO: should this be "direct method?" class DirectMethodParameters(TypedDict):
class MethodParameters(TypedDict):
methodName: str methodName: str
payload: str payload: JSONSerializable
connectTimeoutInSeconds: int connectTimeoutInSeconds: int
responseTimeoutInSeconds: int responseTimeoutInSeconds: int
class DirectMethodResult(TypedDict):
status: int
payload: JSONSerializable
class StorageInfo(TypedDict): class StorageInfo(TypedDict):
correlationId: str correlationId: str
hostName: str hostName: str

8
v3_async_wip/v3_async_wip/iot_exceptions.py
Просмотреть файл

@ -7,17 +7,21 @@
class IoTHubError(Exception): class IoTHubError(Exception):
"""Represents a failure reported by IoTHub""" """Represents a failure reported by IoT Hub"""
pass pass
class IoTEdgeError(Exception): class IoTEdgeError(Exception):
"""Represents a failure reported by IoTEdge""" """Represents a failure reported by IoT Edge"""
pass pass
class IoTEdgeEnvironmentError(Exception):
"""Represents a failure retrieving data from the IoT Edge environment"""
class IoTHubClientError(Exception): class IoTHubClientError(Exception):
"""Represents a failure from the IoTHub Client""" """Represents a failure from the IoTHub Client"""

619
v3_async_wip/v3_async_wip/iothub_client.py Normal file
Просмотреть файл

@ -0,0 +1,619 @@
# -------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
# license information.
# --------------------------------------------------------------------------
import abc
import logging
import os
import ssl
from typing import Optional, Union, cast
from .custom_typing import FunctionOrCoroutine
from .iot_exceptions import IoTEdgeEnvironmentError
from . import config, edge_hsm
from . import connection_string as cs
from . import iothub_http_client as http
from . import iothub_mqtt_client as mqtt
from . import sastoken as st
from . import signing_mechanism as sm
logger = logging.getLogger(__name__)
# TODO: finalize documentation
class IoTHubClient(abc.ABC):
"""Abstract parent class for IoTHubDeviceClient and IoTHubModuleClient containing
partial implementation.
"""
def __init__(self, client_config: config.IoTHubClientConfig) -> None:
"""Initializer for a generic IoTHubClient.
Do not directly use as the end user, use a factory method instead.
:param client_config: The IoTHubClientConfig object
:type client_config: :class:`IoTHubClientConfig`
"""
# Internal clients
self._mqtt_client = mqtt.IoTHubMQTTClient(client_config)
self._http_client = http.IoTHubHTTPClient(client_config)
# Keep a reference to the SAS Token Provider so it can be shut down later
self._sastoken_provider = client_config.sastoken_provider
async def shutdown(self) -> None:
"""Shut down the client
Call only when completely done with the client for graceful exit.
Cannot be cancelled - if you try, the client will still fully shut down as much as
possible (although the CancelledError will still be raised)
"""
cached_exception: Optional[Union[Exception, BaseException]] = None
logger.debug("Beginning IoTHubClient shutdown procedure")
try:
logger.debug("Shutting down IoTHubMQTTClient...")
await self._mqtt_client.shutdown()
logger.debug("IoTHubMQTTClient shutdown complete")
except (Exception, BaseException) as e:
logger.warning(
"Unexpected error during shutdown of IoTHubMQTTClient suppressed - still completing the rest of shutdown procedure"
)
cached_exception = e
try:
logger.debug("Shutting down IoTHubHTTPClient...")
await self._http_client.shutdown()
logger.debug("IoTHubHTTPClient shutdown complete")
except (Exception, BaseException) as e:
logger.warning(
"Unexpected error during shutdown of IoTHubHTTPClient suppressed - still completing the rest of shutdown procedure"
)
cached_exception = e
if self._sastoken_provider:
try:
logger.debug("Shutting down SasTokenProvider...")
await self._sastoken_provider.shutdown()
logger.debug("SasTokenProvider shutdown complete")
except (Exception, BaseException) as e:
logger.warning(
"Unexpected error during shutdown of SasTokenProvider suppressed - still completing the rest of shutdown procedure"
)
cached_exception = e
logger.debug("IoTHubClient shutdown procedure complete")
if cached_exception:
# NOTE: In the case of multiple failures, only the last one gets raised.
# Not much way around it, and besides, this is all an extreme edge case anyway.
logger.warning(
"Raising previously suppressed error now that shutdown procedure is complete"
)
raise cached_exception
# ~~~~~ Abstract declarations ~~~~~
# NOTE: rigid typechecking doesn't like when the signature changes in the child class
# implementation of an abstract method. This creates problems, given that Device/Module
# clients have some methods with different signatures. It may be worth considering
# dropping abstract definitions altogether if their use is too inconsistent, or at least
# paring them back to only the crucial ones (connect, shutdown)
# @abc.abstractmethod
# async def connect(self) -> None:
# raise NotImplementedError
# @abc.abstractmethod
# async def disconnect(self) -> None:
# raise NotImplementedError
# @abc.abstractmethod
# async def send_message(self) -> None:
# raise NotImplementedError
# @abc.abstractmethod
# async def send_direct_method_response(self) -> None:
# raise NotImplementedError
# @abc.abstractmethod
# async def send_twin_reported_properties_patch(self) -> None:
# raise NotImplementedError
# @abc.abstractmethod
# async def get_twin(self) -> Twin:
# raise NotImplementedError
# ~~~~~~ Shared implementations ~~~~~
@classmethod
async def _shared_client_create(
cls,
*,
device_id: str,
module_id: Optional[str] = None,
hostname: str,
ssl_context: Optional[ssl.SSLContext] = None,
symmetric_key: Optional[str] = None,
sastoken_fn: Optional[FunctionOrCoroutine] = None, # TODO: need more rigid definition
# sastoken_fn: Optional[FunctionOrCoroutine[[], str]] = None,
**kwargs,
) -> "IoTHubClient":
"""Agnostic implementation of .create() shared between Devices and Modules
:raises: ValueError if one of 'ssl_context', 'symmetric_key' or 'sastoken_fn' is not
provided
:raises: ValueError if both 'symmetric_key' and 'sastoken_fn' are provided
:raises: ValueError if an invalid 'symmetric_key' is provided
:raises: SasTokenError if there is a failure generating a SAS Token
"""
# Validate Parameters
_validate_kwargs(**kwargs)
if symmetric_key and sastoken_fn:
raise ValueError(
"Incompatible authentication - cannot provide both 'symmetric_key' and 'sastoken_fn'"
)
if not symmetric_key and not sastoken_fn and not ssl_context:
raise ValueError(
"Missing authentication - must provide one of 'symmetric_key', 'sastoken_fn' or 'ssl_context'"
)
if symmetric_key:
signing_mechanism = sm.SymmetricKeySigningMechanism(symmetric_key)
else:
signing_mechanism = None
return await cls._internal_factory(
device_id=device_id,
module_id=module_id,
hostname=hostname,
ssl_context=ssl_context,
sas_signing_mechanism=signing_mechanism,
sastoken_fn=sastoken_fn,
**kwargs,
)
@classmethod
async def _shared_client_create_from_connection_string(
cls, cs_obj: cs.ConnectionString, ssl_context: Optional[ssl.SSLContext] = None, **kwargs
) -> "IoTHubClient":
"""Agnostic implementation of .create_from_connection_string() shared between Devices
and Modules. Uses a ConnectionString object rather than a string, since the outer
client-specific implementation already converted it to validate
:raises: ValueError if the provided connection string is invalid
:raises: SasTokenError if there is a failure generating a SAS Token"""
# ssl_context is required if x509 is indicated by the connection string
if cs_obj.get(cs.X509, "").lower() == "true" and not ssl_context:
raise ValueError(
"Connection string indicates X509 certificate authentication, but no ssl_context provided"
)
# If the Gateway Hostname exists, use it instead of the Hostname
hostname = cs_obj.get(cs.GATEWAY_HOST_NAME, cs_obj[cs.HOST_NAME])
if cs.SHARED_ACCESS_KEY in cs_obj:
signing_mechanism = sm.SymmetricKeySigningMechanism(cs_obj[cs.SHARED_ACCESS_KEY])
else:
signing_mechanism = None
return await cls._internal_factory(
device_id=cs_obj[cs.DEVICE_ID],
module_id=cs_obj.get(cs.MODULE_ID),
hostname=hostname,
sas_signing_mechanism=signing_mechanism,
ssl_context=ssl_context,
**kwargs,
)
@classmethod
async def _internal_factory(
cls,
*,
device_id: str,
module_id: Optional[str] = None,
hostname: str,
ssl_context: Optional[ssl.SSLContext] = None,
sas_signing_mechanism: Optional[sm.SigningMechanism] = None,
sastoken_fn: Optional[FunctionOrCoroutine] = None, # TODO: need more rigid definition
# sastoken_fn: Optional[FunctionOrCoroutine[[], str]] = None,
**kwargs,
) -> "IoTHubClient":
"""Internal factory method that creates a client for a all configurations
:raises: SasTokenError if there is a failure generating a SAS Token
"""
# NOTE: Validation is assumed to have been done by the time this method is called.
# Internal SAS Generation
sastoken_generator: st.SasTokenGenerator
if sas_signing_mechanism:
uri = _format_sas_uri(hostname=hostname, device_id=device_id, module_id=module_id)
sastoken_generator = st.InternalSasTokenGenerator(
signing_mechanism=sas_signing_mechanism,
uri=uri,
)
sastoken_provider = await st.SasTokenProvider.create_from_generator(sastoken_generator)
# External SAS Generation
elif sastoken_fn:
sastoken_generator = st.ExternalSasTokenGenerator(sastoken_fn)
sastoken_provider = await st.SasTokenProvider.create_from_generator(sastoken_generator)
# No SAS Auth
else:
sastoken_provider = None
# SSL
if not ssl_context:
ssl_context = _default_ssl_context()
# Config setup
client_config = config.IoTHubClientConfig(
hostname=hostname,
device_id=device_id,
module_id=module_id,
sastoken_provider=sastoken_provider,
ssl_context=ssl_context,
**kwargs,
)
return cls(client_config)
class IoTHubDeviceClient(IoTHubClient):
"""A client for connecting a device to an instance of IoT Hub"""
@classmethod
async def create(
cls,
device_id: str,
hostname: str,
ssl_context: Optional[ssl.SSLContext] = None,
symmetric_key: Optional[str] = None,
sastoken_fn: Optional[FunctionOrCoroutine] = None, # TODO: more rigid definition
# sastoken_fn: Optional[FunctionOrCoroutine[[], str]] = None,
**kwargs,
) -> "IoTHubDeviceClient":
"""
Instantiate an IoTHubDeviceClient
- To use symmetric key authentication, provide the symmetric key as the 'symmetric_key'
parameter
- To use your own SAS tokens for authentication, provide a function or coroutine function
that returns SAS Tokens as the 'sastoken_fn' parameter
- To use X509 certificate authentication, configure an SSLContext for the certificate, and
provide it as the 'ssl_context' parameter
One of the these three types of authentication is required to instantiate the client.
:param str device_id: The device identity for the IoT Hub device
:param str hostname: Hostname of the IoT Hub or IoT Edge the device should connect to
:param ssl_context: Custom SSL context to be used by the client
If not provided, a default one will be used
:type ssl_context: :class:`ssl.SSLContext`
:param str symmetric_key: A symmetric key that can be used to generate SAS Tokens
:param sastoken_fn: A function or coroutine function that takes no arguments and returns
a SAS token string when invoked
:keyword bool connection_retry: Indicates whether to use built-in connection retry policy.
Default is 'True'
:keyword int keep_alive: Maximum period in seconds between MQTT communications. If no
communications are exchanged for this period, a ping exchange will occur.
Default is 60 seconds
:keyword str product_info: Arbitrary product information which will be included in the
User-Agent string
:keyword proxy_options: Configuration structure for sending traffic through a proxy server
:type: proxy_options: :class:`ProxyOptions`
:keyword bool websockets: Set to 'True' to use WebSockets over MQTT. Default is 'False'
:raises: ValueError if one of 'ssl_context', 'symmetric_key' or 'sastoken_fn' is not
provided
:raises: ValueError if both 'symmetric_key' and 'sastoken_fn' are provided
:raises: ValueError if an invalid 'symmetric_key' is provided
:raises: SasTokenError if there is a failure generating a SAS Token
:return: An IoTHubDeviceClient instance
"""
client = await cls._shared_client_create(
device_id=device_id,
hostname=hostname,
ssl_context=ssl_context,
symmetric_key=symmetric_key,
sastoken_fn=sastoken_fn,
**kwargs,
)
return cast(IoTHubDeviceClient, client)
@classmethod
async def create_from_connection_string(
cls, connection_string: str, ssl_context: Optional[ssl.SSLContext] = None, **kwargs
) -> "IoTHubDeviceClient":
"""Instantiate an IoTHubDeviceClient using a IoT Hub device connection string
:param str connection_string: The IoT Hub device connection string
:param ssl_context: Custom SSL context to be used by the client
If not provided, a default one will be used
:type ssl_context: :class:`ssl.SSLContext`
:keyword bool connection_retry: Indicates whether to use built-in connection retry policy.
Default is 'True'
:keyword int keep_alive: Maximum period in seconds between MQTT communications. If no
communications are exchanged for this period, a ping exchange will occur.
Default is 60 seconds
:keyword str product_info: Arbitrary product information which will be included in the
User-Agent string
:keyword proxy_options: Configuration structure for sending traffic through a proxy server
:type: proxy_options: :class:`ProxyOptions`
:keyword bool websockets: Set to 'True' to use WebSockets over MQTT. Default is 'False'
:raises: ValueError if the provided connection string is invalid
:raises: SasTokenError if there is a failure generating a SAS Token
:return: An IoTHubDeviceClient instance
"""
# Validate connection string is for Device
cs_obj = cs.ConnectionString(connection_string)
if cs.MODULE_ID in cs_obj:
raise ValueError("IoT Hub module connection string provided for IoTHubDeviceClient")
client = await cls._shared_client_create_from_connection_string(
cs_obj, ssl_context, **kwargs
)
return cast(IoTHubDeviceClient, client)
class IoTHubModuleClient(IoTHubClient):
"""A client for connecting a module to an instance of IoT Hub"""
@classmethod
async def create(
cls,
device_id: str,
module_id: str,
hostname: str,
ssl_context: Optional[ssl.SSLContext] = None,
symmetric_key: Optional[str] = None,
# sastoken_fn: Optional[FunctionOrCoroutine[[], str]] = None,
sastoken_fn: Optional[FunctionOrCoroutine] = None, # TODO: more rigid definition
**kwargs,
) -> "IoTHubModuleClient":
"""
Instantiate an IoTHubModuleClient
- To use symmetric key authentication, provide the symmetric key as the 'symmetric_key'
parameter
- To use your own SAS tokens for authentication, provide a function or coroutine function
that returns SAS Tokens as the 'sastoken_fn' parameter
- To use X509 certificate authentication, configure an SSLContext for the certificate, and
provide it as the 'ssl_context' parameter
One of the these three types of authentication is required to instantiate the client.
:param str device_id: The device identity for the IoT Hub device containing the
IoT Hub module
:param str module_id: The module identity for the IoT Hub module
:param str hostname: Hostname of the IoT Hub or IoT Edge the device should connect to
:param ssl_context: Custom SSL context to be used by the client
If not provided, a default one will be used
:type ssl_context: :class:`ssl.SSLContext`
:param str symmetric_key: A symmetric key that can be used to generate SAS Tokens
:param sastoken_fn: A function or coroutine function that takes no arguments and returns
a SAS token string when invoked
:keyword bool connection_retry: Indicates whether to use built-in connection retry policy.
Default is 'True'
:keyword int keep_alive: Maximum period in seconds between MQTT communications. If no
communications are exchanged for this period, a ping exchange will occur.
Default is 60 seconds
:keyword str product_info: Arbitrary product information which will be included in the
User-Agent string
:keyword proxy_options: Configuration structure for sending traffic through a proxy server
:type: proxy_options: :class:`ProxyOptions`
:keyword bool websockets: Set to 'True' to use WebSockets over MQTT. Default is 'False'
:raises: ValueError if one of 'ssl_context', 'symmetric_key' or 'sastoken_fn' is not
provided
:raises: ValueError if both 'symmetric_key' and 'sastoken_fn' are provided
:raises: ValueError if an invalid 'symmetric_key' is provided
:raises: SasTokenError if there is a failure generating a SAS Token
:return: An IoTHubModuleClient instance
"""
client = await cls._shared_client_create(
device_id=device_id,
module_id=module_id,
hostname=hostname,
ssl_context=ssl_context,
symmetric_key=symmetric_key,
sastoken_fn=sastoken_fn,
**kwargs,
)
return cast(IoTHubModuleClient, client)
@classmethod
async def create_from_connection_string(
cls, connection_string: str, ssl_context: Optional[ssl.SSLContext] = None, **kwargs
) -> "IoTHubModuleClient":
"""Instantiate an IoTHubModuleClient using a IoT Hub module connection string
:param str connection_string: The IoT Hub module connection string
:param ssl_context: Custom SSL context to be used by the client
If not provided, a default one will be used
:type ssl_context: :class:`ssl.SSLContext`
:keyword bool connection_retry: Indicates whether to use built-in connection retry policy.
Default is 'True'
:keyword int keep_alive: Maximum period in seconds between MQTT communications. If no
communications are exchanged for this period, a ping exchange will occur.
Default is 60 seconds
:keyword str product_info: Arbitrary product information which will be included in the
User-Agent string
:keyword proxy_options: Configuration structure for sending traffic through a proxy server
:type: proxy_options: :class:`ProxyOptions`
:keyword bool websockets: Set to 'True' to use WebSockets over MQTT. Default is 'False'
:raises: ValueError if the provided connection string is invalid
:raises: SasTokenError if there is a failure generating a SAS Token
:return: An IoTHubModuleClient instance
"""
# Validate connection string is for Module
cs_obj = cs.ConnectionString(connection_string)
if cs.MODULE_ID not in cs_obj:
raise ValueError("IoT Hub device connection string provided for IoTHubModuleClient")
client = await cls._shared_client_create_from_connection_string(
cs_obj, ssl_context, **kwargs
)
return cast(IoTHubModuleClient, client)
@classmethod
async def create_from_edge_environment(cls, **kwargs) -> "IoTHubModuleClient":
"""Instantiate an IoTHubModuleClient using information from an IoT Edge environment
This method can only be run from inside an IoT Edge environment, or in a debugging
environment configured for Edge development (e.g. Visual Studio Code)
:keyword bool connection_retry: Indicates whether to use built-in connection retry policy.
Default is 'True'
:keyword int keep_alive: Maximum period in seconds between MQTT communications. If no
communications are exchanged for this period, a ping exchange will occur.
Default is 60 seconds
:keyword str product_info: Arbitrary product information which will be included in the
User-Agent string
:keyword proxy_options: Configuration structure for sending traffic through a proxy server
:type: proxy_options: :class:`ProxyOptions`
:keyword bool websockets: Set to 'True' to use WebSockets over MQTT. Default is 'False'
:raises: IoTEdgeEnvironmentError if the required environment variables are not present or
cannot be accessed
:raises: IoTEdgeError if there is a failure with the IoT Edge
:raises: SasTokenError if there is a failure generating a SAS Token
:raises: ValueError if IoT Edge environment variable values are invalid
:raises: TypeError if IoT Edge environment variable values are of the wrong format
:return: An IoTHubModuleClient instance
"""
_validate_kwargs(**kwargs)
try:
# First, try to find the regular IoT Edge environment variables
return await cls._create_from_real_edge_environment(**kwargs)
except IoTEdgeEnvironmentError as original_exception:
try:
# If they can't be found, try looking for the IoT Edge simulator variables
return await cls._create_from_simulated_edge_environment(**kwargs)
except IoTEdgeEnvironmentError:
# Raise the original error if the IoT Edge simulator variables also cannot be found
raise original_exception
@classmethod
async def _create_from_real_edge_environment(cls, **kwargs) -> "IoTHubModuleClient":
"""Instantiate an IoTHubModuleClient from values stored in environment variables
in a IoT Edge deployment environment.
:raises: IoTEdgeEnvironmentError if IoT Edge environment variables are not present or
cannot be accessed
:raises: IoTEdgeError if there is a failure communicating with IoT Edge
:raises: SasTokenError if there is a failure generating a SAS Token
:raises: ValueError if IoT Edge environment variables values are invalid
:raises: TypeError if IoT Edge environment variable values are of the wrong format
"""
# Read values from the IoT Edge environment variables
try:
device_id = os.environ["IOTEDGE_DEVICEID"]
module_id = os.environ["IOTEDGE_MODULEID"]
hostname = os.environ["IOTEDGE_GATEWAYHOSTNAME"]
module_generation_id = os.environ["IOTEDGE_MODULEGENERATIONID"]
workload_uri = os.environ["IOTEDGE_WORKLOADURI"]
api_version = os.environ["IOTEDGE_APIVERSION"]
except KeyError as e:
raise IoTEdgeEnvironmentError("Could not retrieve Edge environment variables") from e
# The IoT Edge HSM will be used to get the verification certs, as well as to sign data
# for making SAS Tokens
hsm = edge_hsm.IoTEdgeHsm(
module_id=module_id,
generation_id=module_generation_id,
workload_uri=workload_uri,
api_version=api_version,
)
# Set up Edge SSL context by loading the cert data
server_verification_cert = await hsm.get_certificate()
ssl_context = _default_ssl_context()
ssl_context.load_verify_locations(cadata=server_verification_cert)
# Send to the internal factory
client = await cls._internal_factory(
device_id=device_id,
module_id=module_id,
hostname=hostname,
ssl_context=ssl_context,
sas_signing_mechanism=hsm,
**kwargs,
)
return cast(IoTHubModuleClient, client)
@classmethod
async def _create_from_simulated_edge_environment(cls, **kwargs) -> "IoTHubModuleClient":
"""Instantiate an IoTHubModuleClient from values stored in environment variables
in a simulated IoT Edge environment
:raises: IoTEdgeEnvironmentError if IoT Edge environment variables are not present or
cannot be accessed
:raises: ValueError if the connection string in the environment is invalid
:raises: SasTokenError if there is a failure generating a SAS Token
"""
# Read values from the IoT Edge Simulator environment variables
try:
connection_string = os.environ["EdgeHubConnectionString"]
ca_cert_filepath = os.environ["EdgeModuleCACertificateFile"]
except KeyError as e:
raise IoTEdgeEnvironmentError("Could not retrieve Edge environment variables") from e
# Set up Edge SSL context by loading the cert file
ssl_context = _default_ssl_context()
ssl_context.load_verify_locations(cafile=ca_cert_filepath)
# Since we have a connection string, just use the connection string factory
return await cls.create_from_connection_string(
connection_string, ssl_context=ssl_context, **kwargs
)
def _validate_kwargs(exclude=[], **kwargs):
"""Helper function to validate user provided kwargs.
Raises TypeError if an invalid option has been provided"""
valid_kwargs = [
"auto_reconnect",
"keep_alive",
"product_info",
"proxy_options",
"websockets",
]
for kwarg in kwargs:
if (kwarg not in valid_kwargs) or (kwarg in exclude):
# NOTE: TypeError is the conventional error that is returned when an invalid kwarg is
# supplied. It feels like it should be a ValueError, but it's not.
raise TypeError("Unsupported keyword argument: '{}'".format(kwarg))
def _default_ssl_context() -> ssl.SSLContext:
"""Return a default SSLContext"""
ssl_context = ssl.SSLContext(protocol=ssl.PROTOCOL_TLS_CLIENT)
ssl_context.verify_mode = ssl.CERT_REQUIRED
ssl_context.check_hostname = True
ssl_context.load_default_certs()
return ssl_context
def _format_sas_uri(hostname: str, device_id: str, module_id: Optional[str] = None) -> str:
if module_id:
return "{hostname}/devices/{device_id}/modules/{module_id}".format(
hostname=hostname, device_id=device_id, module_id=module_id
)
else:
return "{hostname}/devices/{device_id}".format(hostname=hostname, device_id=device_id)

45
v3_async_wip/v3_async_wip/iothub_http_client.py
Просмотреть файл

@ -8,7 +8,7 @@ import asyncio
import logging import logging
import urllib.parse import urllib.parse
from typing import Optional, cast from typing import Optional, cast
from .custom_typing import MethodParameters, StorageInfo from .custom_typing import DirectMethodParameters, DirectMethodResult, StorageInfo
from .iot_exceptions import IoTHubClientError, IoTHubError, IoTEdgeError from .iot_exceptions import IoTHubClientError, IoTHubError, IoTEdgeError
from . import config, constant, user_agent from . import config, constant, user_agent
from . import http_path_iothub as http_path from . import http_path_iothub as http_path
@ -30,7 +30,6 @@ HTTP_TIMEOUT = 10
# TODO: document aiohttp exceptions that can be raised # TODO: document aiohttp exceptions that can be raised
# TODO: URL Encoding logic # TODO: URL Encoding logic
# TODO: Proxy support # TODO: Proxy support
# TODO: Hostname/Gateway Hostname split (E2E test to see what works)
# TODO: Should direct method responses be a DirectMethodResponse object? If so, what is the rid? # TODO: Should direct method responses be a DirectMethodResponse object? If so, what is the rid?
# See specific inline commentary for more details on what is required # See specific inline commentary for more details on what is required
@ -64,14 +63,8 @@ class IoTHubHTTPClient:
""" """
self._device_id = client_config.device_id self._device_id = client_config.device_id
self._module_id = client_config.module_id self._module_id = client_config.module_id
self._edge_module_id = _format_edge_module_id( self._edge_module_id = _format_edge_module_id(self._device_id, self._module_id)
self._device_id, self._module_id, client_config.is_edge_module
)
self._user_agent_string = user_agent.get_iothub_user_agent() + client_config.product_info self._user_agent_string = user_agent.get_iothub_user_agent() + client_config.product_info
if client_config.gateway_hostname:
self._hostname = client_config.gateway_hostname
else:
self._hostname = client_config.hostname
# TODO: add proxy support # TODO: add proxy support
# Doing so will require building a custom "Connector" that can be injected into the # Doing so will require building a custom "Connector" that can be injected into the
@ -84,7 +77,7 @@ class IoTHubHTTPClient:
logger.warning("Proxy use with .get_storage_info_for_blob() not supported") logger.warning("Proxy use with .get_storage_info_for_blob() not supported")
logger.warning("Proxy use with .notify_blob_upload_status() not supported") logger.warning("Proxy use with .notify_blob_upload_status() not supported")
self._session = _create_client_session(self._hostname) self._session = _create_client_session(client_config.hostname)
self._ssl_context = client_config.ssl_context self._ssl_context = client_config.ssl_context
self._sastoken_provider = client_config.sastoken_provider self._sastoken_provider = client_config.sastoken_provider
@ -98,28 +91,34 @@ class IoTHubHTTPClient:
# See: https://docs.aiohttp.org/en/stable/client_advanced.html#graceful-shutdown # See: https://docs.aiohttp.org/en/stable/client_advanced.html#graceful-shutdown
await asyncio.sleep(0.25) await asyncio.sleep(0.25)
# TODO: Should this return type be a MethodResponse? Or should we get rid of those objects entirely?
# TODO: Either way, need a better rtype than "dict"
async def invoke_direct_method( async def invoke_direct_method(
self, *, device_id: str, module_id: Optional[str] = None, method_params: MethodParameters self,
) -> dict: *,
device_id: str,
module_id: Optional[str] = None,
method_params: DirectMethodParameters
) -> DirectMethodResult:
"""Send a request to invoke a direct method on a target device or module """Send a request to invoke a direct method on a target device or module
:param str device_id: The target device ID :param str device_id: The target device ID
:param str module_id: The target module ID :param str module_id: The target module ID
:param dict method_params: The parameters for the direct method invocation :param dict method_params: The parameters for the direct method invocation
:returns: A dictionary containing a status and payload reported by the target device
:rtype: dict
:raises: :class:`IoTHubClientError` if not using an IoT Edge Module :raises: :class:`IoTHubClientError` if not using an IoT Edge Module
:raises: :class:`IoTHubClientError` if the direct method response cannot be parsed :raises: :class:`IoTHubClientError` if the direct method response cannot be parsed
:raises: :class:`IoTEdgeError` if IoT Edge responds with failure :raises: :class:`IoTEdgeError` if IoT Edge responds with failure
""" """
if not self._edge_module_id: if not self._edge_module_id:
# NOTE: The Edge Module ID will be exist for any Module, it doesn't actually indicate
# if it is an Edge Module or not. There's no way to tell, unfortunately.
raise IoTHubClientError(".invoke_direct_method() only available for Edge Modules") raise IoTHubClientError(".invoke_direct_method() only available for Edge Modules")
path = http_path.get_direct_method_invoke_path(device_id, module_id) path = http_path.get_direct_method_invoke_path(device_id, module_id)
query_params = {PARAM_API_VERISON: constant.IOTHUB_API_VERSION} query_params = {PARAM_API_VERISON: constant.IOTHUB_API_VERSION}
# NOTE: Other headers are auto-generated by aiohttp # NOTE: Other headers are auto-generated by aiohttp
# TODO: we may need to explicitly add the Host header depending on how host/gateway host works out
headers = { headers = {
HEADER_USER_AGENT: urllib.parse.quote_plus(self._user_agent_string), HEADER_USER_AGENT: urllib.parse.quote_plus(self._user_agent_string),
HEADER_EDGE_MODULE_ID: self._edge_module_id, # TODO: I assume this isn't supposed to be URI encoded just like in MQTT? HEADER_EDGE_MODULE_ID: self._edge_module_id, # TODO: I assume this isn't supposed to be URI encoded just like in MQTT?
@ -152,9 +151,9 @@ class IoTHubHTTPClient:
logger.debug( logger.debug(
"Successfully received response from IoT Edge for direct method invocation" "Successfully received response from IoT Edge for direct method invocation"
) )
dm_response_json = await response.json() dm_result = cast(DirectMethodResult, await response.json())
return dm_response_json return dm_result
async def get_storage_info_for_blob(self, *, blob_name: str) -> StorageInfo: async def get_storage_info_for_blob(self, *, blob_name: str) -> StorageInfo:
"""Request information for uploading blob file via the Azure Storage SDK """Request information for uploading blob file via the Azure Storage SDK
@ -257,16 +256,10 @@ class IoTHubHTTPClient:
return None return None
def _format_edge_module_id( def _format_edge_module_id(device_id: str, module_id: Optional[str]) -> Optional[str]:
device_id: str, module_id: Optional[str], is_edge_module
) -> Optional[str]:
"""Returns the edge module identifier""" """Returns the edge module identifier"""
if is_edge_module: if module_id:
if module_id: return "{device_id}/{module_id}".format(device_id=device_id, module_id=module_id)
return "{device_id}/{module_id}".format(device_id=device_id, module_id=module_id)
else:
# This shouldn't ever happen
raise ValueError("Invalid configuration - Edge Module with no Module ID")
else: else:
return None return None

12
v3_async_wip/v3_async_wip/iothub_mqtt_client.py
Просмотреть файл

@ -44,7 +44,6 @@ class IoTHubMQTTClient:
self._module_id = client_config.module_id self._module_id = client_config.module_id
self._client_id = _format_client_id(self._device_id, self._module_id) self._client_id = _format_client_id(self._device_id, self._module_id)
self._username = _format_username( self._username = _format_username(
# NOTE: Always use the original hostname, even if gateway hostname is set
hostname=client_config.hostname, hostname=client_config.hostname,
client_id=self._client_id, client_id=self._client_id,
product_info=client_config.product_info, product_info=client_config.product_info,
@ -501,18 +500,13 @@ def _create_mqtt_client(
) -> mqtt.MQTTClient: ) -> mqtt.MQTTClient:
logger.debug("Creating MQTTClient") logger.debug("Creating MQTTClient")
logger.debug("Using {} as hostname".format(client_config.hostname))
if client_config.module_id: if client_config.module_id:
logger.debug("Using IoTHub Module. Client ID is {}".format(client_id)) logger.debug("Using IoTHub Module. Client ID is {}".format(client_id))
else: else:
logger.debug("Using IoTHub Device. Client ID is {}".format(client_id)) logger.debug("Using IoTHub Device. Client ID is {}".format(client_id))
if client_config.gateway_hostname:
logger.debug("Gateway Hostname is present. Using Gateway Hostname as Hostname")
hostname = client_config.gateway_hostname
else:
logger.debug("Gateway Hostname not present. Using Hostname as Hostname")
hostname = client_config.hostname
if client_config.websockets: if client_config.websockets:
logger.debug("Using MQTT over websockets") logger.debug("Using MQTT over websockets")
transport = "websockets" transport = "websockets"
@ -526,7 +520,7 @@ def _create_mqtt_client(
client = mqtt.MQTTClient( client = mqtt.MQTTClient(
client_id=client_id, client_id=client_id,
hostname=hostname, hostname=client_config.hostname,
port=port, port=port,
transport=transport, transport=transport,
keep_alive=client_config.keep_alive, keep_alive=client_config.keep_alive,

7
v3_async_wip/v3_async_wip/sastoken.py
Просмотреть файл

@ -10,9 +10,11 @@ import asyncio
import logging import logging
import time import time
import urllib.parse import urllib.parse
from typing import Dict, List, Union, Awaitable, Callable, cast from typing import Dict, List, Awaitable, Callable, cast
from .custom_typing import FunctionOrCoroutine
from .signing_mechanism import SigningMechanism from .signing_mechanism import SigningMechanism
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
DEFAULT_TOKEN_UPDATE_MARGIN: int = 120 DEFAULT_TOKEN_UPDATE_MARGIN: int = 120
@ -99,7 +101,8 @@ class InternalSasTokenGenerator(SasTokenGenerator):
class ExternalSasTokenGenerator(SasTokenGenerator): class ExternalSasTokenGenerator(SasTokenGenerator):
def __init__(self, generator_fn: Union[Callable[[], str], Callable[[], Awaitable[str]]]): # TODO: need more specificity in generator_fn
def __init__(self, generator_fn: FunctionOrCoroutine):
"""An object that can generate SasTokens by invoking a provided callable. """An object that can generate SasTokens by invoking a provided callable.
This callable can be a function or a coroutine function. This callable can be a function or a coroutine function.