зеркало из https://github.com/microsoft/CCF.git
Move Python client code out of `ccf` package and back to infra (#3386)
This commit is contained in:
Julien Maffre
GitHub
Родитель
08b6740d21
Коммит
f302ddade3
|
|
@ -1 +1 @@
|
|||
Trigger daily
|
||||
Daily please
|
||||
|
|
|
|||
|
|
@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
|
|||
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
|
||||
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Remove
|
||||
|
||||
- The `ccf` Python package no longer provides utilities to issue requests to a running CCF service. This is because CCF supports widely-used client-server protocols (TLS, HTTP) that should already be provided by libraries for all programming languages. The `ccf` Python package can still be used to audit the ledger and snapshot files (#3386).
|
||||
|
||||
## [2.0.0-dev8]
|
||||
|
||||
### Added
|
||||
|
|
|
|||
|
|
@ -677,8 +677,6 @@ if(BUILD_TESTS)
|
|||
ADDITIONAL_ARGS
|
||||
--schema-dir
|
||||
${CMAKE_SOURCE_DIR}/doc/schemas
|
||||
--client-tutorial
|
||||
${CMAKE_SOURCE_DIR}/python/tutorial.py
|
||||
--ledger-tutorial
|
||||
${CMAKE_SOURCE_DIR}/python/ledger_tutorial.py
|
||||
--config-samples-dir
|
||||
|
|
|
|||
|
|
@ -24,7 +24,13 @@ Audit
|
|||
|
||||
---
|
||||
|
||||
:fa:`code` :doc:`python_library`
|
||||
.. image:: ../img/python.svg
|
||||
:width: 22
|
||||
:alt: Python
|
||||
:align: left
|
||||
|
||||
|
||||
:doc:`python_library`
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Utility library that can be used to write ledger audit scripts quickly and easily.
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
Python Library
|
||||
==============
|
||||
|
||||
This page describes the Python API of the :py:class:`ccf.ledger` module which can be used by auditors to parse a CCF ledger. To install the ``ccf`` Python package, run:
|
||||
This page describes the Python API of the :py:class:`ccf.ledger` module which can be used by auditors to parse a CCF ledger.
|
||||
|
||||
The latest version of the CCF Python tools package is `available on PyPi <https://pypi.org/project/ccf/>`_ and can be installed as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
|
|
@ -22,7 +24,7 @@ First, the paths to the ledger directories should be set:
|
|||
|
||||
ledger_dirs = ["</path/to/ledger/dir>"] # List of a single ledger directory
|
||||
|
||||
.. note:: By default, ledger directories are created under the node directory.
|
||||
.. note:: By default, ledger directories are created under the node directory (See :ref:`operations/configuration:``ledger``` configuration entry).
|
||||
|
||||
Then, import the ledger module and instantiate a :py:class:`ccf.ledger` object:
|
||||
|
||||
|
|
|
|||
|
|
@ -79,7 +79,7 @@ Additionally, the ``cchost`` binary must be told that the enclave type is debug,
|
|||
Integration Tests
|
||||
-----------------
|
||||
|
||||
The ``sandbox.sh`` script can be a helpful element of infrastructure to execute Integration Tests against a CCF test network running a particular application. The `test_install.sh <https://github.com/microsoft/CCF/blob/main/tests/test_install.sh>`_ script is a good example of that, using the sandbox to run `tutorial.py <https://github.com/microsoft/CCF/blob/main/python/tutorial.py>`_ on a release package.
|
||||
The ``sandbox.sh`` script can be a helpful element of infrastructure to execute Integration Tests against a CCF test network running a particular application (see `test_install.sh <https://github.com/microsoft/CCF/blob/main/tests/test_install.sh>`_ script as example).
|
||||
|
||||
``test_install.sh`` illustrates how to wait for the sandbox to be `ready <https://github.com/microsoft/CCF/blob/main/tests/test_install.sh#L33>`_ before issuing application transactions, how to shut it down cleanly, and how to trigger a recovery. Recovering a test network can be a useful way to inspect post-test application test.
|
||||
|
||||
|
|
|
|||
|
|
@ -94,29 +94,6 @@ Some of these subcommands require additional arguments, such as the node ID or u
|
|||
|
||||
These proposals and votes should be sent as the body of HTTP requests as described below.
|
||||
|
||||
Creating Proposals in Python
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
``ccf.proposal_generator`` can also be imported and used in a Python application instead of as a command-line tool.
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET: import_proposal_generator
|
||||
:lines: 1
|
||||
|
||||
The proposal generation functions return dictionaries that can be submitted to a ``CCFClient``.
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET_START: dict_proposal
|
||||
:end-before: SNIPPET_END: dict_proposal
|
||||
|
||||
You may wish to write these proposals to files so they can be examined or modified further. These proposal files can be submitted directly --- ``CCFClient`` will treat string request bodies beginning with an ``@`` as file paths in the same way that ``curl`` does, and use the content of the file when sending.
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET_START: json_proposal_with_file
|
||||
:end-before: SNIPPET_END: json_proposal_with_file
|
||||
|
||||
Submitting a New Proposal
|
||||
-------------------------
|
||||
|
||||
|
|
|
|||
|
|
@ -28,25 +28,13 @@ This section describes how :term:`Users` can issue transactions to CCF applicati
|
|||
|
||||
.. image:: ../img/python.svg
|
||||
:width: 22
|
||||
:alt: C++
|
||||
:alt: Python
|
||||
:align: left
|
||||
|
||||
:doc:`python_tutorial`
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Common user operations from a Python client.
|
||||
|
||||
---
|
||||
|
||||
.. image:: ../img/python.svg
|
||||
:width: 22
|
||||
:alt: C++
|
||||
:align: left
|
||||
|
||||
:doc:`python_api`
|
||||
^^^^^^^^^^^^^^^^^
|
||||
|
||||
Python client API reference.
|
||||
**[Removed]** Common user operations from a Python client.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,13 +0,0 @@
|
|||
Python Client API
|
||||
=================
|
||||
|
||||
.. autoclass:: ccf.clients.Identity
|
||||
:members:
|
||||
|
||||
.. autoclass:: ccf.clients.CCFClient
|
||||
:members:
|
||||
|
||||
.. autoclass:: ccf.clients.Response
|
||||
:members:
|
||||
|
||||
.. autoexception:: ccf.clients.CCFConnectionException
|
||||
|
|
@ -1,137 +1,4 @@
|
|||
Python Client Tutorial
|
||||
======================
|
||||
Python Client
|
||||
=============
|
||||
|
||||
Install
|
||||
-------
|
||||
|
||||
The CCF Python tools package can be used to interact with an existing running service and provides utilities to:
|
||||
|
||||
- Issue HTTP requests over TLS to a CCF service
|
||||
- Build custom governance proposals and votes
|
||||
- Parse and verify the integrity of a CCF ledger
|
||||
|
||||
The latest version of the CCF Python tools package is `available on PyPi <https://pypi.org/project/ccf/>`_ and can be installed as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install ccf
|
||||
|
||||
.. note:: The CCF Python tools package does `not` provide utilities to build and deploy CCF applications.
|
||||
|
||||
A step-by-step tutorial on how to use the CCF Python package is available :ref:`here <use_apps/python_tutorial:Python Client Tutorial>`.
|
||||
|
||||
Uninstall
|
||||
---------
|
||||
|
||||
To uninstall the CCF Python package, run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip uninstall ccf
|
||||
|
||||
|
||||
Tutorial
|
||||
--------
|
||||
|
||||
.. note:: The CCF Python client module uses `Python Requests <https://requests.readthedocs.io/en/master/>`_ by default, but can be switched to a ``curl``-based client (printing each command to stdout) by running with the environment variable ``CURL_CLIENT`` set.
|
||||
|
||||
This tutorial describes how a Python client can securely issue requests to a running CCF network. It is assumed that the CCF network has already been started (e.g. after having :doc:`deployed a sandbox service </build_apps/run_app>`).
|
||||
|
||||
.. note:: See :ref:`Python Client API <use_apps/python_api:Python Client API>` for the complete API specification.
|
||||
|
||||
In the Python interpreter or new file:
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET: import_clients
|
||||
:lines: 1
|
||||
|
||||
Set the following CCF node variables:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
host = "<node-host>" # Node address or domain (str)
|
||||
port = <node-port> # Node port (int)
|
||||
ca = "<path/to/network/cert>" # Network certificate path
|
||||
|
||||
.. note:: :doc:`When starting a CCF sandbox </build_apps/run_app>`, use any node's IP address and port number. All certificates and keys can be found in the associated ``common_dir`` folder.
|
||||
|
||||
Create a new :py:class:`ccf.clients.CCFClient` instance which will create a secure TLS connection to the target node part of the network specified via ``ca``:
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET: anonymous_client
|
||||
:lines: 1
|
||||
|
||||
You can then use the ``anonymous_client`` to issue requests that do not require authentication (typically, ``GET`` endpoints under ``/node``). Every call returns a :py:class:`ccf.clients.Response` object associated with the HTTP response.
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET_START: anonymous_requests
|
||||
:end-before: SNIPPET_END: anonymous_requests
|
||||
|
||||
TLS Session Authentication
|
||||
--------------------------
|
||||
|
||||
To create a client authenticated via TLS and issue application or governance requests, the session identity (certificate and private key) should be specified:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
cert = "</path/to/client/cert>" # Client certificate path
|
||||
key = "</path/to/client/private/key>" # Private key certificate path
|
||||
|
||||
Create a new instance of :py:class:`ccf.clients.CCFClient`, this time specifying the client's certificate and private key as :py:class:`ccf.clients.Identity`:
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET_START: session_authenticated_client
|
||||
:end-before: SNIPPET_END: session_authenticated_client
|
||||
|
||||
The authenticated client can then be used to issue ``POST`` requests, e.g. registering new public and private messages to the default logging application:
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET_START: authenticated_post_requests
|
||||
:end-before: SNIPPET_END: authenticated_post_requests
|
||||
|
||||
It is possible to use the same :py:class:`ccf.clients.CCFClient` instance to wait for the transaction to be committed by the network:
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET: wait_for_commit
|
||||
:lines: 1
|
||||
|
||||
In fact, even an anonymous client can be used to verify that a transaction is committed. This is because only the sequence number and view associated with the transaction are required to verify that a transaction is committed.
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET: any_client_can_wait
|
||||
:lines: 1
|
||||
|
||||
.. warning:: This does not imply that the content of a confidential transaction issued by an authenticated client is visible by an unauthenticated client. Access control to the confidential resource is handled by the CCF application logic.
|
||||
|
||||
Finally, the authenticated client can be used to issue ``GET`` requests and verify that the previous messages have successfully been recorded:
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET_START: authenticated_get_requests
|
||||
:end-before: SNIPPET_END: authenticated_get_requests
|
||||
|
||||
Request Signature Authentication
|
||||
--------------------------------
|
||||
|
||||
Alternatively, the client's identity can be derived from the client signature on the request:
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET_START: signature_authenticated_client
|
||||
:end-before: SNIPPET_END: signature_authenticated_client
|
||||
|
||||
Signed requests can then be submitted the usual way, e.g.:
|
||||
|
||||
.. literalinclude:: ../../python/tutorial.py
|
||||
:language: py
|
||||
:start-after: SNIPPET_START: signed_request
|
||||
:end-before: SNIPPET_END: signed_request
|
||||
|
||||
.. note:: It is possible to set different session and signing identities for a :py:class:`ccf.clients.CCFClient` instance. If the triggered CCF application endpoint has registered both authentication policies, it is up to the application logic to check for the identity type.
|
||||
.. warning:: As of CCF 2.0, the ``ccf`` Python package no longer provides utilities to issue requests to a running CCF service. This is because CCF supports widely-used client-server protocols (TLS, HTTP) that should already be provided by libraries for all programming languages. The ``ccf`` Python package can still be used to audit the ledger and snapshot files (see :doc:`/audit/python_library`).
|
||||
|
|
@ -23,12 +23,12 @@
|
|||
"@types/node": "17.0.8",
|
||||
"@types/node-forge": "^0.10.9",
|
||||
"chai": "^4.3.4",
|
||||
"colors": "1.4.0",
|
||||
"cross-env": "^7.0.3",
|
||||
"mocha": "^9.1.3",
|
||||
"node-forge": "^1.2.0",
|
||||
"ts-node": "^10.4.0",
|
||||
"typedoc": "^0.22.7",
|
||||
"typescript": "^4.2.4",
|
||||
"colors": "1.4.0"
|
||||
"typescript": "^4.2.4"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,122 +0,0 @@
|
|||
# Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
# Licensed under the Apache 2.0 License.
|
||||
|
||||
import sys
|
||||
import http
|
||||
import json
|
||||
import os
|
||||
from loguru import logger as LOG
|
||||
|
||||
# Change default log format
|
||||
LOG.remove()
|
||||
LOG.add(
|
||||
sys.stdout,
|
||||
format="<green>[{time:HH:mm:ss.SSS}]</green> {message}",
|
||||
)
|
||||
|
||||
# SNIPPET: import_clients
|
||||
import ccf.clients
|
||||
|
||||
# Load client info file.
|
||||
if len(sys.argv) < 2:
|
||||
print("Error: Common directory should be specified as first argument")
|
||||
sys.exit(1)
|
||||
|
||||
common_dir = sys.argv[1]
|
||||
|
||||
# Assumes sandbox started with at least one node
|
||||
host = "127.0.0.1"
|
||||
port = 8000
|
||||
ca = os.path.join(common_dir, "networkcert.pem")
|
||||
cert = os.path.join(common_dir, "user0_cert.pem")
|
||||
key = os.path.join(common_dir, "user0_privk.pem")
|
||||
# User client info loaded.
|
||||
|
||||
member_cert = os.path.join(common_dir, "member0_cert.pem")
|
||||
member_key = os.path.join(common_dir, "member0_privk.pem")
|
||||
# Member client info loaded.
|
||||
|
||||
# Tutorial starts below.
|
||||
|
||||
# SNIPPET: anonymous_client
|
||||
anonymous_client = ccf.clients.CCFClient(host, port, ca)
|
||||
|
||||
# SNIPPET_START: anonymous_requests
|
||||
r = anonymous_client.get("/node/state")
|
||||
assert r.status_code == http.HTTPStatus.OK
|
||||
r = anonymous_client.get("/node/network")
|
||||
assert r.status_code == http.HTTPStatus.OK
|
||||
# SNIPPET_END: anonymous_requests
|
||||
|
||||
# SNIPPET_START: session_authenticated_client
|
||||
user_client = ccf.clients.CCFClient(
|
||||
host, port, ca, session_auth=ccf.clients.Identity(key, cert, "session client")
|
||||
)
|
||||
# SNIPPET_END: session_authenticated_client
|
||||
|
||||
# SNIPPET_START: authenticated_post_requests
|
||||
r = user_client.post("/app/log/private", body={"id": 0, "msg": "Private message"})
|
||||
assert r.status_code == http.HTTPStatus.OK
|
||||
r = user_client.post("/app/log/public", body={"id": 0, "msg": "Public message"})
|
||||
assert r.status_code == http.HTTPStatus.OK
|
||||
# SNIPPET_END: authenticated_post_requests
|
||||
|
||||
# SNIPPET: wait_for_commit
|
||||
user_client.wait_for_commit(r)
|
||||
|
||||
# SNIPPET: any_client_can_wait
|
||||
anonymous_client.wait_for_commit(r)
|
||||
|
||||
# SNIPPET_START: authenticated_get_requests
|
||||
r = user_client.get("/app/log/private?id=0")
|
||||
assert r.status_code == http.HTTPStatus.OK
|
||||
assert r.body.json() == {"msg": "Private message"}
|
||||
r = user_client.get("/app/log/public?id=0")
|
||||
assert r.status_code == http.HTTPStatus.OK
|
||||
assert r.body.json() == {"msg": "Public message"}
|
||||
# SNIPPET_END: authenticated_get_requests
|
||||
|
||||
# SNIPPET_START: signature_authenticated_client
|
||||
member_client = ccf.clients.CCFClient(
|
||||
host,
|
||||
port,
|
||||
ca,
|
||||
session_auth=None,
|
||||
signing_auth=ccf.clients.Identity(member_key, member_cert, "sign member client"),
|
||||
)
|
||||
# SNIPPET_END: signature_authenticated_client
|
||||
|
||||
# SNIPPET_START: signed_request
|
||||
r = member_client.post("/gov/ack/update_state_digest")
|
||||
assert r.status_code == http.HTTPStatus.OK
|
||||
# SNIPPET_END: signed_request
|
||||
|
||||
# SNIPPET: import_proposal_generator
|
||||
import ccf.proposal_generator
|
||||
|
||||
# SNIPPET_START: dict_proposal
|
||||
proposal, vote = ccf.proposal_generator.transition_service_to_open()
|
||||
|
||||
member_client = ccf.clients.CCFClient(
|
||||
host,
|
||||
port,
|
||||
ca,
|
||||
session_auth=ccf.clients.Identity(member_key, member_cert, "member"),
|
||||
signing_auth=ccf.clients.Identity(member_key, member_cert, "member"),
|
||||
)
|
||||
response = member_client.post(
|
||||
"/gov/proposals",
|
||||
body=proposal,
|
||||
)
|
||||
# SNIPPET_END: dict_proposal
|
||||
|
||||
# SNIPPET_START: json_proposal_with_file
|
||||
with open("my_open_network_proposal.json", "w", encoding="utf-8") as f:
|
||||
f.write(json.dumps(proposal, indent=2))
|
||||
|
||||
# The contents of `my_open_network_proposal.json` are submitted as the request body.
|
||||
response = member_client.post(
|
||||
"/gov/proposals",
|
||||
body="@my_open_network_proposal.json",
|
||||
)
|
||||
# SNIPPET_END: json_proposal_with_file
|
||||
|
|
@ -5,7 +5,7 @@ import infra.network
|
|||
import infra.proc
|
||||
import time
|
||||
import http
|
||||
from ccf.tx_status import TxStatus
|
||||
from infra.tx_status import TxStatus
|
||||
|
||||
from loguru import logger as LOG
|
||||
|
||||
|
|
|
|||
|
|
@ -9,8 +9,8 @@ import infra.interfaces
|
|||
import contextlib
|
||||
import resource
|
||||
import psutil
|
||||
from ccf.log_capture import flush_info
|
||||
from ccf.clients import CCFConnectionException
|
||||
from infra.log_capture import flush_info
|
||||
from infra.clients import CCFConnectionException
|
||||
import random
|
||||
import http
|
||||
import functools
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import infra.network
|
|||
import suite.test_requirements as reqs
|
||||
import infra.logging_app as app
|
||||
import infra.e2e_args
|
||||
from ccf.tx_status import TxStatus
|
||||
from infra.tx_status import TxStatus
|
||||
import infra.checker
|
||||
import infra.jwt_issuer
|
||||
import inspect
|
||||
|
|
@ -17,8 +17,8 @@ from collections import defaultdict
|
|||
import time
|
||||
import json
|
||||
import hashlib
|
||||
import ccf.clients
|
||||
from ccf.log_capture import flush_info
|
||||
import infra.clients
|
||||
from infra.log_capture import flush_info
|
||||
import ccf.receipt
|
||||
from ccf.tx_id import TxID
|
||||
from cryptography.x509 import load_pem_x509_certificate
|
||||
|
|
@ -592,7 +592,7 @@ def test_historical_query(network, args):
|
|||
with primary.client("user0") as c:
|
||||
r = c.get(
|
||||
"/app/log/private/historical",
|
||||
headers={ccf.clients.CCF_TX_ID_HEADER: "99999.1"},
|
||||
headers={infra.clients.CCF_TX_ID_HEADER: "99999.1"},
|
||||
)
|
||||
assert r.status_code == http.HTTPStatus.NOT_FOUND, r
|
||||
assert r.body.json()["error"]["code"] == "TransactionInvalid", r
|
||||
|
|
@ -601,7 +601,7 @@ def test_historical_query(network, args):
|
|||
with primary.client("user0") as c:
|
||||
r = c.get(
|
||||
"/app/log/private/historical",
|
||||
headers={ccf.clients.CCF_TX_ID_HEADER: "99999.999999"},
|
||||
headers={infra.clients.CCF_TX_ID_HEADER: "99999.999999"},
|
||||
)
|
||||
assert r.status_code == http.HTTPStatus.NOT_FOUND, r
|
||||
assert r.body.json()["error"]["code"] == "TransactionPendingOrUnknown", r
|
||||
|
|
@ -751,7 +751,7 @@ def test_historical_query_range(network, args):
|
|||
|
||||
last_seqno = seqno
|
||||
|
||||
ccf.commit.wait_for_commit(c, seqno=last_seqno, view=view, timeout=3)
|
||||
infra.commit.wait_for_commit(c, seqno=last_seqno, view=view, timeout=3)
|
||||
|
||||
entries_a, _ = get_all_entries(c, id_a)
|
||||
entries_b, _ = get_all_entries(c, id_b)
|
||||
|
|
@ -827,7 +827,7 @@ def test_historical_query_sparse(network, args):
|
|||
|
||||
seqnos.append(seqno)
|
||||
|
||||
ccf.commit.wait_for_commit(c, seqno=seqnos[-1], view=view, timeout=3)
|
||||
infra.commit.wait_for_commit(c, seqno=seqnos[-1], view=view, timeout=3)
|
||||
|
||||
def get_sparse(client, target_id, seqnos, timeout=3):
|
||||
seqnos_s = ",".join(str(n) for n in seqnos)
|
||||
|
|
|
|||
|
|
@ -14,19 +14,12 @@ def run(args):
|
|||
network.start_and_join(args)
|
||||
primary, _ = network.find_primary()
|
||||
|
||||
cmd = [
|
||||
"python",
|
||||
args.client_tutorial,
|
||||
network.common_dir,
|
||||
]
|
||||
rc = infra.proc.ccall(*cmd).returncode
|
||||
assert rc == 0, f"Failed to run tutorial script: {rc}"
|
||||
|
||||
_, committed_ledger_dirs = primary.get_ledger()
|
||||
uncommitted_ledger_dir, committed_ledger_dirs = list(primary.get_ledger())
|
||||
cmd = [
|
||||
"python",
|
||||
args.ledger_tutorial,
|
||||
committed_ledger_dirs[0],
|
||||
*committed_ledger_dirs,
|
||||
uncommitted_ledger_dir,
|
||||
]
|
||||
rc = infra.proc.ccall(*cmd).returncode
|
||||
assert rc == 0, f"Failed to run tutorial script: {rc}"
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
import infra.e2e_args
|
||||
import infra.network
|
||||
import infra.proc
|
||||
import ccf.commit
|
||||
import infra.commit
|
||||
import http
|
||||
from e2e_logging import get_all_entries
|
||||
import cimetrics.upload
|
||||
|
|
@ -87,7 +87,7 @@ def test_historical_query_range(network, args):
|
|||
last_seqno = max(res[2] for res in results)
|
||||
|
||||
with primary.client("user0") as c:
|
||||
ccf.commit.wait_for_commit(c, seqno=last_seqno, view=view, timeout=3)
|
||||
infra.commit.wait_for_commit(c, seqno=last_seqno, view=view, timeout=3)
|
||||
|
||||
LOG.info(
|
||||
f"Total ledger contains {last_seqno} entries, of which we expect our transactions to be spread over a range of ~{last_seqno - first_seqno} transactions"
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
# Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
# Licensed under the Apache 2.0 License.
|
||||
|
||||
from ccf.commit import wait_for_commit
|
||||
from infra.commit import wait_for_commit
|
||||
import pprint
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -26,8 +26,8 @@ from ccf.tx_id import TxID
|
|||
import httpx
|
||||
from loguru import logger as LOG # type: ignore
|
||||
|
||||
import ccf.commit
|
||||
from ccf.log_capture import flush_info
|
||||
import infra.commit
|
||||
from infra.log_capture import flush_info
|
||||
|
||||
|
||||
class HttpSig(httpx.Auth):
|
||||
|
|
@ -115,7 +115,7 @@ class Request:
|
|||
@dataclass
|
||||
class Identity:
|
||||
"""
|
||||
Identity (as private key and corresponding certificate) for a :py:class:`ccf.clients.CCFClient` client.
|
||||
Identity (as private key and corresponding certificate) for a :py:class:`infra.clients.CCFClient` client.
|
||||
"""
|
||||
|
||||
#: Path to file containing private key
|
||||
|
|
@ -191,7 +191,7 @@ class RawResponseBody(ResponseBody):
|
|||
@dataclass
|
||||
class Response:
|
||||
"""
|
||||
Response to request sent via :py:class:`ccf.clients.CCFClient`
|
||||
Response to request sent via :py:class:`infra.clients.CCFClient`
|
||||
"""
|
||||
|
||||
#: Response HTTP status code
|
||||
|
|
@ -268,7 +268,7 @@ def human_readable_size(n):
|
|||
|
||||
class CCFConnectionException(Exception):
|
||||
"""
|
||||
Exception raised if a :py:class:`ccf.clients.CCFClient` instance cannot successfully establish
|
||||
Exception raised if a :py:class:`infra.clients.CCFClient` instance cannot successfully establish
|
||||
a connection with a target CCF node.
|
||||
"""
|
||||
|
||||
|
|
@ -590,7 +590,7 @@ class CCFClient:
|
|||
:param list log_capture: Rather than emit to default handler, capture log lines to list (optional).
|
||||
:param bool allow_redirects: Select whether redirects are followed.
|
||||
|
||||
:return: :py:class:`ccf.clients.Response`
|
||||
:return: :py:class:`infra.clients.Response`
|
||||
"""
|
||||
if not path.startswith("/"):
|
||||
raise ValueError(f"URL path '{path}' is invalid, must start with /")
|
||||
|
|
@ -629,9 +629,9 @@ class CCFClient:
|
|||
def get(self, *args, **kwargs) -> Response:
|
||||
"""
|
||||
Issue ``GET`` request.
|
||||
See :py:meth:`ccf.clients.CCFClient.call`.
|
||||
See :py:meth:`infra.clients.CCFClient.call`.
|
||||
|
||||
:return: :py:class:`ccf.clients.Response`
|
||||
:return: :py:class:`infra.clients.Response`
|
||||
"""
|
||||
if "http_verb" in kwargs:
|
||||
raise ValueError('"http_verb" should not be specified')
|
||||
|
|
@ -642,9 +642,9 @@ class CCFClient:
|
|||
def post(self, *args, **kwargs) -> Response:
|
||||
"""
|
||||
Issue ``POST`` request.
|
||||
See :py:meth:`ccf.clients.CCFClient.call`.
|
||||
See :py:meth:`infra.clients.CCFClient.call`.
|
||||
|
||||
:return: :py:class:`ccf.clients.Response`
|
||||
:return: :py:class:`infra.clients.Response`
|
||||
"""
|
||||
if "http_verb" in kwargs:
|
||||
raise ValueError('"http_verb" should not be specified')
|
||||
|
|
@ -655,9 +655,9 @@ class CCFClient:
|
|||
def put(self, *args, **kwargs) -> Response:
|
||||
"""
|
||||
Issue ``PUT`` request.
|
||||
See :py:meth:`ccf.clients.CCFClient.call`.
|
||||
See :py:meth:`infra.clients.CCFClient.call`.
|
||||
|
||||
:return: :py:class:`ccf.clients.Response`
|
||||
:return: :py:class:`infra.clients.Response`
|
||||
"""
|
||||
if "http_verb" in kwargs:
|
||||
raise ValueError('"http_verb" should not be specified')
|
||||
|
|
@ -668,9 +668,9 @@ class CCFClient:
|
|||
def delete(self, *args, **kwargs) -> Response:
|
||||
"""
|
||||
Issue ``DELETE`` request.
|
||||
See :py:meth:`ccf.clients.CCFClient.call`.
|
||||
See :py:meth:`infra.clients.CCFClient.call`.
|
||||
|
||||
:return: :py:class:`ccf.clients.Response`
|
||||
:return: :py:class:`infra.clients.Response`
|
||||
"""
|
||||
if "http_verb" in kwargs:
|
||||
raise ValueError('"http_verb" should not be specified')
|
||||
|
|
@ -681,9 +681,9 @@ class CCFClient:
|
|||
def head(self, *args, **kwargs) -> Response:
|
||||
"""
|
||||
Issue ``HEAD`` request.
|
||||
See :py:meth:`ccf.clients.CCFClient.call`.
|
||||
See :py:meth:`infra.clients.CCFClient.call`.
|
||||
|
||||
:return: :py:class:`ccf.clients.Response`
|
||||
:return: :py:class:`infra.clients.Response`
|
||||
"""
|
||||
if "http_verb" in kwargs:
|
||||
raise ValueError('"http_verb" should not be specified')
|
||||
|
|
@ -694,9 +694,9 @@ class CCFClient:
|
|||
def options(self, *args, **kwargs) -> Response:
|
||||
"""
|
||||
Issue ``OPTIONS`` request.
|
||||
See :py:meth:`ccf.clients.CCFClient.call`.
|
||||
See :py:meth:`infra.clients.CCFClient.call`.
|
||||
|
||||
:return: :py:class:`ccf.clients.Response`
|
||||
:return: :py:class:`infra.clients.Response`
|
||||
"""
|
||||
if "http_verb" in kwargs:
|
||||
raise ValueError('"http_verb" should not be specified')
|
||||
|
|
@ -708,12 +708,12 @@ class CCFClient:
|
|||
self, response: Response, timeout: int = DEFAULT_COMMIT_TIMEOUT_SEC
|
||||
):
|
||||
"""
|
||||
Given a :py:class:`ccf.clients.Response`, this functions waits
|
||||
Given a :py:class:`infra.clients.Response`, this functions waits
|
||||
for the associated sequence number and view to be committed by the CCF network.
|
||||
|
||||
The client will poll the ``/node/tx`` endpoint until ``COMMITTED`` is returned.
|
||||
|
||||
:param ccf.clients.Response response: Response returned by :py:meth:`ccf.clients.CCFClient.call`
|
||||
:param infra.clients.Response response: Response returned by :py:meth:`infra.clients.CCFClient.call`
|
||||
:param int timeout: Maximum time (secs) to wait for commit before giving up.
|
||||
|
||||
A ``TimeoutError`` exception is raised if the transaction is not committed within ``timeout`` seconds.
|
||||
|
|
@ -721,7 +721,7 @@ class CCFClient:
|
|||
if response.seqno is None or response.view is None:
|
||||
raise ValueError("Response seqno and view should not be None")
|
||||
|
||||
ccf.commit.wait_for_commit(self, response.seqno, response.view, timeout)
|
||||
infra.commit.wait_for_commit(self, response.seqno, response.view, timeout)
|
||||
|
||||
def close(self):
|
||||
self.client_impl.close()
|
||||
|
|
@ -7,8 +7,8 @@ import pprint
|
|||
|
||||
from typing import Optional, List
|
||||
|
||||
from ccf.tx_status import TxStatus
|
||||
from ccf.log_capture import flush_info
|
||||
from infra.tx_status import TxStatus
|
||||
from infra.log_capture import flush_info
|
||||
|
||||
|
||||
def wait_for_commit(
|
||||
|
|
@ -18,7 +18,7 @@ def wait_for_commit(
|
|||
Waits for a specific seqno/view pair to be committed by the network,
|
||||
as per the node to which client is connected to.
|
||||
|
||||
:param client: Instance of :py:class:`ccf.clients.CCFClient`
|
||||
:param client: Instance of :py:class:`infra.clients.CCFClient`
|
||||
:param int seqno: Transaction sequence number.
|
||||
:param int view: Consensus view.
|
||||
:param str timeout: Maximum time to wait for this seqno/view pair to be committed before giving up.
|
||||
|
|
@ -261,7 +261,7 @@ class Consortium:
|
|||
# Wait for proposal completion to be committed, even if no votes are issued
|
||||
if wait_for_global_commit:
|
||||
with remote_node.client() as c:
|
||||
ccf.commit.wait_for_commit(c, seqno, view, timeout=timeout)
|
||||
infra.commit.wait_for_commit(c, seqno, view, timeout=timeout)
|
||||
|
||||
if proposal.state == ProposalState.ACCEPTED:
|
||||
proposal.set_completed(seqno, view)
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ from contextlib import AbstractContextManager
|
|||
import tempfile
|
||||
import json
|
||||
import time
|
||||
from ccf.log_capture import flush_info
|
||||
from infra.log_capture import flush_info
|
||||
from loguru import logger as LOG
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -6,8 +6,8 @@ import infra.jwt_issuer
|
|||
import time
|
||||
import http
|
||||
import random
|
||||
import ccf.clients
|
||||
import ccf.commit
|
||||
import infra.clients
|
||||
import infra.commit
|
||||
from collections import defaultdict
|
||||
from ccf.tx_id import TxID
|
||||
|
||||
|
|
@ -225,7 +225,7 @@ class LoggingTxs:
|
|||
cmd = "/app/log/private/historical"
|
||||
headers.update(
|
||||
{
|
||||
ccf.clients.CCF_TX_ID_HEADER: f"{view}.{seqno}",
|
||||
infra.clients.CCF_TX_ID_HEADER: f"{view}.{seqno}",
|
||||
}
|
||||
)
|
||||
|
||||
|
|
@ -233,7 +233,7 @@ class LoggingTxs:
|
|||
start_time = time.time()
|
||||
while time.time() < (start_time + timeout):
|
||||
with node.client(self.user) as c:
|
||||
ccf.commit.wait_for_commit(
|
||||
infra.commit.wait_for_commit(
|
||||
c, seqno, view, timeout, log_capture=log_capture
|
||||
)
|
||||
|
||||
|
|
@ -278,7 +278,7 @@ class LoggingTxs:
|
|||
headers = self._get_headers_base()
|
||||
headers.update(
|
||||
{
|
||||
ccf.clients.CCF_TX_ID_HEADER: f"{view}.{seqno}",
|
||||
infra.clients.CCF_TX_ID_HEADER: f"{view}.{seqno}",
|
||||
}
|
||||
)
|
||||
|
||||
|
|
@ -286,7 +286,7 @@ class LoggingTxs:
|
|||
start_time = time.time()
|
||||
while time.time() < (start_time + timeout):
|
||||
with node.client(self.user) as c:
|
||||
ccf.commit.wait_for_commit(c, seqno, view, timeout)
|
||||
infra.commit.wait_for_commit(c, seqno, view, timeout)
|
||||
|
||||
rep = c.get(f"{cmd}?id={idx}", headers=headers)
|
||||
if rep.status_code == http.HTTPStatus.OK:
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ from enum import Enum
|
|||
import infra.proc
|
||||
import infra.proposal
|
||||
import infra.crypto
|
||||
import ccf.clients
|
||||
import infra.clients
|
||||
import http
|
||||
import os
|
||||
import base64
|
||||
|
|
@ -218,4 +218,4 @@ class Member:
|
|||
log_output=True,
|
||||
)
|
||||
res.check_returncode()
|
||||
return ccf.clients.Response.from_raw(res.stdout)
|
||||
return infra.clients.Response.from_raw(res.stdout)
|
||||
|
|
|
|||
|
|
@ -5,13 +5,13 @@ import time
|
|||
import logging
|
||||
from contextlib import contextmanager
|
||||
from enum import Enum, IntEnum, auto
|
||||
from ccf.clients import CCFConnectionException, flush_info
|
||||
from infra.clients import CCFConnectionException, flush_info
|
||||
import infra.path
|
||||
import infra.proc
|
||||
import infra.node
|
||||
import infra.consortium
|
||||
from ccf.ledger import NodeStatus, Ledger, COMMITTED_FILE_SUFFIX
|
||||
from ccf.tx_status import TxStatus
|
||||
from infra.tx_status import TxStatus
|
||||
from ccf.tx_id import TxID
|
||||
import random
|
||||
from dataclasses import dataclass
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ from datetime import datetime, timedelta
|
|||
import infra.net
|
||||
import infra.path
|
||||
import infra.interfaces
|
||||
import ccf.clients
|
||||
import infra.clients
|
||||
import ccf.ledger
|
||||
import os
|
||||
import socket
|
||||
|
|
@ -417,7 +417,7 @@ class Node:
|
|||
rep.status_code == 200
|
||||
), f"An error occured after node {self.local_node_id} joined the network: {rep.body}"
|
||||
self.network_state = infra.node.NodeNetworkState.joined
|
||||
except ccf.clients.CCFConnectionException as e:
|
||||
except infra.clients.CCFConnectionException as e:
|
||||
raise TimeoutError(
|
||||
f"Node {self.local_node_id} failed to join the network"
|
||||
) from e
|
||||
|
|
@ -472,7 +472,7 @@ class Node:
|
|||
|
||||
def identity(self, name=None):
|
||||
if name is not None:
|
||||
return ccf.clients.Identity(
|
||||
return infra.clients.Identity(
|
||||
os.path.join(self.common_dir, f"{name}_privk.pem"),
|
||||
os.path.join(self.common_dir, f"{name}_cert.pem"),
|
||||
name,
|
||||
|
|
@ -528,7 +528,7 @@ class Node:
|
|||
)
|
||||
raise
|
||||
|
||||
return ccf.clients.client(
|
||||
return infra.clients.client(
|
||||
rpc_interface.public_host, rpc_interface.public_port, **akwargs
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ def test_missing_signature_header(network, args):
|
|||
|
||||
|
||||
def make_signature_corrupter(fn):
|
||||
class SignatureCorrupter(ccf.clients.HttpSig):
|
||||
class SignatureCorrupter(infra.clients.HttpSig):
|
||||
def auth_flow(self, request):
|
||||
yield fn(next(super(SignatureCorrupter, self).auth_flow(request)))
|
||||
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ import infra.logging_app as app
|
|||
import suite.test_requirements as reqs
|
||||
from infra.checker import check_can_progress, check_does_not_progress
|
||||
import pprint
|
||||
from ccf.tx_status import TxStatus
|
||||
from infra.tx_status import TxStatus
|
||||
import time
|
||||
import http
|
||||
import contextlib
|
||||
|
|
|
|||
|
|
@ -158,11 +158,6 @@ if __name__ == "__main__":
|
|||
help="List all discovered methods at the end of the run",
|
||||
action="store_true",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--client-tutorial",
|
||||
help="Path to client tutorial file",
|
||||
type=str,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--ledger-tutorial",
|
||||
help="Path to ledger tutorial file",
|
||||
|
|
|
|||
|
|
@ -59,12 +59,10 @@ if poll_for_service_open ${network_live_time}; then
|
|||
exit 1
|
||||
fi
|
||||
|
||||
# Issue tutorial transactions to ephemeral network
|
||||
python3.8 -m venv env
|
||||
# shellcheck source=/dev/null
|
||||
source env/bin/activate
|
||||
python -m pip install -e ../../../python
|
||||
python ../../../python/tutorial.py ./workspace/sandbox_common/
|
||||
|
||||
# Test Python package CLI
|
||||
../../../tests/test_python_cli.sh > test_python_cli.out
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче