Azure
/
azure-sdk-for-cpp
зеркало из https://github.com/Azure/azure-sdk-for-cpp.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Create EventHubs stress and performance test basics. (#4818) 

* Scaffolding for eventhubs perf test

* Added eventhubs 'stress' test, which is really a totally broken perf test'

* Added stress test skeleton

* EventHubs Stress test is an actual eventhubs test now

* EventHubs throws exceptions on producer client failure. Rationalized error reporting from AMQP - message sender and receiver generate the same errors.

* Rationalize AMQP error results; throw exceptions on eventhubs errors

* Fixed amqp management test; added HTTP status code for management triggered exceptions

* Event Data binary no longer takes an AMQP binary; Preliminary readme.md for eventhubs (still many go constructs in the readme)

* Created the first eventhubs samples - they don't do much but they work

Co-authored-by: Ahson Khan <ahkha@microsoft.com>

Co-authored-by: Rick Winter <rick.winter@microsoft.com>

* API Review feedback

* Added isTransient flag to eventhubs exception based on Java implementation

* Updated AMQP changelog to reflect recent changes

---------

Co-authored-by: Ahson Khan <ahkha@microsoft.com>
Co-authored-by: Rick Winter <rick.winter@microsoft.com>
This commit is contained in:
Larry Osterman 2023-08-04 11:40:30 -07:00 коммит произвёл GitHub
Родитель 6d583899d0
Коммит 9fbc3856ad
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
90 изменённых файлов: 3530 добавлений и 1022 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

6
.gitignore поставляемый
Просмотреть файл

@ -10,6 +10,12 @@
*.sln.docstates
*.runsettings
# Stress deploy.ps1 script generated files
generatedValues.yaml
stress-test-resources.json
stress-test-addons*.tgz
Chart.lock
# User-specific files (MonoDevelop/Xamarin Studio)
*.userprefs

3
sdk/core/azure-core-amqp/CHANGELOG.md
Просмотреть файл

@ -9,10 +9,13 @@
- Collection types (`AmqpArray`, `AmqpMap`, `AmqpList`, `AmqpBinaryData`, `AmqpSymbol` and `AmqpComposite`):
- Added explicit cast operator to underlying collection type.
- Added `find()`.
- Rationalized the return code for AMQP MessageSender and MessageReceiver and Management APIs to use AmqpError for error codes.
- Added additional AMQP Error values.
### Breaking Changes
- Renamed `Azure::Core::Amqp::Models::AmqpMessageFormatValue` to `AmqpDefaultMessageFormatValue`.
- Changed the return values for the MessageSender, MessageReceiver and Management APIs.
### Bugs Fixed

1
sdk/core/azure-core-amqp/inc/ApiViewSettings.json
Просмотреть файл

@ -16,7 +16,6 @@
"-D_ALLOW_COMPILER_AND_STL_VERSION_MISMATCH"
],
"allowInternal": true,
"includeInternal": true,
"includeDetail": false,
"includePrivate": false,
"filterNamespace": "Azure::",

6
sdk/core/azure-core-amqp/inc/azure/core/amqp/management.hpp
Просмотреть файл

@ -109,15 +109,17 @@ namespace Azure { namespace Core { namespace Amqp { namespace _internal {
* @brief The status of the operation.
*/
ManagementOperationStatus Status = ManagementOperationStatus::Invalid;
/**
* @brief The response message from the operation, if Status is ManagementOperationStatus::Ok.
*/
Models::AmqpMessage Message;
/**
* @brief The description of the operation, if Status is ManagementOperationStatus::Error.
* @brief The error code associated with the message, if Status is
* ManagementOperationStatus::Error.
*/
std::string Description;
Models::_internal::AmqpError Error;
/**
* @brief The HTTP status code of the operation, if Status is ManagementOperationStatus::Error.

3
sdk/core/azure-core-amqp/inc/azure/core/amqp/message_receiver.hpp
Просмотреть файл

@ -74,6 +74,9 @@ namespace Azure { namespace Core { namespace Amqp { namespace _internal {
/** @brief The Maximum message size for the link associated with the message receiver. */
Nullable<uint64_t> MaxMessageSize;
/** @brief Attach properties for the link associated with the message receiver. */
Models::AmqpMap Properties;
/** @brief If true, the message receiver will generate low level events */
bool EnableTrace{false};

2
sdk/core/azure-core-amqp/inc/azure/core/amqp/message_sender.hpp
Просмотреть файл

@ -137,7 +137,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace _internal {
*
* @return A tuple containing the status of the send operation and the send disposition.
*/
std::tuple<MessageSendStatus, Models::AmqpValue> Send(
std::tuple<MessageSendStatus, Models::_internal::AmqpError> Send(
Models::AmqpMessage const& message,
Context const& context = {});

155
sdk/core/azure-core-amqp/inc/azure/core/amqp/models/amqp_error.hpp
Просмотреть файл

@ -158,6 +158,161 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models { namespace
*
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition FrameSizeTooSmall;
/**
* The link has been attached elsewhere, causing the existing attachment to be forcibly
* closed.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition LinkStolen;
/**
* The peer sent a larger message than is supported on the link.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition LinkPayloadSizeExceeded;
/**
* An operator intervened to detach for some reason.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition LinkDetachForced;
/**
* An operator intervened to close the connection for some reason. The client could retry at
* some later date.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition ConnectionForced;
// These are errors that are specific to Azure services.
/**
* The server is busy.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition ServerBusyError;
/**
* One or more arguments supplied to the method are invalid.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition ArgumentError;
/**
* One or more arguments supplied to the method are invalid.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition ArgumentOutOfRangeError;
/**
* Request for a runtime operation on a disabled entity.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition EntityDisabledError;
/**
* Partition is not owned.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition PartitionNotOwnedError;
/**
* Lock token associated with the message or session has expired, or the lock token is not
* found.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition StoreLockLostError;
/**
* The TokenProvider object could not acquire a token, the token is invalid, or the token
* does not contain the claims required to perform the operation.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition PublisherRevokedError;
/**
* The server did not respond to the requested operation within the specified time. The
* server may have completed the requested operation. This can happen due to network or
* other infrastructure delays.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition TimeoutError;
/**
* Tracking Id for an exception.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition TrackingIdProperty;
/**
* IO exceptions that occur in proton-j library.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition ProtonIo;
/**
* A connection error occurred. A valid frame header cannot be formed from the incoming byte
* stream.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition ConnectionFramingError;
/**
* The operation was cancelled.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition OperationCancelled;
/**
* Error condition when receiver attempts {@code complete}, {@code abandon}, {@code
* renewLock}, {@code deadLetter}, or {@code defer} on a peek-locked message whose lock had
* already expired.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition MessageLockLost;
/**
* Error condition when a session receiver performs an operation on a session after its lock
* is expired. When a client accepts a session, the session is locked to the receiver for a
* duration specified in the entity definition. When the accepted session remains idle for
* the duration of lock, that is no operations performed on the session, the lock expires
* and the session is made available to other clients.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition SessionLockLost;
/**
* Error condition when a client attempts to accept a session that is already locked by
* another client.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition SessionCannotBeLocked;
/**
* Error condition when a receiver attempts to receive a message with sequence number and
* the message with that sequence number is not available in the queue or subscription.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition MessageNotFound;
/**
* Error condition when a receiver attempts to receive from a session that does not exist.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition SessionNotFound;
/**
* Error condition when a subscription client tries to create a rule with the name of an
* already existing rule.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition EntityAlreadyExists;
/**
* The container is no longer available on the current connection. The peer SHOULD attempt
* reconnection to the container using the details provided in the info map.
*
* The address provided cannot be resolved to a terminus at the current container. The info
* map MAY contain the following information to allow the client to locate the attach to the
* terminus.
*
* hostname:
* the hostname of the container. This is the value that SHOULD be supplied in the hostname
* field of the open frame, and during the SASL and TLS negotiation (if used).
*
* network-host:
* the DNS hostname or IP address of the machine hosting the container.
*
* port:
* the port number on the machine hosting the container.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition ConnectionRedirect;
/**
* The address provided cannot be resolved to a terminus at the current container. The info
* map MAY contain the following information to allow the client to locate the attach to the
* terminus.
*
* hostname:
* the hostname of the container hosting the terminus. This is the value that SHOULD be
* supplied in the hostname field of the open frame, and during SASL and TLS negotiation (if
* used).
*
* network-host:
* the DNS hostname or IP address of the machine hosting the container.
*
* port:
* the port number on the machine hosting the container.
*
* address:
* the address of the terminus at the container.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition LinkRedirect;
/**
* The peer sent more message transfers than currently allowed on the link.
*/
AZ_CORE_AMQP_DLLEXPORT static const AmqpErrorCondition TransferLimitExceeded;
};
struct AmqpError final

2
sdk/core/azure-core-amqp/samples/internal/eventhub_get_properties_sample/eventhub_get_eventhub_properties_sample.cpp
Просмотреть файл

@ -122,7 +122,7 @@ std::tuple<bool, EventHubPartitionProperties> GetPartitionProperties(
bool error{false};
if (result.Status == Azure::Core::Amqp::_internal::ManagementOperationStatus::Error)
{
std::cerr << "Error: " << result.Description;
std::cerr << "Error: " << result.Error.Description;
error = true;
}
else

1
sdk/core/azure-core-amqp/src/amqp/connection.cpp
Просмотреть файл

@ -143,7 +143,6 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
}
else if (options.Port == _internal::AmqpTlsPort)
{
Log::Write(Logger::Level::Informational, "Creating TLS socket connection transport.");
m_transport = Network::_internal::TlsTransportFactory::Create(m_hostName, m_port).GetImpl();
}
else

43
sdk/core/azure-core-amqp/src/amqp/management.cpp
Просмотреть файл

@ -240,12 +240,16 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
messageToSend,
[&](_internal::MessageSendStatus sendStatus, Models::AmqpValue const& deliveryState) {
m_sendCompleted = true;
Log::Stream(Logger::Level::Informational)
<< "Management operation send complete. Status: " << static_cast<int>(sendStatus)
<< ", DeliveryState: " << deliveryState;
if (m_options.EnableTrace)
{
Log::Stream(Logger::Level::Informational)
<< "Management operation send complete. Status: " << static_cast<int>(sendStatus)
<< ", DeliveryState: " << deliveryState;
}
Models::_internal::AmqpError error;
if (sendStatus != _internal::MessageSendStatus::Ok)
{
std::string errorDescription = "Send failed.";
error.Description = "Send failed.";
auto deliveryStateAsList{deliveryState.AsList()};
Models::AmqpValue firstState{deliveryStateAsList[0]};
ERROR_HANDLE errorHandle;
@ -253,15 +257,11 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
{
Models::_internal::UniqueAmqpErrorHandle uniqueError{
errorHandle}; // This will free the error handle when it goes out of scope.
Models::_internal::AmqpError error{
Models::_internal::AmqpErrorFactory::FromUamqp(errorHandle)};
errorDescription = error.Description;
error = Models::_internal::AmqpErrorFactory::FromUamqp(errorHandle);
}
m_messageQueue.CompleteOperation(
_internal::ManagementOperationStatus::Error,
500,
errorDescription,
Models::AmqpMessage{});
_internal::ManagementOperationStatus::Error, 500, error, Models::AmqpMessage{});
}
},
context);
@ -271,7 +271,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
_internal::ManagementOperationResult rv;
rv.Status = std::get<0>(*result);
rv.StatusCode = std::get<1>(*result);
rv.Description = std::get<2>(*result);
rv.Error = std::get<2>(*result);
rv.Message = std::get<3>(*result);
return rv;
}
@ -583,20 +583,21 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
std::string const& condition,
std::string const& description)
{
Models::_internal::AmqpError error;
error.Condition = Models::_internal::AmqpErrorCondition(condition);
error.Description = "Message Delivery Rejected: " + description;
Log::Stream(Logger::Level::Error)
<< "Indicate Management Error: " << condition << " " << description;
<< "Indicate Management Error: " << condition << " - " << description;
if (m_eventHandler)
{
// Let external callers know that the error was triggered.
Models::_internal::AmqpError error;
error.Condition = Models::_internal::AmqpErrorCondition(condition);
error.Description = "Message Delivery Rejected: " + description;
m_eventHandler->OnError(error);
}
// Complete any outstanding receives with an error.
m_messageQueue.CompleteOperation(
_internal::ManagementOperationStatus::Error, 500, description, Models::AmqpMessage());
_internal::ManagementOperationStatus::Error, 500, error, Models::AmqpMessage());
return Models::_internal::Messaging::DeliveryRejected(condition, description);
}
@ -668,17 +669,21 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
Log::Stream(Logger::Level::Error) << "Received message before send completed.";
}
Models::_internal::AmqpError messageError;
messageError.Description = description;
messageError.Condition = Models::_internal::AmqpErrorCondition::NotAllowed;
// AMQP management statusCode values are [RFC
// 2616](https://www.rfc-editor.org/rfc/rfc2616#section-6.1.1) status codes.
if ((statusCode < 200) || (statusCode > 299))
{
m_messageQueue.CompleteOperation(
_internal::ManagementOperationStatus::FailedBadStatus, statusCode, description, message);
_internal::ManagementOperationStatus::FailedBadStatus, statusCode, messageError, message);
}
else
{
m_messageQueue.CompleteOperation(
_internal::ManagementOperationStatus::Ok, statusCode, description, message);
_internal::ManagementOperationStatus::Ok, statusCode, messageError, message);
}
return Models::_internal::Messaging::DeliveryAccepted();
}

11
sdk/core/azure-core-amqp/src/amqp/message_receiver.cpp
Просмотреть файл

@ -133,6 +133,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
{
m_link->SetMaxMessageSize(std::numeric_limits<uint64_t>::max());
}
m_link->SetAttachProperties(static_cast<Models::AmqpValue>(m_options.Properties));
}
AMQP_VALUE MessageReceiverImpl::OnMessageReceivedFn(const void* context, MESSAGE_HANDLE message)
@ -241,10 +242,12 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
}
else
{
Log::Stream(Logger::Level::Verbose)
<< "Message receiver changed state. New: " << MESSAGE_RECEIVER_STATEStrings[newState]
<< " Old: " << MESSAGE_RECEIVER_STATEStrings[oldState];
;
if (receiver->m_options.EnableTrace)
{
Log::Stream(Logger::Level::Verbose)
<< "Message receiver changed state. New: " << MESSAGE_RECEIVER_STATEStrings[newState]
<< " Old: " << MESSAGE_RECEIVER_STATEStrings[oldState];
}
}
// If we are transitioning to the error state, we want to stick a response on the incoming

41
sdk/core/azure-core-amqp/src/amqp/message_sender.cpp
Просмотреть файл

@ -37,7 +37,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace _internal {
void MessageSender::Open(Context const& context) { m_impl->Open(context); }
void MessageSender::Close() { m_impl->Close(); }
std::tuple<MessageSendStatus, Models::AmqpValue> MessageSender::Send(
std::tuple<MessageSendStatus, Models::_internal::AmqpError> MessageSender::Send(
Models::AmqpMessage const& message,
Context const& context)
{
@ -129,7 +129,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
// Cache the error we received in the OnDetach notification so we can return it to the user
// on the next send which fails.
m_savedMessageError = Models::_internal::AmqpErrorFactory::ToAmqp(error);
m_savedMessageError = error;
});
}
@ -281,19 +281,22 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
(void)context;
}
std::tuple<_internal::MessageSendStatus, Models::AmqpValue> MessageSenderImpl::Send(
std::tuple<_internal::MessageSendStatus, Models::_internal::AmqpError> MessageSenderImpl::Send(
Models::AmqpMessage const& message,
Context const& context)
{
Azure::Core::Amqp::Common::_internal::
AsyncOperationQueue<Azure::Core::Amqp::_internal::MessageSendStatus, Models::AmqpValue>
sendCompleteQueue;
Azure::Core::Amqp::Common::_internal::AsyncOperationQueue<
Azure::Core::Amqp::_internal::MessageSendStatus,
Models::_internal::AmqpError>
sendCompleteQueue;
QueueSend(
message,
[&sendCompleteQueue, this](
Azure::Core::Amqp::_internal::MessageSendStatus sendResult,
Models::AmqpValue deliveryStatus) {
Models::_internal::AmqpError error;
// If the send failed. then we need to return the error. If the send completed because
// of an error, it's possible that the deliveryStatus provided is null. In that case,
// we use the cached saved error because it is highly likely to be better than
@ -302,16 +305,36 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
{
if (deliveryStatus.IsNull())
{
deliveryStatus = m_savedMessageError;
error = m_savedMessageError;
}
else
{
if (deliveryStatus.GetType() != Models::AmqpValueType::List)
{
throw std::runtime_error("Delivery status is not a list");
}
auto deliveryStatusAsList{deliveryStatus.AsList()};
if (deliveryStatusAsList.size() != 1)
{
throw std::runtime_error("Delivery Status list is not of size 1");
}
Models::AmqpValue firstState{deliveryStatusAsList[0]};
ERROR_HANDLE errorHandle;
if (!amqpvalue_get_error(firstState, &errorHandle))
{
Models::_internal::UniqueAmqpErrorHandle uniqueError{
errorHandle}; // This will free the error handle when it goes out of scope.
error = Models::_internal::AmqpErrorFactory::FromUamqp(errorHandle);
}
}
}
else
{
// If we successfully sent the message, then whatever saved error should be cleared,
// it's no longer valid.
m_savedMessageError = Models::AmqpValue();
m_savedMessageError = Models::_internal::AmqpError();
}
sendCompleteQueue.CompleteOperation(sendResult, deliveryStatus);
sendCompleteQueue.CompleteOperation(sendResult, error);
},
context);
auto result = sendCompleteQueue.WaitForPolledResult(context, *m_session->GetConnection());

2
sdk/core/azure-core-amqp/src/amqp/private/management_impl.hpp
Просмотреть файл

@ -125,7 +125,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
Azure::Core::Amqp::Common::_internal::AsyncOperationQueue<
_internal::ManagementOperationStatus,
std::uint32_t,
std::string,
Models::_internal::AmqpError,
Models::AmqpMessage>
m_messageQueue;

4
sdk/core/azure-core-amqp/src/amqp/private/message_sender_impl.hpp
Просмотреть файл

@ -54,7 +54,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
void Open(Context const& context);
void Close();
std::tuple<_internal::MessageSendStatus, Models::AmqpValue> Send(
std::tuple<_internal::MessageSendStatus, Models::_internal::AmqpError> Send(
Models::AmqpMessage const& message,
Context const& context);
void QueueSend(
@ -75,7 +75,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
UniqueMessageSender m_messageSender{};
std::shared_ptr<_detail::LinkImpl> m_link;
_internal::MessageSenderEvents* m_events;
Models::AmqpValue m_savedMessageError;
Models::_internal::AmqpError m_savedMessageError;
Azure::Core::Amqp::Common::_internal::AsyncOperationQueue<Models::AmqpMessage> m_messageQueue;

2
sdk/core/azure-core-amqp/src/amqp/session.cpp
Просмотреть файл

@ -238,8 +238,6 @@ namespace Azure { namespace Core { namespace Amqp { namespace _detail {
m_claimsBasedSecurity = std::make_shared<ClaimsBasedSecurityImpl>(shared_from_this());
}
auto accessToken = GetConnection()->GetSecurityToken(audience, context);
Log::Stream(Logger::Level::Informational)
<< "Authenticate with audience: " << audience << ", token: " << accessToken;
m_claimsBasedSecurity->SetTrace(GetConnection()->EnableTrace());
if (!m_cbsOpen)

37
sdk/core/azure-core-amqp/src/models/amqp_error.cpp
Просмотреть файл

@ -91,4 +91,41 @@ namespace Azure { namespace Core { namespace Amqp { namespace Models { namespace
const AmqpErrorCondition AmqpErrorCondition::ResourceLocked(amqp_error_resource_locked);
const AmqpErrorCondition AmqpErrorCondition::UnauthorizedAccess(amqp_error_unauthorized_access);
const AmqpErrorCondition AmqpErrorCondition::LinkStolen("amqp:link:stolen");
const AmqpErrorCondition AmqpErrorCondition::LinkPayloadSizeExceeded(
"amqp:link:message-size-exceeded");
const AmqpErrorCondition AmqpErrorCondition::LinkDetachForced("amqp:link:detach-forced");
const AmqpErrorCondition AmqpErrorCondition::ConnectionForced("amqp:connection:forced");
// These are errors that are specific to Azure services.
const AmqpErrorCondition AmqpErrorCondition::ServerBusyError("com.microsoft:server-busy");
const AmqpErrorCondition AmqpErrorCondition::ArgumentError("com.microsoft:argument-error");
const AmqpErrorCondition AmqpErrorCondition::ArgumentOutOfRangeError(
"com.microsoft:argument-out-of-range");
const AmqpErrorCondition AmqpErrorCondition::EntityDisabledError("com.microsoft:entity-disabled");
const AmqpErrorCondition AmqpErrorCondition::PartitionNotOwnedError(
"com.microsoft:partition-not-owned");
const AmqpErrorCondition AmqpErrorCondition::StoreLockLostError("com.microsoft:store-lock-lost");
const AmqpErrorCondition AmqpErrorCondition::PublisherRevokedError(
"com.microsoft:publisher-revoked");
const AmqpErrorCondition AmqpErrorCondition::TimeoutError("com.microsoft:timeout");
const AmqpErrorCondition AmqpErrorCondition::TrackingIdProperty("com.microsoft:tracking-id");
const AmqpErrorCondition AmqpErrorCondition::ProtonIo("proton:io");
const AmqpErrorCondition AmqpErrorCondition::ConnectionFramingError(
"amqp:connection:framing-error");
const AmqpErrorCondition AmqpErrorCondition::OperationCancelled(
"com.microsoft:operation-cancelled");
const AmqpErrorCondition AmqpErrorCondition::MessageLockLost("com.microsoft:message-lock-lost");
const AmqpErrorCondition AmqpErrorCondition::SessionLockLost("com.microsoft:session-lock-lost");
const AmqpErrorCondition AmqpErrorCondition::SessionCannotBeLocked(
"com.microsoft:session-cannot-be-locked");
const AmqpErrorCondition AmqpErrorCondition::MessageNotFound("com.microsoft:message-not-found");
const AmqpErrorCondition AmqpErrorCondition::SessionNotFound("com.microsoft:session-not-found");
const AmqpErrorCondition AmqpErrorCondition::EntityAlreadyExists(
"com.microsoft:entity-already-exists");
const AmqpErrorCondition AmqpErrorCondition::ConnectionRedirect("amqp:connection:redirect");
const AmqpErrorCondition AmqpErrorCondition::LinkRedirect("amqp:link:redirect");
const AmqpErrorCondition AmqpErrorCondition::TransferLimitExceeded(
"amqp:link:transfer-limit-exceeded");
}}}}} // namespace Azure::Core::Amqp::Models::_internal

10
sdk/core/azure-core-amqp/test/ut/amqp_value_tests.cpp
Просмотреть файл

@ -1013,7 +1013,7 @@ TEST_F(TestValueSerialization, SerializeBinary)
EXPECT_EQ(value.GetType(), AmqpValueType::Binary);
EXPECT_EQ(value.AsBinary().size(), 16);
auto binary(value.AsBinary());
std::array<uint8_t, 16> valueAsArray;
std::array<uint8_t, 16> valueAsArray{};
std::copy_n(binary.begin(), 16, valueAsArray.begin());
EXPECT_EQ(valueAsArray, testUuid.AsArray());
@ -1030,7 +1030,7 @@ TEST_F(TestValueSerialization, SerializeBinary)
EXPECT_EQ(value.GetType(), AmqpValueType::Binary);
EXPECT_EQ(value.AsBinary().size(), 16);
auto binary(value.AsBinary());
std::array<uint8_t, 16> valueAsArray;
std::array<uint8_t, 16> valueAsArray{};
std::copy_n(binary.begin(), 16, valueAsArray.begin());
EXPECT_EQ(valueAsArray, testUuid.AsArray());
@ -1434,7 +1434,8 @@ TEST_F(TestValueSerialization, SerializeArray)
{
values.push_back(AmqpValue(static_cast<int64_t>(GenerateRandomValue<long long>())));
}
size_t totalSize = 4 + 1; // Include the size of the list count in the size
size_t totalSize
= (static_cast<size_t>(4) + 1); // Include the size of the list count in the size
for (auto const& val : values)
{
totalSize
@ -1479,7 +1480,8 @@ TEST_F(TestValueSerialization, SerializeArray)
{
values.push_back(AmqpValue(static_cast<int64_t>(GenerateRandomValue<long long>())));
}
size_t totalSize = 4 + 1; // Include the size of the list count in the size
size_t totalSize
= (static_cast<size_t>(4) + 1); // Include the size of the list count in the size
for (auto const& val : values)
{
totalSize += (AmqpValue::GetSerializedSize(val) - 1);

16
sdk/core/azure-core-amqp/test/ut/management_tests.cpp
Просмотреть файл

@ -217,7 +217,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace Tests {
auto response = management.ExecuteOperation("Test", "Test", "Test", messageToSend);
EXPECT_EQ(response.Status, ManagementOperationStatus::Ok);
EXPECT_EQ(response.StatusCode, 200);
EXPECT_EQ(response.Description, "Successful");
EXPECT_EQ(response.Error.Description, "Successful");
management.Close();
mockServer.StopListening();
@ -252,7 +252,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace Tests {
auto response = management.ExecuteOperation("Test", "Test", "Test", messageToSend);
EXPECT_EQ(response.Status, ManagementOperationStatus::FailedBadStatus);
EXPECT_EQ(response.StatusCode, 500);
EXPECT_EQ(response.Description, "Bad Things Happened.");
EXPECT_EQ(response.Error.Description, "Bad Things Happened.");
management.Close();
mockServer.StopListening();
@ -288,7 +288,9 @@ namespace Azure { namespace Core { namespace Amqp { namespace Tests {
auto response = management.ExecuteOperation("Test", "Type", "Locales", messageToSend);
EXPECT_EQ(response.Status, ManagementOperationStatus::Error);
EXPECT_EQ(response.StatusCode, 500);
EXPECT_EQ(response.Description, "Received message statusCode value is not an int.");
EXPECT_EQ(
response.Error.Description,
"Message Delivery Rejected: Received message statusCode value is not an int.");
management.Close();
mockServer.StopListening();
@ -339,7 +341,9 @@ namespace Azure { namespace Core { namespace Amqp { namespace Tests {
EXPECT_EQ(response.Status, ManagementOperationStatus::Error);
EXPECT_EQ(response.StatusCode, 500);
EXPECT_EQ(
response.Description, "Received message does not have a statusCode status code key.");
response.Error.Description,
"Message Delivery Rejected: Received message does not have a statusCode status code "
"key.");
EXPECT_TRUE(managementEvents.Error);
management.Close();
@ -379,7 +383,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace Tests {
auto response = management.ExecuteOperation("Test", "Type", "Locales", messageToSend);
EXPECT_EQ(response.Status, ManagementOperationStatus::Ok);
EXPECT_EQ(response.StatusCode, 235);
EXPECT_EQ(response.Description, "Bad Things Happened..");
EXPECT_EQ(response.Error.Description, "Bad Things Happened..");
management.Close();
mockServer.StopListening();
@ -419,7 +423,7 @@ namespace Azure { namespace Core { namespace Amqp { namespace Tests {
std::chrono::system_clock::now() + std::chrono::seconds(10)));
EXPECT_EQ(response.Status, ManagementOperationStatus::Error);
EXPECT_EQ(response.StatusCode, 500);
EXPECT_EQ(response.Description, "Unknown Request operation");
EXPECT_EQ(response.Error.Description, "Unknown Request operation");
management.Close();
mockServer.StopListening();

3
sdk/core/ci.yml
Просмотреть файл

@ -126,6 +126,9 @@ stages:
Value: https://NotRealAttestationInstanceiso.wus.attest.azure.net
- Name: ATTESTATION_AAD_URL
Value: https://NotRealAttestationInstanceaad.wus.attest.azure.net
# EventHubs
- Name: CHECKPOINTSTORE_STORAGE_CONNECTION_STRING
Value: "DefaultEndpointsProtocol=https;AccountName=notReal;AccountKey=3333333333333333333333333333333333333333333333333333333333333333333333333333333333333333;EndpointSuffix=core.windows.net"
CMakeTestOptions:
- Name: Default

2
sdk/eventhubs/assets.json
Просмотреть файл

@ -2,5 +2,5 @@
"AssetsRepo": "Azure/azure-sdk-assets",
"AssetsRepoPrefixPath": "cpp",
"TagPrefix": "cpp/eventhubs",
"Tag": ""
"Tag": "cpp/eventhubs_3d146641d9"
}

22
sdk/eventhubs/azure-messaging-eventhubs/CMakeLists.txt
Просмотреть файл

@ -59,10 +59,10 @@ set(
AZURE_MESSAGING_EVENTHUBS_HEADER
inc/azure/messaging/eventhubs.hpp
inc/azure/messaging/eventhubs/checkpoint_store.hpp
inc/azure/messaging/eventhubs/eventhub_constants.hpp
inc/azure/messaging/eventhubs/consumer_client.hpp
inc/azure/messaging/eventhubs/dll_import_export.hpp
inc/azure/messaging/eventhubs/event_data_batch.hpp
inc/azure/messaging/eventhubs/eventhubs_exception.hpp
inc/azure/messaging/eventhubs/models/checkpoint_store_models.hpp
inc/azure/messaging/eventhubs/models/consumer_client_models.hpp
inc/azure/messaging/eventhubs/models/event_data.hpp
@ -84,16 +84,22 @@ set(
src/consumer_client.cpp
src/event_data.cpp
src/event_data_batch.cpp
src/partition_client.cpp
src/partition_client_models.cpp
src/private/package_version.hpp
src/private/eventhubs_constants.hpp
src/private/eventhubs_utilities.hpp
src/processor_load_balancer.cpp
src/processor_partition_client.cpp
src/producer_client.cpp
src/retry_operation.cpp
src/eventhubs_utilities.cpp
)
add_library(
azure-messaging-eventhubs
${AZURE_MESSAGING_EVENTHUBS_HEADER} ${AZURE_MESSAGING_EVENTHUBS_SOURCE}
)
)
create_per_service_target_build(eventhubs azure-messaging-eventhubs)
add_library(Azure::azure-messaging-eventhubs ALIAS azure-messaging-eventhubs)
@ -112,14 +118,10 @@ create_code_coverage(eventhubs azure-messaging-eventhubs azure-messaging-eventhu
get_az_version("${CMAKE_CURRENT_SOURCE_DIR}/src/private/package_version.hpp")
generate_documentation(azure-messaging-eventhubs ${AZ_LIBRARY_VERSION})
if(BUILD_TESTING)
add_compile_definitions(TESTING_BUILD_AMQP)
if (NOT AZ_ALL_LIBRARIES OR FETCH_SOURCE_DEPS)
include(AddGoogleTest)
enable_testing ()
endif()
add_subdirectory(test/ut)
add_subdirectory(test)
if (BUILD_SAMPLES)
add_subdirectory(samples)
endif()
az_vcpkg_export(

171
sdk/eventhubs/azure-messaging-eventhubs/README.md
Просмотреть файл

@ -1,3 +1,170 @@
# Azure Messaging EventHub service
<!-- cspell:words azeventhubs -->
# Azure Event Hubs Client Package for C++
C++ SDK for Azure Messaging EventHub.
[Azure Event Hubs](https://azure.microsoft.com/services/event-hubs/) is a big data streaming platform and event ingestion service from Microsoft. For more information about Event Hubs see: [link](https://docs.microsoft.com/azure/event-hubs/event-hubs-about).
Use the client library `github.com/Azure/azure-sdk-for-cpp/sdk/eventhubs` in your application to:
- Send events to an event hub.
- Consume events from an event hub.
Key links:
- [Source code][source]
- [API Reference Documentation][cppdoc]
- [Product documentation](https://azure.microsoft.com/services/event-hubs/)
- [Samples][cppdoc_examples]
## Getting started
### Install the package
Install the Azure Event Hubs client package for C++ with `vcpkg`:
```bash
vcpkg install azure-messaging-eventhubs-cpp
```
### Prerequisites
- A C++ Compiler with C++14 support
- An [Azure subscription](https://azure.microsoft.com/free/)
- An [Event Hub namespace](https://docs.microsoft.com/azure/event-hubs/).
- An Event Hub. You can create an event hub in your Event Hubs Namespace using the [Azure Portal](https://docs.microsoft.com/azure/event-hubs/event-hubs-create), or the [Azure CLI](https://docs.microsoft.com/azure/event-hubs/event-hubs-quickstart-cli).
### Authenticate the client
Event Hub clients are created using a credential from the [Azure Identity package][azure_identity_pkg], like [DefaultAzureCredential][default_azure_credential].
Alternatively, you can create a client using a connection string.
<!-- NOTE: Fix dead Links -->
#### Using a service principal
- ConsumerClient: [link](https://azure.github.io/azure-sdk-for-cpp/storage.html)
- ProducerClient: [link](https://azure.github.io/azure-sdk-for-cpp/storage.html)
<!-- NOTE: Fix dead links -->
#### Using a connection string
- ConsumerClient: [link](https://azure.github.io/azure-sdk-for-cpp/storage.html)
- ProducerClient: [link](https://azure.github.io/azure-sdk-for-cpp/storage.html)
# Key concepts
An Event Hub [**namespace**](https://docs.microsoft.com/azure/event-hubs/event-hubs-features#namespace) can have multiple event hubs.
Each event hub, in turn, contains [**partitions**](https://docs.microsoft.com/azure/event-hubs/event-hubs-features#partitions) which
store events.
<!-- NOTE: Fix dead links -->
Events are published to an event hub using an [event publisher](https://docs.microsoft.com/azure/event-hubs/event-hubs-features#event-publishers). In this package, the event publisher is the [ProducerClient](https://azure.github.io/azure-sdk-for-cpp/storage.html)
Events can be consumed from an event hub using an [event consumer](https://docs.microsoft.com/azure/event-hubs/event-hubs-features#event-consumers). In this package there are two types for consuming events:
- The basic event consumer is the PartitionClient, in the [ConsumerClient](https://azure.github.io/azure-sdk-for-cpp/storage.html). This consumer is useful if you already known which partitions you want to receive from.
- A distributed event consumer, which uses Azure Blobs for checkpointing and coordination. This is implemented in the [Processor](https://azure.github.io/azure-sdk-for-cpp/storage.html).
The Processor is useful when you want to have the partition assignment be dynamically chosen, and balanced with other Processor instances.
More information about Event Hubs features and terminology can be found here: [link](https://docs.microsoft.com/azure/event-hubs/event-hubs-features)
# Examples
<!-- NOTE: Fix dead links -->
Examples for various scenarios can be found on [azure.github.io](https://azure.github.io/azure-sdk-for-cpp/storage.html) or in the samples directory in our GitHub repo for
[EventHubs](https://github.com/Azure/azure-sdk-for-cpp/blob/main/sdk/eventhubs).
## Send events
The following example shows how to send events to an event hub:
```cpp
// Your Event Hubs namespace connection string is available in the Azure portal.
std::string connectionString = "<connection_string>";
std::string eventHubName = "<event_hub_name>";
Azure::Messaging::EventHubs::EventDataBatchOptions batchOptions;
batchOptions.PartitionId = "1";
Azure::Messaging::EventHubs::EventDataBatch eventBatch(batchOptions);
Azure::Messaging::EventHubs::Models::EventData message;
message.Body.Data = {'H', 'e', 'l', 'l', 'o', '2'};
eventBatch.AddMessage(message);
Azure::Messaging::EventHubs::ProducerClientOptions producerOptions;
producerOptions.Name = "sender-link";
producerOptions.ApplicationID = "some";
auto client = Azure::Messaging::EventHubs::ProducerClient(
connectionString, eventHubName, producerOptions);
auto result = client.SendEventDataBatch(eventBatch);
```
## Receive events
The following example shows how to receive events from partition 1 on an event hub:
```cpp
// Your Event Hubs namespace connection string is available in the Azure portal.
std::string connectionString = "<connection_string>";
std::string eventHubName = "<event_hub_name>";
auto client = Azure::Messaging::EventHubs::ConsumerClient(
connectionString, eventHubName);
Azure::Messaging::EventHubs::PartitionClient partitionClient
= client.CreatePartitionClient("1");
auto events = partitionClient.ReceiveEvents(1);
```
# Troubleshooting
## Logging
The EventHubs SDK client uses the [Azure SDK log message](https://github.com/Azure/azure-sdk-for-cpp/tree/main/sdk/core/azure-core#sdk-log-messages) functionality to
enable diagnostics.
## Contributing
For details on contributing to this repository, see the [contributing guide][azure_sdk_for_cpp_contributing].
This project welcomes contributions and suggestions. Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
### Additional Helpful Links for Contributors
Many people all over the world have helped make this project better. You'll want to check out:
* [What are some good first issues for new contributors to the repo?](https://github.com/azure/azure-sdk-for-cpp/issues?q=is%3Aopen+is%3Aissue+label%3A%22up+for+grabs%22)
* [How to build and test your change][azure_sdk_for_cpp_contributing_developer_guide]
* [How you can make a change happen!][azure_sdk_for_cpp_contributing_pull_requests]
* Frequently Asked Questions (FAQ) and Conceptual Topics in the detailed [Azure SDK for C++ wiki](https://github.com/azure/azure-sdk-for-cpp/wiki).
<!-- ### Community-->
### Reporting security issues and security bugs
Security issues and bugs should be reported privately, via email, to the Microsoft Security Response Center (MSRC) <secure@microsoft.com>. You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Further information, including the MSRC PGP key, can be found in the [Security TechCenter](https://www.microsoft.com/msrc/faqs-report-an-issue).
### License
Azure SDK for C++ is licensed under the [MIT](https://github.com/Azure/azure-sdk-for-cpp/blob/main/LICENSE.txt) license.
<!-- LINKS -->
[azure_sdk_for_cpp_contributing]: https://github.com/Azure/azure-sdk-for-cpp/blob/main/CONTRIBUTING.md
[azure_sdk_for_cpp_contributing_developer_guide]: https://github.com/Azure/azure-sdk-for-cpp/blob/main/CONTRIBUTING.md#developer-guide
[azure_sdk_for_cpp_contributing_pull_requests]: https://github.com/Azure/azure-sdk-for-cpp/blob/main/CONTRIBUTING.md#pull-requests
[source]: https://github.com/Azure/azure-sdk-for-cpp/tree/main/sdk/eventhubs
[azure_identity_pkg]: https://azuresdkdocs.blob.core.windows.net/$web/cpp/azure-identity/latest/index.html
[default_azure_credential]: https://azuresdkdocs.blob.core.windows.net/$web/cpp/azure-identity/latest/index.html#defaultazurecredential
<!-- TODO: Fix go links to refer to C++ documentation when it is published.-->
[cppdoc]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs
[cppdoc_examples]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs#pkg-examples
![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-cpp%2Fsdk%2Feventhubs%2FREADME.png)

1
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs.hpp
Просмотреть файл

@ -11,6 +11,7 @@
#include "azure/messaging/eventhubs/consumer_client.hpp"
#include "azure/messaging/eventhubs/dll_import_export.hpp"
#include "azure/messaging/eventhubs/event_data_batch.hpp"
#include "azure/messaging/eventhubs/eventhubs_exception.hpp"
#include "azure/messaging/eventhubs/models/checkpoint_store_models.hpp"
#include "azure/messaging/eventhubs/models/consumer_client_models.hpp"
#include "azure/messaging/eventhubs/models/event_data.hpp"

48
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/checkpoint_store.hpp
Просмотреть файл

@ -28,17 +28,14 @@ namespace Azure { namespace Messaging { namespace EventHubs {
/** @brief Construct a CheckpointStore from another CheckpointStore.
*/
CheckpointStore& operator=(CheckpointStore const& other) = default;
/**@brief ClaimOwnership attempts to claim ownership of the partitions in partitionOwnership and
* returns the actual partitions that were claimed.
* returns the actual partitions that were claimed.
*/
virtual std::vector<Models::Ownership> ClaimOwnership(
std::vector<Models::Ownership> partitionOwnership,
std::vector<Models::Ownership> const& partitionOwnership,
Core::Context const& context = {})
{
(void)partitionOwnership;
(void)context;
throw std::runtime_error("Not Implemented");
}
= 0;
/**@brief ListCheckpoints lists all the available checkpoints.
*/
@ -47,13 +44,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
std::string const& eventHubName,
std::string const& consumerGroup,
Core::Context const& context = {})
{
(void)fullyQualifiedNamespace;
(void)consumerGroup;
(void)eventHubName;
(void)context;
throw std::runtime_error("Not Implemented");
}
= 0;
/**@brief ListOwnership lists all ownerships.
*/
@ -62,24 +53,14 @@ namespace Azure { namespace Messaging { namespace EventHubs {
std::string const& eventHubName,
std::string const& consumerGroup,
Core::Context const& context = {})
{
(void)fullyQualifiedNamespace;
(void)eventHubName;
(void)consumerGroup;
(void)context;
throw std::runtime_error("Not Implemented");
}
= 0;
/**@brief UpdateCheckpoint updates a specific checkpoint with a sequence and offset.
*/
virtual void UpdateCheckpoint(
Models::Checkpoint const& checkpoint,
Core::Context const& context = {})
{
(void)checkpoint;
(void)context;
throw std::runtime_error("Not Implemented");
}
= 0;
virtual ~CheckpointStore() = default;
};
@ -88,9 +69,6 @@ namespace Azure { namespace Messaging { namespace EventHubs {
* Storage.
*/
class BlobCheckpointStore final : public CheckpointStore {
std::string m_connectionString;
std::string m_containerName;
Azure::Storage::Blobs::BlobContainerClient m_containerClient;
void UpdateCheckpointImpl(
@ -120,20 +98,16 @@ namespace Azure { namespace Messaging { namespace EventHubs {
/**@brief Construct a BlobCheckpointStore.
*
* @param connectionString The connection string of an Azure Storage account.
* @param containerName The name of the blob container.
* @param containerClient An Azure Blob ContainerClient used to hold the checkpoints.
*/
BlobCheckpointStore(std::string const& connectionString, std::string const& containerName)
: CheckpointStore(), m_connectionString(connectionString), m_containerName(containerName),
m_containerClient(Azure::Storage::Blobs::BlobContainerClient::CreateFromConnectionString(
connectionString,
containerName))
BlobCheckpointStore(Azure::Storage::Blobs::BlobContainerClient const& containerClient)
: CheckpointStore(), m_containerClient(containerClient)
{
m_containerClient.CreateIfNotExists();
}
std::vector<Models::Ownership> ClaimOwnership(
std::vector<Models::Ownership> partitionOwnership,
std::vector<Models::Ownership> const& partitionOwnership,
Core::Context const& context = {}) override;
std::vector<Models::Checkpoint> ListCheckpoints(

92
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/consumer_client.hpp
Просмотреть файл

@ -4,7 +4,6 @@
// cspell: word myeventhub
#pragma once
#include "eventhub_constants.hpp"
#include "models/consumer_client_models.hpp"
#include "models/management_models.hpp"
#include "partition_client.hpp"
@ -16,6 +15,10 @@
#include <azure/core/http/policies/policy.hpp>
#include <azure/core/internal/diagnostics/log.hpp>
namespace Azure { namespace Messaging { namespace EventHubs {
/// @brief The default consumer group name.
constexpr const char* DefaultConsumerGroup = "$Default";
/**@brief Contains options for the ConsumerClient creation
*/
struct ConsumerClientOptions final
@ -29,9 +32,11 @@ namespace Azure { namespace Messaging { namespace EventHubs {
*/
Azure::Core::Http::Policies::RetryOptions RetryOptions{};
/**@brief Message sender options.
*/
Azure::Core::Amqp::_internal::MessageReceiverOptions ReceiverOptions{};
/** @brief Maximum message size for messages being sent. */
Azure::Nullable<std::uint64_t> MaxMessageSize;
/** @brief Name of the consumer client. */
std::string Name{};
};
/**
@ -44,32 +49,6 @@ namespace Azure { namespace Messaging { namespace EventHubs {
* managing the connection to the Event Hub and will reconnect as necessary.
*/
class ConsumerClient final {
/// The connection string for the Event Hubs namespace
std::string m_connectionString;
/// the Event Hubs namespace name (ex: myeventhub.servicebus.windows.net)
std::string m_hostName;
/// The name of the Event Hub
std::string m_eventHub;
/// The name of the consumer group
std::string m_consumerGroup;
/// Credentials to be used to authenticate the client.
std::shared_ptr<Core::Credentials::TokenCredential> m_credential;
/// The URL to the Event Hubs namespace
std::string m_hostUrl;
/// @brief The message receivers used to receive messages for a given partition.
std::map<std::string, Azure::Core::Amqp::_internal::MessageReceiver> m_receivers;
/// @brief The AMQP Sessions used to receive messages for a given partition.
std::map<std::string, Azure::Core::Amqp::_internal::Session> m_sessions;
/// @brief The options used to configure the consumer client.
ConsumerClientOptions m_consumerClientOptions;
public:
/** Create a new ConsumerClient from an existing one. */
ConsumerClient(ConsumerClient const& other) = default;
@ -89,18 +68,6 @@ namespace Azure { namespace Messaging { namespace EventHubs {
*/
std::string const& GetConsumerGroup() const { return m_consumerGroup; }
/** @brief Getter FQDN
*
* @returns FQDN client
*/
std::string const& GetHostName() const { return m_hostName; }
/** @brief Getter for client id
*
* @returns Clientid for client
*/
std::string const& GetClientId() const { return m_consumerClientOptions.ApplicationID; }
/** @brief Getter for client details
*
* @returns Client details for client
@ -108,10 +75,10 @@ namespace Azure { namespace Messaging { namespace EventHubs {
Models::ConsumerClientDetails GetDetails() const
{
Models::ConsumerClientDetails details;
details.ClientID = GetClientId();
details.ConsumerGroup = GetConsumerGroup();
details.EventHubName = GetEventHubName();
details.HostName = GetHostName();
details.ClientId = m_consumerClientOptions.ApplicationID;
details.ConsumerGroup = m_consumerGroup;
details.EventHubName = m_eventHub;
details.FullyQualifiedNamespace = m_fullyQualifiedNamespace;
return details;
}
/** @brief Getter for retry options
@ -142,7 +109,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
ConsumerClient(
std::string const& connectionString,
std::string const& eventHub = {},
std::string const& consumerGroup = _detail::DefaultConsumerGroup,
std::string const& consumerGroup = DefaultConsumerGroup,
ConsumerClientOptions const& options = {});
/** @brief creates a ConsumerClient from a token credential.
@ -159,7 +126,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
std::string const& fullyQualifiedNamespace,
std::string const& eventHub,
std::shared_ptr<Azure::Core::Credentials::TokenCredential> credential,
std::string const& consumerGroup = _detail::DefaultConsumerGroup,
std::string const& consumerGroup = DefaultConsumerGroup,
ConsumerClientOptions const& options = {});
/** @brief Create new Partition client
@ -187,5 +154,34 @@ namespace Azure { namespace Messaging { namespace EventHubs {
Models::EventHubPartitionProperties GetPartitionProperties(
std::string const& partitionID,
Core::Context const& context = {});
private:
/// The connection string for the Event Hubs namespace
std::string m_connectionString;
/// the Event Hubs namespace name (ex: myeventhub.servicebus.windows.net)
std::string m_fullyQualifiedNamespace;
/// The name of the Event Hub
std::string m_eventHub;
/// The name of the consumer group
std::string m_consumerGroup;
/// Credentials to be used to authenticate the client.
std::shared_ptr<Core::Credentials::TokenCredential> m_credential;
/// The URL to the Event Hubs namespace
std::string m_hostUrl;
/// @brief The message receivers used to receive messages for a given partition.
std::map<std::string, Azure::Core::Amqp::_internal::MessageReceiver> m_receivers;
/// @brief The AMQP Sessions used to receive messages for a given partition.
std::map<std::string, Azure::Core::Amqp::_internal::Session> m_sessions;
/// @brief The options used to configure the consumer client.
ConsumerClientOptions m_consumerClientOptions;
std::string GetStartExpression(Models::StartPosition const& startPosition);
};
}}} // namespace Azure::Messaging::EventHubs

127
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/event_data_batch.hpp
Просмотреть файл

@ -2,7 +2,6 @@
// Licensed under the MIT License.
#pragma once
#include "eventhub_constants.hpp"
#include "models/event_data.hpp"
#include <azure/core/amqp/models/amqp_message.hpp>
@ -19,7 +18,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
/** @brief EventDataBatchOptions contains optional parameters for the
* [ProducerClient.CreateEventDataBatch] function.
*
* @remark If both PartitionKey and PartitionID are nil, Event Hubs will choose an arbitrary
* @remark If both PartitionKey and PartitionId are empty, Event Hubs will choose an arbitrary
* partition for any events in this [EventDataBatch].
*/
struct EventDataBatchOptions final
@ -32,14 +31,14 @@ namespace Azure { namespace Messaging { namespace EventHubs {
/** @brief PartitionKey is hashed to calculate the partition assignment.Messages and message
* batches with the same PartitionKey are guaranteed to end up in the same partition.
* Note that if you use this option then PartitionID cannot be set.
* Note that if you use this option then PartitionId cannot be set.
*/
std::string PartitionKey;
/** @brief PartitionID is the ID of the partition to send these messages to.
/** @brief PartitionId is the ID of the partition to send these messages to.
* Note that if you use this option then PartitionKey cannot be set.
*/
std::string PartitionID;
std::string PartitionId;
};
/**@brief EventDataBatch is used to efficiently pack up EventData before sending it to Event Hubs.
@ -53,7 +52,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
const std::string anyPartitionId = "";
std::mutex m_rwMutex;
std::string m_partitionID;
std::string m_partitionId;
std::string m_partitionKey;
uint64_t m_maxBytes;
std::vector<std::vector<uint8_t>> m_marshalledMessages;
@ -70,7 +69,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
*/
EventDataBatch(EventDataBatch const& other)
// Copy constructor cannot be defaulted because of m_rwMutex.
: m_rwMutex{}, m_partitionID{other.m_partitionID}, m_partitionKey{other.m_partitionKey},
: m_rwMutex{}, m_partitionId{other.m_partitionId}, m_partitionKey{other.m_partitionKey},
m_maxBytes{other.m_maxBytes}, m_marshalledMessages{other.m_marshalledMessages},
m_batchEnvelope{other.m_batchEnvelope}, m_currentSize(other.m_currentSize){};
@ -80,7 +79,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
// Assignment operator cannot be defaulted because of m_rwMutex.
if (this != &other)
{
m_partitionID = other.m_partitionID;
m_partitionId = other.m_partitionId;
m_partitionKey = other.m_partitionKey;
m_maxBytes = other.m_maxBytes;
m_marshalledMessages = other.m_marshalledMessages;
@ -95,53 +94,26 @@ namespace Azure { namespace Messaging { namespace EventHubs {
* @param options Options settings for creating the data batch
*/
EventDataBatch(EventDataBatchOptions options = {})
: m_partitionId{options.PartitionId}, m_partitionKey{options.PartitionKey},
m_maxBytes{options.MaxBytes ? options.MaxBytes : std::numeric_limits<uint16_t>::max()},
m_marshalledMessages{}, m_batchEnvelope{}, m_currentSize{0}
{
SetPartitionID(anyPartitionId);
if (!options.PartitionID.empty() && !options.PartitionKey.empty())
if (!options.PartitionId.empty() && !options.PartitionKey.empty())
{
throw std::runtime_error("Either PartionID or PartitionKey can be set.");
throw std::runtime_error("Either PartionID or PartitionKey can be set, but not both.");
}
if (!options.PartitionID.empty())
if (options.PartitionId.empty())
{
SetPartitionID(options.PartitionID);
}
else if (!options.PartitionKey.empty())
{
SetPartitionKey(options.PartitionKey);
}
if (options.MaxBytes == 0)
{
SetMaxBytes(std::numeric_limits<uint16_t>::max());
}
else
{
SetMaxBytes(options.MaxBytes);
m_partitionId = anyPartitionId;
}
};
/** @brief Sets the partition ID for the data batch
*
* @param partitionID The partition ID to set
*/
void SetPartitionID(std::string partitionID) { m_partitionID = partitionID; }
/** @brief Sets the partition key for the data batch
*
* @param partitionKey The partition key to set
*/
void SetPartitionKey(std::string partitionKey) { m_partitionKey = partitionKey; }
/** @brief Sets the maximum size of the data batch */
void SetMaxBytes(uint64_t maxBytes) { m_maxBytes = maxBytes; }
/** @brief Gets the partition ID for the data batch
*
* @return std::string
*/
std::string GetPartitionID() const { return m_partitionID; }
std::string GetPartitionId() const { return m_partitionId; }
/** @brief Gets the partition key for the data batch
* @return std::string
@ -158,7 +130,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
*
* @param message The message to add to the batch
*/
void AddMessage(Azure::Core::Amqp::Models::AmqpMessage& message) { AddAmqpMessage(message); }
void AddMessage(Azure::Core::Amqp::Models::AmqpMessage message) { AddAmqpMessage(message); }
/** @brief Adds a message to the data batch
*
@ -180,73 +152,10 @@ namespace Azure { namespace Messaging { namespace EventHubs {
*
* @return Azure::Core::Amqp::Models::AmqpMessage
*/
Azure::Core::Amqp::Models::AmqpMessage ToAmqpMessage() const
{
Azure::Core::Amqp::Models::AmqpMessage returnValue{m_batchEnvelope};
if (m_marshalledMessages.size() == 0)
{
throw std::runtime_error("No messages added to the batch.");
}
// Make sure that the partition key in the message is the current partition key.
if (!m_partitionKey.empty())
{
returnValue.DeliveryAnnotations.emplace(
_detail::PartitionKeyAnnotation, Azure::Core::Amqp::Models::AmqpValue(m_partitionKey));
}
std::vector<Azure::Core::Amqp::Models::AmqpBinaryData> messageList;
for (auto const& marshalledMessage : m_marshalledMessages)
{
Azure::Core::Amqp::Models::AmqpBinaryData data(marshalledMessage);
messageList.push_back(data);
}
returnValue.SetBody(messageList);
Azure::Core::Diagnostics::_internal::Log::Stream(
Azure::Core::Diagnostics::Logger::Level::Informational)
<< "EventDataBatch::ToAmqpMessage: " << returnValue << std::endl;
return returnValue;
}
Azure::Core::Amqp::Models::AmqpMessage ToAmqpMessage() const;
private:
void AddAmqpMessage(Azure::Core::Amqp::Models::AmqpMessage& message)
{
std::lock_guard<std::mutex> lock(m_rwMutex);
if (!message.Properties.MessageId.HasValue())
{
message.Properties.MessageId
= Azure::Core::Amqp::Models::AmqpValue(Azure::Core::Uuid::CreateUuid().ToString());
}
if (!m_partitionKey.empty())
{
message.MessageAnnotations.emplace(
_detail::PartitionKeyAnnotation, Azure::Core::Amqp::Models::AmqpValue(m_partitionKey));
}
auto serializedMessage = Azure::Core::Amqp::Models::AmqpMessage::Serialize(message);
if (m_marshalledMessages.size() == 0)
{
// The first message is special - we use its properties and annotations on the envelope for
// the batch message.
m_batchEnvelope = CreateBatchEnvelope(message);
m_currentSize = serializedMessage.size();
}
auto actualPayloadSize = CalculateActualSizeForPayload(serializedMessage);
if (m_currentSize + actualPayloadSize > m_maxBytes)
{
m_currentSize = 0;
m_batchEnvelope = nullptr;
throw std::runtime_error("EventDataBatch size is too large.");
}
m_currentSize += actualPayloadSize;
m_marshalledMessages.push_back(serializedMessage);
}
void AddAmqpMessage(Azure::Core::Amqp::Models::AmqpMessage message);
size_t CalculateActualSizeForPayload(std::vector<uint8_t> const& payload)
{

69
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/eventhubs_exception.hpp Normal file
Просмотреть файл

@ -0,0 +1,69 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#pragma once
#include <azure/core/amqp/models/amqp_error.hpp>
#include <stdexcept>
#include <string>
namespace Azure { namespace Messaging { namespace EventHubs { namespace _detail {
class EventHubsExceptionFactory;
}}}} // namespace Azure::Messaging::EventHubs::_detail
namespace Azure { namespace Messaging { namespace EventHubs {
/**
* @brief An exception thrown when an EventHubs service operation fails.
*/
class EventHubsException final : public std::runtime_error {
public:
/**
* @brief Constructs a #EventHubsException with a message.
*
* @param what An explanatory string.
*/
explicit EventHubsException(const std::string& what)
: std::runtime_error(what), ErrorDescription{what}
{
}
/** @brief A symbolic value indicating the error condition.
*
* @remarks For more information, see [AMQP
* Section 2.8.14](https://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-transport-v1.0-os.html#type-error).
*
*/
std::string ErrorCondition{};
/**
* @brief A description of the error intended for the developer to understand what the error
* refers to and how to fix it.
*
* @remarks For more information, see [AMQP
* Section 2.8.15](https://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-transport-v1.0-os.html#type-error).
* */
std::string ErrorDescription;
/**
* @brief The status code associated with the error, if any.
*
* If this field has a value, then it will typically be an HTTP status code with additional
* information about the failure. This property is only filled in for the GetEventHubProperties
* and GetEventHubPartitionProperties operations.
*
*/
Azure::Nullable<std::uint32_t> StatusCode{};
/**
* @brief Indicates whether the error is transient in nature.
*
* If this field is set to true, then retrying the operation may succeed at a later time.
*
*/
bool IsTransient;
friend _detail::EventHubsExceptionFactory;
};
}}} // namespace Azure::Messaging::EventHubs

8
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/models/checkpoint_store_models.hpp
Просмотреть файл

@ -22,9 +22,9 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
/// @brief The fully qualified namespace for the event hub.
std::string FullyQualifiedNamespace;
/// @brief The partition ID for the corresponding ownership.
std::string PartitionID{};
std::string PartitionId{};
/// @brief The owner ID for the corresponding ownership.
std::string OwnerID{};
std::string OwnerId{};
/// the ETag, used when attempting to claim or update ownership of a partition.
Azure::Nullable<Azure::ETag> ETag{};
@ -48,9 +48,9 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
/// @brief The event hub name.
std::string EventHubName;
/// @brief The fully qualified namespace for the event hub.
std::string EventHubHostName;
std::string FullyQualifiedNamespaceName;
/// @brief The partition ID for the corresponding checkpoint.
std::string PartitionID{};
std::string PartitionId{};
/// @brief The offset of the last successfully processed event.
Azure::Nullable<int64_t> Offset{};
/// @brief The sequence number of the last successfully processed event.

11
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/models/consumer_client_models.hpp
Просмотреть файл

@ -3,12 +3,7 @@
#pragma once
// cspell: word myservicebus
#include <azure/core/amqp.hpp>
#include <azure/core/context.hpp>
#include <azure/core/credentials/credentials.hpp>
#include <azure/core/http/policies/policy.hpp>
#include <azure/messaging/eventhubs.hpp>
#include <string>
namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
@ -18,7 +13,7 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
{
/**@brief The Fully Qualified Namespace that the Event Hub exists in.
*/
std::string HostName;
std::string FullyQualifiedNamespace;
/**@brief The name of the consumer group that this consumer is associated with. Events will be
* read only in the context of this group.
@ -31,7 +26,7 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
/**@brief A unique name used to identify this consumer.
*/
std::string ClientID;
std::string ClientId;
};
}}}} // namespace Azure::Messaging::EventHubs::Models

80
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/models/event_data.hpp
Просмотреть файл

@ -4,37 +4,13 @@
#include <azure/core/amqp.hpp>
namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
/** @brief The type of the body of an EventData Message.
*
*/
struct EventDataBody final
{
/** @brief Value is encoded / decoded as the amqp - value section in the body.
*
* The type of Value can be any of the AMQP simple types, as listed in the comment for
* AmqpMessage, as well as slices or maps of AMQP simple types.
*/
Azure::Core::Amqp::Models::AmqpValue Value;
/** @brief Sequence is encoded/decoded as one or more amqp-sequence sections in the body.
*
* The values of the slices are are restricted to AMQP simple types, as listed in the comment
* for AmqpMessage.
*/
Azure::Core::Amqp::Models::AmqpList Sequence;
/** @brief Data is encoded decoded as multiple data sections in the body.
*/
Azure::Core::Amqp::Models::AmqpBinaryData Data;
};
/** @brief Represents an event sent to the Azure Event Hubs service.
*/
struct EventData
{
class EventData {
public:
/** @brief The body of the event data.
*/
EventDataBody Body;
std::vector<uint8_t> Body;
/** Represents the MIME ContentType of the event data. */
Azure::Nullable<std::string> ContentType;
@ -60,7 +36,33 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
*/
std::map<std::string, Azure::Core::Amqp::Models::AmqpValue> Properties;
EventData() = default;
/** @brief Construct a default EventData object. */
EventData() : m_message{nullptr} {};
/** @brief Construct a new EventData object from an AMQP message.
*
* @param message - AMQP message to construct the EventData from.
*/
EventData(Azure::Core::Amqp::Models::AmqpMessage const& message);
/** @brief Construct a new EventData object from an initializer list of bytes
*
* @param body - Body for the newly created EventData.
*/
EventData(std::initializer_list<uint8_t> const& body) : Body(body), m_message{nullptr} {}
/** @brief Construct a new EventData object from a vector of bytes.
*
* @param body - Body for the newly created EventData.
*/
EventData(std::vector<uint8_t> const& body) : Body(body), m_message{nullptr} {}
/** @brief Construct a new EventData object from a string.
*
* @param body - Body for the newly created EventData.
*/
EventData(std::string const& body) : Body(body.begin(), body.end()), m_message{nullptr} {}
virtual ~EventData() = default;
/** Copy an EventData to another.
@ -78,7 +80,23 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
/** Move an EventData to another.
*/
EventData& operator=(EventData&&) = default;
/** @brief Get the AMQP message associated with this EventData.
*
* Returns an underlying AMQP message corresponding to this EventData object.
*
* @note When this method is called on an EventData object, the returned message is
* constructed from the fields of the EventData object and does NOT reflect the value received
* from the service.
*
*/
virtual Azure::Core::Amqp::Models::AmqpMessage const GetRawAmqpMessage() const;
protected:
/** The incoming AMQP message, if one was received. */
Azure::Core::Amqp::Models::AmqpMessage m_message;
};
std::ostream& operator<<(std::ostream&, EventData const&);
/** @brief Represents an event received from the Azure Event Hubs service.
*
@ -131,9 +149,7 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
*
* Returns the underlying AMQP message that was received from the Event Hubs service.
*/
Azure::Core::Amqp::Models::AmqpMessage const& RawAmqpMessage() const { return m_message; }
private:
Azure::Core::Amqp::Models::AmqpMessage const m_message;
Azure::Core::Amqp::Models::AmqpMessage const GetRawAmqpMessage() const { return m_message; }
};
std::ostream& operator<<(std::ostream&, ReceivedEventData const&);
}}}} // namespace Azure::Messaging::EventHubs::Models

4
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/models/management_models.hpp
Просмотреть файл

@ -20,8 +20,9 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
std::string Name;
/// A list of the partitions in the Event Hub.
std::vector<std::string> PartitionIDs;
std::vector<std::string> PartitionIds;
};
std::ostream& operator<<(std::ostream&, EventHubProperties const&);
/**@brief EventHubPartitionProperties represents properties of an Event Hub partition
*/
@ -48,5 +49,6 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
/** Indicates whether or not the partition is currently empty. */
bool IsEmpty{};
};
std::ostream& operator<<(std::ostream&, EventHubPartitionProperties const&);
}}}} // namespace Azure::Messaging::EventHubs::Models

7
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/models/partition_client_models.hpp
Просмотреть файл

@ -1,10 +1,12 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#pragma once
#include <azure/core/amqp.hpp>
#include <azure/core/datetime.hpp>
#include <azure/core/http/policies/policy.hpp>
#include <azure/core/nullable.hpp>
namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
/**@brief StartPosition indicates the position to start receiving events within a partition.
@ -33,9 +35,9 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
/**@brief Inclusive configures whether the events directly at Offset,
* SequenceNumber or EnqueuedTime will be included (true) or excluded
* (false).
* (false). The default is false.
*/
bool Inclusive;
bool Inclusive{false};
/**@brief Earliest will start the consumer at the earliest event.
*/
@ -45,5 +47,6 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
*/
Azure::Nullable<bool> Latest;
};
std::ostream& operator<<(std::ostream&, StartPosition const&);
}}}} // namespace Azure::Messaging::EventHubs::Models

6
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/models/processor_models.hpp
Просмотреть файл

@ -1,12 +1,10 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#pragma once
#include "checkpoint_store_models.hpp"
#include <azure/core/context.hpp>
#include <azure/messaging/eventhubs.hpp>
#include "partition_client_models.hpp"
#include <chrono>
#include <string>
namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
/**@brief StartPositions are used if there is no checkpoint for a partition in

31
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/partition_client.hpp
Просмотреть файл

@ -1,6 +1,7 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#pragma once
#include "eventhubs_exception.hpp"
#include "models/event_data.hpp"
#include "models/partition_client_models.hpp"
@ -9,7 +10,8 @@
#include <azure/core/http/policies/policy.hpp>
#include <azure/core/nullable.hpp>
namespace Azure { namespace Messaging { namespace EventHubs {
/**brief PartitionClientOptions provides options for the CreatePartitionClient function.
/**brief PartitionClientOptions provides options for the ConsumerClient::CreatePartitionClient
* function.
*/
struct PartitionClientOptions final
{
@ -28,7 +30,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
* from partition clients with a lower OwnerLevel.
* Default is off.
*/
int64_t OwnerLevel;
Azure::Nullable<std::int64_t> OwnerLevel{};
/**@brief Prefetch represents the size of the internal prefetch buffer. When set,
* this client will attempt to always maintain an internal cache of events of
@ -50,19 +52,12 @@ namespace Azure { namespace Messaging { namespace EventHubs {
*/
class PartitionClient final {
protected:
/// The message receivers used to receive events from the partition.
std::vector<Azure::Core::Amqp::_internal::MessageReceiver> m_receivers{};
/// The name of the offset to start receiving events from.
std::string m_offsetExpression;
/// The level of the ownership.
uint64_t m_ownerLevel;
/// The number of events to prefetch at any time.
int32_t m_prefetchCount;
/// The options used to create the PartitionClient.
PartitionClientOptions m_partitionOptions;
@ -89,22 +84,8 @@ namespace Azure { namespace Messaging { namespace EventHubs {
*
*/
std::vector<Models::ReceivedEventData> ReceiveEvents(
uint32_t const& maxMessages,
Core::Context const& context = {})
{
std::vector<Models::ReceivedEventData> messages;
// bool prefetchDisabled = m_prefetchCount < 0;
while (messages.size() < maxMessages && !context.IsCancelled())
{
auto message = m_receivers[0].WaitForIncomingMessage(context);
if (message.first.HasValue())
{
messages.push_back(Models::ReceivedEventData{message.first.Value()});
}
}
return messages;
}
uint32_t maxMessages,
Core::Context const& context = {});
/** @brief Closes the connection to the Event Hub service.
*/

37
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/processor.hpp
Просмотреть файл

@ -71,7 +71,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
Models::StartPositions m_defaultStartPositions;
std::shared_ptr<CheckpointStore> m_checkpointStore;
int32_t m_prefetch;
std::shared_ptr<ConsumerClient> m_ConsumerClient;
std::shared_ptr<ConsumerClient> m_consumerClient;
std::vector<std::shared_ptr<ProcessorPartitionClient>> m_nextPartitionClients;
uint32_t m_currentPartitionClient;
Models::ConsumerClientDetails m_consumerClientDetails;
@ -92,13 +92,13 @@ namespace Azure { namespace Messaging { namespace EventHubs {
std::shared_ptr<CheckpointStore> checkpointStore,
ProcessorOptions const& options = {})
: m_defaultStartPositions(options.StartPositions), m_checkpointStore(checkpointStore),
m_prefetch(options.Prefetch), m_ConsumerClient(consumerClient)
m_prefetch(options.Prefetch), m_consumerClient(consumerClient)
{
m_ownershipUpdateInterval = options.UpdateInterval == Azure::DateTime::duration::zero()
? std::chrono::seconds(10)
: options.UpdateInterval;
m_consumerClientDetails = m_ConsumerClient->GetDetails();
m_consumerClientDetails = m_consumerClient->GetDetails();
m_loadBalancer = std::make_shared<ProcessorLoadBalancer>(
m_checkpointStore,
m_consumerClientDetails,
@ -133,7 +133,8 @@ namespace Azure { namespace Messaging { namespace EventHubs {
*/
void Run(Core::Context const& context = {})
{
Models::EventHubProperties eventHubProperties = m_ConsumerClient->GetEventHubProperties();
Models::EventHubProperties eventHubProperties
= m_consumerClient->GetEventHubProperties(context);
ConsumersType consumers;
Dispatch(eventHubProperties, consumers, context);
// time_t timeNowSeconds
@ -160,7 +161,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
Core::Context const& context)
{
std::vector<Models::Ownership> ownerships
= m_loadBalancer->LoadBalance(eventHubProperties.PartitionIDs, context);
= m_loadBalancer->LoadBalance(eventHubProperties.PartitionIds, context);
std::map<std::string, Models::Checkpoint> checkpoints = GetCheckpointsMap(context);
@ -192,16 +193,16 @@ namespace Azure { namespace Messaging { namespace EventHubs {
std::shared_ptr<ProcessorPartitionClient> processorPartitionClient
= std::make_shared<ProcessorPartitionClient>(
ownership.PartitionID,
m_ConsumerClient->CreatePartitionClient(
ownership.PartitionID, {startPosition, m_processorOwnerLevel, m_prefetch}),
ownership.PartitionId,
m_consumerClient->CreatePartitionClient(
ownership.PartitionId, {startPosition, m_processorOwnerLevel, m_prefetch}),
m_checkpointStore,
m_consumerClientDetails,
[&]() { consumers.erase(ownership.PartitionID); });
[&]() { consumers.erase(ownership.PartitionId); });
if (consumers.find(ownership.PartitionID) == consumers.end())
if (consumers.find(ownership.PartitionId) == consumers.end())
{
consumers.emplace(ownership.PartitionID, processorPartitionClient);
consumers.emplace(ownership.PartitionId, processorPartitionClient);
}
m_nextPartitionClients.push_back(processorPartitionClient);
@ -213,9 +214,9 @@ namespace Azure { namespace Messaging { namespace EventHubs {
{
Models::StartPosition startPosition = m_defaultStartPositions.Default;
if (checkpoints.find(ownership.PartitionID) != checkpoints.end())
if (checkpoints.find(ownership.PartitionId) != checkpoints.end())
{
Models::Checkpoint checkpoint = checkpoints.at(ownership.PartitionID);
Models::Checkpoint checkpoint = checkpoints.at(ownership.PartitionId);
if (checkpoint.Offset.HasValue())
{
@ -228,14 +229,14 @@ namespace Azure { namespace Messaging { namespace EventHubs {
else
{
throw std::runtime_error(
"invalid checkpoint" + ownership.PartitionID + "no offset or sequence number");
"invalid checkpoint" + ownership.PartitionId + "no offset or sequence number");
}
}
else if (
m_defaultStartPositions.PerPartition.find(ownership.PartitionID)
m_defaultStartPositions.PerPartition.find(ownership.PartitionId)
!= m_defaultStartPositions.PerPartition.end())
{
startPosition = m_defaultStartPositions.PerPartition.at(ownership.PartitionID);
startPosition = m_defaultStartPositions.PerPartition.at(ownership.PartitionId);
}
return startPosition;
}
@ -243,7 +244,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
std::map<std::string, Models::Checkpoint> GetCheckpointsMap(Core::Context const& context)
{
std::vector<Models::Checkpoint> checkpoints = m_checkpointStore->ListCheckpoints(
m_consumerClientDetails.HostName,
m_consumerClientDetails.FullyQualifiedNamespace,
m_consumerClientDetails.EventHubName,
m_consumerClientDetails.ConsumerGroup,
context);
@ -251,7 +252,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
std::map<std::string, Models::Checkpoint> checkpointsMap;
for (auto& checkpoint : checkpoints)
{
checkpointsMap.emplace(checkpoint.PartitionID, checkpoint);
checkpointsMap.emplace(checkpoint.PartitionId, checkpoint);
}
return checkpointsMap;

39
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/processor_partition_client.hpp
Просмотреть файл

@ -3,7 +3,6 @@
#pragma once
#include "checkpoint_store.hpp"
#include "consumer_client.hpp"
#include "eventhub_constants.hpp"
#include <azure/core/amqp.hpp>
namespace Azure { namespace Messaging { namespace EventHubs {
@ -74,43 +73,7 @@ namespace Azure { namespace Messaging { namespace EventHubs {
private:
void UpdateCheckpoint(
Azure::Core::Amqp::Models::AmqpMessage const& amqpMessage,
Core::Context const& context = {})
{
Azure::Nullable<int64_t> sequenceNumber;
Azure::Nullable<int64_t> offsetNumber;
for (auto const& pair : amqpMessage.MessageAnnotations)
{
if (pair.first == _detail::SequenceNumberAnnotation)
{
if (pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Int
|| pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Uint
|| pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Long
|| pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Ulong)
sequenceNumber = static_cast<int64_t>(pair.second);
}
if (pair.first == _detail::OffsetNumberAnnotation)
{
if (pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Int
|| pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Uint
|| pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Long
|| pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Ulong)
offsetNumber = static_cast<int64_t>(pair.second);
}
}
Models::Checkpoint checkpoint
= {m_consumerClientDetails.ConsumerGroup,
m_consumerClientDetails.EventHubName,
m_consumerClientDetails.HostName,
m_partitionId,
sequenceNumber,
offsetNumber};
m_checkpointStore->UpdateCheckpoint(checkpoint, context);
}
Core::Context const& context = {});
std::string GetPartitionId() { return m_partitionId; }
};
}}} // namespace Azure::Messaging::EventHubs

8
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/producer_client.hpp
Просмотреть файл

@ -30,9 +30,13 @@ namespace Azure { namespace Messaging { namespace EventHubs {
*/
Azure::Core::Http::Policies::RetryOptions RetryOptions{};
/**@brief Message sender options.
/** @brief The name of the producer client link, used in diagnostics.
*/
Azure::Core::Amqp::_internal::MessageSenderOptions SenderOptions{};
std::string Name{};
/**@brief The maximum size of the message that can be sent.
*/
Azure::Nullable<std::uint64_t> MaxMessageSize{};
};
/**@brief ProducerClient can be used to send events to an Event Hub.

12
sdk/eventhubs/azure-messaging-eventhubs/samples/CMakeLists.txt Normal file
Просмотреть файл

@ -0,0 +1,12 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
cmake_minimum_required (VERSION 3.13)
if(MSVC)
add_compile_definitions(_CRT_SECURE_NO_WARNINGS)
endif()
add_subdirectory(basic-operations)
add_subdirectory(produce-events)
add_subdirectory(consume-events)

37
sdk/eventhubs/azure-messaging-eventhubs/samples/README.md Normal file
Просмотреть файл

@ -0,0 +1,37 @@
# Eventhub Samples
This repository contains samples for the Azure Event Hubs service.
## Sample Requirements
These samples are written with the assumption that the following environment
variables have been set by the user:
* EVENTHUBS_CONNECTION_STRING - The service connection string for the eventhubs instance.
* EVENTHUB_NAME - Name of the eventhubs instance to communicate with.
* EVENTHUBS_HOST - Fully qualified domain name for the eventhubs instance.
* AZURE_TENANT_ID - The tenant ID for the user or service principal which has
been granted access to the eventhubs service instance.
* AZURE_CLIENT_ID - The client ID for the user or service principal which has been
granted access to the eventhubs service instance.
* AZURE_CLIENT_SECRET - The client secret for the user or service principal
which has been granted access to the eventhubs service instance.
The tests also assume that the currently logged on user is authorized to call
into the Event Hubs service instance because they use [Azure::Core::Credentials::TokenCredential](https://azuresdkdocs.blob.core.windows.net/$web/cpp/azure-core/1.3.1/class_azure_1_1_core_1_1_credentials_1_1_token_credential.html) for authorization.
## Samples
| Sample | Description |
|--------|-------------|
| basic-operations/create_producer.cpp | This sample demonstrates how to create an `EventHubProducerClient` using a connection string. |
| basic-operation/create_consumer.cpp | This sample demonstrates how to create an `EventHubConsumerClient` using a connection string. |
| basic-operations/create_producer-aad.cpp | This sample demonstrates how to create an `EventHubProducerClient` using an Azure Active Directory account. |
| basic-operation/create_consumer-aad.cpp | This sample demonstrates how to create an `EventHubConsumerClient` using an Azure Active Directory account. |
| | |
| produce-events/produce_events.cpp | This sample demonstrates how to send events to an Event Hub using the `EventHubProducerClient`. |
| produce-events/produce_events_aad.cpp | This sample demonstrates how to send events to an Event Hub using the `EventHubProducerClient` using an Azure Active Directory account. |
| consume-events/consume_events.cpp | This sample demonstrates how to receive events from an Event Hub using the `EventHubConsumerClient`. |
| consume-events/consume_events_aad.cpp | This sample demonstrates how to receive events from an Event Hub using the `EventHubConsumerClient` using an Azure Active Directory account. |

23
sdk/eventhubs/azure-messaging-eventhubs/samples/basic-operations/CMakeLists.txt Normal file
Просмотреть файл

@ -0,0 +1,23 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
cmake_minimum_required (VERSION 3.13)
project (eventhubs-basic-operations LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED True)
macro (define_sample samplename)
add_executable (
eventhubs-${samplename}
${samplename}.cpp)
CREATE_PER_SERVICE_TARGET_BUILD_FOR_SAMPLE(eventhubs eventhubs-${samplename})
target_link_libraries(eventhubs-${samplename} PRIVATE azure-messaging-eventhubs azure-identity)
endmacro()
define_sample(create_producer)
define_sample(create_consumer)
define_sample(create_consumer_aad)
define_sample(create_producer_aad)

38
sdk/eventhubs/azure-messaging-eventhubs/samples/basic-operations/create_consumer.cpp Normal file
Просмотреть файл

@ -0,0 +1,38 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
// Minimal sample showing how to create an Event Hubs consumer using a connection string retrieved
// from the Azure portal.
// This sample expects that the following environment variables exist:
// * EVENTHUB_CONNECTION_STRING - contains the connection string to a specific Event Hub instance.
// * EVENTHUB_NAME - the name of the Event Hub instance.
//
// Both of these should be available from the Azure portal.
//
#include <azure/messaging/eventhubs.hpp>
#include <iostream>
int main()
{
char* const eventhubConnectionString{std::getenv("EVENTHUB_CONNECTION_STRING")};
char* const eventhubName{std::getenv("EVENTHUB_NAME")};
if (eventhubConnectionString == nullptr)
{
std::cerr << "Missing environment variable EVENTHUB_CONNECTION_STRING" << std::endl;
return 1;
}
if (eventhubName == nullptr)
{
std::cerr << "Missing environment variable EVENTHUB_NAME" << std::endl;
}
Azure::Messaging::EventHubs::ConsumerClient consumerClient(
eventhubConnectionString, eventhubName);
// Retrieve properties about the EventHubs instance just created.
auto eventhubProperties{consumerClient.GetEventHubProperties()};
std::cout << "Created event hub, properties: " << eventhubProperties << std::endl;
}

50
sdk/eventhubs/azure-messaging-eventhubs/samples/basic-operations/create_consumer_aad.cpp Normal file
Просмотреть файл

@ -0,0 +1,50 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
// Minimal sample demonstrating how to create an Event Hubs consumer using an AAD token credential
// obtained from using the Azure Identity library.
// This sample expects that the following environment variables exist:
// * EVENTHUBS_HOST - contains the fully qualified domain name for the eventhub service instance.
// * EVENTHUB_NAME - the name of the Event Hub instance.
//
// The following environment variables are required to authenticate the request using
// Azure::Identity::EnvironmentCredential:
// * AZURE_CLIENT_ID - the application client ID used to authenticate the request.
// * AZURE_TENANT_ID - the tenant ID or domain used to authenticate the request.
// * AZURE_CLIENT_SECRET - the application client secret used to authenticate the request.
//
// Both of these should be available from the Azure portal.
//
#include <azure/identity/client_secret_credential.hpp>
#include <azure/identity/environment_credential.hpp>
#include <azure/messaging/eventhubs.hpp>
#include <iostream>
int main()
{
char const* const eventhubsHost{std::getenv("EVENTHUBS_HOST")};
char const* const eventhubName{std::getenv("EVENTHUB_NAME")};
if (eventhubsHost == nullptr)
{
std::cerr << "Missing environment variable EVENTHUBS_HOST" << std::endl;
return 1;
}
if (eventhubName == nullptr)
{
std::cerr << "Missing environment variable EVENTHUB_NAME" << std::endl;
return 1;
}
std::shared_ptr<Azure::Core::Credentials::TokenCredential> credential
= std::make_shared<Azure::Identity::EnvironmentCredential>();
Azure::Messaging::EventHubs::ConsumerClient consumerClient(
eventhubsHost, eventhubName, credential);
auto eventhubProperties{consumerClient.GetEventHubProperties()};
std::cout << "Created event hub, properties: " << eventhubProperties << std::endl;
}

38
sdk/eventhubs/azure-messaging-eventhubs/samples/basic-operations/create_producer.cpp Normal file
Просмотреть файл

@ -0,0 +1,38 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
// Minimal sample showing how to create an Event Hubs producer using a connection string retrieved
// from the Azure portal.
// This sample expects that the following environment variables exist:
// * EVENTHUB_CONNECTION_STRING - contains the connection string to a specific Event Hub instance.
// * EVENTHUB_NAME - the name of the Event Hub instance.
//
// Both of these should be available from the Azure portal.
//
#include <azure/messaging/eventhubs.hpp>
#include <iostream>
int main()
{
char* const eventhubConnectionString{std::getenv("EVENTHUB_CONNECTION_STRING")};
char* const eventhubName{std::getenv("EVENTHUB_NAME")};
if (eventhubConnectionString == nullptr)
{
std::cerr << "Missing environment variable EVENTHUB_CONNECTION_STRING" << std::endl;
return 1;
}
if (eventhubName == nullptr)
{
std::cerr << "Missing environment variable EVENTHUB_NAME" << std::endl;
}
Azure::Messaging::EventHubs::ProducerClient producerClient(
eventhubConnectionString, eventhubName);
auto eventhubProperties{producerClient.GetEventHubProperties()};
std::cout << "Created event hub, properties: " << eventhubProperties << std::endl;
}

51
sdk/eventhubs/azure-messaging-eventhubs/samples/basic-operations/create_producer_aad.cpp Normal file
Просмотреть файл

@ -0,0 +1,51 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
// Minimal sample demonstrating how to create an Event Hubs producer using an AAD token credential
// obtained from using the Azure Identity library.
// This sample expects that the following environment variables exist:
// * EVENTHUBS_HOST - contains the fully qualified domain name for the eventhub service instance.
// * EVENTHUB_NAME - the name of the Event Hub instance.
//
// The following environment variables are required to authenticate the request using
// Azure::Identity::EnvironmentCredential:
// * AZURE_CLIENT_ID - the application client ID used to authenticate the request.
// * AZURE_TENANT_ID - the tenant ID or domain used to authenticate the request.
// * AZURE_CLIENT_SECRET - the application client secret used to authenticate the request.
//
// Both of these should be available from the Azure portal.
//
#include <azure/identity/environment_credential.hpp>
#include <azure/messaging/eventhubs.hpp>
#include <iostream>
int main()
{
char const* const eventhubsHost{std::getenv("EVENTHUBS_HOST")};
char const* const eventhubName{std::getenv("EVENTHUB_NAME")};
if (eventhubsHost == nullptr)
{
std::cerr << "Missing environment variable EVENTHUBS_HOST" << std::endl;
return 1;
}
if (eventhubName == nullptr)
{
std::cerr << "Missing environment variable EVENTHUB_NAME" << std::endl;
return 1;
}
std::shared_ptr<Azure::Core::Credentials::TokenCredential> credential
= std::make_shared<Azure::Identity::EnvironmentCredential>();
Azure::Messaging::EventHubs::ProducerClient producerClient(
eventhubsHost, eventhubName, credential);
auto eventhubProperties{producerClient.GetEventHubProperties()};
std::cout << "Created event hub, properties: " << eventhubProperties << std::endl;
}

21
sdk/eventhubs/azure-messaging-eventhubs/samples/consume-events/CMakeLists.txt Normal file
Просмотреть файл

@ -0,0 +1,21 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
cmake_minimum_required (VERSION 3.13)
project (eventhubs-consume-events LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED True)
macro (define_sample samplename)
add_executable (
eventhubs-${samplename}
${samplename}.cpp)
CREATE_PER_SERVICE_TARGET_BUILD_FOR_SAMPLE(eventhubs eventhubs-${samplename})
target_link_libraries(eventhubs-${samplename} PRIVATE azure-messaging-eventhubs azure-identity)
endmacro()
define_sample(consume_events)
define_sample(consume_events_aad)

56
sdk/eventhubs/azure-messaging-eventhubs/samples/consume-events/consume_events.cpp Normal file
Просмотреть файл

@ -0,0 +1,56 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
// Minimal sample showing how to create an Event Hubs consumer using a connection string retrieved
// from the Azure portal.
// This sample expects that the following environment variables exist:
// * EVENTHUB_CONNECTION_STRING - contains the connection string to a specific Event Hub instance.
// * EVENTHUB_NAME - the name of the Event Hub instance.
//
// Both of these should be available from the Azure portal.
//
#include <azure/messaging/eventhubs.hpp>
#include <iostream>
int main()
{
char* const eventhubConnectionString{std::getenv("EVENTHUB_CONNECTION_STRING")};
char* const eventhubName{std::getenv("EVENTHUB_NAME")};
if (eventhubConnectionString == nullptr)
{
std::cerr << "Missing environment variable EVENTHUB_CONNECTION_STRING" << std::endl;
return 1;
}
if (eventhubName == nullptr)
{
std::cerr << "Missing environment variable EVENTHUB_NAME" << std::endl;
}
/* Create a sample EventHubs application using a PartitionClient to read all the messages from an
* EventHubs instance. */
Azure::Messaging::EventHubs::ConsumerClient consumerClient(
eventhubConnectionString, eventhubName);
// Retrieve properties about the EventHubs instance just created.
auto eventhubProperties{consumerClient.GetEventHubProperties()};
std::cout << "Created event hub, properties: " << eventhubProperties << std::endl;
// Retrieve properties about the EventHubs instance just created.
auto partitionProperties{
consumerClient.GetPartitionProperties(eventhubProperties.PartitionIds[0])};
Azure::Messaging::EventHubs::PartitionClient partitionClient{
consumerClient.CreatePartitionClient(eventhubProperties.PartitionIds[0])};
std::vector<Azure::Messaging::EventHubs::Models::ReceivedEventData> events
= partitionClient.ReceiveEvents(4);
// Dump the contents of each event received.
for (const auto& event : events)
{
std::cout << "Event: " << event << std::endl;
}
}

66
sdk/eventhubs/azure-messaging-eventhubs/samples/consume-events/consume_events_aad.cpp Normal file
Просмотреть файл

@ -0,0 +1,66 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
// Minimal sample showing how to create an Event Hubs event consumer using AAD credentials and then
// consume events from an EventHub partition.
// This sample expects that the following environment variables exist:
// * EVENTHUBS_HOST - contains the host name of to a specific Event Hubs instance.
// * EVENTHUB_NAME - the name of the Event Hub instance.
//
// Both of these should be available from the Azure portal.
//
// In addition, the following environment variables are required to authenticate the request using
// Azure::Identity::EnvironmentCredential:
// * AZURE_TENANT_ID - contains the tenant id used to authenticate the request.
// * AZURE_CLIENT_ID - contains the client id used to authenticate the request.
// * AZURE_CLIENT_SECRET - contains the client secret used to authenticate the request.
//
#include <azure/identity/environment_credential.hpp>
#include <azure/messaging/eventhubs.hpp>
#include <iostream>
int main()
{
char const* const eventhubsHost{std::getenv("EVENTHUBS_HOST")};
char const* const eventhubName{std::getenv("EVENTHUB_NAME")};
if (eventhubsHost == nullptr)
{
std::cerr << "Missing environment variable EVENTHUBS_HOST" << std::endl;
return 1;
}
if (eventhubName == nullptr)
{
std::cerr << "Missing environment variable EVENTHUB_NAME" << std::endl;
return 1;
}
std::shared_ptr<Azure::Core::Credentials::TokenCredential> credential
= std::make_shared<Azure::Identity::EnvironmentCredential>();
Azure::Messaging::EventHubs::ConsumerClient consumerClient(
eventhubsHost, eventhubName, credential);
// Retrieve properties about the EventHubs instance just created.
auto eventhubProperties{consumerClient.GetEventHubProperties()};
std::cout << "Created event hub, properties: " << eventhubProperties << std::endl;
// Retrieve properties about the EventHubs instance just created.
auto partitionProperties{
consumerClient.GetPartitionProperties(eventhubProperties.PartitionIds[0])};
Azure::Messaging::EventHubs::PartitionClient partitionClient{
consumerClient.CreatePartitionClient(eventhubProperties.PartitionIds[0])};
std::vector<Azure::Messaging::EventHubs::Models::ReceivedEventData> events
= partitionClient.ReceiveEvents(4);
// Dump the contents of each event received.
for (const auto& event : events)
{
std::cout << "Event: " << event << std::endl;
}
}

21
sdk/eventhubs/azure-messaging-eventhubs/samples/produce-events/CMakeLists.txt Normal file
Просмотреть файл

@ -0,0 +1,21 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
cmake_minimum_required (VERSION 3.13)
project (eventhubs-produce-events LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED True)
macro (define_sample samplename)
add_executable (
eventhubs-${samplename}
${samplename}.cpp)
CREATE_PER_SERVICE_TARGET_BUILD_FOR_SAMPLE(eventhubs eventhubs-${samplename})
target_link_libraries(eventhubs-${samplename} PRIVATE azure-messaging-eventhubs azure-identity)
endmacro()
define_sample(produce_events)
define_sample(produce_events_aad)

84
sdk/eventhubs/azure-messaging-eventhubs/samples/produce-events/produce_events.cpp Normal file
Просмотреть файл

@ -0,0 +1,84 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
// Minimal sample showing how to create an Event Hubs producer using a connection string retrieved
// from the Azure portal.
// This sample expects that the following environment variables exist:
// * EVENTHUB_CONNECTION_STRING - contains the connection string to a specific Event Hub instance.
// * EVENTHUB_NAME - the name of the Event Hub instance.
//
// Both of these should be available from the Azure portal.
//
#include <azure/messaging/eventhubs.hpp>
#include <iostream>
int main()
{
char* const eventhubConnectionString{std::getenv("EVENTHUB_CONNECTION_STRING")};
char* const eventhubName{std::getenv("EVENTHUB_NAME")};
if (eventhubConnectionString == nullptr)
{
std::cerr << "Missing environment variable EVENTHUB_CONNECTION_STRING" << std::endl;
return 1;
}
if (eventhubName == nullptr)
{
std::cerr << "Missing environment variable EVENTHUB_NAME" << std::endl;
}
Azure::Messaging::EventHubs::ProducerClient producerClient(
eventhubConnectionString, eventhubName);
Azure::Messaging::EventHubs::Models::EventHubProperties eventhubProperties
= producerClient.GetEventHubProperties();
// By default, the producer will round-robin amongst all available partitions. You can use the
// same producer instance to send to a specific partition.
// To do so, specify the partition ID in the options when creating the batch.
//
// The event consumer sample reads from the 0th partition ID in the eventhub properties, so
// configure this batch processor to send to that partition.
Azure::Messaging::EventHubs::EventDataBatchOptions batchOptions;
batchOptions.PartitionId = eventhubProperties.PartitionIds[0];
Azure::Messaging::EventHubs::EventDataBatch batch(batchOptions);
// Send an event with a simple binary body.
{
Azure::Messaging::EventHubs::Models::EventData event;
event.Body = {1, 3, 5, 7};
event.MessageId = "test-message-id";
batch.AddMessage(event);
}
{
Azure::Messaging::EventHubs::Models::EventData event;
event.Body = {2, 4, 6, 8, 10};
batch.AddMessage(event);
}
// Send an event with a body initialized at EventData constructor time.
{
Azure::Messaging::EventHubs::Models::EventData event{1, 1, 2, 3, 5, 8};
event.MessageId = "test-message-id-fibonacci";
batch.AddMessage(event);
}
// Send an event with a UTF-8 encoded string body.
{
Azure::Messaging::EventHubs::Models::EventData event{"Hello Eventhubs!"};
event.MessageId = "test-message-id-hellowworld";
batch.AddMessage(event);
}
if (!producerClient.SendEventDataBatch(batch))
{
std::cerr << "Failed to send message to the Event Hub instance." << std::endl;
}
else
{
std::cout << "Sent message to the Event Hub instance." << std::endl;
}
}

91
sdk/eventhubs/azure-messaging-eventhubs/samples/produce-events/produce_events_aad.cpp Normal file
Просмотреть файл

@ -0,0 +1,91 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
// Minimal sample showing how to create an Event Hubs producer using AAD credentials. It then
// creates 4 events in a single batch and sends those messages to eventhubs instance 0.
// This sample expects that the following environment variables exist:
// * EVENTHUBS_HOST - contains the host name of to a specific Event Hubs instance.
// * EVENTHUB_NAME - the name of the Event Hub instance.
//
// Both of these should be available from the Azure portal.
//
// In addition, the following environment variables are required to authenticate the request using
// Azure::Identity::EnvironmentCredential:
// * AZURE_TENANT_ID - contains the tenant id used to authenticate the request.
// * AZURE_CLIENT_ID - contains the client id used to authenticate the request.
// * AZURE_CLIENT_SECRET - contains the client secret used to authenticate the request.
//
#include <azure/identity.hpp>
#include <azure/messaging/eventhubs.hpp>
#include <iostream>
int main()
{
char const* const eventhubsHost{std::getenv("EVENTHUBS_HOST")};
char const* const eventhubName{std::getenv("EVENTHUB_NAME")};
if (eventhubsHost == nullptr)
{
std::cerr << "Missing environment variable EVENTHUBS_HOST" << std::endl;
return 1;
}
if (eventhubName == nullptr)
{
std::cerr << "Missing environment variable EVENTHUB_NAME" << std::endl;
return 1;
}
std::shared_ptr<Azure::Core::Credentials::TokenCredential> credential
= std::make_shared<Azure::Identity::EnvironmentCredential>();
Azure::Messaging::EventHubs::ProducerClient producerClient(
eventhubsHost, eventhubName, credential);
Azure::Messaging::EventHubs::Models::EventHubProperties eventhubProperties
= producerClient.GetEventHubProperties();
// By default, the producer will round-robin amongst all available partitions. You can use the
// same producer instance to send to a specific partition.
// To do so, specify the partition ID in the options when creating the batch.
//
// The event consumer sample reads from the 0th partition ID in the eventhub properties, so
// configure this batch processor to send to that partition.
Azure::Messaging::EventHubs::EventDataBatchOptions batchOptions;
batchOptions.PartitionId = eventhubProperties.PartitionIds[0];
Azure::Messaging::EventHubs::EventDataBatch batch(batchOptions);
// Send an event with a simple binary body.
{
Azure::Messaging::EventHubs::Models::EventData event;
event.Body = {1, 3, 5, 7};
event.MessageId = "test-message-id";
batch.AddMessage(event);
}
{
Azure::Messaging::EventHubs::Models::EventData event;
event.Body = {2, 4, 6, 8, 10};
event.MessageId = "test-message-id-2";
batch.AddMessage(event);
}
{
Azure::Messaging::EventHubs::Models::EventData event{1, 1, 2, 3, 5, 8};
event.MessageId = "test-message-id5";
batch.AddMessage(event);
}
{
Azure::Messaging::EventHubs::Models::EventData event{"Hello Eventhubs via AAD!"};
event.MessageId = "test-message-id4";
batch.AddMessage(event);
}
if (!producerClient.SendEventDataBatch(batch))
{
std::cerr << "Failed to send message to the Event Hub instance." << std::endl;
}
else
{
std::cout << "Sent message to the Event Hub instance." << std::endl;
}
}

19
sdk/eventhubs/azure-messaging-eventhubs/src/checkpoint_store.cpp
Просмотреть файл

@ -10,12 +10,12 @@ using namespace Azure::Messaging::EventHubs::Models;
std::string Azure::Messaging::EventHubs::Models::Ownership::GetOwnershipName() const
{
if (PartitionID.empty())
if (PartitionId.empty())
{
throw std::runtime_error("missing ownership fields");
}
std::stringstream strstr;
strstr << GetOwnershipPrefixName() << PartitionID;
strstr << GetOwnershipPrefixName() << PartitionId;
return strstr.str();
}
@ -33,23 +33,24 @@ std::string Azure::Messaging::EventHubs::Models::Ownership::GetOwnershipPrefixNa
std::string Azure::Messaging::EventHubs::Models::Checkpoint::GetCheckpointBlobPrefixName() const
{
if (EventHubHostName.empty() || EventHubName.empty() || ConsumerGroup.empty())
if (FullyQualifiedNamespaceName.empty() || EventHubName.empty() || ConsumerGroup.empty())
{
throw std::runtime_error("missing checkpoint fields");
}
std::stringstream strstr;
strstr << EventHubHostName << "/" << EventHubName << "/" << ConsumerGroup << "/checkpoint/";
strstr << FullyQualifiedNamespaceName << "/" << EventHubName << "/" << ConsumerGroup
<< "/checkpoint/";
return strstr.str();
}
std::string Azure::Messaging::EventHubs::Models::Checkpoint::GetCheckpointBlobName() const
{
if (PartitionID.empty())
if (PartitionId.empty())
{
throw std::runtime_error("missing checkpoint fields");
}
return GetCheckpointBlobPrefixName() + PartitionID;
return GetCheckpointBlobPrefixName() + PartitionId;
}
void Azure::Messaging::EventHubs::BlobCheckpointStore::UpdateCheckpointImpl(
@ -81,7 +82,7 @@ void Azure::Messaging::EventHubs::BlobCheckpointStore::UpdateOwnership(
{
throw std::runtime_error("missing sequence number");
}
ownership.OwnerID = temp;
ownership.OwnerId = temp;
ownership.LastModifiedTime = blob.Details.LastModified;
ownership.ETag = blob.Details.ETag;
}
@ -105,7 +106,7 @@ Azure::Messaging::EventHubs::BlobCheckpointStore::CreateCheckpointBlobMetadata(
}
std::vector<Ownership> Azure::Messaging::EventHubs::BlobCheckpointStore::ClaimOwnership(
std::vector<Ownership> partitionOwnership,
std::vector<Ownership> const& partitionOwnership,
Core::Context const& context)
{
std::vector<Ownership> newOwnerships;
@ -114,7 +115,7 @@ std::vector<Ownership> Azure::Messaging::EventHubs::BlobCheckpointStore::ClaimOw
{
std::string blobName = ownership.GetOwnershipName();
Azure::Storage::Metadata metadata;
metadata["ownerId"] = ownership.OwnerID;
metadata["ownerId"] = ownership.OwnerId;
try
{
std::pair<Azure::DateTime, Azure::ETag> result

390
sdk/eventhubs/azure-messaging-eventhubs/src/consumer_client.cpp
Просмотреть файл

@ -1,208 +1,226 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#include <azure/core/amqp.hpp>
#include "private/eventhubs_utilities.hpp"
#include "private/package_version.hpp"
#include <azure/core/amqp/message_receiver.hpp>
#include <azure/core/platform.hpp>
#include <azure/messaging/eventhubs.hpp>
using namespace Azure::Core::Diagnostics::_internal;
using namespace Azure::Core::Diagnostics;
using namespace Azure::Messaging::EventHubs::Models;
using namespace Azure::Core::Amqp::_internal;
Azure::Messaging::EventHubs::ConsumerClient::ConsumerClient(
std::string const& connectionString,
std::string const& eventHub,
std::string const& consumerGroup,
Azure::Messaging::EventHubs::ConsumerClientOptions const& options)
: m_connectionString{connectionString}, m_eventHub{eventHub}, m_consumerGroup{consumerGroup},
m_consumerClientOptions(options)
{
auto sasCredential
= std::make_shared<Azure::Core::Amqp::_internal::ServiceBusSasConnectionStringCredential>(
m_connectionString);
m_credential = sasCredential;
if (!sasCredential->GetEntityPath().empty())
namespace Azure { namespace Messaging { namespace EventHubs {
ConsumerClient::ConsumerClient(
std::string const& connectionString,
std::string const& eventHub,
std::string const& consumerGroup,
ConsumerClientOptions const& options)
: m_connectionString{connectionString}, m_eventHub{eventHub}, m_consumerGroup{consumerGroup},
m_consumerClientOptions(options)
{
m_eventHub = sasCredential->GetEntityPath();
}
m_hostName = sasCredential->GetHostName();
m_hostUrl = "amqps://" + m_hostName + "/" + m_eventHub + "/ConsumerGroups/" + m_consumerGroup;
}
auto sasCredential
= std::make_shared<ServiceBusSasConnectionStringCredential>(m_connectionString);
Azure::Messaging::EventHubs::ConsumerClient::ConsumerClient(
std::string const& hostName,
std::string const& eventHub,
std::shared_ptr<Azure::Core::Credentials::TokenCredential> credential,
std::string const& consumerGroup,
Azure::Messaging::EventHubs::ConsumerClientOptions const& options)
: m_hostName{hostName}, m_eventHub{eventHub}, m_consumerGroup{consumerGroup},
m_credential{credential}, m_consumerClientOptions(options)
{
m_hostUrl = "amqps://" + m_hostName + "/" + m_eventHub + "/ConsumerGroups/" + m_consumerGroup;
}
Azure::Messaging::EventHubs::PartitionClient
Azure::Messaging::EventHubs::ConsumerClient::CreatePartitionClient(
std::string partitionId,
Azure::Messaging::EventHubs::PartitionClientOptions const& options)
{
Azure::Messaging::EventHubs::PartitionClient partitionClient(
options, m_consumerClientOptions.RetryOptions);
std::string suffix = !partitionId.empty() ? "/Partitions/" + partitionId : "";
std::string hostUrl = m_hostUrl + suffix;
Azure::Core::Amqp::_internal::ConnectionOptions connectOptions;
connectOptions.ContainerId = m_consumerClientOptions.ApplicationID;
connectOptions.EnableTrace = m_consumerClientOptions.ReceiverOptions.EnableTrace;
Azure::Core::Amqp::_internal::Connection connection(m_hostName, m_credential, connectOptions);
Azure::Core::Amqp::_internal::SessionOptions sessionOptions;
sessionOptions.InitialIncomingWindowSize
= static_cast<uint32_t>(m_consumerClientOptions.ReceiverOptions.MaxMessageSize.ValueOr(
std::numeric_limits<int32_t>::max()));
Azure::Core::Amqp::_internal::Session session{connection.CreateSession(sessionOptions)};
Azure::Core::Amqp::_internal::MessageReceiver receiver
= session.CreateMessageReceiver(hostUrl, m_consumerClientOptions.ReceiverOptions);
// Open the connection to the remote.
receiver.Open();
m_sessions.emplace(partitionId, session);
partitionClient.PushBackReceiver(receiver);
Log::Write(Logger::Level::Informational, "Created new partition client");
return partitionClient;
}
Azure::Messaging::EventHubs::Models::EventHubProperties
Azure::Messaging::EventHubs::ConsumerClient::GetEventHubProperties(Core::Context const& context)
{
std::shared_ptr<PartitionClient> client;
if (m_sessions.size() == 0 && m_sessions.find("0") == m_sessions.end())
{
client = std::make_shared<PartitionClient>(CreatePartitionClient("0"));
}
// Create a management client off the session.
// Eventhubs management APIs return a status code in the "status-code" application properties.
Azure::Core::Amqp::_internal::ManagementClientOptions managementClientOptions;
managementClientOptions.EnableTrace = false;
managementClientOptions.ExpectedStatusCodeKeyName = "status-code";
Azure::Core::Amqp::_internal::ManagementClient managementClient(
m_sessions.at("0").CreateManagementClient(m_eventHub, managementClientOptions));
managementClient.Open();
// Send a message to the management endpoint to retrieve the properties of the eventhub.
Azure::Core::Amqp::Models::AmqpMessage message;
message.ApplicationProperties["name"]
= static_cast<Azure::Core::Amqp::Models::AmqpValue>(m_eventHub);
message.SetBody(Azure::Core::Amqp::Models::AmqpValue{});
auto result = managementClient.ExecuteOperation(
"READ" /* operation */,
"com.microsoft:eventhub" /* type of operation */,
"" /* locales */,
message,
context);
Models::EventHubProperties properties;
if (result.Status == Azure::Core::Amqp::_internal::ManagementOperationStatus::Error)
{
std::string ss = "Error: "
+ static_cast<std::string>(result.Message.ApplicationProperties["status-description"]);
Log::Write(Logger::Level::Error, ss);
}
else
{
std::cout << "Management endpoint properties message: " << result.Message;
if (result.Message.BodyType != Azure::Core::Amqp::Models::MessageBodyType::Value)
m_credential = sasCredential;
if (!sasCredential->GetEntityPath().empty())
{
throw std::runtime_error("Unexpected body type");
m_eventHub = sasCredential->GetEntityPath();
}
m_fullyQualifiedNamespace = sasCredential->GetHostName();
m_hostUrl = "amqps://" + m_fullyQualifiedNamespace + "/" + m_eventHub + "/ConsumerGroups/"
+ m_consumerGroup;
}
ConsumerClient::ConsumerClient(
std::string const& fullyQualifiedNamespace,
std::string const& eventHub,
std::shared_ptr<Azure::Core::Credentials::TokenCredential> credential,
std::string const& consumerGroup,
ConsumerClientOptions const& options)
: m_fullyQualifiedNamespace{fullyQualifiedNamespace}, m_eventHub{eventHub},
m_consumerGroup{consumerGroup}, m_credential{credential}, m_consumerClientOptions(options)
{
m_hostUrl = "amqps://" + m_fullyQualifiedNamespace + "/" + m_eventHub + "/ConsumerGroups/"
+ m_consumerGroup;
}
namespace {
struct FilterDescription
{
std::string Name;
std::uint64_t Code;
};
void AddFilterElementToSourceOptions(
Azure::Core::Amqp::Models::_internal::MessageSourceOptions& sourceOptions,
FilterDescription description,
Azure::Core::Amqp::Models::AmqpValue const& filterValue)
{
Azure::Core::Amqp::Models::AmqpDescribed value{description.Code, filterValue};
sourceOptions.Filter.emplace(description.Name, value);
}
auto const& body = result.Message.GetBodyAsAmqpValue();
if (body.GetType() != Azure::Core::Amqp::Models::AmqpValueType::Map)
{
throw std::runtime_error("Unexpected body type");
}
auto bodyMap = body.AsMap();
properties.Name = static_cast<std::string>(bodyMap["name"]);
properties.CreatedOn = Azure::DateTime(std::chrono::system_clock::from_time_t(
static_cast<std::chrono::milliseconds>(bodyMap["created_at"].AsTimestamp()).count()));
auto partitions = bodyMap["partition_ids"].AsArray();
for (const auto& partition : partitions)
{
properties.PartitionIDs.push_back(static_cast<std::string>(partition));
}
}
managementClient.Close();
FilterDescription SelectorFilter{"apache.org:selector-filter:string", 0x0000468c00000004};
} // namespace
return properties;
}
Azure::Messaging::EventHubs::Models::EventHubPartitionProperties
Azure::Messaging::EventHubs::ConsumerClient::GetPartitionProperties(
std::string const& partitionID,
Core::Context const& context)
{
if (m_sessions.find(partitionID) == m_sessions.end())
PartitionClient ConsumerClient::CreatePartitionClient(
std::string partitionId,
PartitionClientOptions const& options)
{
CreatePartitionClient(partitionID);
}
PartitionClient partitionClient(options, m_consumerClientOptions.RetryOptions);
// Create a management client off the session.
// Eventhubs management APIs return a status code in the "status-code" application properties.
Azure::Core::Amqp::_internal::ManagementClientOptions managementClientOptions;
managementClientOptions.EnableTrace = false;
managementClientOptions.ExpectedStatusCodeKeyName = "status-code";
Azure::Core::Amqp::_internal::ManagementClient managementClient{
m_sessions.at(partitionID).CreateManagementClient(m_eventHub, managementClientOptions)};
std::string suffix = !partitionId.empty() ? "/Partitions/" + partitionId : "";
std::string hostUrl = m_hostUrl + suffix;
managementClient.Open();
ConnectionOptions connectOptions;
connectOptions.ContainerId = m_consumerClientOptions.ApplicationID;
connectOptions.EnableTrace = true;
connectOptions.AuthenticationScopes = {"https://eventhubs.azure.net/.default"};
// Send a message to the management endpoint to retrieve the properties of the eventhub.
Azure::Core::Amqp::Models::AmqpMessage message;
message.ApplicationProperties["name"]
= static_cast<Azure::Core::Amqp::Models::AmqpValue>(m_eventHub);
message.ApplicationProperties["partition"] = Azure::Core::Amqp::Models::AmqpValue{partitionID};
message.SetBody(Azure::Core::Amqp::Models::AmqpValue{});
auto result = managementClient.ExecuteOperation(
"READ" /* operation */,
"com.microsoft:partition" /* type of operation */,
"" /* locales */,
message,
context);
// Set the user agent related properties in the connectOptions based on the package information
// and application ID.
_detail::EventHubsUtilities::SetUserAgent(
connectOptions, m_consumerClientOptions.ApplicationID);
Models::EventHubPartitionProperties properties;
if (result.Status == Azure::Core::Amqp::_internal::ManagementOperationStatus::Error)
{
std::cerr << "Error: " << result.Message.ApplicationProperties["status-description"];
}
else
{
std::cout << "Partition properties message: " << result.Message;
if (result.Message.BodyType != Azure::Core::Amqp::Models::MessageBodyType::Value)
Connection connection(m_fullyQualifiedNamespace, m_credential, connectOptions);
SessionOptions sessionOptions;
sessionOptions.InitialIncomingWindowSize = static_cast<uint32_t>(
m_consumerClientOptions.MaxMessageSize.ValueOr(std::numeric_limits<int32_t>::max()));
Session session{connection.CreateSession(sessionOptions)};
Azure::Core::Amqp::Models::_internal::MessageSourceOptions sourceOptions;
sourceOptions.Address = static_cast<Azure::Core::Amqp::Models::AmqpValue>(hostUrl);
AddFilterElementToSourceOptions(
sourceOptions,
SelectorFilter,
static_cast<Azure::Core::Amqp::Models::AmqpValue>(
GetStartExpression(options.StartPosition)));
Azure::Core::Amqp::Models::_internal::MessageSource messageSource(sourceOptions);
Azure::Core::Amqp::_internal::MessageReceiverOptions receiverOptions;
if (m_consumerClientOptions.MaxMessageSize)
{
throw std::runtime_error("Unexpected body type");
receiverOptions.MaxMessageSize = m_consumerClientOptions.MaxMessageSize.Value();
}
receiverOptions.EnableTrace = true;
// receiverOptions.MessageTarget = m_consumerClientOptions.MessageTarget;
receiverOptions.Name = m_consumerClientOptions.Name;
receiverOptions.Properties.emplace("com.microsoft:receiver-name", m_consumerClientOptions.Name);
if (options.OwnerLevel.HasValue())
{
receiverOptions.Properties.emplace("com.microsoft:epoch", options.OwnerLevel.Value());
}
auto const& body = result.Message.GetBodyAsAmqpValue();
if (body.GetType() != Azure::Core::Amqp::Models::AmqpValueType::Map)
{
throw std::runtime_error("Unexpected body type");
}
auto bodyMap = body.AsMap();
properties.Name = static_cast<std::string>(bodyMap["name"]);
properties.PartitionId = static_cast<std::string>(bodyMap["partition"]);
properties.BeginningSequenceNumber = bodyMap["begin_sequence_number"];
properties.LastEnqueuedSequenceNumber = bodyMap["last_enqueued_sequence_number"];
properties.LastEnqueuedOffset = static_cast<std::string>(bodyMap["last_enqueued_offset"]);
properties.LastEnqueuedTimeUtc = Azure::DateTime(std::chrono::system_clock::from_time_t(
std::chrono::duration_cast<std::chrono::seconds>(
static_cast<std::chrono::milliseconds>(bodyMap["last_enqueued_time_utc"].AsTimestamp()))
.count()));
properties.IsEmpty = bodyMap["is_partition_empty"];
}
managementClient.Close();
MessageReceiver receiver = session.CreateMessageReceiver(messageSource, receiverOptions);
return properties;
}
// Open the connection to the remote.
receiver.Open();
m_sessions.emplace(partitionId, session);
partitionClient.PushBackReceiver(receiver);
return partitionClient;
}
std::string ConsumerClient::GetStartExpression(Models::StartPosition const& startPosition)
{
std::string greaterThan = ">";
if (startPosition.Inclusive)
{
greaterThan = ">=";
}
constexpr const char* expressionErrorText
= "Only a single start point can be set: Earliest, EnqueuedTime, "
"Latest, Offset, or SequenceNumber";
std::string returnValue;
if (startPosition.EnqueuedTime.HasValue())
{
returnValue = "amqp.annotation.x--opt-enqueued-time " + greaterThan + "'"
+ std::to_string(std::chrono::duration_cast<std::chrono::milliseconds>(
startPosition.EnqueuedTime.Value().time_since_epoch())
.count())
+ "'";
}
if (startPosition.Offset.HasValue())
{
if (!returnValue.empty())
{
throw std::runtime_error(expressionErrorText);
}
returnValue = "amqp.annotation.x-opt-offset " + greaterThan + "'"
+ std::to_string(startPosition.Offset.Value()) + "'";
}
if (startPosition.SequenceNumber.HasValue())
{
if (!returnValue.empty())
{
throw std::runtime_error(expressionErrorText);
}
returnValue = "amqp.annotation.x-opt-sequence-number " + greaterThan + "'"
+ std::to_string(startPosition.SequenceNumber.Value()) + "'";
}
if (startPosition.Latest.HasValue())
{
if (!returnValue.empty())
{
throw std::runtime_error(expressionErrorText);
}
returnValue = "amqp.annotation.x-opt-offset > '@latest'";
}
if (startPosition.Earliest.HasValue())
{
if (!returnValue.empty())
{
throw std::runtime_error(expressionErrorText);
}
returnValue = "amqp.annotation.x-opt-offset > '-1'";
}
// If we don't have a filter value, then default to the start.
if (returnValue.empty())
{
return "amqp.annotation.x-opt-offset > '@latest'";
}
else
{
return returnValue;
}
}
Models::EventHubProperties ConsumerClient::GetEventHubProperties(Core::Context const& context)
{
// We need to capture the partition client here, because we need to keep it alive across the
// call to GetEventHubsProperties.
//
// If we don't keep the PartitionClient alive, the message receiver inside the partition client
// will be disconnected AFTER the outgoing ATTACH frame is sent. When the response for the
// ATTACH frame is received, it creates a new link_endpoint which is in the half attached state.
// This runs into a uAMQP bug where an incoming link detach frame will cause a crash if the
// corresponding link_endpoint is in the half attached state.
std::shared_ptr<PartitionClient> client;
if (m_sessions.find("0") == m_sessions.end())
{
client = std::make_shared<PartitionClient>(CreatePartitionClient("0"));
}
return _detail::EventHubsUtilities::GetEventHubsProperties(
m_sessions.at("0"), m_eventHub, context);
}
Models::EventHubPartitionProperties ConsumerClient::GetPartitionProperties(
std::string const& partitionId,
Core::Context const& context)
{
if (m_sessions.find(partitionId) == m_sessions.end())
{
CreatePartitionClient(partitionId);
}
return _detail::EventHubsUtilities::GetEventHubsPartitionProperties(
m_sessions.at(partitionId), m_eventHub, partitionId, context);
}
}}} // namespace Azure::Messaging::EventHubs

191
sdk/eventhubs/azure-messaging-eventhubs/src/event_data.cpp
Просмотреть файл

@ -3,49 +3,37 @@
#include "azure/messaging/eventhubs/models/event_data.hpp"
#include "azure/messaging/eventhubs/eventhub_constants.hpp"
#include "private/event_data_models_private.hpp"
#include "private/eventhubs_constants.hpp"
#include "private/eventhubs_utilities.hpp"
#include <iomanip>
#include <iostream>
#include <sstream>
namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
ReceivedEventData::ReceivedEventData(Azure::Core::Amqp::Models::AmqpMessage const& message)
: EventData(), m_message{message}
EventData::EventData(Azure::Core::Amqp::Models::AmqpMessage const& message)
: // Promote the specific message properties into ReceivedEventData.
ContentType{message.Properties.ContentType},
CorrelationId{message.Properties.CorrelationId}, MessageId{message.Properties.MessageId},
Properties{message.ApplicationProperties}, m_message{message}
{
// Promote the specific message properties into ReceivedEventData.
Properties = message.ApplicationProperties;
ContentType = message.Properties.ContentType;
CorrelationId = message.Properties.CorrelationId;
// If the message's body type is a single value, capture it in the ReceivedEventData.Body.
// Otherwise we can't express the message body as a single value, so we'll leave
// ReceivedEventData.Body as null.
switch (message.BodyType)
// If the message's body type is a single binary value, capture it in the
// EventData.Body. Otherwise we can't express the message body as a single value, so
// we'll leave EventData.Body as null.
if (message.BodyType == Azure::Core::Amqp::Models::MessageBodyType::Data)
{
case Azure::Core::Amqp::Models::MessageBodyType::Value:
Body.Value = message.GetBodyAsAmqpValue();
break;
case Azure::Core::Amqp::Models::MessageBodyType::Sequence: {
auto sequence = message.GetBodyAsAmqpList();
if (sequence.size() == 1)
{
Body.Sequence = sequence[0];
}
break;
auto& binaryData = message.GetBodyAsBinary();
if (binaryData.size() == 1)
{
Body = std::vector<uint8_t>(binaryData[0]);
}
case Azure::Core::Amqp::Models::MessageBodyType::Data: {
auto binaryData = message.GetBodyAsBinary();
if (binaryData.size() == 1)
{
Body.Data = binaryData[0];
}
break;
}
default:
break;
}
}
ReceivedEventData::ReceivedEventData(Azure::Core::Amqp::Models::AmqpMessage const& message)
: EventData(message)
{
// Copy the message annotations into the ReceivedEventData.SystemProperties. There are 3
// eventhubs specific annotations which are promoted in the ReceivedEventData, so promote them
// as well.
@ -59,8 +47,9 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
auto key = item.first.AsSymbol();
if (key == _detail::EnqueuedTimeAnnotation)
{
EnqueuedTime = Azure::DateTime{std::chrono::system_clock::time_point{
static_cast<std::chrono::milliseconds>(item.second.AsTimestamp())}};
auto timePoint = static_cast<std::chrono::milliseconds>(item.second.AsTimestamp());
auto dateTime = Azure::DateTime{Azure::DateTime::time_point{timePoint}};
EnqueuedTime = dateTime;
}
else if (key == _detail::OffsetNumberAnnotation)
{
@ -101,56 +90,116 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
}
}
}
}}}} // namespace Azure::Messaging::EventHubs::Models
namespace Azure { namespace Messaging { namespace EventHubs { namespace _detail {
Azure::Core::Amqp::Models::AmqpMessage EventDataFactory::EventDataToAmqpMessage(
Models::EventData const& eventData)
Azure::Core::Amqp::Models::AmqpMessage const EventData::GetRawAmqpMessage() const
{
// If the underlying message is already populated, return it. This will typically happen when a
// client attempts to send a raw AMQP message.
if (m_message)
{
return m_message;
}
Azure::Core::Amqp::Models::AmqpMessage rv;
rv.Properties.ContentType = eventData.ContentType;
rv.Properties.CorrelationId = eventData.CorrelationId;
rv.Properties.ContentType = ContentType;
rv.Properties.CorrelationId = CorrelationId;
rv.Properties.MessageId = MessageId;
rv.ApplicationProperties = eventData.Properties;
EventBodyToAmqpMessageBody(eventData.Body, rv);
rv.ApplicationProperties = Properties;
if (!Body.empty())
{
rv.SetBody(Body);
}
return rv;
}
/** @brief Set the body of the AMQP message.
*
* Throws an exception if the caller has set more than one of Value, Sequence, or Data.
*
* @param message The AMQP message to set the body on.
*/
void EventDataFactory::EventBodyToAmqpMessageBody(
Models::EventDataBody const& body,
Azure::Core::Amqp::Models::AmqpMessage& message)
std::ostream& operator<<(std::ostream& os, EventData const& data)
{
if (!body.Data.empty())
os << "EventData: [" << std::endl;
os << " Body: ";
Azure::Messaging::EventHubs::_detail::EventHubsUtilities::LogRawBuffer(os, data.Body);
os << std::endl;
if (!data.Properties.empty())
{
if (!body.Sequence.empty() || !body.Value.IsNull())
os << " Properties: [";
for (auto const& item : data.Properties)
{
throw std::runtime_error("Message body cannot contain both data and value/sequence.");
os << " " << item.first << ": " << item.second << std::endl;
}
message.SetBody(body.Data);
os << " ]" << std::endl;
}
else if (!body.Sequence.empty())
if (data.ContentType.HasValue())
{
if (!body.Value.IsNull() || !body.Data.empty())
{
throw std::runtime_error("Message body cannot contain both sequence and data/value.");
}
message.SetBody(body.Sequence);
os << " ContentType: " << data.ContentType.Value() << std::endl;
}
else
if (data.CorrelationId.HasValue())
{
if (!body.Sequence.empty() || !body.Data.empty())
{
throw std::runtime_error("Message body cannot contain both value and data/sequence.");
}
os << " CorrelationId: " << data.CorrelationId.Value() << std::endl;
}
if (data.MessageId.HasValue())
{
os << " MessageId: " << data.MessageId.Value() << std::endl;
}
message.SetBody(body.Value);
os << "]" << std::endl;
return os;
}
std::ostream& operator<<(std::ostream& os, ReceivedEventData const& data)
{
os << "EventData: [" << std::endl;
os << " Body: ";
Azure::Messaging::EventHubs::_detail::EventHubsUtilities::LogRawBuffer(os, data.Body);
os << std::endl;
if (!data.Properties.empty())
{
os << " Properties: [";
for (auto const& item : data.Properties)
{
os << " " << item.first << ": " << item.second << std::endl;
}
os << " ]" << std::endl;
}
if (!data.SystemProperties.empty())
{
os << " SystemProperties: [" << std::endl;
for (auto const& item : data.SystemProperties)
{
os << " " << item.first << ": " << item.second << std::endl;
}
os << " ]" << std::endl;
}
if (data.ContentType.HasValue())
{
os << " ContentType: " << data.ContentType.Value() << std::endl;
}
if (data.CorrelationId.HasValue())
{
os << " CorrelationId: " << data.CorrelationId.Value() << std::endl;
}
if (data.PartitionKey.HasValue())
{
os << " PartitionKey: " << data.PartitionKey.Value() << std::endl;
}
if (data.SequenceNumber.HasValue())
{
os << " SequenceNumber: " << data.SequenceNumber.Value() << std::endl;
}
if (data.MessageId.HasValue())
{
os << " MessageId: " << data.MessageId.Value() << std::endl;
}
if (data.Offset.HasValue())
{
os << " Offset: " << data.Offset.Value() << std::endl;
}
if (data.EnqueuedTime.HasValue())
{
os << " EnqueuedTime: " << data.EnqueuedTime.Value().ToString() << std::endl;
}
os << "Raw Message" << data.GetRawAmqpMessage();
os << "]" << std::endl;
return os;
}
}}}} // namespace Azure::Messaging::EventHubs::_detail
}}}} // namespace Azure::Messaging::EventHubs::Models

80
sdk/eventhubs/azure-messaging-eventhubs/src/event_data_batch.cpp
Просмотреть файл

@ -3,14 +3,88 @@
#include "azure/messaging/eventhubs/event_data_batch.hpp"
#include "private/event_data_models_private.hpp"
#include "private/eventhubs_constants.hpp"
#include "private/eventhubs_utilities.hpp"
#include <azure/core/diagnostics/logger.hpp>
#include <azure/core/internal/diagnostics/log.hpp>
using namespace Azure::Core::Diagnostics::_internal;
using namespace Azure::Core::Diagnostics;
namespace Azure { namespace Messaging { namespace EventHubs {
void EventDataBatch::AddMessage(Azure::Messaging::EventHubs::Models::EventData& message)
{
auto amqpMessage = _detail::EventDataFactory::EventDataToAmqpMessage(message);
AddAmqpMessage(amqpMessage);
AddAmqpMessage(message.GetRawAmqpMessage());
}
Azure::Core::Amqp::Models::AmqpMessage EventDataBatch::ToAmqpMessage() const
{
Azure::Core::Amqp::Models::AmqpMessage returnValue{m_batchEnvelope};
if (m_marshalledMessages.size() == 0)
{
throw std::runtime_error("No messages added to the batch.");
}
// Make sure that the partition key in the message is the current partition key.
if (!m_partitionKey.empty())
{
returnValue.DeliveryAnnotations.emplace(
_detail::PartitionKeyAnnotation, Azure::Core::Amqp::Models::AmqpValue(m_partitionKey));
}
std::vector<Azure::Core::Amqp::Models::AmqpBinaryData> messageList;
for (auto const& marshalledMessage : m_marshalledMessages)
{
std::stringstream ss;
_detail::EventHubsUtilities::LogRawBuffer(ss, marshalledMessage);
Log::Stream(Logger::Level::Informational) << "Add marshalled AMQP message:" << ss.str();
Azure::Core::Amqp::Models::AmqpBinaryData data(marshalledMessage);
messageList.push_back(data);
}
returnValue.SetBody(messageList);
return returnValue;
}
void EventDataBatch::AddAmqpMessage(Azure::Core::Amqp::Models::AmqpMessage message)
{
std::lock_guard<std::mutex> lock(m_rwMutex);
if (!message.Properties.MessageId.HasValue())
{
message.Properties.MessageId
= Azure::Core::Amqp::Models::AmqpValue(Azure::Core::Uuid::CreateUuid().ToString());
}
if (!m_partitionKey.empty())
{
message.MessageAnnotations.emplace(
_detail::PartitionKeyAnnotation, Azure::Core::Amqp::Models::AmqpValue(m_partitionKey));
}
Log::Stream(Logger::Level::Informational) << "Insert AMQP message: " << message;
auto serializedMessage = Azure::Core::Amqp::Models::AmqpMessage::Serialize(message);
if (m_marshalledMessages.size() == 0)
{
// The first message is special - we use its properties and annotations on the envelope for
// the batch message.
m_batchEnvelope = CreateBatchEnvelope(message);
m_currentSize = serializedMessage.size();
}
auto actualPayloadSize = CalculateActualSizeForPayload(serializedMessage);
if (m_currentSize + actualPayloadSize > m_maxBytes)
{
m_currentSize = 0;
m_batchEnvelope = nullptr;
throw std::runtime_error("EventDataBatch size is too large.");
}
m_currentSize += actualPayloadSize;
m_marshalledMessages.push_back(serializedMessage);
}
}}} // namespace Azure::Messaging::EventHubs

109
sdk/eventhubs/azure-messaging-eventhubs/src/eventhubs_utilities.cpp Normal file
Просмотреть файл

@ -0,0 +1,109 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#include "private/eventhubs_utilities.hpp"
#include <azure/core/amqp/models/amqp_error.hpp>
#include <iomanip>
#include <iostream>
#include <sstream>
using namespace Azure::Core::Amqp::Models::_internal;
namespace Azure { namespace Messaging { namespace EventHubs { namespace _detail {
namespace {
constexpr const size_t bytesPerLine = 0x10;
size_t LogRawData(std::ostream& os, size_t startOffset, const uint8_t* const data, size_t count)
{
// scratch buffer which will hold the data being logged.
std::stringstream ss;
size_t bytesToWrite = (count < bytesPerLine ? count : bytesPerLine);
ss << std::hex << std::right << std::setw(8) << std::setfill('0') << startOffset << ": ";
// Write the buffer data out in hex.
for (size_t i = 0; i < bytesToWrite; i += 1)
{
ss << std::hex << std::right << std::setw(2) << std::setfill('0')
<< static_cast<int>(data[i]) << " ";
}
// Now write the data in string format (similar to what the debugger does).
// Start by padding partial lines to a fixed end.
for (size_t i = bytesToWrite; i < bytesPerLine; i += 1)
{
ss << " ";
}
// Start of text marker.
ss << " * ";
for (size_t i = 0; i < bytesToWrite; i += 1)
{
if (isprint(data[i]))
{
ss << data[i];
}
else
{
ss << ".";
}
}
for (size_t i = bytesToWrite; i < 0x10; i += 1)
{
ss << " ";
}
// End of text marker.
ss << " *";
os << ss.str();
return bytesToWrite;
}
} // namespace
// Log the vector `value` in a structured format, bytesPerLine at a time.
void EventHubsUtilities::LogRawBuffer(std::ostream& os, std::vector<uint8_t> const& value)
{
const uint8_t* data = value.data();
size_t count = value.size();
size_t currentOffset = 0;
do
{
auto countLogged = LogRawData(os, currentOffset, data, count);
data += countLogged;
count -= countLogged;
currentOffset += countLogged;
if (count)
{
os << std::endl;
}
} while (count);
}
bool EventHubsExceptionFactory::IsErrorTransient(AmqpErrorCondition const& condition)
{
bool isTransient = false;
if ((condition == AmqpErrorCondition::TimeoutError)
|| (condition == AmqpErrorCondition::ServerBusyError)
|| (condition == AmqpErrorCondition::InternalError)
|| (condition == AmqpErrorCondition::LinkDetachForced)
|| (condition == AmqpErrorCondition::ConnectionForced)
|| (condition == AmqpErrorCondition::ConnectionFramingError)
|| (condition == AmqpErrorCondition::ProtonIo))
{
isTransient = true;
}
else if (condition == AmqpErrorCondition::NotFound)
{
// Note: Java has additional processing here, it looks for the regex:
// "The messaging entity .* could not be found" in the error description and if it is
// found it treats the error as not transient. For now, just treat NotFound as transient.
isTransient = true;
}
return isTransient;
}
}}}} // namespace Azure::Messaging::EventHubs::_detail

41
sdk/eventhubs/azure-messaging-eventhubs/src/partition_client.cpp Normal file
Просмотреть файл

@ -0,0 +1,41 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#include "azure/messaging/eventhubs/partition_client.hpp"
#include "azure/messaging/eventhubs/eventhubs_exception.hpp"
#include "private/eventhubs_utilities.hpp"
#include "private/retry_operation.hpp"
#include <azure/core/amqp.hpp>
namespace Azure { namespace Messaging { namespace EventHubs {
/** Receive events from the partition.
*
* @param maxMessages The maximum number of messages to receive.
* @param context A context to control the request lifetime.
* @return A vector of received events.
*
*/
std::vector<Models::ReceivedEventData> PartitionClient::ReceiveEvents(
uint32_t maxMessages,
Core::Context const& context)
{
std::vector<Models::ReceivedEventData> messages;
// bool prefetchDisabled = m_prefetchCount < 0;
while (messages.size() < maxMessages && !context.IsCancelled())
{
auto message = m_receivers[0].WaitForIncomingMessage(context);
if (message.first.HasValue())
{
messages.push_back(Models::ReceivedEventData{message.first.Value()});
}
else
{
throw _detail::EventHubsExceptionFactory::CreateEventHubsException(message.second);
}
}
return messages;
}
}}} // namespace Azure::Messaging::EventHubs

69
sdk/eventhubs/azure-messaging-eventhubs/src/partition_client_models.cpp Normal file
Просмотреть файл

@ -0,0 +1,69 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#include "azure/messaging/eventhubs/models/partition_client_models.hpp"
#include "azure/messaging/eventhubs/models/management_models.hpp"
#include <iomanip>
namespace Azure { namespace Messaging { namespace EventHubs { namespace Models {
std::ostream& operator<<(std::ostream& os, StartPosition const& sp)
{
os << "StartPosition:[";
if (sp.Offset.HasValue())
{
os << "Off: " << sp.Offset.Value() << std::endl;
}
if (sp.SequenceNumber.HasValue())
{
os << "Seq: " << sp.SequenceNumber.Value() << std::endl;
}
if (sp.EnqueuedTime.HasValue())
{
os << "Enq: " << sp.EnqueuedTime.Value().ToString();
}
os << " Inclusive: " << std::boolalpha << sp.Inclusive;
if (sp.Earliest.HasValue())
{
os << " Earliest: " << std::boolalpha << sp.Earliest.Value() << std::endl;
}
if (sp.Latest.HasValue())
{
os << "Latest: " << std::boolalpha << sp.Latest.Value() << std::endl;
}
os << "]";
return os;
}
std::ostream& operator<<(std::ostream& os, EventHubPartitionProperties const& pp)
{
os << "PartitionProperties:[[" << pp.Name << "]: ";
os << "Id: " << pp.PartitionId << std::endl;
os << "BeginningSequenceNumber: " << pp.BeginningSequenceNumber << std::endl;
os << "LastEnqueuedSequenceNumber: " << pp.LastEnqueuedSequenceNumber << std::endl;
os << "LastEnqueuedOffset: " << pp.LastEnqueuedOffset << std::endl;
os << "LastEnqueuedTimeUtc: " << pp.LastEnqueuedTimeUtc.ToString() << std::endl;
os << "IsEmpty" << std::boolalpha << pp.IsEmpty << std::endl;
os << "]" << std::endl;
return os;
}
std::ostream& operator<<(std::ostream& os, EventHubProperties const& ep)
{
os << "Properties:[[" << ep.Name << "]: ";
os << "createdOn: " << ep.CreatedOn.ToString();
os << " partitionCount: " << ep.PartitionIds.size();
os << " partitionIds: [";
for (auto const& id : ep.PartitionIds)
{
os << id << " ";
}
os << "]]" << std::endl;
return os;
}
}}}} // namespace Azure::Messaging::EventHubs::Models

19
sdk/eventhubs/azure-messaging-eventhubs/src/private/event_data_models_private.hpp
Просмотреть файл

@ -1,19 +0,0 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#include "azure/messaging/eventhubs/models/event_data.hpp"
namespace Azure { namespace Messaging { namespace EventHubs { namespace _detail {
class EventDataFactory {
public:
static Azure::Core::Amqp::Models::AmqpMessage EventDataToAmqpMessage(
Models::EventData const& message);
static void EventBodyToAmqpMessageBody(
Models::EventDataBody const& body,
Azure::Core::Amqp::Models::AmqpMessage& messageToUpdate);
private:
EventDataFactory() = delete;
};
}}}} // namespace Azure::Messaging::EventHubs::_detail

2
sdk/eventhubs/azure-messaging-eventhubs/inc/azure/messaging/eventhubs/eventhub_constants.hpp → sdk/eventhubs/azure-messaging-eventhubs/src/private/eventhubs_constants.hpp
Просмотреть файл

@ -10,8 +10,6 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace _detail
/// @brief The default maximum size for a single receive operation.
constexpr const std::uint32_t DefaultMaxSize = 5000;
/// @brief The default consumer group name.
constexpr const char* DefaultConsumerGroup = "$Default";
constexpr const char* PartitionKeyAnnotation = "x-opt-partition-key";
constexpr const char* SequenceNumberAnnotation = "x-opt-sequence-number";

218
sdk/eventhubs/azure-messaging-eventhubs/src/private/eventhubs_utilities.hpp Normal file
Просмотреть файл

@ -0,0 +1,218 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
// Useful utilities for the Event Hubs Clients.
#pragma once
#include "azure/messaging/eventhubs/eventhubs_exception.hpp"
#include "azure/messaging/eventhubs/models/management_models.hpp"
#include "package_version.hpp"
#include <azure/core/amqp/management.hpp>
#include <azure/core/amqp/session.hpp>
#include <azure/core/context.hpp>
#include <azure/core/internal/http/user_agent.hpp>
#include <chrono>
namespace Azure { namespace Messaging { namespace EventHubs { namespace _detail {
class EventHubsExceptionFactory {
public:
/**
* @brief Constructs a #EventHubsException with a message, an error condition, and an HTTP
* status code.
*
* This constructor is primarily intended for use by the EventHubs Properties events, which
* report their status using HTTP status codes.
*
* @param error The AMQP error indicating the error.
* @param statusCode The HTTP status code associated with the error.
*/
static EventHubsException CreateEventHubsException(
Azure::Core::Amqp::Models::_internal::AmqpError const& error,
std::uint32_t statusCode)
{
EventHubsException rv(error.Description);
rv.ErrorCondition = error.Condition.ToString();
rv.ErrorDescription = error.Description;
rv.StatusCode = statusCode;
rv.IsTransient = IsErrorTransient(error.Condition);
return rv;
}
/**
* @brief Constructs a #EventHubsException with an error condition.
*
* @param error The AMQP Error indicating the error.
*/
static EventHubsException CreateEventHubsException(
Azure::Core::Amqp::Models::_internal::AmqpError const& error)
{
EventHubsException rv(error.Description);
rv.ErrorCondition = error.Condition.ToString();
rv.ErrorDescription = error.Description;
rv.IsTransient = IsErrorTransient(error.Condition);
return rv;
}
private:
static bool IsErrorTransient(
Azure::Core::Amqp::Models::_internal::AmqpErrorCondition const& condition);
};
class EventHubsUtilities {
public:
template <typename T> static void SetUserAgent(T& options, std::string const& applicationId)
{
constexpr const char* packageName = "cpp-azure-messaging-eventhubs-cpp";
options.Properties.emplace("Product", +packageName);
options.Properties.emplace("Version", PackageVersion::ToString());
#if defined(AZ_PLATFORM_WINDOWS)
options.Properties.emplace("Platform", "Windows");
#elif defined(AZ_PLATFORM_LINUX)
options.Properties.emplace("Platform", "Linux");
#elif defined(AZ_PLATFORM_MAC)
options.Properties.emplace("Platform", "Mac");
#endif
options.Properties.emplace(
"User-Agent",
Azure::Core::Http::_detail::UserAgentGenerator::GenerateUserAgent(
packageName, PackageVersion::ToString(), applicationId));
}
static Models::EventHubProperties GetEventHubsProperties(
Azure::Core::Amqp::_internal::Session const& session,
std::string const& eventHubName,
Core::Context const& context)
{
// Create a management client off the session.
// Eventhubs management APIs return a status code in the "status-code" application properties.
Azure::Core::Amqp::_internal::ManagementClientOptions managementClientOptions;
managementClientOptions.EnableTrace = false;
managementClientOptions.ExpectedStatusCodeKeyName = "status-code";
Azure::Core::Amqp::_internal::ManagementClient managementClient{
session.CreateManagementClient(eventHubName, managementClientOptions)};
managementClient.Open();
// Send a message to the management endpoint to retrieve the properties of the eventhub.
Azure::Core::Amqp::Models::AmqpMessage message;
message.ApplicationProperties["name"]
= static_cast<Azure::Core::Amqp::Models::AmqpValue>(eventHubName);
message.SetBody(Azure::Core::Amqp::Models::AmqpValue{});
auto result = managementClient.ExecuteOperation(
"READ" /* operation */,
"com.microsoft:eventhub" /* type of operation */,
"" /* locales */,
message,
context);
Models::EventHubProperties properties;
if (result.Status != Azure::Core::Amqp::_internal::ManagementOperationStatus::Ok)
{
throw _detail::EventHubsExceptionFactory::CreateEventHubsException(
result.Error, result.StatusCode);
}
else
{
if (result.Message.BodyType != Azure::Core::Amqp::Models::MessageBodyType::Value)
{
throw std::runtime_error("Unexpected body type");
}
auto const& body = result.Message.GetBodyAsAmqpValue();
if (body.GetType() != Azure::Core::Amqp::Models::AmqpValueType::Map)
{
throw std::runtime_error("Unexpected body type");
}
auto bodyMap = body.AsMap();
properties.Name = static_cast<std::string>(bodyMap["name"]);
properties.CreatedOn = Azure::DateTime(std::chrono::system_clock::from_time_t(
std::chrono::duration_cast<std::chrono::seconds>(
static_cast<std::chrono::milliseconds>(bodyMap["created_at"].AsTimestamp()))
.count()));
auto partitions = bodyMap["partition_ids"].AsArray();
for (const auto& partition : partitions)
{
properties.PartitionIds.push_back(static_cast<std::string>(partition));
}
}
managementClient.Close();
return properties;
}
static Models::EventHubPartitionProperties GetEventHubsPartitionProperties(
Azure::Core::Amqp::_internal::Session const& session,
std::string const& eventHubName,
std::string const& partitionId,
Core::Context const& context)
{
// Create a management client off the session.
// Eventhubs management APIs return a status code in the "status-code" application properties.
Azure::Core::Amqp::_internal::ManagementClientOptions managementClientOptions;
managementClientOptions.EnableTrace = false;
managementClientOptions.ExpectedStatusCodeKeyName = "status-code";
Azure::Core::Amqp::_internal::ManagementClient managementClient{
session.CreateManagementClient(eventHubName, managementClientOptions)};
managementClient.Open();
// Send a message to the management endpoint to retrieve the properties of the eventhub.
Azure::Core::Amqp::Models::AmqpMessage message;
message.ApplicationProperties["name"]
= static_cast<Azure::Core::Amqp::Models::AmqpValue>(eventHubName);
message.ApplicationProperties["partition"]
= Azure::Core::Amqp::Models::AmqpValue{partitionId};
message.SetBody(Azure::Core::Amqp::Models::AmqpValue{});
auto result = managementClient.ExecuteOperation(
"READ" /* operation */,
"com.microsoft:partition" /* type of operation */,
"" /* locales */,
message,
context);
Models::EventHubPartitionProperties properties;
if (result.Status != Azure::Core::Amqp::_internal::ManagementOperationStatus::Ok)
{
throw _detail::EventHubsExceptionFactory::CreateEventHubsException(
result.Error, result.StatusCode);
}
else
{
if (result.Message.BodyType != Azure::Core::Amqp::Models::MessageBodyType::Value)
{
throw std::runtime_error("Unexpected body type");
}
auto const& body = result.Message.GetBodyAsAmqpValue();
if (body.GetType() != Azure::Core::Amqp::Models::AmqpValueType::Map)
{
throw std::runtime_error("Unexpected body type");
}
auto bodyMap = body.AsMap();
properties.Name = static_cast<std::string>(bodyMap["name"]);
properties.PartitionId = static_cast<std::string>(bodyMap["partition"]);
properties.BeginningSequenceNumber = bodyMap["begin_sequence_number"];
properties.LastEnqueuedSequenceNumber = bodyMap["last_enqueued_sequence_number"];
properties.LastEnqueuedOffset = static_cast<std::string>(bodyMap["last_enqueued_offset"]);
properties.LastEnqueuedTimeUtc = Azure::DateTime(std::chrono::system_clock::from_time_t(
std::chrono::duration_cast<std::chrono::seconds>(
static_cast<std::chrono::milliseconds>(
bodyMap["last_enqueued_time_utc"].AsTimestamp()))
.count()));
properties.IsEmpty = bodyMap["is_partition_empty"];
}
managementClient.Close();
return properties;
}
static void LogRawBuffer(std::ostream& os, std::vector<uint8_t> const& buffer);
~EventHubsUtilities() = delete;
};
}}}} // namespace Azure::Messaging::EventHubs::_detail

9
sdk/eventhubs/azure-messaging-eventhubs/src/private/retry_operation.hpp
Просмотреть файл

@ -1,9 +1,10 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#pragma once
#include <azure/core/amqp.hpp>
#include <azure/core/http/policies/policy.hpp>
#include <azure/messaging/eventhubs/eventhubs_exception.hpp>
#include <chrono>
#include <functional>
#if defined(TESTING_BUILD_AMQP)
@ -52,6 +53,12 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace _detail
std::chrono::milliseconds& retryAfter,
double jitterFactor = -1);
bool ShouldRetry(
Azure::Messaging::EventHubs::EventHubsException const& exception,
int32_t attempt,
std::chrono::milliseconds& retryAfter,
double jitterFactor = -1);
public:
explicit RetryOperation(Azure::Core::Http::Policies::RetryOptions& retryOptions)
: m_retryOptions(std::move(retryOptions))

20
sdk/eventhubs/azure-messaging-eventhubs/src/processor_load_balancer.cpp
Просмотреть файл

@ -18,23 +18,23 @@ Azure::Messaging::EventHubs::ProcessorLoadBalancer::GetAvailablePartitions(
std::vector<Models::Ownership> ownerships = m_checkpointStore->ListOwnership(
m_consumerClientDetails.EventHubName,
m_consumerClientDetails.ConsumerGroup,
m_consumerClientDetails.ClientID,
m_consumerClientDetails.ClientId,
context);
std::vector<Models::Ownership> unownedOrExpired;
std::map<std::string, bool> alreadyProcessed;
std::map<std::string, std::vector<Models::Ownership>> groupedByOwner;
groupedByOwner[m_consumerClientDetails.ClientID] = std::vector<Models::Ownership>();
groupedByOwner[m_consumerClientDetails.ClientId] = std::vector<Models::Ownership>();
for (auto& ownership : ownerships)
{
if (alreadyProcessed.find(ownership.PartitionID) != alreadyProcessed.end())
if (alreadyProcessed.find(ownership.PartitionId) != alreadyProcessed.end())
{
continue;
}
alreadyProcessed[ownership.PartitionID] = true;
alreadyProcessed[ownership.PartitionId] = true;
Azure::DateTime now(Azure::_detail::Clock::now());
if (ownership.OwnerID.empty()
if (ownership.OwnerId.empty()
|| std::chrono::duration_cast<std::chrono::minutes>(
now - ownership.LastModifiedTime.Value())
> m_duration)
@ -42,7 +42,7 @@ Azure::Messaging::EventHubs::ProcessorLoadBalancer::GetAvailablePartitions(
unownedOrExpired.push_back(ownership);
continue;
}
groupedByOwner[ownership.OwnerID].push_back(ownership);
groupedByOwner[ownership.OwnerId].push_back(ownership);
}
for (auto& partitionID : partitionIDs)
@ -55,9 +55,9 @@ Azure::Messaging::EventHubs::ProcessorLoadBalancer::GetAvailablePartitions(
unownedOrExpired.push_back(Ownership{
m_consumerClientDetails.ConsumerGroup,
m_consumerClientDetails.EventHubName,
m_consumerClientDetails.HostName,
m_consumerClientDetails.FullyQualifiedNamespace,
partitionID,
m_consumerClientDetails.ClientID,
m_consumerClientDetails.ClientId,
});
}
@ -82,7 +82,7 @@ Azure::Messaging::EventHubs::ProcessorLoadBalancer::GetAvailablePartitions(
}
return Models::LoadBalancerInfo{
groupedByOwner[m_consumerClientDetails.ClientID],
groupedByOwner[m_consumerClientDetails.ClientId],
unownedOrExpired,
aboveMax,
maxAllowed,
@ -113,7 +113,7 @@ Azure::Messaging::EventHubs::ProcessorLoadBalancer::GetRandomOwnerships(
Azure::Messaging::EventHubs::Models::Ownership
Azure::Messaging::EventHubs::ProcessorLoadBalancer::ResetOwnership(Models::Ownership ownership)
{
ownership.OwnerID = m_consumerClientDetails.ClientID;
ownership.OwnerId = m_consumerClientDetails.ClientId;
return ownership;
}

47
sdk/eventhubs/azure-messaging-eventhubs/src/processor_partition_client.cpp Normal file
Просмотреть файл

@ -0,0 +1,47 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#include "azure/messaging/eventhubs/processor_partition_client.hpp"
#include "private/eventhubs_constants.hpp"
namespace Azure { namespace Messaging { namespace EventHubs {
void ProcessorPartitionClient::UpdateCheckpoint(
Azure::Core::Amqp::Models::AmqpMessage const& amqpMessage,
Core::Context const& context)
{
Azure::Nullable<int64_t> sequenceNumber;
Azure::Nullable<int64_t> offsetNumber;
for (auto const& pair : amqpMessage.MessageAnnotations)
{
if (pair.first == _detail::SequenceNumberAnnotation)
{
if (pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Int
|| pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Uint
|| pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Long
|| pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Ulong)
sequenceNumber = static_cast<int64_t>(pair.second);
}
if (pair.first == _detail::OffsetNumberAnnotation)
{
if (pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Int
|| pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Uint
|| pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Long
|| pair.second.GetType() == Azure::Core::Amqp::Models::AmqpValueType::Ulong)
offsetNumber = static_cast<int64_t>(pair.second);
}
}
Models::Checkpoint checkpoint
= {m_consumerClientDetails.ConsumerGroup,
m_consumerClientDetails.EventHubName,
m_consumerClientDetails.FullyQualifiedNamespace,
m_partitionId,
sequenceNumber,
offsetNumber};
m_checkpointStore->UpdateCheckpoint(checkpoint, context);
}
}}} // namespace Azure::Messaging::EventHubs

154
sdk/eventhubs/azure-messaging-eventhubs/src/producer_client.cpp
Просмотреть файл

@ -1,7 +1,10 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#include "azure/messaging/eventhubs/producer_client.hpp"
#include "azure/messaging/eventhubs/eventhubs_exception.hpp"
#include "private/eventhubs_utilities.hpp"
#include "private/retry_operation.hpp"
#include <azure/core/amqp.hpp>
@ -54,10 +57,13 @@ void Azure::Messaging::EventHubs::ProducerClient::CreateSender(std::string const
Azure::Core::Amqp::_internal::ConnectionOptions connectOptions;
connectOptions.ContainerId = m_producerClientOptions.ApplicationID;
connectOptions.EnableTrace = m_producerClientOptions.SenderOptions.EnableTrace;
std::string hostName;
hostName = m_fullyQualifiedNamespace;
connectOptions.EnableTrace = true;
connectOptions.AuthenticationScopes = {"https://eventhubs.azure.net/.default"};
// Set the UserAgent related properties on this message sender.
_detail::EventHubsUtilities::SetUserAgent(connectOptions, m_producerClientOptions.ApplicationID);
std::string fullyQualifiedNamespace{m_fullyQualifiedNamespace};
std::string targetUrl = m_targetUrl;
if (!partitionId.empty())
@ -65,7 +71,8 @@ void Azure::Messaging::EventHubs::ProducerClient::CreateSender(std::string const
targetUrl += "/Partitions/" + partitionId;
}
Azure::Core::Amqp::_internal::Connection connection(hostName, m_credential, connectOptions);
Azure::Core::Amqp::_internal::Connection connection(
fullyQualifiedNamespace, m_credential, connectOptions);
Azure::Core::Amqp::_internal::SessionOptions sessionOptions;
sessionOptions.InitialIncomingWindowSize = std::numeric_limits<int32_t>::max();
@ -73,8 +80,14 @@ void Azure::Messaging::EventHubs::ProducerClient::CreateSender(std::string const
Azure::Core::Amqp::_internal::Session session{connection.CreateSession(sessionOptions)};
m_sessions.emplace(partitionId, session);
Azure::Core::Amqp::_internal::MessageSenderOptions senderOptions;
senderOptions.Name = m_producerClientOptions.Name;
senderOptions.EnableTrace = true;
senderOptions.MaxMessageSize = m_producerClientOptions.MaxMessageSize;
Azure::Core::Amqp::_internal::MessageSender sender
= session.CreateMessageSender(targetUrl, m_producerClientOptions.SenderOptions, nullptr);
= session.CreateMessageSender(targetUrl, senderOptions, nullptr);
sender.Open();
m_senders.emplace(partitionId, sender);
}
@ -88,8 +101,15 @@ bool Azure::Messaging::EventHubs::ProducerClient::SendEventDataBatch(
Azure::Messaging::EventHubs::_detail::RetryOperation retryOp(
m_producerClientOptions.RetryOptions);
return retryOp.Execute([&]() -> bool {
auto result = GetSender(eventDataBatch.GetPartitionID()).Send(message, context);
return std::get<0>(result) == Azure::Core::Amqp::_internal::MessageSendStatus::Ok;
auto result = GetSender(eventDataBatch.GetPartitionId()).Send(message, context);
auto sendStatus = std::get<0>(result);
if (sendStatus == Azure::Core::Amqp::_internal::MessageSendStatus::Ok)
{
return true;
}
// Throw an exception about the error we just received.
throw Azure::Messaging::EventHubs::_detail::EventHubsExceptionFactory::CreateEventHubsException(
std::get<1>(result));
});
}
@ -101,125 +121,19 @@ Azure::Messaging::EventHubs::ProducerClient::GetEventHubProperties(Core::Context
CreateSender("");
}
// Create a management client off the session.
// Eventhubs management APIs return a status code in the "status-code" application properties.
Azure::Core::Amqp::_internal::ManagementClientOptions managementClientOptions;
managementClientOptions.EnableTrace = false;
managementClientOptions.ExpectedStatusCodeKeyName = "status-code";
Azure::Core::Amqp::_internal::ManagementClient managementClient{
m_sessions.at("").CreateManagementClient(m_eventHub, managementClientOptions)};
managementClient.Open();
// Send a message to the management endpoint to retrieve the properties of the eventhub.
Azure::Core::Amqp::Models::AmqpMessage message;
message.ApplicationProperties["name"]
= static_cast<Azure::Core::Amqp::Models::AmqpValue>(m_eventHub);
message.SetBody(Azure::Core::Amqp::Models::AmqpValue{});
auto result = managementClient.ExecuteOperation(
"READ" /* operation */,
"com.microsoft:eventhub" /* type of operation */,
"" /* locales */,
message,
context);
Models::EventHubProperties properties;
if (result.Status == Azure::Core::Amqp::_internal::ManagementOperationStatus::Error)
{
std::cerr << "Error: " << result.Message.ApplicationProperties["status-description"];
}
else
{
std::cout << "Management endpoint properties message: " << result.Message;
if (result.Message.BodyType != Azure::Core::Amqp::Models::MessageBodyType::Value)
{
throw std::runtime_error("Unexpected body type");
}
auto const& body = result.Message.GetBodyAsAmqpValue();
if (body.GetType() != Azure::Core::Amqp::Models::AmqpValueType::Map)
{
throw std::runtime_error("Unexpected body type");
}
auto bodyMap = body.AsMap();
properties.Name = static_cast<std::string>(bodyMap["name"]);
properties.CreatedOn = Azure::DateTime(std::chrono::system_clock::from_time_t(
static_cast<std::chrono::milliseconds>(bodyMap["created_at"].AsTimestamp()).count()));
auto partitions = bodyMap["partition_ids"].AsArray();
for (const auto& partition : partitions)
{
properties.PartitionIDs.push_back(static_cast<std::string>(partition));
}
}
managementClient.Close();
return properties;
return _detail::EventHubsUtilities::GetEventHubsProperties(
m_sessions.at(""), m_eventHub, context);
}
Azure::Messaging::EventHubs::Models::EventHubPartitionProperties
Azure::Messaging::EventHubs::ProducerClient::GetPartitionProperties(
std::string const& partitionID,
std::string const& partitionId,
Core::Context const& context)
{
if (m_senders.find(partitionID) == m_senders.end())
if (m_senders.find(partitionId) == m_senders.end())
{
CreateSender(partitionID);
CreateSender(partitionId);
}
// Create a management client off the session.
// Eventhubs management APIs return a status code in the "status-code" application properties.
Azure::Core::Amqp::_internal::ManagementClientOptions managementClientOptions;
managementClientOptions.EnableTrace = false;
managementClientOptions.ExpectedStatusCodeKeyName = "status-code";
Azure::Core::Amqp::_internal::ManagementClient managementClient{
m_sessions.at(partitionID).CreateManagementClient(m_eventHub, managementClientOptions)};
managementClient.Open();
// Send a message to the management endpoint to retrieve the properties of the eventhub.
Azure::Core::Amqp::Models::AmqpMessage message;
message.ApplicationProperties["name"]
= static_cast<Azure::Core::Amqp::Models::AmqpValue>(m_eventHub);
message.ApplicationProperties["partition"] = Azure::Core::Amqp::Models::AmqpValue{partitionID};
message.SetBody(Azure::Core::Amqp::Models::AmqpValue{});
auto result = managementClient.ExecuteOperation(
"READ" /* operation */,
"com.microsoft:partition" /* type of operation */,
"" /* locales */,
message,
context);
Models::EventHubPartitionProperties properties;
if (result.Status == Azure::Core::Amqp::_internal::ManagementOperationStatus::Error)
{
std::cerr << "Error: " << result.Message.ApplicationProperties["status-description"];
}
else
{
std::cout << "Partition properties message: " << result.Message;
if (result.Message.BodyType != Azure::Core::Amqp::Models::MessageBodyType::Value)
{
throw std::runtime_error("Unexpected body type");
}
auto const& body = result.Message.GetBodyAsAmqpValue();
if (body.GetType() != Azure::Core::Amqp::Models::AmqpValueType::Map)
{
throw std::runtime_error("Unexpected body type");
}
auto bodyMap = body.AsMap();
properties.Name = static_cast<std::string>(bodyMap["name"]);
properties.PartitionId = static_cast<std::string>(bodyMap["partition"]);
properties.BeginningSequenceNumber = bodyMap["begin_sequence_number"];
properties.LastEnqueuedSequenceNumber = bodyMap["last_enqueued_sequence_number"];
properties.LastEnqueuedOffset = static_cast<std::string>(bodyMap["last_enqueued_offset"]);
properties.LastEnqueuedTimeUtc = Azure::DateTime(std::chrono::system_clock::from_time_t(
std::chrono::duration_cast<std::chrono::seconds>(
static_cast<std::chrono::milliseconds>(bodyMap["last_enqueued_time_utc"].AsTimestamp()))
.count()));
properties.IsEmpty = bodyMap["is_partition_empty"];
}
managementClient.Close();
return properties;
return _detail::EventHubsUtilities::GetEventHubsPartitionProperties(
m_sessions.at(partitionId), m_eventHub, partitionId, context);
}

41
sdk/eventhubs/azure-messaging-eventhubs/src/retry_operation.cpp
Просмотреть файл

@ -3,7 +3,26 @@
#include "private/retry_operation.hpp"
#include "azure/core/internal/diagnostics/log.hpp"
#include "azure/messaging/eventhubs/eventhubs_exception.hpp"
#include <thread>
namespace {
// The set of AMQP error conditions that should be treated as fatal conditions.
constexpr const char* AmqpFatalConditions[] = {"amqp:link:message-size-exceeded"};
bool IsFatalException(Azure::Messaging::EventHubs::EventHubsException const& ex)
{
for (auto const& condition : AmqpFatalConditions)
{
if (ex.ErrorCondition == condition)
{
return true;
}
}
return false;
}
} // namespace
bool Azure::Messaging::EventHubs::_detail::RetryOperation::Execute(std::function<bool()> operation)
{
using Azure::Core::Diagnostics::Logger;
@ -27,17 +46,39 @@ bool Azure::Messaging::EventHubs::_detail::RetryOperation::Execute(std::function
return result;
}
}
catch (EventHubsException const& e)
{
if (Log::ShouldWrite(Logger::Level::Warning))
{
Log::Stream(Logger::Level::Warning)
<< "Exception thrown. " << e.ErrorCondition << " - " << e.ErrorDescription << std::endl;
}
if (ShouldRetry(IsFatalException(e), retryCount, retryAfter))
{
retryCount++;
std::this_thread::sleep_for(retryAfter);
}
else
{
throw;
}
}
catch (std::exception const& e)
{
if (Log::ShouldWrite(Logger::Level::Warning))
{
Log::Write(Logger::Level::Warning, std::string("Exception while trying ") + e.what());
}
// We assume that all exceptions other than EventHubs exceptions might be retriable.
if (ShouldRetry(false, retryCount, retryAfter))
{
retryCount++;
std::this_thread::sleep_for(retryAfter);
}
else
{
throw;
}
}
}
return false;

25
sdk/eventhubs/azure-messaging-eventhubs/test/CMakeLists.txt Normal file
Просмотреть файл

@ -0,0 +1,25 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# CMakeList.txt : Top-level CMake project file, do global configuration
# and include sub-projects here.
#
cmake_minimum_required (VERSION 3.13)
if(BUILD_TESTING)
add_compile_definitions(TESTING_BUILD_AMQP)
if (NOT AZ_ALL_LIBRARIES OR FETCH_SOURCE_DEPS)
include(AddGoogleTest)
enable_testing ()
endif()
endif()
# Include sub-projects.
if (BUILD_TESTING)
add_subdirectory ("ut")
# stress tests are categorized as normal tests.
add_subdirectory ("eventhubs-stress-test")
endif()
if (BUILD_PERFORMANCE_TESTS)
add_subdirectory ("perf")
endif()

2
sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/.dockerignore Normal file
Просмотреть файл

@ -0,0 +1,2 @@
out/*

10
sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/.helmignore Normal file
Просмотреть файл

@ -0,0 +1,10 @@
Dockerfile
.dockerignore
*.go
*.bat
*.txt
*.ps1
*.cpp
*.md
azure-messaging-eventhubs-stress-test
[Bb]in/

19
sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/CMakeLists.txt Normal file
Просмотреть файл

@ -0,0 +1,19 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
cmake_minimum_required(VERSION 3.13)
project(azure-messaging-eventhubs-stress-test LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED True)
add_executable(
azure-messaging-eventhubs-stress-test
eventhubs_stress_test.cpp
)
target_link_libraries(azure-messaging-eventhubs-stress-test PRIVATE azure-messaging-eventhubs azure-core azure-identity)
create_map_file(azure-messaging-eventhubs-stress-test azure-messaging-eventhubs-stress-test.map)
file(COPY ${CMAKE_CURRENT_BINARY_DIR}
DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}/bin)

6
sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/Chart.lock Normal file
Просмотреть файл

@ -0,0 +1,6 @@
dependencies:
- name: stress-test-addons
repository: https://stresstestcharts.blob.core.windows.net/helm/
version: 0.2.0
digest: sha256:59fff3930e78c4ca9f9c0120433c7695d31db63f36ac61d50abcc91b1f1835a0
generated: "2023-07-24T13:06:32.3351603-07:00"

16
sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/Chart.yaml Normal file
Просмотреть файл

@ -0,0 +1,16 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
apiVersion: v2
name: eventhubs-stress-test
description: C++ Stress Test for Azure Event Hubs.
version: 0.0.2
appVersion: v0.1
annotations:
stressTest: 'true' # enable auto-discovery of this test via `find-all-stress-packages.ps1`
namespace: 'azuresdkforcpp'
dependencies:
- name: stress-test-addons
version: ~0.2.0
repository: https://stresstestcharts.blob.core.windows.net/helm/

39
sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/Dockerfile Normal file
Просмотреть файл

@ -0,0 +1,39 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
FROM mcr.microsoft.com/mirror/docker/library/ubuntu:22.04 as build
# FROM mcr.microsoft.com/mirror/docker/library/ubuntu:22.04
# install the mem check tool along side the other deps
RUN apt-get update -y
RUN apt-get install -y gcc cmake make g++ git zip unzip build-essential pkg-config wget curl valgrind
RUN wget -O vcpkg.tar.gz https://github.com/microsoft/vcpkg/archive/master.tar.gz
RUN mkdir /opt/vcpkg
RUN tar xf vcpkg.tar.gz --strip-components=1 -C /opt/vcpkg
RUN /opt/vcpkg/bootstrap-vcpkg.sh
RUN ln -s /opt/vcpkg/vcpkg /usr/local/bin/vcpkg
ADD . /src
WORKDIR /build
# During CMake generate step VCPKG runs in manifest mode, as such it will sync the packages to the level
# of the hash specified in src/azure-sdk-for-cpp/cmake-modules/AzureVcpkg.cmake in the VCPKG_COMMIT_STRING
# environment variable thus the packages we run with are not the latest versions but the ones the code
# was developed against. If the builtin-baseline is specified in the vcpkg file then that is the top most
# version of the packages that will be fetched.
# So when building from root we need to match the two values. When not building from root if the vcpkg file
# does not specify a baseline the value set in the cmake file will ensure that we are at the desired level.
RUN cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DBUILD_TRANSPORT_CURL=ON /src
RUN cmake --build . --target azure-messaging-eventhubs-stress-test
FROM mcr.microsoft.com/mirror/docker/library/ubuntu:22.04
RUN apt-get update -y
RUN apt-get install -y valgrind
WORKDIR /
# copy the target binary
COPY --from=build ./build/sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/azure-messaging-eventhubs-stress-test ./azure-messaging-eventhubs-stress-test
RUN chmod +x ./azure-messaging-eventhubs-stress-test
CMD ./azure-messaging-eventhubs-stress-test
# this should be run by the scenarios matrix , run valgrind only when needed since it impacts performance and resources.

29
sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/README.md Normal file
Просмотреть файл

@ -0,0 +1,29 @@
# Stress test prototype
This is work in progress. It's a prototype of how a stress test would look.
## Components
### Code (https://en.wikipedia.org/wiki/C%2B%2B)
The cpp file represents the code for the test, it will generate a number of invalid URLs and then issue CURL send commands. The requests are expected to fail. The point was that it exposes memory leaks in handling the error cases, which we fixed since.
### Dockerfile (https://www.docker.com/)
Represents the build file for the container in which the test runs, it is based on Ubuntu 22.04 , from MCR.
The main change from default Ubuntu is making sure we have the Valgrind tool installed. Valgrind is a heap monitoring tool that helps identify potential stack traces that might leak memory. While not 100% effective is great at reducing the surface are for investigations.
### Helm chart (https://helm.sh/)
Chart.yaml together with the bicep file(https://docs.microsoft.com/azure/azure-resource-manager/bicep/overview?tabs=bicep) and the deploy job file , represent the helm chart needed to deploy to the docker image built from the dockerfile to the stress cluster and execute the stress test.
The helm chart creates a pod with a container based on the docker image, and executes the test under Valgrind.
To deploy the chart you will need to run "azure-sdk-for-cpp\eng\common\scripts\stress-testing> .\deploy-stress-tests.ps1 -Namespace azuresdkforcpp -SearchDirectory E:\src\azure-sdk-for-cpp\sdk\core\azure-core\test -PushImage"
Where namespace will be created if missing , search directory can be any folder where it will search for charts in it and all it's sub dirs, push image will call it to build the docker image.
ATM the docker image is build by hand and hard-coded in the chart to simplify matters.
To build the image run "docker build -t <acr>/azuresdkforcpp/curlstress:v8 --build-arg targetTest=azure-core-libcurl-stress-test --build-arg build=on ."
To push to mcr : "docker push <acr>/azuresdkforcpp/curlstress:v8"
Obviously after logging in to the acr "az acr login -n <acr>"
To use another image you will need to go to line 12 in deploy job and update with your new file.
Once the deploy succeeds run " kubectl logs -n azuresdkforcpp -f libcurl-stress-test" to grab the logs in real time .

2
sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/deploy.ps1 Normal file
Просмотреть файл

@ -0,0 +1,2 @@
Set-Location $PSScriptRoot
pwsh "../../../../../eng/common/scripts/stress-testing/deploy-stress-tests.ps1"

240
sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/eventhubs_stress_test.cpp Normal file
Просмотреть файл

@ -0,0 +1,240 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
/**
* @brief Validates the Azure Core transport adapters with fault responses from server.
*
* @note This test requires the Http-fault-injector
* (https://github.com/Azure/azure-sdk-tools/tree/main/tools/http-fault-injector) running. Follow
* the instructions to install and run the server before running this test.
*
*/
#define REQUESTS 100
#define WARMUP 100
#define ROUNDS 100
#include <azure/core.hpp>
#include <azure/core/internal/environment.hpp>
#include <azure/identity/client_secret_credential.hpp>
#include <azure/messaging/eventhubs/consumer_client.hpp>
#include <azure/messaging/eventhubs/producer_client.hpp>
#include <iostream>
using namespace Azure::Messaging::EventHubs;
class EventHubsStress {
public:
EventHubsStress()
{
m_eventHubName = Azure::Core::_internal::Environment::GetVariable("EVENTHUB_NAME");
m_eventHubConnectionString
= Azure::Core::_internal::Environment::GetVariable("EVENTHUB_CONNECTION_STRING");
m_checkpointStoreConnectionString
= Azure::Core::_internal::Environment::GetVariable("CHECKPOINT_STORE_CONNECTION_STRING");
m_numberToSend = 100;
m_batchSize = 100;
m_prefetchCount = 10;
m_messageBodySize = 1024;
m_tenantId = Azure::Core::_internal::Environment::GetVariable("AZURE_TENANT_ID");
m_clientId = Azure::Core::_internal::Environment::GetVariable("AZURE_CLIENT_ID");
m_secret = Azure::Core::_internal::Environment::GetVariable("AZURE_CLIENT_SECRET");
if (m_eventHubConnectionString.empty())
{
m_credential = std::make_shared<Azure::Identity::ClientSecretCredential>(
m_tenantId, m_clientId, m_secret);
m_client = std::make_unique<Azure::Messaging::EventHubs::ProducerClient>(
m_eventHubConnectionString, m_eventHubName, m_credential);
}
else
{
m_client = std::make_unique<Azure::Messaging::EventHubs::ProducerClient>(
m_eventHubConnectionString, m_eventHubName);
}
}
void Warmup(int repetitions)
{
for (int i = 0; i < repetitions; i++)
{
std::cout << "Warmup " << i << std::endl;
SendMessages();
ReceiveMessages();
}
}
void Run(int repetitions)
{
for (int i = 0; i < repetitions; i++)
{
std::cout << "Run " << i << std::endl;
SendMessages();
ReceiveMessages();
}
}
void Cleanup() {}
private:
std::string m_eventHubName;
std::string m_eventHubConnectionString;
std::string m_checkpointStoreConnectionString;
std::string m_partitionId{"0"};
std::string m_tenantId;
std::string m_clientId;
std::string m_secret;
uint32_t m_numberToSend;
uint32_t m_batchSize;
uint32_t m_prefetchCount;
size_t m_messageBodySize;
int m_rounds{10};
std::unique_ptr<Azure::Messaging::EventHubs::ProducerClient> m_client;
std::shared_ptr<Azure::Core::Credentials::TokenCredential> m_credential;
Models::StartPosition m_receiveStartPosition;
void SendEventsToPartition(Azure::Core::Context const& context)
{
auto beforeSendProps = m_client->GetPartitionProperties(m_partitionId, context);
std::vector<uint8_t> bodyData(m_messageBodySize, 'a');
Azure::Messaging::EventHubs::EventDataBatchOptions batchOptions;
batchOptions.PartitionId = m_partitionId;
Azure::Messaging::EventHubs::EventDataBatch batch(batchOptions);
for (uint32_t j = 0; j < m_numberToSend; ++j)
{
Azure::Messaging::EventHubs::Models::EventData event;
event.Body = bodyData;
event.Properties["Number"] = j;
event.Properties["PartitionId"]
= static_cast<Azure::Core::Amqp::Models::AmqpValue>(m_partitionId);
AddEndProperty(event, m_numberToSend);
batch.AddMessage(event);
}
m_client->SendEventDataBatch(batch, context);
auto afterSendProps = m_client->GetPartitionProperties(m_partitionId, context);
m_receiveStartPosition.Inclusive = false;
m_receiveStartPosition.SequenceNumber = beforeSendProps.LastEnqueuedSequenceNumber;
}
void AddEndProperty(Azure::Messaging::EventHubs::Models::EventData& event, uint64_t expectedCount)
{
event.Properties["End"] = expectedCount;
}
void SendMessages()
{
try
{
Azure::Core::Context context;
SendEventsToPartition(context);
}
catch (std::exception const& ex)
{
std::cerr << "Exception " << ex.what();
throw;
}
}
void ReceiveMessages()
{
try
{
Azure::Core::Context context;
ConsumerClientOptions clientOptions;
clientOptions.ApplicationID = "StressConsumerClient";
ConsumerClient consumerClient(
m_eventHubConnectionString, m_eventHubName, DefaultConsumerGroup, clientOptions);
auto consumerProperties = consumerClient.GetEventHubProperties(context);
std::cout << "Starting receive tests for partition " << m_partitionId << std::endl;
std::cout << " Start position: " << m_receiveStartPosition << std::endl;
for (auto round = 0; round < m_rounds; round += 1)
{
ConsumeForBatchTester(round, consumerClient, m_receiveStartPosition, context);
}
}
catch (std::exception const& ex)
{
std::cerr << "Exception " << ex.what();
throw;
}
}
void ConsumeForBatchTester(
uint32_t round,
ConsumerClient& client,
Models::StartPosition const& startPosition,
Azure::Core::Context const& context)
{
PartitionClientOptions partitionOptions;
partitionOptions.StartPosition = startPosition;
partitionOptions.Prefetch = m_prefetchCount;
PartitionClient partitionClient{client.CreatePartitionClient(m_partitionId, partitionOptions)};
std::cout << "[r: " << round << "/" << m_rounds << "p: " << m_partitionId
<< "] Starting to receive messages from partition" << std::endl;
size_t total = 0;
// uint32_t numCancels = 0;
// constexpr const uint32_t cancelLimit = 5;
auto events = partitionClient.ReceiveEvents(m_batchSize, context);
total += events.size();
}
};
int main(int argc, char**)
{
try
{
EventHubsStress stressTest;
// some param was passed to the program, doesn't matter what it is,
// it is meant for the moment to just run a quick iteration to check for sanity of the test.
// since prototype TODO: pass in warmup/rounds/requests as params.
if (argc != 1)
{
std::cout << "--------------\tBUILD TEST\t--------------" << std::endl;
stressTest.Warmup(1);
stressTest.Run(5);
stressTest.Cleanup();
std::cout << "--------------\tEND BUILD TEST\t--------------" << std::endl;
return 0;
}
std::cout << "--------------\tSTARTING TEST\t--------------" << std::endl;
std::cout << "--------------\tPRE WARMUP\t--------------" << std::endl;
stressTest.Warmup(WARMUP);
std::cout << "--------------\tPOST WARMUP\t--------------" << std::endl;
for (int i = 0; i < ROUNDS; i++)
{
std::cout << "--------------\tTEST ITERATION:" << i << "\t--------------" << std::endl;
stressTest.Run(REQUESTS);
std::cout << "--------------\tDONE ITERATION:" << i << "\t--------------" << std::endl;
}
stressTest.Cleanup();
}
catch (std::exception const& ex)
{
std::cerr << "Test failed due to exception thrown: " << ex.what() << std::endl;
}
return 0;
}

17
sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/scenarios-matrix.yaml Normal file
Просмотреть файл

@ -0,0 +1,17 @@
displayNames:
# this makes it so these don't show up in the scenario names,
# since they're just clutter.
1.5Gi": ""
4Gi": ""
image: ""
matrix:
images:
cpp:
image: Dockerfile
imageBuildDir: "../../../../../"
scenarios:
constantDetach:
testTarget: azure-messaging-eventhubs-stress-test
memory: "1.5Gi"

128
sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/stress-test-resources.bicep Normal file
Просмотреть файл

@ -0,0 +1,128 @@
@description('The base resource name.')
param baseName string = resourceGroup().name
@description('The location for all resources.')
param location string = resourceGroup().location
@description('Storage endpoint suffix. The default value uses Azure Public Cloud (core.windows.net)')
param storageEndpointSuffix string = 'core.windows.net'
var apiVersion = '2017-04-01'
var storageApiVersion = '2019-04-01'
//var iotApiVersion = '2018-04-01'
var namespaceName = baseName
var storageAccountName = 'storage${baseName}'
var containerName = 'container'
//var iotName = 'iot${baseName}'
var authorizationName = '${baseName}/RootManageSharedAccessKey'
var eventHubName = 'eventhub'
var eventHubFullName = '${baseName}/eventhub'
//var location = resourceGroup().location
resource namespace 'Microsoft.EventHub/namespaces@2017-04-01' = {
name: namespaceName
location: location
sku: {
name: 'Standard'
tier: 'Standard'
capacity: 5
}
properties: {
// zoneRedundant: false
isAutoInflateEnabled: false
maximumThroughputUnits: 0
}
}
resource authorization 'Microsoft.EventHub/namespaces/AuthorizationRules@2017-04-01' = {
name: authorizationName
// location: location
properties: {
rights: [
'Listen'
'Manage'
'Send'
]
}
dependsOn: [
namespace
]
}
resource eventHubNameFull 'Microsoft.EventHub/namespaces/eventhubs@2017-04-01' = {
name: eventHubFullName
// location: location
properties: {
messageRetentionInDays: 7
partitionCount: 32
}
dependsOn: [
namespace
]
}
resource namespaceName_default 'Microsoft.EventHub/namespaces/networkRuleSets@2017-04-01' = {
parent:namespace
name: 'default'
// location: location
properties: {
defaultAction: 'Deny'
virtualNetworkRules: []
ipRules: []
}
dependsOn: [
]
}
resource eventHubNameFull_Default 'Microsoft.EventHub/namespaces/eventhubs/consumergroups@2017-04-01' = {
parent: eventHubNameFull
name: '$Default'
// location: location
properties: {}
dependsOn: [
namespace
]
}
resource storageAccount 'Microsoft.Storage/storageAccounts@2019-04-01' = {
name: storageAccountName
location: location
sku: {
name: 'Standard_RAGRS'
// tier: 'Standard'
}
kind: 'StorageV2'
properties: {
networkAcls: {
bypass: 'AzureServices'
virtualNetworkRules: []
ipRules: []
defaultAction: 'Allow'
}
supportsHttpsTrafficOnly: true
encryption: {
services: {
file: {
enabled: true
}
blob: {
enabled: true
}
}
keySource: 'Microsoft.Storage'
}
accessTier: 'Hot'
}
}
resource storageAccountName_default_container 'Microsoft.Storage/storageAccounts/blobServices/containers@2019-04-01' = {
name: '${storageAccountName}/default/${containerName}'
dependsOn: [
storageAccount
]
}
output EVENTHUB_NAME string = eventHubName
output EVENTHUB_CONNECTION_STRING string = listKeys(resourceId('Microsoft.EventHub/namespaces/authorizationRules', namespaceName, 'RootManageSharedAccessKey'), apiVersion).primaryConnectionString
output CHECKPOINTSTORE_STORAGE_CONNECTION_STRING string = 'DefaultEndpointsProtocol=https;AccountName=${storageAccountName};AccountKey=${listKeys(storageAccount.id, storageApiVersion).keys[0].value};EndpointSuffix=${storageEndpointSuffix}'

25
sdk/eventhubs/azure-messaging-eventhubs/test/eventhubs-stress-test/templates/deploy-job.yaml Normal file
Просмотреть файл

@ -0,0 +1,25 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
{{- include "stress-test-addons.deploy-job-template.from-pod" (list . "stress.deploy-example") -}}
{{- define "stress.deploy-example" -}}
metadata:
labels:
chaos: "{{ default false .Stress.chaos }}"
spec:
containers:
- name: main
image: {{ .Stress.imageTag }}
imagePullPolicy: Always
command:
[
"valgrind",
"--tool=memcheck",
"-s",
"./azure-messaging-eventhubs-stress-test",
]
{{- include "stress-test-addons.container-env" . | nindent 6 }}
resources:
limits:
memory: {{.Stress.memory }}
cpu: "1"
{{- end -}}

38
sdk/eventhubs/azure-messaging-eventhubs/test/perf/CMakeLists.txt Normal file
Просмотреть файл

@ -0,0 +1,38 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# Configure CMake project.
cmake_minimum_required (VERSION 3.13)
project(azure-messaging-eventhubs-perf LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED True)
set(
AZURE_EVENTHUBS_PERF_TEST_HEADER
inc/azure/messaging/eventhubs/test/eventhubs_batch_perf_test.hpp
)
set(
AZURE_EVENTHUBS_PERF_TEST_SOURCE
src/azure_eventhubs_perf_test.cpp
)
# Name the binary to be created.
add_executable (
azure-messaging-eventhubs-perf
${AZURE_EVENTHUBS_PERF_TEST_HEADER} ${AZURE_EVENTHUBS_PERF_TEST_SOURCE}
)
create_per_service_target_build(eventhubs azure-messaging-eventhubs-perf)
create_map_file(azure-messaging-eventhubs-perf azure-messaging-eventhubs-perf.map)
# Include the headers from the project.
target_include_directories(
azure-messaging-eventhubs-perf
PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/inc>
)
# link the `azure-perf` lib together with any other library which will be used for the tests.
target_link_libraries(azure-messaging-eventhubs-perf PRIVATE azure-identity azure-messaging-eventhubs azure-perf)
# Make sure the project will appear in the test folder for Visual Studio CMake view
set_target_properties(azure-messaging-eventhubs-perf PROPERTIES FOLDER "Tests/eventhubs")

272
sdk/eventhubs/azure-messaging-eventhubs/test/perf/inc/azure/messaging/eventhubs/test/eventhubs_batch_perf_test.hpp Normal file
Просмотреть файл

@ -0,0 +1,272 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
/**
* @file
* @brief Test batch sends
*
*/
#pragma once
#include <azure/core/internal/environment.hpp>
#include <azure/identity.hpp>
#include <azure/messaging/eventhubs/consumer_client.hpp>
#include <azure/messaging/eventhubs/models/partition_client_models.hpp>
#include <azure/messaging/eventhubs/producer_client.hpp>
#include <azure/perf.hpp>
#include <memory>
#include <string>
#include <vector>
namespace Azure { namespace Messaging { namespace EventHubs { namespace PerfTest { namespace Batch {
/**
* @brief A test to measure getting a key performance.
*
*/
class BatchTest : public Azure::Perf::PerfTest {
private:
std::string m_eventHubName;
std::string m_eventHubConnectionString;
std::string m_partitionId;
std::string m_checkpointStoreConnectionString;
std::string m_tenantId;
std::string m_clientId;
std::string m_secret;
uint32_t m_numberToSend;
uint32_t m_batchSize;
uint32_t m_prefetchCount;
uint64_t m_rounds;
uint32_t m_paddingBytes;
uint32_t m_maxDeadlineExceeded;
std::shared_ptr<Azure::Identity::ClientSecretCredential> m_credential;
std::unique_ptr<Azure::Messaging::EventHubs::ProducerClient> m_client;
public:
/**
* @brief Get the Ids and secret
*
*/
void Setup() override
{
m_eventHubName = m_options.GetOptionOrDefault<std::string>(
"EventHubName", Azure::Core::_internal::Environment::GetVariable("EVENTHUB_NAME"));
m_eventHubConnectionString = m_options.GetOptionOrDefault<std::string>(
"EventHubConnectionString",
Azure::Core::_internal::Environment::GetVariable("EVENTHUB_CONNECTION_STRING"));
m_checkpointStoreConnectionString = m_options.GetOptionOrDefault<std::string>(
"CheckpointStoreConnectionString",
Azure::Core::_internal::Environment::GetVariable("CHECKPOINT_STORE_CONNECTION_STRING"));
m_numberToSend = m_options.GetOptionOrDefault<uint32_t>("NumberToSend", 1000);
m_batchSize = m_options.GetOptionOrDefault<uint32_t>("BatchSize", 1000);
m_prefetchCount = m_options.GetOptionOrDefault<uint32_t>("PrefetchCount", 1000);
m_rounds = m_options.GetOptionOrDefault<uint64_t>("Rounds", 100);
m_paddingBytes = m_options.GetOptionOrDefault<uint32_t>("PaddingBytes", 1024);
m_partitionId = m_options.GetOptionOrDefault<std::string>("PartitionId", "0");
m_maxDeadlineExceeded = m_options.GetOptionOrDefault<uint32_t>("MaxTimeouts", 10);
m_tenantId = m_options.GetOptionOrDefault<std::string>(
"TenantId", Azure::Core::_internal::Environment::GetVariable("AZURE_TENANT_ID"));
m_clientId = m_options.GetOptionOrDefault<std::string>(
"ClientId", Azure::Core::_internal::Environment::GetVariable("AZURE_CLIENT_ID"));
m_secret = m_options.GetOptionOrDefault<std::string>(
"Secret", Azure::Core::_internal::Environment::GetVariable("AZURE_CLIENT_SECRET"));
if (m_eventHubConnectionString.empty())
{
m_credential = std::make_shared<Azure::Identity::ClientSecretCredential>(
m_tenantId, m_clientId, m_secret);
m_client = std::make_unique<Azure::Messaging::EventHubs::ProducerClient>(
m_eventHubConnectionString, m_eventHubName, m_credential);
}
else
{
m_client = std::make_unique<Azure::Messaging::EventHubs::ProducerClient>(
m_eventHubConnectionString, m_eventHubName);
}
}
/**
* @brief Construct a new Event Hubs performance test.
*
* @param options The test options.
*/
BatchTest(Azure::Perf::TestOptions options) : PerfTest(options) {}
/**
* @brief Define the test
*
*/
void Run(Azure::Core::Context const& context) override
{
try
{
std::cout << "Starting test with: batch size: " << m_batchSize
<< " Prefetch: " << m_prefetchCount << std::endl;
// Warm up the connection to the remote instance.
auto properties = m_client->GetEventHubProperties(context);
std::cout << "Sending messages to partition " << m_partitionId << std::endl;
std::tuple<Models::StartPosition, Models::EventHubPartitionProperties> sendResult
= SendEventsToPartition(context);
ConsumerClientOptions clientOptions;
clientOptions.ApplicationID = "StressConsumerClient";
ConsumerClient consumerClient(
m_eventHubConnectionString, m_eventHubName, DefaultConsumerGroup, clientOptions);
auto consumerProperties = consumerClient.GetEventHubProperties(context);
std::cout << "Starting receive tests for partition " << m_partitionId << std::endl;
std::cout << " Start position: " << std::get<0>(sendResult) << std::endl;
std::cout << " Partition properties: " << std::get<1>(sendResult) << std::endl;
for (auto i = 0ul; i < m_rounds; i += 1)
{
ConsumeForBatchTester(i, consumerClient, std::get<0>(sendResult), context);
}
}
catch (std::exception const& ex)
{
std::cerr << "Exception thrown processing batch: " << ex.what() << std::endl;
}
}
std::tuple<Models::StartPosition, Models::EventHubPartitionProperties> SendEventsToPartition(
Core::Context const& context)
{
auto beforeSendProps = m_client->GetPartitionProperties(m_partitionId, context);
std::vector<uint8_t> bodyData(m_paddingBytes, 'a');
Azure::Messaging::EventHubs::EventDataBatchOptions batchOptions;
batchOptions.PartitionId = m_partitionId;
Azure::Messaging::EventHubs::EventDataBatch batch(batchOptions);
for (uint32_t j = 0; j < m_numberToSend; ++j)
{
Azure::Messaging::EventHubs::Models::EventData event;
event.Body = bodyData;
event.Properties["Number"] = j;
event.Properties["PartitionId"]
= static_cast<Azure::Core::Amqp::Models::AmqpValue>(m_partitionId);
AddEndProperty(event, m_numberToSend);
batch.AddMessage(event);
}
m_client->SendEventDataBatch(batch, context);
auto afterSendProps = m_client->GetPartitionProperties(m_partitionId, context);
Models::StartPosition startPosition;
startPosition.Inclusive = false;
startPosition.SequenceNumber = beforeSendProps.LastEnqueuedSequenceNumber;
return std::make_tuple(startPosition, afterSendProps);
}
void ConsumeForBatchTester(
uint32_t round,
ConsumerClient& client,
Models::StartPosition const& startPosition,
Core::Context const& context)
{
PartitionClientOptions partitionOptions;
partitionOptions.StartPosition = startPosition;
partitionOptions.Prefetch = m_prefetchCount;
PartitionClient partitionClient{
client.CreatePartitionClient(m_partitionId, partitionOptions)};
std::cout << "[r: " << round << "/" << m_rounds << "p: " << m_partitionId
<< "] Starting to receive messages from partition" << std::endl;
size_t total = 0;
// uint32_t numCancels = 0;
// constexpr const uint32_t cancelLimit = 5;
auto events = partitionClient.ReceiveEvents(m_batchSize, context);
total += events.size();
}
void AddEndProperty(
Azure::Messaging::EventHubs::Models::EventData& event,
uint64_t expectedCount)
{
event.Properties["End"] = expectedCount;
}
/**
* @brief Define the test options for the test.
*
* @return The list of test options.
*/
std::vector<Azure::Perf::TestOption> GetTestOptions() override
{
return {
{"EventHubName", {"--eventHubName"}, "The EventHub name.", 1, false},
{"EventHubConnectionString",
{"--eventHubConnectionString"},
"The EventHub connection string.",
1,
false,
true},
{"CheckpointStoreConnectionString",
{"--checkpointStoreConnectionString"},
"The checkpoint store connection string.",
1,
false},
{"NumberToSend", {"--numberToSend"}, "The number of events to send.", 1, false},
{"BatchSize",
{"--batchSize"},
"Size to request each time we call ReceiveEvents(). Higher batch sizes will require "
"higher amounts of memory for this test.",
1,
false},
{"Timeout", {"--timeout"}, "Time to wait for each batch (ie. 1m, 30s, etc...)", 1, false},
{"PrefetchCount",
{"--prefetchCount"},
"The number of events to set for the prefetch. Negative numbers disable prefetch "
"altogether. 0 uses the default for the package.",
1,
false},
{"Rounds",
{"--rounds"},
"The number of rounds to run with these parameters. -1 means MAX_UINT64.",
1,
false},
{"PaddingBytes",
{"--paddingBytes"},
"THe number of bytes to send in each message body.",
1,
false},
{"partitionId",
{"--partitionId"},
"The partition Id to send and receive events to.",
1,
false},
{"MaxTimeouts", {"--maxTimeouts"}, "The max number of timeouts.", 1, false},
{"TenantId", {"--tenantId"}, "The tenant Id for the authentication.", 1, false},
{"ClientId", {"--clientId"}, "The client Id for the authentication.", 1, false},
{"Secret", {"--secret"}, "The secret for authentication.", 1, false, true}};
}
/**
* @brief Get the static Test Metadata for the test.
*
* @return Azure::Perf::TestMetadata describing the test.
*/
static Azure::Perf::TestMetadata GetTestMetadata()
{
return {"Batch", "Batch Processing", [](Azure::Perf::TestOptions options) {
return std::make_unique<Azure::Messaging::EventHubs::PerfTest::Batch::BatchTest>(
options);
}};
}
};
}}}}} // namespace Azure::Messaging::EventHubs::PerfTest::Batch

18
sdk/eventhubs/azure-messaging-eventhubs/test/perf/src/azure_eventhubs_perf_test.cpp Normal file
Просмотреть файл

@ -0,0 +1,18 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#include "azure/messaging/eventhubs/test/eventhubs_batch_perf_test.hpp"
#include <azure/perf.hpp>
int main(int argc, char** argv)
{
// Create the test list
std::vector<Azure::Perf::TestMetadata> tests{
Azure::Messaging::EventHubs::PerfTest::Batch::BatchTest::GetTestMetadata()};
Azure::Perf::Program::Run(Azure::Core::Context::ApplicationContext, tests, argc, argv);
return 0;
}

62
sdk/eventhubs/azure-messaging-eventhubs/test/ut/checkpoint_store_test.cpp
Просмотреть файл

@ -13,22 +13,39 @@
namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
class CheckpointStoreTest : public EventHubsTestBase {
virtual void SetUp() override
{
EventHubsTestBase::SetUp();
m_blobClientOptions = InitClientOptions<Azure::Storage::Blobs::BlobClientOptions>();
}
protected:
std::string GetRandomName()
{
std::string name = "checkpoint";
if (m_testContext.IsLiveMode())
{
name.append(Azure::Core::Uuid::CreateUuid().ToString());
}
else
{
name.append("-recording");
}
return name;
}
Azure::Storage::Blobs::BlobClientOptions m_blobClientOptions;
};
std::string GetRandomName()
{
std::string name = "checkpoint";
name.append(Azure::Core::Uuid::CreateUuid().ToString());
return name;
}
TEST_F(CheckpointStoreTest, TestCheckpoints_LIVEONLY_)
TEST_F(CheckpointStoreTest, TestCheckpoints)
{
std::string const testName = GetRandomName();
Azure::Messaging::EventHubs::BlobCheckpointStore checkpointStore(
auto containerClient{Azure::Storage::Blobs::BlobContainerClient::CreateFromConnectionString(
Azure::Core::_internal::Environment::GetVariable(
"CHECKPOINTSTORE_STORAGE_CONNECTION_STRING"),
testName);
testName,
m_blobClientOptions)};
Azure::Messaging::EventHubs::BlobCheckpointStore checkpointStore(containerClient);
auto checkpoints = checkpointStore.ListCheckpoints(
"fully-qualified-namespace", "event-hub-name", "consumer-group");
@ -49,8 +66,8 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
EXPECT_EQ(checkpoints.size(), 1ul);
EXPECT_EQ("$Default", checkpoints[0].ConsumerGroup);
EXPECT_EQ("event-hub-name", checkpoints[0].EventHubName);
EXPECT_EQ("ns.servicebus.windows.net", checkpoints[0].EventHubHostName);
EXPECT_EQ("partition-id", checkpoints[0].PartitionID);
EXPECT_EQ("ns.servicebus.windows.net", checkpoints[0].FullyQualifiedNamespaceName);
EXPECT_EQ("partition-id", checkpoints[0].PartitionId);
EXPECT_EQ(202, checkpoints[0].SequenceNumber.Value());
EXPECT_EQ(101, checkpoints[0].Offset.Value());
@ -68,19 +85,22 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
EXPECT_EQ(checkpoints.size(), 1ul);
EXPECT_EQ("$Default", checkpoints[0].ConsumerGroup);
EXPECT_EQ("event-hub-name", checkpoints[0].EventHubName);
EXPECT_EQ("ns.servicebus.windows.net", checkpoints[0].EventHubHostName);
EXPECT_EQ("partition-id", checkpoints[0].PartitionID);
EXPECT_EQ("ns.servicebus.windows.net", checkpoints[0].FullyQualifiedNamespaceName);
EXPECT_EQ("partition-id", checkpoints[0].PartitionId);
EXPECT_EQ(203, checkpoints[0].SequenceNumber.Value());
EXPECT_EQ(102, checkpoints[0].Offset.Value());
}
TEST_F(CheckpointStoreTest, TestOwnerships_LIVEONLY_)
TEST_F(CheckpointStoreTest, TestOwnerships)
{
std::string const testName = GetRandomName();
Azure::Messaging::EventHubs::BlobCheckpointStore checkpointStore(
auto containerClient{Azure::Storage::Blobs::BlobContainerClient::CreateFromConnectionString(
Azure::Core::_internal::Environment::GetVariable(
"CHECKPOINTSTORE_STORAGE_CONNECTION_STRING"),
testName);
testName,
m_blobClientOptions)};
Azure::Messaging::EventHubs::BlobCheckpointStore checkpointStore(containerClient);
auto ownerships = checkpointStore.ListOwnership(
"fully-qualified-namespace", "event-hub-name", "consumer-group");
@ -104,8 +124,8 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
EXPECT_EQ("$Default", ownerships[0].ConsumerGroup);
EXPECT_EQ("event-hub-name", ownerships[0].EventHubName);
EXPECT_EQ("ns.servicebus.windows.net", ownerships[0].FullyQualifiedNamespace);
EXPECT_EQ("partition-id", ownerships[0].PartitionID);
EXPECT_EQ("owner-id", ownerships[0].OwnerID);
EXPECT_EQ("partition-id", ownerships[0].PartitionId);
EXPECT_EQ("owner-id", ownerships[0].OwnerId);
EXPECT_TRUE(ownerships[0].ETag.HasValue());
EXPECT_TRUE(ownerships[0].LastModifiedTime.HasValue());
Azure::ETag validEtag = ownerships[0].ETag.Value();
@ -139,7 +159,7 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
EXPECT_EQ("$Default", ownerships[0].ConsumerGroup);
EXPECT_EQ("event-hub-name", ownerships[0].EventHubName);
EXPECT_EQ("ns.servicebus.windows.net", ownerships[0].FullyQualifiedNamespace);
EXPECT_EQ("partition-id", ownerships[0].PartitionID);
EXPECT_EQ("owner-id", ownerships[0].OwnerID);
EXPECT_EQ("partition-id", ownerships[0].PartitionId);
EXPECT_EQ("owner-id", ownerships[0].OwnerId);
}
}}}} // namespace Azure::Messaging::EventHubs::Test

47
sdk/eventhubs/azure-messaging-eventhubs/test/ut/consumer_client_test.cpp
Просмотреть файл

@ -17,7 +17,7 @@ int i = 0;
void ProcessMessageSuccess(Azure::Core::Amqp::Models::AmqpMessage const& message)
{
(void)message;
std::cout << "Message Id: " << i++ << std::endl;
GTEST_LOG_(INFO) << "Message Id: " << i++ << std::endl;
}
} // namespace LocalTest
namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
@ -46,10 +46,9 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
TEST_F(ConsumerClientTest, ConnectionStringEntityPathNoConsumerGroup_LIVEONLY_)
{
std::string const connStringNoEntityPath
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=hehe";
std::string const connStringNoEntityPath = GetEnv("EVENTHUB_CONNECTION_STRING");
auto client = Azure::Messaging::EventHubs::ConsumerClient(connStringNoEntityPath, "eventhub");
EXPECT_EQ("hehe", client.GetEventHubName());
EXPECT_EQ("eventhub", client.GetEventHubName());
EXPECT_EQ("$Default", client.GetConsumerGroup());
}
@ -64,27 +63,26 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
TEST_F(ConsumerClientTest, ConnectToPartition_LIVEONLY_)
{
std::string eventHubName{GetEnv("EVENTHUB_NAME")};
std::string const connStringNoEntityPath
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + GetEnv("EVENTHUB_NAME");
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + eventHubName;
Azure::Messaging::EventHubs::ConsumerClientOptions options;
options.ApplicationID = "unit-test";
options.ReceiverOptions.Name = "unit-test";
options.ReceiverOptions.SettleMode = Azure::Core::Amqp::_internal::ReceiverSettleMode::First;
options.ReceiverOptions.MessageTarget = "ingress";
options.ReceiverOptions.EnableTrace = true;
options.ReceiverOptions.MaxMessageSize = std::numeric_limits<uint16_t>::max();
options.Name = "unit-test";
auto client = Azure::Messaging::EventHubs::ConsumerClient(
connStringNoEntityPath, GetEnv("EVENTHUB_NAME"), "$Default", options);
connStringNoEntityPath, eventHubName, "$Default", options);
Azure::Messaging::EventHubs::PartitionClientOptions partitionOptions;
partitionOptions.StartPosition.Inclusive = true;
// We want to consume all messages from the earliest.
partitionOptions.StartPosition.Earliest = true;
Azure::Messaging::EventHubs::PartitionClient partitionClient
= client.CreatePartitionClient("1", partitionOptions);
auto events = partitionClient.ReceiveEvents(1);
EXPECT_EQ(events.size(), 1ul);
GTEST_LOG_(INFO) << "Received message " << events[0].RawAmqpMessage();
GTEST_LOG_(INFO) << "Received message " << events[0].GetRawAmqpMessage();
EXPECT_TRUE(events[0].EnqueuedTime.HasValue());
EXPECT_TRUE(events[0].SequenceNumber.HasValue());
EXPECT_TRUE(events[0].Offset.HasValue());
@ -92,17 +90,14 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
TEST_F(ConsumerClientTest, GetEventHubProperties_LIVEONLY_)
{
std::string eventHubName{GetEnv("EVENTHUB_NAME")};
std::string const connStringEntityPath
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + GetEnv("EVENTHUB_NAME");
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + eventHubName;
Azure::Messaging::EventHubs::ConsumerClientOptions options;
options.ApplicationID = "unit-test";
options.ReceiverOptions.Name = "unit-test";
options.ReceiverOptions.SettleMode = Azure::Core::Amqp::_internal::ReceiverSettleMode::First;
options.ReceiverOptions.MessageTarget = "ingress";
options.ReceiverOptions.EnableTrace = true;
options.ReceiverOptions.MaxMessageSize = std::numeric_limits<uint16_t>::max();
options.Name = "unit-test";
auto client = Azure::Messaging::EventHubs::ConsumerClient(connStringEntityPath);
Azure::Messaging::EventHubs::PartitionClientOptions partitionOptions;
partitionOptions.StartPosition.Inclusive = true;
@ -111,24 +106,22 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
= client.CreatePartitionClient("0", partitionOptions);
auto result = client.GetEventHubProperties();
EXPECT_EQ(result.Name, "eventhub");
EXPECT_TRUE(result.PartitionIDs.size() > 0);
EXPECT_EQ(result.Name, eventHubName);
EXPECT_TRUE(result.PartitionIds.size() > 0);
}
TEST_F(ConsumerClientTest, GetPartitionProperties_LIVEONLY_)
{
std::string eventHubName{GetEnv("EVENTHUB_NAME")};
std::string const connStringEntityPath
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + GetEnv("EVENTHUB_NAME");
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + eventHubName;
Azure::Messaging::EventHubs::ConsumerClientOptions options;
options.ApplicationID = "unit-test";
options.ReceiverOptions.Name = "unit-test";
options.ReceiverOptions.SettleMode = Azure::Core::Amqp::_internal::ReceiverSettleMode::First;
options.ReceiverOptions.MessageTarget = "ingress";
options.ReceiverOptions.EnableTrace = true;
options.ReceiverOptions.MaxMessageSize = std::numeric_limits<uint16_t>::max();
options.Name = "unit-test";
options.MaxMessageSize = std::numeric_limits<uint16_t>::max();
auto client = Azure::Messaging::EventHubs::ConsumerClient(connStringEntityPath);
Azure::Messaging::EventHubs::PartitionClientOptions partitionOptions;
@ -138,7 +131,7 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
= client.CreatePartitionClient("0", partitionOptions);
auto result = client.GetPartitionProperties("0");
EXPECT_EQ(result.Name, "eventhub");
EXPECT_EQ(result.Name, eventHubName);
EXPECT_EQ(result.PartitionId, "0");
}

322
sdk/eventhubs/azure-messaging-eventhubs/test/ut/event_data_test.cpp
Просмотреть файл

@ -1,9 +1,9 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
#include "../src/private/eventhubs_constants.hpp"
#include "azure/messaging/eventhubs.hpp"
#include "eventhubs_test_base.hpp"
#include "private/event_data_models_private.hpp"
#include <gtest/gtest.h>
@ -19,38 +19,41 @@ TEST_F(EventDataTest, EventDataNew)
{
Azure::Messaging::EventHubs::Models::EventData eventData;
auto message
= Azure::Messaging::EventHubs::_detail::EventDataFactory::EventDataToAmqpMessage(eventData);
auto message{eventData.GetRawAmqpMessage()};
EXPECT_EQ(0ul, message.ApplicationProperties.size());
EXPECT_FALSE(message.Properties.ContentType.HasValue());
EXPECT_FALSE(message.Properties.CorrelationId.HasValue());
EXPECT_FALSE(message.Properties.MessageId.HasValue());
EventData newData;
newData.ContentType = "application/xml";
{
EventData newData;
newData.ContentType = "application/xml";
{
EventData copyData{newData};
EXPECT_EQ(copyData.ContentType.Value(), newData.ContentType.Value());
{
EventData copyData{newData};
EXPECT_EQ(copyData.ContentType.Value(), newData.ContentType.Value());
}
{
EventData moveData{std::move(newData)};
// The contents of newData should be moved to moveData. The state of newData is undefined.
EXPECT_TRUE(moveData.ContentType.HasValue());
EXPECT_EQ(moveData.ContentType.Value(), "application/xml");
}
}
{
EventData moveData{std::move(newData)};
// The contents of newData should be moved to moveData. The state of newData is undefined.
EXPECT_TRUE(moveData.ContentType.HasValue());
EXPECT_EQ(moveData.ContentType.Value(), "application/xml");
}
newData.ContentType = "application/json";
{
EventData copyData;
copyData = newData;
EXPECT_EQ(copyData.ContentType.Value(), newData.ContentType.Value());
}
{
EventData moveData;
moveData = std::move(newData);
EXPECT_TRUE(moveData.ContentType.HasValue());
EventData newData;
newData.ContentType = "application/json";
{
EventData copyData;
copyData = newData;
EXPECT_EQ(copyData.ContentType.Value(), newData.ContentType.Value());
}
{
EventData moveData;
moveData = std::move(newData);
EXPECT_TRUE(moveData.ContentType.HasValue());
}
}
}
@ -58,98 +61,225 @@ TEST_F(EventDataTest, EventData1)
{
Azure::Messaging::EventHubs::Models::EventData eventData;
eventData.Body.Value = Azure::Core::Amqp::Models::AmqpValue(5);
eventData.Body = {1, 2};
eventData.ContentType = "ct";
eventData.Properties.emplace("abc", AmqpValue(23));
eventData.CorrelationId = AmqpValue("ci");
eventData.MessageId = AmqpValue("mi");
auto message
= Azure::Messaging::EventHubs::_detail::EventDataFactory::EventDataToAmqpMessage(eventData);
GTEST_LOG_(INFO) << "Message: " << eventData;
auto message{eventData.GetRawAmqpMessage()};
EXPECT_EQ(1ul, message.ApplicationProperties.size());
EXPECT_EQ(eventData.Body.Value, message.GetBodyAsAmqpValue());
EXPECT_EQ(eventData.Body, static_cast<std::vector<uint8_t>>(message.GetBodyAsBinary()[0]));
EXPECT_EQ("ct", message.Properties.ContentType.Value());
EXPECT_EQ(AmqpValue("ci"), message.Properties.CorrelationId.Value());
EXPECT_TRUE(message.Properties.MessageId.HasValue());
Azure::Messaging::EventHubs::Models::ReceivedEventData receivedEventData(message);
EXPECT_EQ(eventData.Body, receivedEventData.Body);
EXPECT_EQ(eventData.ContentType.HasValue(), receivedEventData.ContentType.HasValue());
if (eventData.ContentType.HasValue())
{
EXPECT_EQ(eventData.ContentType.Value(), receivedEventData.ContentType.Value());
}
EXPECT_EQ(eventData.Properties, receivedEventData.Properties);
EXPECT_EQ(eventData.CorrelationId.HasValue(), receivedEventData.CorrelationId.HasValue());
if (eventData.CorrelationId.HasValue())
{
EXPECT_EQ(eventData.CorrelationId.Value(), receivedEventData.CorrelationId.Value());
}
EXPECT_EQ(eventData.MessageId.HasValue(), receivedEventData.MessageId.HasValue());
if (eventData.MessageId.HasValue())
{
EXPECT_EQ(eventData.MessageId.Value(), receivedEventData.MessageId.Value());
}
GTEST_LOG_(INFO) << "Received message: " << receivedEventData;
}
TEST_F(EventDataTest, EventDataStringBody)
{
Azure::Messaging::EventHubs::Models::EventData eventData{"String Body Message."};
auto message{eventData.GetRawAmqpMessage()};
EXPECT_FALSE(message.Properties.MessageId.HasValue());
EXPECT_EQ(message.BodyType, Azure::Core::Amqp::Models::MessageBodyType::Data);
EXPECT_EQ(message.GetBodyAsBinary().size(), 1ul);
EXPECT_EQ(
message.GetBodyAsBinary()[0],
std::vector<uint8_t>(eventData.Body.begin(), eventData.Body.end()));
}
TEST_F(EventDataTest, EventDataBodyTest)
{
{
Azure::Messaging::EventHubs::Models::EventData msg;
msg.Body.Value = AmqpValue("3");
auto message
= Azure::Messaging::EventHubs::_detail::EventDataFactory::EventDataToAmqpMessage(msg);
EXPECT_EQ(msg.Body.Value, message.GetBodyAsAmqpValue());
}
{
Azure::Messaging::EventHubs::Models::EventData msg;
// Note that Data is an AMQP BinaryData value.
msg.Body.Data = AmqpBinaryData{1, 3, 5, 7, 9};
msg.Body = {1, 3, 5, 7, 9};
auto message
= Azure::Messaging::EventHubs::_detail::EventDataFactory::EventDataToAmqpMessage(msg);
auto message{msg.GetRawAmqpMessage()};
EXPECT_EQ(message.GetBodyAsBinary().size(), 1ul);
EXPECT_EQ(msg.Body.Data, message.GetBodyAsBinary()[0]);
}
{
Azure::Messaging::EventHubs::Models::EventData msg;
// Note that Sequence is an array of AmqpLists.
msg.Body.Sequence = {3, "Foo", AmqpValue{AmqpBinaryData{1, 3, 5, 7, 9}}};
auto message
= Azure::Messaging::EventHubs::_detail::EventDataFactory::EventDataToAmqpMessage(msg);
EXPECT_EQ(message.GetBodyAsAmqpList().size(), 1ul);
EXPECT_EQ(msg.Body.Sequence, message.GetBodyAsAmqpList()[0]);
}
// Data and Value set - error.
{
Azure::Messaging::EventHubs::Models::EventData msg;
msg.Body.Data = AmqpBinaryData{1, 3, 5, 7};
msg.Body.Value = 27;
EXPECT_ANY_THROW(
Azure::Messaging::EventHubs::_detail::EventDataFactory::EventDataToAmqpMessage(msg));
}
// Data and Sequence set - error.
{
Azure::Messaging::EventHubs::Models::EventData msg;
msg.Body.Data = AmqpBinaryData{1, 3, 5, 7};
msg.Body.Sequence = {27, 3.25};
EXPECT_ANY_THROW(
Azure::Messaging::EventHubs::_detail::EventDataFactory::EventDataToAmqpMessage(msg));
}
// Value and Sequence set - error.
{
Azure::Messaging::EventHubs::Models::EventData msg;
msg.Body.Value = "AmqpBinaryData{1, 3, 5, 7}";
msg.Body.Sequence = {27, 3.25};
EXPECT_ANY_THROW(
Azure::Messaging::EventHubs::_detail::EventDataFactory::EventDataToAmqpMessage(msg));
}
// Value, Data, and Sequence set - error.
{
Azure::Messaging::EventHubs::Models::EventData msg;
msg.Body.Data = AmqpBinaryData{1, 7, 32, 127, 255};
msg.Body.Value = "AmqpBinaryData{1, 3, 5, 7}";
msg.Body.Sequence = {27, 3.25};
EXPECT_ANY_THROW(
Azure::Messaging::EventHubs::_detail::EventDataFactory::EventDataToAmqpMessage(msg));
EXPECT_EQ(msg.Body, static_cast<std::vector<uint8_t>>(message.GetBodyAsBinary()[0]));
}
}
TEST_F(EventDataTest, EventDataArrayBody)
{
Azure::Messaging::EventHubs::Models::EventData eventData{1, 3, 5, 7, 9};
auto message{eventData.GetRawAmqpMessage()};
EXPECT_FALSE(message.Properties.MessageId.HasValue());
EXPECT_EQ(message.BodyType, Azure::Core::Amqp::Models::MessageBodyType::Data);
EXPECT_EQ(message.GetBodyAsBinary().size(), 1ul);
EXPECT_EQ(
message.GetBodyAsBinary()[0],
std::vector<uint8_t>(eventData.Body.begin(), eventData.Body.end()));
}
TEST_F(EventDataTest, EventDataVectorBody)
{
std::vector<uint8_t> vector{2, 4, 6, 8, 10};
Azure::Messaging::EventHubs::Models::EventData eventData{vector};
auto message{eventData.GetRawAmqpMessage()};
EXPECT_FALSE(message.Properties.MessageId.HasValue());
EXPECT_EQ(message.BodyType, Azure::Core::Amqp::Models::MessageBodyType::Data);
EXPECT_EQ(message.GetBodyAsBinary().size(), 1ul);
EXPECT_EQ(
message.GetBodyAsBinary()[0],
std::vector<uint8_t>(eventData.Body.begin(), eventData.Body.end()));
}
TEST_F(EventDataTest, ReceivedEventData)
{
{
Azure::Core::Amqp::Models::AmqpMessage message;
message.MessageAnnotations[static_cast<Azure::Core::Amqp::Models::AmqpValue>(
Azure::Core::Amqp::Models::AmqpSymbol{
Azure::Messaging::EventHubs::_detail::PartitionKeyAnnotation})]
= "PartitionKey";
Azure::Messaging::EventHubs::Models::ReceivedEventData receivedEventData(message);
ASSERT_TRUE(receivedEventData.PartitionKey);
EXPECT_EQ(receivedEventData.PartitionKey.Value(), "PartitionKey");
EXPECT_FALSE(receivedEventData.EnqueuedTime);
EXPECT_FALSE(receivedEventData.Offset);
EXPECT_FALSE(receivedEventData.SequenceNumber);
}
{
Azure::Core::Amqp::Models::AmqpMessage message;
Azure::DateTime timeNow{
std::chrono::time_point_cast<std::chrono::milliseconds>(Azure::DateTime::clock::now())};
GTEST_LOG_(INFO) << "timeNow: " << timeNow.ToString();
message.MessageAnnotations[static_cast<Azure::Core::Amqp::Models::AmqpValue>(
Azure::Core::Amqp::Models::AmqpSymbol{
Azure::Messaging::EventHubs::_detail::EnqueuedTimeAnnotation})]
= static_cast<Azure::Core::Amqp::Models::AmqpValue>(
Azure::Core::Amqp::Models::AmqpTimestamp{
std::chrono::duration_cast<std::chrono::milliseconds>(timeNow.time_since_epoch())});
Azure::Messaging::EventHubs::Models::ReceivedEventData receivedEventData(message);
ASSERT_TRUE(receivedEventData.EnqueuedTime.HasValue());
GTEST_LOG_(INFO) << "EnqueuedTime: " << receivedEventData.EnqueuedTime.Value().ToString();
EXPECT_EQ(receivedEventData.EnqueuedTime.Value(), timeNow);
EXPECT_FALSE(receivedEventData.PartitionKey);
EXPECT_FALSE(receivedEventData.Offset);
EXPECT_FALSE(receivedEventData.SequenceNumber);
}
{
Azure::Core::Amqp::Models::AmqpMessage message;
message.MessageAnnotations[static_cast<Azure::Core::Amqp::Models::AmqpValue>(
Azure::Core::Amqp::Models::AmqpSymbol{
Azure::Messaging::EventHubs::_detail::SequenceNumberAnnotation})]
= static_cast<int64_t>(235);
Azure::Messaging::EventHubs::Models::ReceivedEventData receivedEventData(message);
ASSERT_TRUE(receivedEventData.SequenceNumber);
EXPECT_EQ(receivedEventData.SequenceNumber.Value(), 235);
EXPECT_FALSE(receivedEventData.EnqueuedTime);
EXPECT_FALSE(receivedEventData.PartitionKey);
EXPECT_FALSE(receivedEventData.Offset);
}
{
Azure::Core::Amqp::Models::AmqpMessage message;
message.MessageAnnotations[static_cast<Azure::Core::Amqp::Models::AmqpValue>(
Azure::Core::Amqp::Models::AmqpSymbol{
Azure::Messaging::EventHubs::_detail::OffsetNumberAnnotation})]
= 54644;
Azure::Messaging::EventHubs::Models::ReceivedEventData receivedEventData(message);
ASSERT_TRUE(receivedEventData.Offset);
EXPECT_EQ(receivedEventData.Offset.Value(), 54644);
EXPECT_FALSE(receivedEventData.SequenceNumber);
EXPECT_FALSE(receivedEventData.EnqueuedTime);
EXPECT_FALSE(receivedEventData.PartitionKey);
}
{
Azure::Core::Amqp::Models::AmqpMessage message;
message.MessageAnnotations[static_cast<Azure::Core::Amqp::Models::AmqpValue>(
Azure::Core::Amqp::Models::AmqpSymbol{
Azure::Messaging::EventHubs::_detail::OffsetNumberAnnotation})]
= "54644";
Azure::Messaging::EventHubs::Models::ReceivedEventData receivedEventData(message);
ASSERT_TRUE(receivedEventData.Offset);
EXPECT_EQ(receivedEventData.Offset.Value(), 54644);
EXPECT_FALSE(receivedEventData.SequenceNumber);
EXPECT_FALSE(receivedEventData.EnqueuedTime);
EXPECT_FALSE(receivedEventData.PartitionKey);
}
{
Azure::Core::Amqp::Models::AmqpMessage message;
message.MessageAnnotations[static_cast<Azure::Core::Amqp::Models::AmqpValue>(
Azure::Core::Amqp::Models::AmqpSymbol{
Azure::Messaging::EventHubs::_detail::OffsetNumberAnnotation})]
= static_cast<uint32_t>(53);
Azure::Messaging::EventHubs::Models::ReceivedEventData receivedEventData(message);
ASSERT_TRUE(receivedEventData.Offset);
EXPECT_EQ(receivedEventData.Offset.Value(), 53);
EXPECT_FALSE(receivedEventData.SequenceNumber);
EXPECT_FALSE(receivedEventData.EnqueuedTime);
EXPECT_FALSE(receivedEventData.PartitionKey);
}
{
Azure::Core::Amqp::Models::AmqpMessage message;
message.MessageAnnotations[static_cast<Azure::Core::Amqp::Models::AmqpValue>(
Azure::Core::Amqp::Models::AmqpSymbol{
Azure::Messaging::EventHubs::_detail::OffsetNumberAnnotation})]
= static_cast<int32_t>(57);
Azure::Messaging::EventHubs::Models::ReceivedEventData receivedEventData(message);
EXPECT_TRUE(receivedEventData.Offset);
EXPECT_EQ(receivedEventData.Offset.Value(), 57);
EXPECT_FALSE(receivedEventData.SequenceNumber);
EXPECT_FALSE(receivedEventData.EnqueuedTime);
EXPECT_FALSE(receivedEventData.PartitionKey);
}
{
Azure::Core::Amqp::Models::AmqpMessage message;
message.MessageAnnotations[static_cast<Azure::Core::Amqp::Models::AmqpValue>(
Azure::Core::Amqp::Models::AmqpSymbol{
Azure::Messaging::EventHubs::_detail::OffsetNumberAnnotation})]
= static_cast<uint64_t>(661011);
Azure::Messaging::EventHubs::Models::ReceivedEventData receivedEventData(message);
EXPECT_TRUE(receivedEventData.Offset);
EXPECT_EQ(receivedEventData.Offset.Value(), 661011);
EXPECT_FALSE(receivedEventData.SequenceNumber);
EXPECT_FALSE(receivedEventData.EnqueuedTime);
EXPECT_FALSE(receivedEventData.PartitionKey);
}
{
Azure::Core::Amqp::Models::AmqpMessage message;
message.MessageAnnotations[static_cast<Azure::Core::Amqp::Models::AmqpValue>(
Azure::Core::Amqp::Models::AmqpSymbol{
Azure::Messaging::EventHubs::_detail::OffsetNumberAnnotation})]
= static_cast<int64_t>(1412612);
Azure::Messaging::EventHubs::Models::ReceivedEventData receivedEventData(message);
EXPECT_TRUE(receivedEventData.Offset);
EXPECT_EQ(receivedEventData.Offset.Value(), 1412612);
EXPECT_FALSE(receivedEventData.SequenceNumber);
EXPECT_FALSE(receivedEventData.EnqueuedTime);
EXPECT_FALSE(receivedEventData.PartitionKey);
}
}

7
sdk/eventhubs/azure-messaging-eventhubs/test/ut/eventhubs_test_base.hpp
Просмотреть файл

@ -6,9 +6,16 @@
#include <azure/core/test/test_base.hpp>
class EventHubsTestBase : public Azure::Core::Test::TestBase {
public:
EventHubsTestBase() { TestBase::SetUpTestSuiteLocal(AZURE_TEST_ASSETS_DIR); }
// Create
virtual void SetUp() override
{
Azure::Core::Test::TestBase::SetUpTestBase(AZURE_TEST_RECORDING_DIR);
}
virtual void TearDown() override
{
// Make sure you call the base classes TearDown method to ensure recordings are made.
TestBase::TearDown();
}
};

6
sdk/eventhubs/azure-messaging-eventhubs/test/ut/processor_load_balancer_test.cpp
Просмотреть файл

@ -47,7 +47,7 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
std::map<std::string, std::vector<std::string>> byOwnerID{};
for (auto const& ownership : ownerships)
{
byOwnerID[ownership.OwnerID].push_back(ownership.PartitionID);
byOwnerID[ownership.OwnerId].push_back(ownership.PartitionId);
}
return byOwnerID;
@ -123,8 +123,8 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
auto const& ownerships = loadBalancer.LoadBalance(std::vector<std::string>{"0", "1", "2", "3"});
EXPECT_EQ(ownerships.size(), 2ul);
EXPECT_TRUE(ownerships[0].PartitionID == "1" || ownerships[0].PartitionID == "2");
EXPECT_TRUE(ownerships[1].PartitionID == "1" || ownerships[1].PartitionID == "2");
EXPECT_TRUE(ownerships[0].PartitionId == "1" || ownerships[0].PartitionId == "2");
EXPECT_TRUE(ownerships[1].PartitionId == "1" || ownerships[1].PartitionId == "2");
auto finalOwneships = loadBalancer.m_checkpointStore->ListOwnership(
testEventHubFQDN, testEventHubName, testConsumerGroup);

21
sdk/eventhubs/azure-messaging-eventhubs/test/ut/processor_test.cpp
Просмотреть файл

@ -26,22 +26,23 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
TEST_F(ProcessorTest, LoadBalancing_LIVEONLY_)
{
std::string const testName = GetRandomName();
Azure::Messaging::EventHubs::BlobCheckpointStore checkpointStore(
GetEnv("CHECKPOINTSTORE_STORAGE_CONNECTION_STRING"), testName);
auto containerClient{Azure::Storage::Blobs::BlobContainerClient::CreateFromConnectionString(
Azure::Core::_internal::Environment::GetVariable(
"CHECKPOINTSTORE_STORAGE_CONNECTION_STRING"),
testName)};
Azure::Messaging::EventHubs::BlobCheckpointStore checkpointStore(containerClient);
std::string eventHubName{GetEnv("EVENTHUB_NAME")};
std::string const connStringNoEntityPath
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + GetEnv("EVENTHUB_NAME");
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + eventHubName;
Azure::Messaging::EventHubs::ConsumerClientOptions options;
options.ApplicationID = "unit-test";
options.ApplicationID = "processor unit test";
options.ReceiverOptions.Name = "unit-test";
options.ReceiverOptions.SettleMode = Azure::Core::Amqp::_internal::ReceiverSettleMode::First;
options.ReceiverOptions.MessageTarget = "ingress";
options.ReceiverOptions.EnableTrace = true;
options.ReceiverOptions.MaxMessageSize = std::numeric_limits<uint16_t>::max();
options.Name = "processor unittest";
auto client = Azure::Messaging::EventHubs::ConsumerClient(
connStringNoEntityPath, "eventhub", "$Default", options);
connStringNoEntityPath, eventHubName, "$Default", options);
ProcessorOptions processorOptions;
processorOptions.LoadBalancingStrategy = Models::ProcessorStrategy::ProcessorStrategyBalanced;
processorOptions.UpdateInterval = std::chrono::seconds(2);

68
sdk/eventhubs/azure-messaging-eventhubs/test/ut/producer_client_test.cpp
Просмотреть файл

@ -19,19 +19,20 @@ class ProducerClientTest : public EventHubsTestBase {
TEST_F(ProducerClientTest, ConnectionStringNoEntityPath_LIVEONLY_)
{
std::string const connStringNoEntityPath = GetEnv("EVENTHUB_CONNECTION_STRING");
std::string eventHubName{GetEnv("EVENTHUB_NAME")};
auto client = Azure::Messaging::EventHubs::ProducerClient(connStringNoEntityPath, "eventhub");
EXPECT_EQ("eventhub", client.GetEventHubName());
auto client = Azure::Messaging::EventHubs::ProducerClient(connStringNoEntityPath, eventHubName);
EXPECT_EQ(eventHubName, client.GetEventHubName());
}
TEST_F(ProducerClientTest, ConnectionStringEntityPath_LIVEONLY_)
{
std::string eventHubName{GetEnv("EVENTHUB_NAME")};
std::string const connStringEntityPath
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + GetEnv("EVENTHUB_NAME");
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + eventHubName;
auto client
= Azure::Messaging::EventHubs::ProducerClient(connStringEntityPath, GetEnv("EVENTHUB_NAME"));
EXPECT_EQ("eventhub", client.GetEventHubName());
auto client = Azure::Messaging::EventHubs::ProducerClient(connStringEntityPath, eventHubName);
EXPECT_EQ(eventHubName, client.GetEventHubName());
}
TEST_F(ProducerClientTest, TokenCredential_LIVEONLY_)
@ -40,45 +41,42 @@ TEST_F(ProducerClientTest, TokenCredential_LIVEONLY_)
GetEnv("EVENTHUBS_TENANT_ID"),
GetEnv("EVENTHUBS_CLIENT_ID"),
GetEnv("EVENTHUBS_CLIENT_SECRET"))};
std::string eventHubName{GetEnv("EVENTHUB_NAME")};
Azure::Messaging::EventHubs::ProducerClientOptions producerOptions;
producerOptions.ApplicationID = "appId";
auto client = Azure::Messaging::EventHubs::ProducerClient(
"gearamaeh1.servicebus.windows.net", "eventhub", credential);
EXPECT_EQ("eventhub", client.GetEventHubName());
"gearamaeh1.servicebus.windows.net", eventHubName, credential);
EXPECT_EQ(eventHubName, client.GetEventHubName());
}
TEST_F(ProducerClientTest, SendMessage_LIVEONLY_)
{
std::string eventHubName{GetEnv("EVENTHUB_NAME")};
std::string const connStringEntityPath
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + GetEnv("EVENTHUB_NAME");
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + eventHubName;
Azure::Messaging::EventHubs::ProducerClientOptions producerOptions;
producerOptions.SenderOptions.Name = "sender-link";
producerOptions.SenderOptions.EnableTrace = true;
producerOptions.SenderOptions.MessageSource = "ingress";
producerOptions.SenderOptions.SettleMode
= Azure::Core::Amqp::_internal::SenderSettleMode::Settled;
producerOptions.SenderOptions.MaxMessageSize = std::numeric_limits<uint16_t>::max();
producerOptions.Name = "sender-link";
producerOptions.ApplicationID = "some";
Azure::Core::Amqp::Models::AmqpMessage message2;
Azure::Messaging::EventHubs::Models::EventData message1;
message2.SetBody(Azure::Core::Amqp::Models::AmqpValue("Hello7"));
message1.Body.Data = Azure::Core::Amqp::Models::AmqpBinaryData{'H', 'e', 'l', 'l', 'o', '2'};
message1.Body = {'H', 'e', 'l', 'l', 'o', '2'};
Azure::Messaging::EventHubs::Models::EventData message3;
message3.Body.Sequence = {'H', 'e', 'l', 'l', 'o', '3'};
message3.Body = {'H', 'e', 'l', 'l', 'o', '3'};
Azure::Messaging::EventHubs::EventDataBatchOptions edboptions;
edboptions.MaxBytes = std::numeric_limits<uint16_t>::max();
edboptions.PartitionID = "1";
edboptions.PartitionId = "1";
Azure::Messaging::EventHubs::EventDataBatch eventBatch(edboptions);
Azure::Messaging::EventHubs::EventDataBatchOptions edboptions2;
edboptions2.MaxBytes = std::numeric_limits<uint16_t>::max();
;
edboptions2.PartitionID = "2";
edboptions2.PartitionId = "2";
Azure::Messaging::EventHubs::EventDataBatch eventBatch2(edboptions2);
eventBatch.AddMessage(message1);
@ -88,7 +86,7 @@ TEST_F(ProducerClientTest, SendMessage_LIVEONLY_)
eventBatch2.AddMessage(message2);
auto client = Azure::Messaging::EventHubs::ProducerClient(
connStringEntityPath, "eventhub", producerOptions);
connStringEntityPath, eventHubName, producerOptions);
for (int i = 0; i < 5; i++)
{
auto result = client.SendEventDataBatch(eventBatch);
@ -98,44 +96,36 @@ TEST_F(ProducerClientTest, SendMessage_LIVEONLY_)
TEST_F(ProducerClientTest, GetEventHubProperties_LIVEONLY_)
{
std::string eventHubName{GetEnv("EVENTHUB_NAME")};
std::string const connStringEntityPath
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + GetEnv("EVENTHUB_NAME");
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + eventHubName;
Azure::Messaging::EventHubs::ProducerClientOptions producerOptions;
producerOptions.SenderOptions.Name = "sender-link";
producerOptions.SenderOptions.EnableTrace = true;
producerOptions.SenderOptions.MessageSource = "ingress";
producerOptions.SenderOptions.SettleMode
= Azure::Core::Amqp::_internal::SenderSettleMode::Settled;
producerOptions.SenderOptions.MaxMessageSize = std::numeric_limits<uint16_t>::max();
producerOptions.Name = "sender-link";
producerOptions.ApplicationID = "some";
auto client = Azure::Messaging::EventHubs::ProducerClient(
connStringEntityPath, "eventhub", producerOptions);
connStringEntityPath, eventHubName, producerOptions);
auto result = client.GetEventHubProperties();
EXPECT_EQ(result.Name, "eventhub");
EXPECT_TRUE(result.PartitionIDs.size() > 0);
EXPECT_EQ(result.Name, eventHubName);
EXPECT_TRUE(result.PartitionIds.size() > 0);
}
TEST_F(ProducerClientTest, GetPartitionProperties_LIVEONLY_)
{
std::string eventHubName{GetEnv("EVENTHUB_NAME")};
std::string const connStringEntityPath
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + GetEnv("EVENTHUB_NAME");
= GetEnv("EVENTHUB_CONNECTION_STRING") + ";EntityPath=" + eventHubName;
Azure::Messaging::EventHubs::ProducerClientOptions producerOptions;
producerOptions.SenderOptions.Name = "sender-link";
producerOptions.SenderOptions.EnableTrace = true;
producerOptions.SenderOptions.MessageSource = "ingress";
producerOptions.SenderOptions.SettleMode
= Azure::Core::Amqp::_internal::SenderSettleMode::Settled;
producerOptions.SenderOptions.MaxMessageSize = std::numeric_limits<uint16_t>::max();
producerOptions.Name = "sender-link";
producerOptions.ApplicationID = "some";
auto client = Azure::Messaging::EventHubs::ProducerClient(
connStringEntityPath, "eventhub", producerOptions);
connStringEntityPath, eventHubName, producerOptions);
auto result = client.GetPartitionProperties("0");
EXPECT_EQ(result.Name, "eventhub");
EXPECT_EQ(result.Name, eventHubName);
EXPECT_EQ(result.PartitionId, "0");
}

18
sdk/eventhubs/azure-messaging-eventhubs/test/ut/test_checkpoint_store.hpp
Просмотреть файл

@ -58,12 +58,12 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
}
std::vector<Azure::Messaging::EventHubs::Models::Ownership> ClaimOwnership(
std::vector<Models::Ownership> partitionOwnership,
std::vector<Models::Ownership> const& partitionOwnership,
Core::Context const& context = {}) override
{
(void)context;
std::vector<Models::Ownership> owned;
for (auto& ownership : partitionOwnership)
for (auto const& ownership : partitionOwnership)
{
Azure::Messaging::EventHubs::Models::Ownership newOwnership = UpdateOwnership(ownership);
if (newOwnership.ETag.HasValue())
@ -75,16 +75,16 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
}
Azure::Messaging::EventHubs::Models::Ownership UpdateOwnership(
Azure::Messaging::EventHubs::Models::Ownership ownership)
Azure::Messaging::EventHubs::Models::Ownership const& ownership)
{
if (ownership.ConsumerGroup.empty() || ownership.EventHubName.empty()
|| ownership.FullyQualifiedNamespace.empty() || ownership.PartitionID.empty())
|| ownership.FullyQualifiedNamespace.empty() || ownership.PartitionId.empty())
{
throw std::runtime_error("Invalid ownership");
}
std::string key = ownership.FullyQualifiedNamespace + "/" + ownership.EventHubName + "/"
+ ownership.ConsumerGroup + "/" + ownership.PartitionID;
+ ownership.ConsumerGroup + "/" + ownership.PartitionId;
if (m_ownerships.find(key) != m_ownerships.end())
{
@ -112,7 +112,7 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
temp.LastModifiedTime
= temp.LastModifiedTime.ValueOr(std::chrono::system_clock::now()) - std::chrono::hours(6);
std::string key = temp.FullyQualifiedNamespace + "/" + temp.EventHubName + "/"
+ temp.ConsumerGroup + "/" + temp.PartitionID;
+ temp.ConsumerGroup + "/" + temp.PartitionId;
m_ownerships[key] = temp;
}
@ -122,12 +122,12 @@ namespace Azure { namespace Messaging { namespace EventHubs { namespace Test {
{
(void)context;
if (checkpoint.ConsumerGroup.empty() || checkpoint.EventHubName.empty()
|| checkpoint.EventHubHostName.empty() || checkpoint.PartitionID.empty())
|| checkpoint.FullyQualifiedNamespaceName.empty() || checkpoint.PartitionId.empty())
{
throw std::runtime_error("Invalid checkpoint");
}
std::string key = checkpoint.EventHubHostName + "/" + checkpoint.EventHubName + "/"
+ checkpoint.ConsumerGroup + "/" + checkpoint.PartitionID;
std::string key = checkpoint.FullyQualifiedNamespaceName + "/" + checkpoint.EventHubName + "/"
+ checkpoint.ConsumerGroup + "/" + checkpoint.PartitionId;
m_checkpoints[key] = checkpoint;
}
};

7
sdk/eventhubs/ci.yml
Просмотреть файл

@ -30,8 +30,8 @@ stages:
CtestRegex: "azure-messaging-eventhubs.*"
LiveTestCtestRegex: "azure-messaging-eventhubs.*"
LiveTestTimeoutInMinutes: 120
LineCoverageTarget: 22
BranchCoverageTarget: 8
LineCoverageTarget: 40
BranchCoverageTarget: 20
Artifacts:
- Name: azure-messaging-eventhubs
Path: azure-messaging-eventhubs
@ -49,6 +49,9 @@ stages:
Value: "non-real-secret"
- Name: AZURE_SUBSCRIPTION_ID
Value: "non-real-sub"
- Name: CHECKPOINTSTORE_STORAGE_CONNECTION_STRING
Value: "DefaultEndpointsProtocol=https;AccountName=notReal;AccountKey=3333333333333333333333333333333333333333333333333333333333333333333333333333333333333333;EndpointSuffix=core.windows.net"
CMakeTestOptions:
- Name: Default
Value: ''