Multiworld Testing Exploration Library: Modules

Multiworld Testing Exploration Library 0
  • Main Page
  • Modules
  • Namespaces
  • Classes
  • Files
Modules
Here is a list of all modules:
MultiWorldTestingCsharp | C# implementation, for sample usage see: https://github.com/sidsen/vowpal_wabbit/blob/v0/cs_test/ExploreOnlySample.cs
MultiWorldTestingCpp | C++ implementation, for sample usage see: https://github.com/sidsen/vowpal_wabbit/blob/v0/explore/explore_sample.cpp
Generated on Wed Nov 12 2014 23:28:15 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MultiWorldTestingCsharp

  • Namespaces
  • Classes
  • Files
Namespaces | Classes
MultiWorldTestingCsharp

C# implementation, for sample usage see: https://github.com/sidsen/vowpal_wabbit/blob/v0/cs_test/ExploreOnlySample.cs. More...

Namespaces

MultiWorldTesting

Classes

class | MultiWorldTesting::EpsilonGreedyExplorer< Ctx >
The epsilon greedy exploration class. More...
class | MultiWorldTesting::TauFirstExplorer< Ctx >
The tau-first exploration class. More...
class | MultiWorldTesting::SoftmaxExplorer< Ctx >
The epsilon greedy exploration class. More...
class | MultiWorldTesting::GenericExplorer< Ctx >
The generic exploration class. More...
class | MultiWorldTesting::BootstrapExplorer< Ctx >
The bootstrap exploration class. More...
class | MultiWorldTesting::MwtExplorer< Ctx >
The top level MwtExplorer class. Using this makes sure that the right bits are recorded and good random actions are chosen. More...
struct | MultiWorldTesting::Feature
Represents a feature in a sparse array. More...
class | MultiWorldTesting::SimpleContext
A sample context class that stores a vector of Features. More...
interface | MultiWorldTesting::IRecorder< Ctx >
Represents a recorder that exposes a method to record exploration data based on generic contexts. More...
interface | MultiWorldTesting::IPolicy< Ctx >
Exposes a method for choosing an action given a generic context. IPolicy objects are passed to (and invoked by) exploration algorithms to specify the default policy behavior. More...
interface | MultiWorldTesting::IScorer< Ctx >
Exposes a method for specifying a score (weight) for each action given a generic context. More...
interface | MultiWorldTesting::IExplorer< Ctx >
interface | MultiWorldTesting::IStringContext

Detailed Description

C# implementation, for sample usage see: https://github.com/sidsen/vowpal_wabbit/blob/v0/cs_test/ExploreOnlySample.cs.

Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MultiWorldTesting Namespace Reference

  • Namespace List
  • Namespace Members
Classes | Typedefs | Functions | Variables
MultiWorldTesting Namespace Reference
MultiWorldTestingCsharp

Classes

class | abstract
class | BootstrapExplorer
The bootstrap exploration class. More...
class | EpsilonGreedyExplorer
The epsilon greedy exploration class. More...
struct | Feature
Represents a feature in a sparse array. More...
class | GenericExplorer
The generic exploration class. More...
interface | IExplorer
interface | IPolicy
Exposes a method for choosing an action given a generic context. IPolicy objects are passed to (and invoked by) exploration algorithms to specify the default policy behavior. More...
interface | IRecorder
Represents a recorder that exposes a method to record exploration data based on generic contexts. More...
interface | IScorer
Exposes a method for specifying a score (weight) for each action given a generic context. More...
interface | IStringContext
class | MwtExplorer
The top level MwtExplorer class. Using this makes sure that the right bits are recorded and good random actions are chosen. More...
class | NativeContext
class | NativePolicy
class | NativeRecorder
class | NativeScorer
class | NativeStringContext
class | SimpleContext
A sample context class that stores a vector of Features. More...
class | SoftmaxExplorer
The epsilon greedy exploration class. More...
class | TauFirstExplorer
The tau-first exploration class. More...

Typedefs

typedef u32 | Native_Policy_Callback (void *explorer, void *context, int index)
typedef void | Native_Scorer_Callback (void *explorer, void *context, float *scores[], u32 *size)
typedef void | Native_Recorder_Callback (void *mwt, void *context, u32 action, float probability, void *unique_key)
typedef void | Native_To_String_Callback (void *explorer, void *string_value)

Functions

private delegate UInt32 | ClrPolicyCallback (IntPtr explorerPtr, IntPtr contextPtr, int index)
private delegate void | ClrScorerCallback (IntPtr explorerPtr, IntPtr contextPtr, IntPtr scores, IntPtr size)
private delegate void | ClrRecorderCallback (IntPtr mwtPtr, IntPtr contextPtr, UInt32 action, float probability, IntPtr uniqueKey)
private delegate void | ClrToStringCallback (IntPtr contextPtr, IntPtr stringValue)
Native_To_String_Callback * | GetCallback ()

Variables

template<class Ctx >
where | Ctx
A sample recorder class that converts the exploration tuple into string format. More...
template<class Ctx >
where public | ToStringCallback< Ctx >
IntPtr | toStringCallbackPtr = Marshal::GetFunctionPointerForDelegate(toStringCallback)
m_callback = static_cast<Native_To_String_Callback*>(toStringCallbackPtr.ToPointer())
ClrToStringCallback | toStringCallback

Detailed Description

Interface for C# version of Multiworld Testing library. For sample usage see: https://github.com/sidsen/vowpal_wabbit/blob/v0/cs_test/ExploreOnlySample.cs

Variable Documentation

template<class Ctx >
where MultiWorldTesting::Ctx

A sample recorder class that converts the exploration tuple into string format.

Template Parameters
Ctx The Context type.
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MultiWorldTesting::EpsilonGreedyExplorer< Ctx > Class Template Reference

  • Class List
  • Class Index
  • Class Hierarchy
  • Class Members
  • MultiWorldTesting
  • EpsilonGreedyExplorer
Public Member Functions | Public Attributes | List of all members
MultiWorldTesting::EpsilonGreedyExplorer< Ctx > Class Template Reference
MultiWorldTestingCsharp

The epsilon greedy exploration class. More...

#include <explore_clr_wrapper.h>

Inheritance diagram for MultiWorldTesting::EpsilonGreedyExplorer< Ctx >:
MultiWorldTesting::IExplorer< Ctx >

Public Member Functions

EpsilonGreedyExplorer (IPolicy< Ctx >^defaultPolicy, float epsilon, UInt32 numActions)
The constructor is the only public member, because this should be used with the MwtExplorer. More...
NativeMultiWorldTesting::EpsilonGreedyExplorer
< NativeContext > * 		Get ()

Public Attributes

internal | __pad0__: virtual UInt32 InvokePolicyCallback(Ctx context
internal int index | override

Detailed Description

template<class Ctx>
class MultiWorldTesting::EpsilonGreedyExplorer< Ctx >

The epsilon greedy exploration class.

This is a good choice if you have no idea which actions should be preferred. Epsilon greedy is also computationally cheap.

Template Parameters
Ctx The Context type.

Constructor & Destructor Documentation

template<class Ctx>
MultiWorldTesting::EpsilonGreedyExplorer< Ctx >::EpsilonGreedyExplorer | ( | IPolicy< Ctx >^ | defaultPolicy,
 |  | float | epsilon,
 |  | UInt32 | numActions
 | ) |  |
inline

The constructor is the only public member, because this should be used with the MwtExplorer.

Parameters
defaultPolicy | A default function which outputs an action given a context.
epsilon | The probability of a random exploration.
numActions | The number of actions to randomize over.

Member Data Documentation

template<class Ctx>
internal int index MultiWorldTesting::EpsilonGreedyExplorer< Ctx >::override
Initial value:
{
return defaultPolicy->ChooseAction(context)
The documentation for this class was generated from the following file:
  • clr/explore_clr_wrapper.h
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MultiWorldTesting::TauFirstExplorer< Ctx > Class Template Reference

  • Class Members
  • MultiWorldTesting
  • TauFirstExplorer
Public Member Functions | Public Attributes | List of all members
MultiWorldTesting::TauFirstExplorer< Ctx > Class Template Reference
MultiWorldTestingCsharp

The tau-first exploration class. More...

#include <explore_clr_wrapper.h>

Inheritance diagram for MultiWorldTesting::TauFirstExplorer< Ctx >:
MultiWorldTesting::IExplorer< Ctx >

Public Member Functions

TauFirstExplorer (IPolicy< Ctx >^defaultPolicy, UInt32 tau, UInt32 numActions)
The constructor is the only public member, because this should be used with the MwtExplorer. More...
NativeMultiWorldTesting::TauFirstExplorer
< NativeContext > * 		Get ()

Public Attributes

internal | __pad0__: virtual UInt32 InvokePolicyCallback(Ctx context
internal int index | override

Detailed Description

template<class Ctx>
class MultiWorldTesting::TauFirstExplorer< Ctx >

The tau-first exploration class.

The tau-first explorer collects precisely tau uniform random exploration events, and then uses the default policy.

Template Parameters
Ctx The Context type.

Constructor & Destructor Documentation

template<class Ctx>
MultiWorldTesting::TauFirstExplorer< Ctx >::TauFirstExplorer | ( | IPolicy< Ctx >^ | defaultPolicy,
 |  | UInt32 | tau,
 |  | UInt32 | numActions
 | ) |  |
inline

The constructor is the only public member, because this should be used with the MwtExplorer.

Parameters
defaultPolicy | A default policy after randomization finishes.
tau | The number of events to be uniform over.
numActions | The number of actions to randomize over.

Member Data Documentation

template<class Ctx>
internal int index MultiWorldTesting::TauFirstExplorer< Ctx >::override
Initial value:
{
return defaultPolicy->ChooseAction(context)
The documentation for this class was generated from the following file:
  • clr/explore_clr_wrapper.h


Multiworld Testing Exploration Library: MultiWorldTesting::SoftmaxExplorer< Ctx > Class Template Reference

  • MultiWorldTesting
  • SoftmaxExplorer
Public Member Functions | Public Attributes | List of all members
MultiWorldTesting::SoftmaxExplorer< Ctx > Class Template Reference
MultiWorldTestingCsharp

The epsilon greedy exploration class. More...

#include <explore_clr_wrapper.h>

Inheritance diagram for MultiWorldTesting::SoftmaxExplorer< Ctx >:
MultiWorldTesting::IExplorer< Ctx >

Public Member Functions

SoftmaxExplorer (IScorer< Ctx >^defaultScorer, float lambda, UInt32 numActions)
The constructor is the only public member, because this should be used with the MwtExplorer. More...
NativeMultiWorldTesting::SoftmaxExplorer
< NativeContext > * 		Get ()

Public Attributes

internal __pad0__: virtual List<float>^ InvokeScorerCallback(Ctx context) override { return defaultScorer->ScoreActions(context)

Detailed Description

template<class Ctx>
class MultiWorldTesting::SoftmaxExplorer< Ctx >

The epsilon greedy exploration class.

In some cases, different actions have a different scores, and you would prefer to choose actions with large scores. Softmax allows you to do that.

Template Parameters
Ctx The Context type.

Constructor & Destructor Documentation

template<class Ctx>
MultiWorldTesting::SoftmaxExplorer< Ctx >::SoftmaxExplorer | ( | IScorer< Ctx >^ | defaultScorer,
 |  | float | lambda,
 |  | UInt32 | numActions
 | ) |  |
inline

The constructor is the only public member, because this should be used with the MwtExplorer.

Parameters
defaultScorer | A function which outputs a score for each action.
lambda | lambda = 0 implies uniform distribution. Large lambda is equivalent to a max.
numActions | The number of actions to randomize over.
The documentation for this class was generated from the following file:
  • clr/explore_clr_wrapper.h


Multiworld Testing Exploration Library: MultiWorldTesting::GenericExplorer< Ctx > Class Template Reference

  • MultiWorldTesting
  • GenericExplorer
Public Member Functions | Public Attributes | List of all members
MultiWorldTesting::GenericExplorer< Ctx > Class Template Reference
MultiWorldTestingCsharp

The generic exploration class. More...

#include <explore_clr_wrapper.h>

Inheritance diagram for MultiWorldTesting::GenericExplorer< Ctx >:
MultiWorldTesting::IExplorer< Ctx >

Public Member Functions

GenericExplorer (IScorer< Ctx >^defaultScorer, UInt32 numActions)
The constructor is the only public member, because this should be used with the MwtExplorer. More...
NativeMultiWorldTesting::GenericExplorer
< NativeContext > * 		Get ()

Public Attributes

internal __pad0__: virtual List<float>^ InvokeScorerCallback(Ctx context) override { return defaultScorer->ScoreActions(context)

Detailed Description

template<class Ctx>
class MultiWorldTesting::GenericExplorer< Ctx >

The generic exploration class.

GenericExplorer provides complete flexibility. You can create any distribution over actions desired, and it will draw from that.

Template Parameters
Ctx The Context type.

Constructor & Destructor Documentation

template<class Ctx>
MultiWorldTesting::GenericExplorer< Ctx >::GenericExplorer | ( | IScorer< Ctx >^ | defaultScorer,
 |  | UInt32 | numActions
 | ) |  |
inline

The constructor is the only public member, because this should be used with the MwtExplorer.

Parameters
defaultScorer | A function which outputs the probability of each action.
numActions | The number of actions to randomize over.
The documentation for this class was generated from the following file:
  • clr/explore_clr_wrapper.h
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MultiWorldTesting::BootstrapExplorer< Ctx > Class Template Reference

  • MultiWorldTesting
  • BootstrapExplorer
Public Member Functions | Public Attributes | List of all members
MultiWorldTesting::BootstrapExplorer< Ctx > Class Template Reference
MultiWorldTestingCsharp

The bootstrap exploration class. More...

#include <explore_clr_wrapper.h>

Inheritance diagram for MultiWorldTesting::BootstrapExplorer< Ctx >:
MultiWorldTesting::IExplorer< Ctx >

Public Member Functions

BootstrapExplorer (cli::array< IPolicy< Ctx >^>^defaultPolicies, UInt32 numActions)
The constructor is the only public member, because this should be used with the MwtExplorer. More...
NativeMultiWorldTesting::BootstrapExplorer
< NativeContext > * 		Get ()

Public Attributes

internal | __pad0__: virtual UInt32 InvokePolicyCallback(Ctx context
internal int index | override

Detailed Description

template<class Ctx>
class MultiWorldTesting::BootstrapExplorer< Ctx >

The bootstrap exploration class.

The Bootstrap explorer randomizes over the actions chosen by a set of default policies. This performs well statistically but can be computationally expensive.

Template Parameters
Ctx The Context type.

Constructor & Destructor Documentation

template<class Ctx>
MultiWorldTesting::BootstrapExplorer< Ctx >::BootstrapExplorer | ( | cli::array< IPolicy< Ctx >^>^ | defaultPolicies,
 |  | UInt32 | numActions
 | ) |  |
inline

The constructor is the only public member, because this should be used with the MwtExplorer.

Parameters
defaultPolicies | A set of default policies to be uniform random over.
numActions | The number of actions to randomize over.

Member Data Documentation

template<class Ctx>
internal int index MultiWorldTesting::BootstrapExplorer< Ctx >::override
Initial value:
{
if (index < 0 || index >= defaultPolicies->Length)
{
throw gcnew InvalidDataException("Internal error: Index of interop bag is out of range.");
}
return defaultPolicies[index]->ChooseAction(context)
The documentation for this class was generated from the following file:
  • clr/explore_clr_wrapper.h
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MultiWorldTesting::MwtExplorer< Ctx > Class Template Reference

  • MultiWorldTesting
  • MwtExplorer
Public Member Functions | Public Attributes | List of all members
MultiWorldTesting::MwtExplorer< Ctx > Class Template Reference
MultiWorldTestingCsharp

The top level MwtExplorer class. Using this makes sure that the right bits are recorded and good random actions are chosen. More...

#include <explore_clr_wrapper.h>

Inheritance diagram for MultiWorldTesting::MwtExplorer< Ctx >:

Public Member Functions

MwtExplorer (String^appId, IRecorder< Ctx >^recorder)
Constructor. More...
UInt32 ChooseAction (IExplorer< Ctx >^explorer, String^unique_key, Ctx context)
Choose_Action should be drop-in replacement for any existing policy function. More...

Public Attributes

internal | __pad0__: virtual void InvokeRecorderCallback(Ctx context
internal UInt32 | action
internal UInt32 float | probability
internal UInt32 float String
unique_key 		override

Detailed Description

template<class Ctx>
class MultiWorldTesting::MwtExplorer< Ctx >

The top level MwtExplorer class. Using this makes sure that the right bits are recorded and good random actions are chosen.

Template Parameters
Ctx The Context type.

Constructor & Destructor Documentation

template<class Ctx >
MultiWorldTesting::MwtExplorer< Ctx >::MwtExplorer | ( | String^ | appId,
 |  | IRecorder< Ctx >^ | recorder
 | ) |  |
inline

Constructor.

Parameters
appId | This should be unique to each experiment to avoid correlation bugs.
recorder | A user-specified class for recording the appropriate bits for use in evaluation and learning.

Member Function Documentation

template<class Ctx >
UInt32 MultiWorldTesting::MwtExplorer< Ctx >::ChooseAction | ( | IExplorer< Ctx >^ | explorer,
 |  | String^ | unique_key,
 |  | Ctx | context
 | ) |  |
inline

Choose_Action should be drop-in replacement for any existing policy function.

Parameters
explorer | An existing exploration algorithm (one of the above) which uses the default policy as a callback.
unique_key | A unique identifier for the experimental unit. This could be a user id, a session id, etc...
context | The context upon which a decision is made. See SimpleContext above for an example.
Returns
An unsigned 32-bit integer representing the 1-based chosen action.

Member Data Documentation

template<class Ctx >
internal UInt32 float String unique_key MultiWorldTesting::MwtExplorer< Ctx >::override
Initial value:
{
recorder->Record(context, action, probability, unique_key)
The documentation for this class was generated from the following file:
  • clr/explore_clr_wrapper.h


Multiworld Testing Exploration Library: MultiWorldTesting::Feature Struct Reference

  • MultiWorldTesting
  • Feature
Public Attributes | List of all members
MultiWorldTesting::Feature Struct Reference
MultiWorldTestingCsharp

Represents a feature in a sparse array. More...

#include <explore_clr_wrapper.h>

Public Attributes

float | Value
UInt32 | Id

Detailed Description

Represents a feature in a sparse array.

The documentation for this struct was generated from the following file:
  • clr/explore_clr_wrapper.h
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MultiWorldTesting::SimpleContext Class Reference

  • MultiWorldTesting
  • SimpleContext
Public Member Functions | Public Attributes | List of all members
MultiWorldTesting::SimpleContext Class Reference
MultiWorldTestingCsharp

A sample context class that stores a vector of Features. More...

#include <explore_clr_wrapper.h>

Inheritance diagram for MultiWorldTesting::SimpleContext:
MultiWorldTesting::IStringContext

Public Member Functions

SimpleContext (cli::array< Feature >^features)
String | ToString () override
cli::array< Feature > | GetFeatures ()

Public Attributes

internal __pad0__: cli::array<Feature>^ Features

Detailed Description

A sample context class that stores a vector of Features.

The documentation for this class was generated from the following file:
  • clr/explore_clr_wrapper.h


Multiworld Testing Exploration Library: MultiWorldTesting::IRecorder< Ctx > Interface Template Reference

  • MultiWorldTesting
  • IRecorder
Public Member Functions | List of all members
MultiWorldTesting::IRecorder< Ctx > Interface Template Referenceabstract
MultiWorldTestingCsharp

Represents a recorder that exposes a method to record exploration data based on generic contexts. More...

#include <explore_interface.h>

Public Member Functions

virtual void Record (Ctx context, UInt32 action, float probability, String^uniqueKey)=0
Records the exploration data associated with a given decision. More...

Detailed Description

template<class Ctx>
interface MultiWorldTesting::IRecorder< Ctx >

Represents a recorder that exposes a method to record exploration data based on generic contexts.

Template Parameters
Ctx The Context type.

Exploration data is specified as a set of tuples <context, action, probability, key> as described below. An application passes an IRecorder object to the constructor. See for a sample IRecorder object.

Member Function Documentation

template<class Ctx>
virtual void MultiWorldTesting::IRecorder< Ctx >::Record | ( | Ctx | context,
 |  | UInt32 | action,
 |  | float | probability,
 |  | String^ | uniqueKey
 | ) |  |
pure virtual

Records the exploration data associated with a given decision.

Parameters
context | A user-defined context for the decision.
action | Chosen by an exploration algorithm given context.
probability | The probability of the chosen action given context.
uniqueKey | A user-defined identifer for the decision.
The documentation for this interface was generated from the following file:
  • clr/explore_interface.h
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MultiWorldTesting::IPolicy< Ctx > Interface Template Reference

  • MultiWorldTesting
  • IPolicy
Public Member Functions | List of all members
MultiWorldTesting::IPolicy< Ctx > Interface Template Referenceabstract
MultiWorldTestingCsharp

Exposes a method for choosing an action given a generic context. IPolicy objects are passed to (and invoked by) exploration algorithms to specify the default policy behavior. More...

#include <explore_interface.h>

Public Member Functions

virtual UInt32 ChooseAction (Ctx context)=0
Determines the action to take for a given context. More...

Detailed Description

template<class Ctx>
interface MultiWorldTesting::IPolicy< Ctx >

Exposes a method for choosing an action given a generic context. IPolicy objects are passed to (and invoked by) exploration algorithms to specify the default policy behavior.

Template Parameters
Ctx The Context type.

Member Function Documentation

template<class Ctx>
virtual UInt32 MultiWorldTesting::IPolicy< Ctx >::ChooseAction ( Ctx context )
pure virtual

Determines the action to take for a given context.

Parameters
context A user-defined context for the decision.
Returns
Index of the action to take (1-based)
The documentation for this interface was generated from the following file:
  • clr/explore_interface.h
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MultiWorldTesting::IScorer< Ctx > Interface Template Reference

  • MultiWorldTesting
  • IScorer
Public Member Functions | List of all members
MultiWorldTesting::IScorer< Ctx > Interface Template Referenceabstract
MultiWorldTestingCsharp

Exposes a method for specifying a score (weight) for each action given a generic context. More...

#include <explore_interface.h>

Public Member Functions

virtual List< float > ScoreActions (Ctx context)=0
Determines the score of each action for a given context. More...

Detailed Description

template<class Ctx>
interface MultiWorldTesting::IScorer< Ctx >

Exposes a method for specifying a score (weight) for each action given a generic context.

Template Parameters
Ctx The Context type.

Member Function Documentation

template<class Ctx>
virtual List<float> MultiWorldTesting::IScorer< Ctx >::ScoreActions ( Ctx context )
pure virtual

Determines the score of each action for a given context.

Parameters
context A user-defined context for the decision.
Returns
Vector of scores indexed by action (1-based).
The documentation for this interface was generated from the following file:
  • clr/explore_interface.h


Multiworld Testing Exploration Library: MultiWorldTesting::IExplorer< Ctx > Interface Template Reference

  • MultiWorldTesting
  • IExplorer
MultiWorldTesting::IExplorer< Ctx > Interface Template Reference
MultiWorldTestingCsharp
Inheritance diagram for MultiWorldTesting::IExplorer< Ctx >:
MultiWorldTesting::BootstrapExplorer< Ctx > MultiWorldTesting::EpsilonGreedyExplorer< Ctx > MultiWorldTesting::GenericExplorer< Ctx > MultiWorldTesting::SoftmaxExplorer< Ctx > MultiWorldTesting::TauFirstExplorer< Ctx >
The documentation for this interface was generated from the following file:
  • clr/explore_interface.h
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MultiWorldTesting::IStringContext Interface Reference

  • MultiWorldTesting
  • IStringContext
Public Member Functions | List of all members
MultiWorldTesting::IStringContext Interface Referenceabstract
MultiWorldTestingCsharp
Inheritance diagram for MultiWorldTesting::IStringContext:
MultiWorldTesting::SimpleContext

Public Member Functions

virtual String ToString ()=0
The documentation for this interface was generated from the following file:
  • clr/explore_interface.h


Multiworld Testing Exploration Library: MultiWorldTestingCpp

Namespaces | Classes
MultiWorldTestingCpp

C++ implementation, for sample usage see: https://github.com/sidsen/vowpal_wabbit/blob/v0/explore/explore_sample.cpp. More...

Namespaces

NativeMultiWorldTesting

Classes

class | NativeMultiWorldTesting::MwtExplorer< Ctx >
class | NativeMultiWorldTesting::IRecorder< Ctx >
class | NativeMultiWorldTesting::IExplorer< Ctx >
class | NativeMultiWorldTesting::IPolicy< Ctx >
class | NativeMultiWorldTesting::IScorer< Ctx >
struct | NativeMultiWorldTesting::StringRecorder< Ctx >
struct | NativeMultiWorldTesting::Feature
class | NativeMultiWorldTesting::SimpleContext
class | NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx >
class | NativeMultiWorldTesting::SoftmaxExplorer< Ctx >
class | NativeMultiWorldTesting::GenericExplorer< Ctx >
class | NativeMultiWorldTesting::TauFirstExplorer< Ctx >
class | NativeMultiWorldTesting::BootstrapExplorer< Ctx >
struct | NativeMultiWorldTesting::murmur_hash
class | NativeMultiWorldTesting::HashUtils
class | NativeMultiWorldTesting::NumberUtils
union | NativeMultiWorldTesting::PRG::int_float
struct | NativeMultiWorldTesting::PRG::prg

Detailed Description

C++ implementation, for sample usage see: https://github.com/sidsen/vowpal_wabbit/blob/v0/explore/explore_sample.cpp.

Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: NativeMultiWorldTesting Namespace Reference

  • Namespace List
  • Namespace Members
Classes | Functions | Variables
NativeMultiWorldTesting Namespace Reference
MultiWorldTestingCpp

Classes

class | BootstrapExplorer
class | EpsilonGreedyExplorer
struct | Feature
class | GenericExplorer
class | HashUtils
class | IExplorer
class | IPolicy
class | IRecorder
class | IScorer
struct | murmur_hash
class | MwtExplorer
class | NumberUtils
class | SimpleContext
class | SoftmaxExplorer
struct | StringRecorder
class | TauFirstExplorer

Functions

uint32_t rotl32 (uint32_t x, int8_t r)

Variables

const size_t | max_int = 100000
const float | max_float = max_int
const float | min_float = 0.00001f
const size_t | max_digits = (size_t) roundf((float) (-log(min_float) / log(10.)))

Detailed Description

Interface for C++ version of Multiworld Testing library. For sample usage see: https://github.com/sidsen/vowpal_wabbit/blob/v0/explore/explore_sample.cpp


Multiworld Testing Exploration Library: NativeMultiWorldTesting::MwtExplorer< Ctx > Class Template Reference

  • NativeMultiWorldTesting
  • MwtExplorer
Public Member Functions | List of all members
NativeMultiWorldTesting::MwtExplorer< Ctx > Class Template Reference
MultiWorldTestingCpp

#include <MWTExplorer.h>

Public Member Functions

MwtExplorer (std::string app_id, IRecorder< Ctx > &recorder)
u32 Choose_Action (IExplorer< Ctx > &explorer, string unique_key, Ctx &context)

Detailed Description

template<class Ctx>
class NativeMultiWorldTesting::MwtExplorer< Ctx >

The top-level MwtExplorer class. Using this enables principled and efficient exploration over a set of possible actions, and ensures that the right bits are recorded.

Constructor & Destructor Documentation

template<class Ctx>
NativeMultiWorldTesting::MwtExplorer< Ctx >::MwtExplorer | ( | std::string | app_id,
 |  | IRecorder< Ctx > & | recorder
 | ) |  |
inline

Constructor

Parameters
appid | This should be unique to your experiment or you risk nasty correlation bugs.
recorder | A user-specified class for recording the appropriate bits for use in evaluation and learning.

Member Function Documentation

template<class Ctx>
u32 NativeMultiWorldTesting::MwtExplorer< Ctx >::Choose_Action | ( | IExplorer< Ctx > & | explorer,
 |  | string | unique_key,
 |  | Ctx & | context
 | ) |  |
inline

Chooses an action by invoking an underlying exploration algorithm. This should be a drop-in replacement for any existing policy function.

Parameters
explorer | An existing exploration algorithm (one of the below) which uses the default policy as a callback.
unique_key | A unique identifier for the experimental unit. This could be a user id, a session id, etc..
context | The context upon which a decision is made. See SimpleContext below for an example.
The documentation for this class was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::IRecorder< Ctx > Class Template Reference

  • NativeMultiWorldTesting
  • IRecorder
Public Member Functions | List of all members
NativeMultiWorldTesting::IRecorder< Ctx > Class Template Referenceabstract
MultiWorldTestingCpp

#include <MWTExplorer.h>

Inheritance diagram for NativeMultiWorldTesting::IRecorder< Ctx >:
NativeMultiWorldTesting::StringRecorder< Ctx >

Public Member Functions

virtual void Record (Ctx &context, u32 action, float probability, string unique_key)=0

Detailed Description

template<class Ctx>
class NativeMultiWorldTesting::IRecorder< Ctx >

Exposes a method to record exploration data based on generic contexts. Exploration data is specified as a set of tuples <context, action, probability, key> as described below. An application passes an IRecorder object to the constructor. See for a sample IRecorder object.

Member Function Documentation

template<class Ctx>
virtual void NativeMultiWorldTesting::IRecorder< Ctx >::Record | ( | Ctx & | context,
 |  | u32 | action,
 |  | float | probability,
 |  | string | unique_key
 | ) |  |
pure virtual

Records the exploration data associated with a given decision.

Parameters
context | A user-defined context for the decision
action | The action chosen by an exploration algorithm given context
probability | The probability the exploration algorithm chose said action
unique_key | A user-defined unique identifer for the decision

Implemented in NativeMultiWorldTesting::StringRecorder< Ctx >.

The documentation for this class was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::IExplorer< Ctx > Class Template Reference

  • NativeMultiWorldTesting
  • IExplorer
Public Member Functions | List of all members
NativeMultiWorldTesting::IExplorer< Ctx > Class Template Referenceabstract
MultiWorldTestingCpp

#include <MWTExplorer.h>

Inheritance diagram for NativeMultiWorldTesting::IExplorer< Ctx >:
NativeMultiWorldTesting::BootstrapExplorer< Ctx > NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx > NativeMultiWorldTesting::GenericExplorer< Ctx > NativeMultiWorldTesting::SoftmaxExplorer< Ctx > NativeMultiWorldTesting::TauFirstExplorer< Ctx >

Public Member Functions

virtual std::tuple< u32, float,
bool > 		Choose_Action (u64 salted_seed, Ctx &context)=0

Detailed Description

template<class Ctx>
class NativeMultiWorldTesting::IExplorer< Ctx >

Exposes a method to choose an action given a generic context, and obtain the relevant exploration bits. Invokes IPolicy::Choose_Action internally. Do not implement this interface yourself: instead, use the various exploration algorithms below, which implement it for you.

Member Function Documentation

template<class Ctx>
virtual std::tuple<u32, float, bool> NativeMultiWorldTesting::IExplorer< Ctx >::Choose_Action | ( | u64 | salted_seed,
 |  | Ctx & | context
 | ) |  |
pure virtual

Determines the action to take and the probability with which it was chosen, for a given context.

Parameters
salted_seed | A PRG seed based on a unique id information provided by the user
context | A user-defined context for the decision
Returns
The action to take, the probability it was chosen, and a flag indicating whether to record this decision
The documentation for this class was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::IPolicy< Ctx > Class Template Reference

  • NativeMultiWorldTesting
  • IPolicy
Public Member Functions | List of all members
NativeMultiWorldTesting::IPolicy< Ctx > Class Template Referenceabstract
MultiWorldTestingCpp

#include <MWTExplorer.h>

Public Member Functions

virtual u32 Choose_Action (Ctx &context)=0

Detailed Description

template<class Ctx>
class NativeMultiWorldTesting::IPolicy< Ctx >

Exposes a method to choose an action given a generic context. IPolicy objects are passed to (and invoked by) exploration algorithms to specify the default policy behavior.

Member Function Documentation

template<class Ctx>
virtual u32 NativeMultiWorldTesting::IPolicy< Ctx >::Choose_Action ( Ctx & context )
pure virtual

Determines the action to take for a given context.

Parameters
context A user-defined context for the decision
Returns
The action to take (1-based index)
The documentation for this class was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::IScorer< Ctx > Class Template Reference

  • NativeMultiWorldTesting
  • IScorer
Public Member Functions | List of all members
NativeMultiWorldTesting::IScorer< Ctx > Class Template Referenceabstract
MultiWorldTestingCpp

#include <MWTExplorer.h>

Public Member Functions

virtual vector< float > Score_Actions (Ctx &context)=0

Detailed Description

template<class Ctx>
class NativeMultiWorldTesting::IScorer< Ctx >

Exposes a method for specifying a score (weight) for each action given a generic context.

Member Function Documentation

template<class Ctx>
virtual vector<float> NativeMultiWorldTesting::IScorer< Ctx >::Score_Actions ( Ctx & context )
pure virtual

Determines the score of each action for a given context.

Parameters
context A user-defined context for the decision
Returns
A vector of scores indexed by action (1-based)
The documentation for this class was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::StringRecorder< Ctx > Struct Template Reference

  • NativeMultiWorldTesting
  • StringRecorder
Public Member Functions | List of all members
NativeMultiWorldTesting::StringRecorder< Ctx > Struct Template Reference
MultiWorldTestingCpp

#include <MWTExplorer.h>

Inheritance diagram for NativeMultiWorldTesting::StringRecorder< Ctx >:
NativeMultiWorldTesting::IRecorder< Ctx >

Public Member Functions

void | Record (Ctx &context, u32 action, float probability, string unique_key)
string | Get_Recording (bool flush=true)

Detailed Description

template<class Ctx>
struct NativeMultiWorldTesting::StringRecorder< Ctx >

A sample recorder class that converts the exploration tuple into string format.

Member Function Documentation

template<class Ctx>
void NativeMultiWorldTesting::StringRecorder< Ctx >::Record | ( | Ctx & | context,
 |  | u32 | action,
 |  | float | probability,
 |  | string | unique_key
 | ) |  |
inlinevirtual

Records the exploration data associated with a given decision.

Parameters
context | A user-defined context for the decision
action | The action chosen by an exploration algorithm given context
probability | The probability the exploration algorithm chose said action
unique_key | A user-defined unique identifer for the decision

Implements NativeMultiWorldTesting::IRecorder< Ctx >.

The documentation for this struct was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::Feature Struct Reference

  • NativeMultiWorldTesting
  • Feature
Public Member Functions | Public Attributes | List of all members
NativeMultiWorldTesting::Feature Struct Reference
MultiWorldTestingCpp

#include <MWTExplorer.h>

Public Member Functions

bool operator== (Feature other_feature)

Public Attributes

float | Value
u32 | Id

Detailed Description

Represents a feature in a sparse array.

The documentation for this struct was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::SimpleContext Class Reference

  • NativeMultiWorldTesting
  • SimpleContext
Public Member Functions | List of all members
NativeMultiWorldTesting::SimpleContext Class Reference
MultiWorldTestingCpp

#include <MWTExplorer.h>

Public Member Functions

SimpleContext (vector< Feature > &features)
vector< Feature > & | Get_Features ()
string | To_String ()

Detailed Description

A sample context class that stores a vector of Features.

The documentation for this class was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx > Class Template Reference

  • NativeMultiWorldTesting
  • EpsilonGreedyExplorer
Public Member Functions | Friends | List of all members
NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx > Class Template Reference
MultiWorldTestingCpp

#include <MWTExplorer.h>

Inheritance diagram for NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx >:
NativeMultiWorldTesting::IExplorer< Ctx >

Public Member Functions

EpsilonGreedyExplorer (IPolicy< Ctx > &default_policy, float epsilon, u32 num_actions)

Friends

class MwtExplorer< Ctx >

Detailed Description

template<class Ctx>
class NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx >

The epsilon greedy exploration algorithm. This is a good choice if you have no idea which actions should be preferred. Epsilon greedy is also computationally cheap.

Constructor & Destructor Documentation

template<class Ctx>
NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx >::EpsilonGreedyExplorer | ( | IPolicy< Ctx > & | default_policy,
 |  | float | epsilon,
 |  | u32 | num_actions
 | ) |  |
inline

The constructor is the only public member, because this should be used with the MwtExplorer.

Parameters
default_policy | A default function which outputs an action given a context.
epsilon | The probability of a random exploration.
num_actions | The number of actions to randomize over.
The documentation for this class was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::SoftmaxExplorer< Ctx > Class Template Reference

  • NativeMultiWorldTesting
  • SoftmaxExplorer
Public Member Functions | Friends | List of all members
NativeMultiWorldTesting::SoftmaxExplorer< Ctx > Class Template Reference
MultiWorldTestingCpp

#include <MWTExplorer.h>

Inheritance diagram for NativeMultiWorldTesting::SoftmaxExplorer< Ctx >:
NativeMultiWorldTesting::IExplorer< Ctx >

Public Member Functions

SoftmaxExplorer (IScorer< Ctx > &default_scorer, float lambda, u32 num_actions)

Friends

class MwtExplorer< Ctx >

Detailed Description

template<class Ctx>
class NativeMultiWorldTesting::SoftmaxExplorer< Ctx >

In some cases, different actions have a different scores, and you would prefer to choose actions with large scores. Softmax allows you to do that.

Constructor & Destructor Documentation

template<class Ctx>
NativeMultiWorldTesting::SoftmaxExplorer< Ctx >::SoftmaxExplorer | ( | IScorer< Ctx > & | default_scorer,
 |  | float | lambda,
 |  | u32 | num_actions
 | ) |  |
inline

The constructor is the only public member, because this should be used with the MwtExplorer.

Parameters
default_scorer | A function which outputs a score for each action.
lambda | lambda = 0 implies uniform distribution. Large lambda is equivalent to a max.
num_actions | The number of actions to randomize over.
The documentation for this class was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::GenericExplorer< Ctx > Class Template Reference

  • NativeMultiWorldTesting
  • GenericExplorer
Public Member Functions | Friends | List of all members
NativeMultiWorldTesting::GenericExplorer< Ctx > Class Template Reference
MultiWorldTestingCpp

#include <MWTExplorer.h>

Inheritance diagram for NativeMultiWorldTesting::GenericExplorer< Ctx >:
NativeMultiWorldTesting::IExplorer< Ctx >

Public Member Functions

GenericExplorer (IScorer< Ctx > &default_scorer, u32 num_actions)

Friends

class MwtExplorer< Ctx >

Detailed Description

template<class Ctx>
class NativeMultiWorldTesting::GenericExplorer< Ctx >

GenericExplorer provides complete flexibility. You can create any distribution over actions desired, and it will draw from that.

Constructor & Destructor Documentation

template<class Ctx>
NativeMultiWorldTesting::GenericExplorer< Ctx >::GenericExplorer | ( | IScorer< Ctx > & | default_scorer,
 |  | u32 | num_actions
 | ) |  |
inline

The constructor is the only public member, because this should be used with the MwtExplorer.

Parameters
default_scorer | A function which outputs the probability of each action.
num_actions | The number of actions to randomize over.
The documentation for this class was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::TauFirstExplorer< Ctx > Class Template Reference

  • NativeMultiWorldTesting
  • TauFirstExplorer
Public Member Functions | Friends | List of all members
NativeMultiWorldTesting::TauFirstExplorer< Ctx > Class Template Reference
MultiWorldTestingCpp

#include <MWTExplorer.h>

Inheritance diagram for NativeMultiWorldTesting::TauFirstExplorer< Ctx >:
NativeMultiWorldTesting::IExplorer< Ctx >

Public Member Functions

TauFirstExplorer (IPolicy< Ctx > &default_policy, u32 tau, u32 num_actions)

Friends

class MwtExplorer< Ctx >

Detailed Description

template<class Ctx>
class NativeMultiWorldTesting::TauFirstExplorer< Ctx >

The tau-first explorer collects exactly tau uniform random exploration events, and then uses the default policy thereafter.

Constructor & Destructor Documentation

template<class Ctx>
NativeMultiWorldTesting::TauFirstExplorer< Ctx >::TauFirstExplorer | ( | IPolicy< Ctx > & | default_policy,
 |  | u32 | tau,
 |  | u32 | num_actions
 | ) |  |
inline

The constructor is the only public member, because this should be used with the MwtExplorer.

Parameters
default_policy | A default policy after randomization finishes.
tau | The number of events to be uniform over.
num_actions | The number of actions to randomize over.
The documentation for this class was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::BootstrapExplorer< Ctx > Class Template Reference

  • NativeMultiWorldTesting
  • BootstrapExplorer
Public Member Functions | Friends | List of all members
NativeMultiWorldTesting::BootstrapExplorer< Ctx > Class Template Reference
MultiWorldTestingCpp

#include <MWTExplorer.h>

Inheritance diagram for NativeMultiWorldTesting::BootstrapExplorer< Ctx >:
NativeMultiWorldTesting::IExplorer< Ctx >

Public Member Functions

BootstrapExplorer (vector< unique_ptr< IPolicy< Ctx >>> &default_policy_functions, u32 num_actions)

Friends

class MwtExplorer< Ctx >

Detailed Description

template<class Ctx>
class NativeMultiWorldTesting::BootstrapExplorer< Ctx >

The Bootstrap explorer randomizes over the actions chosen by a set of default policies. This performs well statistically but can be computationally expensive.

Constructor & Destructor Documentation

template<class Ctx>
NativeMultiWorldTesting::BootstrapExplorer< Ctx >::BootstrapExplorer | ( | vector< unique_ptr< IPolicy< Ctx >>> & | default_policy_functions,
 |  | u32 | num_actions
 | ) |  |
inline

The constructor is the only public member, because this should be used with the MwtExplorer.

Parameters
default_policy_functions | A set of default policies to be uniform random over.
num_actions | The number of actions to randomize over.
The documentation for this class was generated from the following file:
  • static/MWTExplorer.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::murmur_hash Struct Reference

  • NativeMultiWorldTesting
  • murmur_hash
Public Member Functions | List of all members
NativeMultiWorldTesting::murmur_hash Struct Reference
MultiWorldTestingCpp

Public Member Functions

uint32_t uniform_hash (const void *key, size_t len, uint32_t seed)
The documentation for this struct was generated from the following file:
  • static/utility.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::HashUtils Class Reference

  • NativeMultiWorldTesting
  • HashUtils
Static Public Member Functions | List of all members
NativeMultiWorldTesting::HashUtils Class Reference
MultiWorldTestingCpp

Static Public Member Functions

static u64 Compute_Id_Hash (const std::string &unique_id)
The documentation for this class was generated from the following file:
  • static/utility.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::NumberUtils Class Reference

  • NativeMultiWorldTesting
  • NumberUtils
Static Public Member Functions | List of all members
NativeMultiWorldTesting::NumberUtils Class Reference
MultiWorldTestingCpp

Static Public Member Functions

static void | Float_To_String (float f, char *str)
template<bool trailing_zeros>
static void | print_mantissa (char *&begin, float f)
static void | print_float (char *begin, float f)
The documentation for this class was generated from the following file:
  • static/utility.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::PRG::int_float Union Reference

  • NativeMultiWorldTesting
  • PRG
  • int_float
Public Attributes | List of all members
NativeMultiWorldTesting::PRG::int_float Union Reference
MultiWorldTestingCpp

Public Attributes

int32_t | i
float | f
The documentation for this union was generated from the following file:
  • static/utility.h


Multiworld Testing Exploration Library: NativeMultiWorldTesting::PRG::prg Struct Reference

  • NativeMultiWorldTesting
  • PRG
  • prg
Public Member Functions | List of all members
NativeMultiWorldTesting::PRG::prg Struct Reference
MultiWorldTestingCpp

Public Member Functions

prg (uint64_t initial)
float | merand48 (uint64_t &initial)
float | Uniform_Unit_Interval ()
uint32_t | Uniform_Int (uint32_t low, uint32_t high)
The documentation for this struct was generated from the following file:
  • static/utility.h


Multiworld Testing Exploration Library: Namespace List

  • Namespace List
  • Namespace Members
Namespace List
Here is a list of all documented namespaces with brief descriptions:
N MultiWorldTesting
N NativeMultiWorldTesting
Generated on Wed Nov 12 2014 23:28:15 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Namespace Members

  • Namespace List
  • Namespace Members
  • All
  • Variables
Here is a list of all documented namespace members with links to the namespaces they belong to:
  • Ctx : MultiWorldTesting


Multiworld Testing Exploration Library: Namespace Members

  • Namespace List
  • Namespace Members
  • All
  • Variables
  • Ctx : MultiWorldTesting


Multiworld Testing Exploration Library: Class List

Class List
Here are the classes, structs, unions and interfaces with brief descriptions:
[detail level 123]
▼N MultiWorldTesting |
C abstract |
C BootstrapExplorer | The bootstrap exploration class.
C EpsilonGreedyExplorer | The epsilon greedy exploration class.
C Feature | Represents a feature in a sparse array.
C GenericExplorer | The generic exploration class.
C IExplorer |
C IPolicy | Exposes a method for choosing an action given a generic context. IPolicy objects are passed to (and invoked by) exploration algorithms to specify the default policy behavior.
C IRecorder | Represents a recorder that exposes a method to record exploration data based on generic contexts.
C IScorer | Exposes a method for specifying a score (weight) for each action given a generic context.
C IStringContext |
C MwtExplorer | The top level MwtExplorer class. Using this makes sure that the right bits are recorded and good random actions are chosen.
C NativeContext |
C NativePolicy |
C NativeRecorder |
C NativeScorer |
C NativeStringContext |
C SimpleContext | A sample context class that stores a vector of Features.
C SoftmaxExplorer | The epsilon greedy exploration class.
C TauFirstExplorer | The tau-first exploration class.
▼N NativeMultiWorldTesting |
▼N PRG |
C int_float |
C prg |
C BootstrapExplorer |
C EpsilonGreedyExplorer |
C Feature |
C GenericExplorer |
C HashUtils |
C IExplorer |
C IPolicy |
C IRecorder |
C IScorer |
C murmur_hash |
C MwtExplorer |
C NumberUtils |
C SimpleContext |
C SoftmaxExplorer |
C StringRecorder |
C TauFirstExplorer |
C MyContext | Example of a custom context
C MyInteraction |
C MyPolicy |
C MyRecorder |
C MyScorer |
C MySimplePolicy |


Multiworld Testing Exploration Library: MultiWorldTesting::abstract< Ctx > Class Template Reference

  • MultiWorldTesting
  • abstract
Public Member Functions | Static Public Member Functions | Public Attributes | List of all members
MultiWorldTesting::abstract< Ctx > Class Template Reference

Public Member Functions

PolicyCallback ()
NativePolicy * | GetNativePolicy ()
vector< unique_ptr
< NativeMultiWorldTesting::IPolicy
< NativeContext > > > * 		GetNativePolicies (int count)
RecorderCallback ()
NativeRecorder * | GetNativeRecorder ()
ScorerCallback ()
NativeScorer * | GetNativeScorer ()

Static Public Member Functions

static UInt32 | InteropInvoke (IntPtr callbackPtr, IntPtr contextPtr, int index)
static void | InteropInvoke (IntPtr mwtPtr, IntPtr contextPtr, UInt32 action, float probability, IntPtr uniqueKeyPtr)
static void | InteropInvoke (IntPtr callbackPtr, IntPtr contextPtr, IntPtr scoresPtr, IntPtr sizePtr)

Public Attributes

internal | __pad0__: virtual UInt32 InvokePolicyCallback(Ctx context
internal int | index = 0
internal UInt32 | action
internal UInt32 float | probability
internal UInt32 float String | unique_key = 0
The documentation for this class was generated from the following file:
  • clr/explore_interop.h
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MultiWorldTesting::NativeContext Class Reference

  • MultiWorldTesting
  • NativeContext
Public Member Functions | List of all members
MultiWorldTesting::NativeContext Class Reference

Public Member Functions

NativeContext (void *clr_mwt, void *clr_explorer, void *clr_context)
void * | Get_Clr_Mwt ()
void * | Get_Clr_Context ()
void * | Get_Clr_Explorer ()
The documentation for this class was generated from the following file:
  • clr/explore_interop.h


Multiworld Testing Exploration Library: MultiWorldTesting::NativePolicy Class Reference

  • MultiWorldTesting
  • NativePolicy
Public Member Functions | List of all members
MultiWorldTesting::NativePolicy Class Reference
Inheritance diagram for MultiWorldTesting::NativePolicy:
NativeMultiWorldTesting::IPolicy< NativeContext >

Public Member Functions

NativePolicy (Native_Policy_Callback *func, int index=-1)
u32 | Choose_Action (NativeContext &context)
- Public Member Functions inherited from NativeMultiWorldTesting::IPolicy< NativeContext >
virtual u32 | Choose_Action (NativeContext &context)=0
The documentation for this class was generated from the following file:
  • clr/explore_interop.h


Multiworld Testing Exploration Library: MultiWorldTesting::NativeRecorder Class Reference

  • MultiWorldTesting
  • NativeRecorder
Public Member Functions | List of all members
MultiWorldTesting::NativeRecorder Class Reference
Inheritance diagram for MultiWorldTesting::NativeRecorder:
NativeMultiWorldTesting::IRecorder< NativeContext >

Public Member Functions

NativeRecorder (Native_Recorder_Callback *native_func)
void | Record (NativeContext &context, u32 action, float probability, string unique_key)
- Public Member Functions inherited from NativeMultiWorldTesting::IRecorder< NativeContext >
virtual void | Record (NativeContext &context, u32 action, float probability, string unique_key)=0
The documentation for this class was generated from the following file:
  • clr/explore_interop.h


Multiworld Testing Exploration Library: MultiWorldTesting::NativeScorer Class Reference

  • MultiWorldTesting
  • NativeScorer
Public Member Functions | List of all members
MultiWorldTesting::NativeScorer Class Reference
Inheritance diagram for MultiWorldTesting::NativeScorer:
NativeMultiWorldTesting::IScorer< NativeContext >

Public Member Functions

NativeScorer (Native_Scorer_Callback *func)
vector< float > | Score_Actions (NativeContext &context)
- Public Member Functions inherited from NativeMultiWorldTesting::IScorer< NativeContext >
virtual vector< float > | Score_Actions (NativeContext &context)=0
The documentation for this class was generated from the following file:
  • clr/explore_interop.h


Multiworld Testing Exploration Library: MultiWorldTesting::NativeStringContext Class Reference

  • MultiWorldTesting
  • NativeStringContext
Public Member Functions | List of all members
MultiWorldTesting::NativeStringContext Class Reference

Public Member Functions

NativeStringContext (void *clr_context, Native_To_String_Callback *func)
string To_String ()
The documentation for this class was generated from the following file:
  • clr/explore_interop.h


Multiworld Testing Exploration Library: MyContext Class Reference

MyContext Class Reference

Example of a custom context. More...

Detailed Description

Example of a custom context.

The documentation for this class was generated from the following file:
  • explore_sample.cpp
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MyInteraction< Ctx > Struct Template Reference

Public Attributes | List of all members
MyInteraction< Ctx > Struct Template Reference

Public Attributes

Ctx | Context
u32 | Action
float | Probability
string | Unique_Key

Detailed Description

template<class Ctx>
struct MyInteraction< Ctx >

Represents a tuple <context, action, probability, key>.

The documentation for this struct was generated from the following file:
  • explore_sample.cpp
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MyPolicy Class Reference

Public Member Functions | List of all members
MyPolicy Class Reference
Inheritance diagram for MyPolicy:
MultiWorldTesting::IPolicy< MyContext >

Public Member Functions

u32 | Choose_Action (MyContext &context)
- Public Member Functions inherited from MultiWorldTesting::IPolicy< MyContext >
virtual UInt32 | ChooseAction (MyContextcontext)=0
Determines the action to take for a given context. More...

Detailed Description

Example of a custom policy which implements the IPolicy<MyContext>, declaring that this policy only interacts with MyContext objects.

The documentation for this class was generated from the following file:
  • explore_sample.cpp
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MyRecorder Class Reference

Public Member Functions | List of all members
MyRecorder Class Reference
Inheritance diagram for MyRecorder:
MultiWorldTesting::IRecorder< MyContext >

Public Member Functions

virtual void | Record (MyContext &context, u32 action, float probability, string unique_key)
- Public Member Functions inherited from MultiWorldTesting::IRecorder< MyContext >
virtual void | Record (MyContextcontext, UInt32 action, float probability, String^uniqueKey)=0
Records the exploration data associated with a given decision. More...

Detailed Description

Example of a custom recorder which implements the IRecorder<MyContext>, declaring that this recorder only interacts with MyContext objects.

The documentation for this class was generated from the following file:
  • explore_sample.cpp
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MyScorer Class Reference

Public Member Functions | List of all members
MyScorer Class Reference
Inheritance diagram for MyScorer:
MultiWorldTesting::IScorer< MyContext >

Public Member Functions

MyScorer (u32 num_actions)
vector< float > | Score_Actions (MyContext &context)
- Public Member Functions inherited from MultiWorldTesting::IScorer< MyContext >
virtual List< float > | ScoreActions (MyContextcontext)=0
Determines the score of each action for a given context. More...

Detailed Description

Example of a custom scorer which implements the IScorer<MyContext>, declaring that this scorer only interacts with MyContext objects.

The documentation for this class was generated from the following file:
  • explore_sample.cpp
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: MySimplePolicy Class Reference

Public Member Functions | List of all members
MySimplePolicy Class Reference
Inheritance diagram for MySimplePolicy:
MultiWorldTesting::IPolicy< SimpleContext > MultiWorldTesting::IPolicy< SimpleContext >

Public Member Functions

u32 | Choose_Action (SimpleContext &context)
u32 | Choose_Action (SimpleContext &context)
- Public Member Functions inherited from MultiWorldTesting::IPolicy< SimpleContext >
virtual UInt32 | ChooseAction (SimpleContextcontext)=0
Determines the action to take for a given context. More...

Detailed Description

Example of a custom policy which implements the IPolicy<SimpleContext>, declaring that this policy only interacts with SimpleContext objects.

The documentation for this class was generated from the following files:
  • explore.cpp
  • explore_sample.cpp
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Class Index

Class Index
A | B | E | F | G | H | I | M | N | P | S | T
B
GenericExplorer (MultiWorldTesting) IStringContext (MultiWorldTesting) NativePolicy (MultiWorldTesting) TauFirstExplorer (NativeMultiWorldTesting)
H
M
NativeRecorder (MultiWorldTesting)
a
BootstrapExplorer (MultiWorldTesting) NativeScorer (MultiWorldTesting)
BootstrapExplorer (NativeMultiWorldTesting) HashUtils (NativeMultiWorldTesting) MwtExplorer (MultiWorldTesting) NativeStringContext (MultiWorldTesting) abstract (MultiWorldTesting)
E
I
MwtExplorer (NativeMultiWorldTesting) NumberUtils (NativeMultiWorldTesting)
i
MyContext
S
EpsilonGreedyExplorer (NativeMultiWorldTesting) IExplorer (MultiWorldTesting) MyInteraction int_float (NativeMultiWorldTesting::PRG)
EpsilonGreedyExplorer (MultiWorldTesting) IExplorer (NativeMultiWorldTesting) MyPolicy SimpleContext (NativeMultiWorldTesting)
m
F
IPolicy (MultiWorldTesting) MyRecorder SimpleContext (MultiWorldTesting)
IPolicy (NativeMultiWorldTesting) MyScorer SoftmaxExplorer (NativeMultiWorldTesting) murmur_hash (NativeMultiWorldTesting)
Feature (NativeMultiWorldTesting) IRecorder (NativeMultiWorldTesting) MySimplePolicy SoftmaxExplorer (MultiWorldTesting)
p
Feature (MultiWorldTesting) IRecorder (MultiWorldTesting)
N
StringRecorder (NativeMultiWorldTesting)
G
IScorer (MultiWorldTesting)
T
prg (NativeMultiWorldTesting::PRG)
IScorer (NativeMultiWorldTesting) NativeContext (MultiWorldTesting)
GenericExplorer (NativeMultiWorldTesting) TauFirstExplorer (MultiWorldTesting)
A | B | E | F | G | H | I | M | N | P | S | T


Multiworld Testing Exploration Library: Class Hierarchy

Class Hierarchy
This inheritance list is sorted roughly, but not completely, alphabetically:
[detail level 12]
C MultiWorldTesting::abstract< Ctx > |
C NativeMultiWorldTesting::Feature |
C MultiWorldTesting::Feature | Represents a feature in a sparse array.
C NativeMultiWorldTesting::HashUtils |
▼C MultiWorldTesting::IExplorer< Ctx > |
C MultiWorldTesting::BootstrapExplorer< Ctx > | The bootstrap exploration class.
C MultiWorldTesting::EpsilonGreedyExplorer< Ctx > | The epsilon greedy exploration class.
C MultiWorldTesting::GenericExplorer< Ctx > | The generic exploration class.
C MultiWorldTesting::SoftmaxExplorer< Ctx > | The epsilon greedy exploration class.
C MultiWorldTesting::TauFirstExplorer< Ctx > | The tau-first exploration class.
▼C NativeMultiWorldTesting::IExplorer< Ctx > |
C NativeMultiWorldTesting::BootstrapExplorer< Ctx > |
C NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx > |
C NativeMultiWorldTesting::GenericExplorer< Ctx > |
C NativeMultiWorldTesting::SoftmaxExplorer< Ctx > |
C NativeMultiWorldTesting::TauFirstExplorer< Ctx > |
▼C NativeMultiWorldTesting::IExplorer< MultiWorldTesting::NativeContext > |
C NativeMultiWorldTesting::BootstrapExplorer< MultiWorldTesting::NativeContext > |
C NativeMultiWorldTesting::EpsilonGreedyExplorer< MultiWorldTesting::NativeContext > |
C NativeMultiWorldTesting::GenericExplorer< MultiWorldTesting::NativeContext > |
C NativeMultiWorldTesting::SoftmaxExplorer< MultiWorldTesting::NativeContext > |
C NativeMultiWorldTesting::TauFirstExplorer< MultiWorldTesting::NativeContext > |
C NativeMultiWorldTesting::PRG::int_float |
C MultiWorldTesting::IPolicy< Ctx > | Exposes a method for choosing an action given a generic context. IPolicy objects are passed to (and invoked by) exploration algorithms to specify the default policy behavior.
C NativeMultiWorldTesting::IPolicy< Ctx > |
C NativeMultiWorldTesting::IPolicy< MultiWorldTesting::NativeContext > |
▼C MultiWorldTesting::IPolicy< MyContext > |
C MyPolicy |
▼C NativeMultiWorldTesting::IPolicy< NativeContext > |
C MultiWorldTesting::NativePolicy |
▼C MultiWorldTesting::IPolicy< SimpleContext > |
C MySimplePolicy |
C MySimplePolicy |
▼C NativeMultiWorldTesting::IRecorder< Ctx > |
C NativeMultiWorldTesting::StringRecorder< Ctx > |
C MultiWorldTesting::IRecorder< Ctx > | Represents a recorder that exposes a method to record exploration data based on generic contexts.
▼C MultiWorldTesting::IRecorder< MyContext > |
C MyRecorder |
▼C NativeMultiWorldTesting::IRecorder< NativeContext > |
C MultiWorldTesting::NativeRecorder |
C MultiWorldTesting::IScorer< Ctx > | Exposes a method for specifying a score (weight) for each action given a generic context.
C NativeMultiWorldTesting::IScorer< Ctx > |
C NativeMultiWorldTesting::IScorer< MultiWorldTesting::NativeContext > |
▼C MultiWorldTesting::IScorer< MyContext > |
C MyScorer |
▼C NativeMultiWorldTesting::IScorer< NativeContext > |
C MultiWorldTesting::NativeScorer |
▼C MultiWorldTesting::IStringContext |
C MultiWorldTesting::SimpleContext | A sample context class that stores a vector of Features.
C NativeMultiWorldTesting::murmur_hash |
C NativeMultiWorldTesting::MwtExplorer< Ctx > |
C MyContext | Example of a custom context
C MyInteraction< Ctx > |
C MultiWorldTesting::NativeContext |
C MultiWorldTesting::NativeStringContext |
C NativeMultiWorldTesting::NumberUtils |
▼C PolicyCallback |
C MultiWorldTesting::BootstrapExplorer< Ctx > | The bootstrap exploration class.
C MultiWorldTesting::EpsilonGreedyExplorer< Ctx > | The epsilon greedy exploration class.
C MultiWorldTesting::TauFirstExplorer< Ctx > | The tau-first exploration class.
C NativeMultiWorldTesting::PRG::prg |
▼C RecorderCallback |
C MultiWorldTesting::MwtExplorer< Ctx > | The top level MwtExplorer class. Using this makes sure that the right bits are recorded and good random actions are chosen.
▼C ScorerCallback |
C MultiWorldTesting::GenericExplorer< Ctx > | The generic exploration class.
C MultiWorldTesting::SoftmaxExplorer< Ctx > | The epsilon greedy exploration class.
C NativeMultiWorldTesting::SimpleContext |


Multiworld Testing Exploration Library: Class Members

  • All
  • Functions
Here is a list of all documented class members with links to the class documentation for each member:
  • BootstrapExplorer()
: MultiWorldTesting::BootstrapExplorer< Ctx >
, NativeMultiWorldTesting::BootstrapExplorer< Ctx >
  • Choose_Action()
: NativeMultiWorldTesting::IExplorer< Ctx >
, NativeMultiWorldTesting::IPolicy< Ctx >
, NativeMultiWorldTesting::MwtExplorer< Ctx >
  • ChooseAction()
: MultiWorldTesting::IPolicy< Ctx >
, MultiWorldTesting::MwtExplorer< Ctx >
  • EpsilonGreedyExplorer()
: MultiWorldTesting::EpsilonGreedyExplorer< Ctx >
, NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx >
  • GenericExplorer()
: MultiWorldTesting::GenericExplorer< Ctx >
, NativeMultiWorldTesting::GenericExplorer< Ctx >
  • MwtExplorer()
: MultiWorldTesting::MwtExplorer< Ctx >
, NativeMultiWorldTesting::MwtExplorer< Ctx >
  • Record()
: MultiWorldTesting::IRecorder< Ctx >
, NativeMultiWorldTesting::IRecorder< Ctx >
, NativeMultiWorldTesting::StringRecorder< Ctx >
  • Score_Actions() : NativeMultiWorldTesting::IScorer< Ctx >
  • ScoreActions() : MultiWorldTesting::IScorer< Ctx >
  • SoftmaxExplorer()
: MultiWorldTesting::SoftmaxExplorer< Ctx >
, NativeMultiWorldTesting::SoftmaxExplorer< Ctx >
  • TauFirstExplorer()
: MultiWorldTesting::TauFirstExplorer< Ctx >
, NativeMultiWorldTesting::TauFirstExplorer< Ctx >


Multiworld Testing Exploration Library: Class Members - Functions

  • All
  • Functions
  • BootstrapExplorer()
: MultiWorldTesting::BootstrapExplorer< Ctx >
, NativeMultiWorldTesting::BootstrapExplorer< Ctx >
  • Choose_Action()
: NativeMultiWorldTesting::IExplorer< Ctx >
, NativeMultiWorldTesting::IPolicy< Ctx >
, NativeMultiWorldTesting::MwtExplorer< Ctx >
  • ChooseAction()
: MultiWorldTesting::IPolicy< Ctx >
, MultiWorldTesting::MwtExplorer< Ctx >
  • EpsilonGreedyExplorer()
: MultiWorldTesting::EpsilonGreedyExplorer< Ctx >
, NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx >
  • GenericExplorer()
: MultiWorldTesting::GenericExplorer< Ctx >
, NativeMultiWorldTesting::GenericExplorer< Ctx >
  • MwtExplorer()
: MultiWorldTesting::MwtExplorer< Ctx >
, NativeMultiWorldTesting::MwtExplorer< Ctx >
  • Record()
: MultiWorldTesting::IRecorder< Ctx >
, NativeMultiWorldTesting::IRecorder< Ctx >
, NativeMultiWorldTesting::StringRecorder< Ctx >
  • Score_Actions() : NativeMultiWorldTesting::IScorer< Ctx >
  • ScoreActions() : MultiWorldTesting::IScorer< Ctx >
  • SoftmaxExplorer()
: MultiWorldTesting::SoftmaxExplorer< Ctx >
, NativeMultiWorldTesting::SoftmaxExplorer< Ctx >
  • TauFirstExplorer()
: MultiWorldTesting::TauFirstExplorer< Ctx >
, NativeMultiWorldTesting::TauFirstExplorer< Ctx >


Multiworld Testing Exploration Library: File List

  • File List
File List
Here is a list of all documented files with brief descriptions:
[detail level 12]
▼ clr
explore_clr_wrapper.h
explore_interface.h
explore_interop.h
▼ static
MWTExplorer.h
utility.h


Multiworld Testing Exploration Library: clr Directory Reference

  • clr
clr Directory Reference

Files

file | explore_clr_wrapper.cpp
file | explore_clr_wrapper.h [code]
file | explore_interface.h [code]
file | explore_interop.h [code]


Multiworld Testing Exploration Library: clr/explore_clr_wrapper.h Source File

  • File List
  • clr
explore_clr_wrapper.h
1 #pragma once
2 #include "explore_interop.h"
3
8 namespace MultiWorldTesting {
9
18 generic <class Ctx>
19 public ref class EpsilonGreedyExplorer : public IExplorer<Ctx>, public PolicyCallback<Ctx>
20 {
21 public:
28 EpsilonGreedyExplorer(IPolicy<Ctx>^ defaultPolicy, float epsilon, UInt32 numActions)
29 {
30 this->defaultPolicy = defaultPolicy;
31 m_explorer = new NativeMultiWorldTesting::EpsilonGreedyExplorer<NativeContext>(*GetNativePolicy(), epsilon, (u32)numActions);
32 }
33
34 ~EpsilonGreedyExplorer()
35 {
36 delete m_explorer;
37 }
38
39 internal:
40 virtual UInt32 InvokePolicyCallback(Ctx context, int index) override
41 {
42 return defaultPolicy->ChooseAction(context);
43 }
44
45 NativeMultiWorldTesting::EpsilonGreedyExplorer<NativeContext>* Get()
46 {
47 return m_explorer;
48 }
49
50 private:
51 IPolicy<Ctx>^ defaultPolicy;
52 NativeMultiWorldTesting::EpsilonGreedyExplorer<NativeContext>* m_explorer;
53 };
54
63 generic <class Ctx>
64 public ref class TauFirstExplorer : public IExplorer<Ctx>, public PolicyCallback<Ctx>
65 {
66 public:
73 TauFirstExplorer(IPolicy<Ctx>^ defaultPolicy, UInt32 tau, UInt32 numActions)
74 {
75 this->defaultPolicy = defaultPolicy;
76 m_explorer = new NativeMultiWorldTesting::TauFirstExplorer<NativeContext>(*GetNativePolicy(), tau, (u32)numActions);
77 }
78
79 ~TauFirstExplorer()
80 {
81 delete m_explorer;
82 }
83
84 internal:
85 virtual UInt32 InvokePolicyCallback(Ctx context, int index) override
86 {
87 return defaultPolicy->ChooseAction(context);
88 }
89
90 NativeMultiWorldTesting::TauFirstExplorer<NativeContext>* Get()
91 {
92 return m_explorer;
93 }
94
95 private:
96 IPolicy<Ctx>^ defaultPolicy;
97 NativeMultiWorldTesting::TauFirstExplorer<NativeContext>* m_explorer;
98 };
99
109 generic <class Ctx>
110 public ref class SoftmaxExplorer : public IExplorer<Ctx>, public ScorerCallback<Ctx>
111 {
112 public:
119 SoftmaxExplorer(IScorer<Ctx>^ defaultScorer, float lambda, UInt32 numActions)
120 {
121 this->defaultScorer = defaultScorer;
122 m_explorer = new NativeMultiWorldTesting::SoftmaxExplorer<NativeContext>(*GetNativeScorer(), lambda, (u32)numActions);
123 }
124
125 ~SoftmaxExplorer()
126 {
127 delete m_explorer;
128 }
129
130 internal:
131 virtual List<float>^ InvokeScorerCallback(Ctx context) override
132 {
133 return defaultScorer->ScoreActions(context);
134 }
135
136 NativeMultiWorldTesting::SoftmaxExplorer<NativeContext>* Get()
137 {
138 return m_explorer;
139 }
140
141 private:
142 IScorer<Ctx>^ defaultScorer;
143 NativeMultiWorldTesting::SoftmaxExplorer<NativeContext>* m_explorer;
144 };
145
154 generic <class Ctx>
155 public ref class GenericExplorer : public IExplorer<Ctx>, public ScorerCallback<Ctx>
156 {
157 public:
163 GenericExplorer(IScorer<Ctx>^ defaultScorer, UInt32 numActions)
164 {
165 this->defaultScorer = defaultScorer;
166 m_explorer = new NativeMultiWorldTesting::GenericExplorer<NativeContext>(*GetNativeScorer(), (u32)numActions);
167 }
168
169 ~GenericExplorer()
170 {
171 delete m_explorer;
172 }
173
174 internal:
175 virtual List<float>^ InvokeScorerCallback(Ctx context) override
176 {
177 return defaultScorer->ScoreActions(context);
178 }
179
180 NativeMultiWorldTesting::GenericExplorer<NativeContext>* Get()
181 {
182 return m_explorer;
183 }
184
185 private:
186 IScorer<Ctx>^ defaultScorer;
187 NativeMultiWorldTesting::GenericExplorer<NativeContext>* m_explorer;
188 };
189
199 generic <class Ctx>
200 public ref class BootstrapExplorer : public IExplorer<Ctx>, public PolicyCallback<Ctx>
201 {
202 public:
208 BootstrapExplorer(cli::array<IPolicy<Ctx>^>^ defaultPolicies, UInt32 numActions)
209 {
210 this->defaultPolicies = defaultPolicies;
211 if (this->defaultPolicies == nullptr)
212 {
213 throw gcnew ArgumentNullException("The specified array of default policy functions cannot be null.");
214 }
215
216 m_explorer = new NativeMultiWorldTesting::BootstrapExplorer<NativeContext>(*GetNativePolicies((u32)defaultPolicies->Length), (u32)numActions);
217 }
218
219 ~BootstrapExplorer()
220 {
221 delete m_explorer;
222 }
223
224 internal:
225 virtual UInt32 InvokePolicyCallback(Ctx context, int index) override
226 {
227 if (index < 0 || index >= defaultPolicies->Length)
228 {
229 throw gcnew InvalidDataException("Internal error: Index of interop bag is out of range.");
230 }
231 return defaultPolicies[index]->ChooseAction(context);
232 }
233
234 NativeMultiWorldTesting::BootstrapExplorer<NativeContext>* Get()
235 {
236 return m_explorer;
237 }
238
239 private:
240 cli::array<IPolicy<Ctx>^>^ defaultPolicies;
241 NativeMultiWorldTesting::BootstrapExplorer<NativeContext>* m_explorer;
242 };
243
249 generic <class Ctx>
250 public ref class MwtExplorer : public RecorderCallback<Ctx>
251 {
252 public:
258 MwtExplorer(String^ appId, IRecorder<Ctx>^ recorder)
259 {
260 this->appId = appId;
261 this->recorder = recorder;
262 }
263
271 UInt32 ChooseAction(IExplorer<Ctx>^ explorer, String^ unique_key, Ctx context)
272 {
273 String^ salt = this->appId;
274 NativeMultiWorldTesting::MwtExplorer<NativeContext> mwt(marshal_as<std::string>(salt), *GetNativeRecorder());
275
276 GCHandle selfHandle = GCHandle::Alloc(this);
277 IntPtr selfPtr = (IntPtr)selfHandle;
278
279 GCHandle contextHandle = GCHandle::Alloc(context);
280 IntPtr contextPtr = (IntPtr)contextHandle;
281
282 GCHandle explorerHandle = GCHandle::Alloc(explorer);
283 IntPtr explorerPtr = (IntPtr)explorerHandle;
284
285 NativeContext native_context(selfPtr.ToPointer(), explorerPtr.ToPointer(), contextPtr.ToPointer());
286 u32 action = 0;
287 if (explorer->GetType() == EpsilonGreedyExplorer<Ctx>::typeid)
288 {
289 EpsilonGreedyExplorer<Ctx>^ epsilonGreedyExplorer = (EpsilonGreedyExplorer<Ctx>^)explorer;
290 action = mwt.Choose_Action(*epsilonGreedyExplorer->Get(), marshal_as<std::string>(unique_key), native_context);
291 }
292 else if (explorer->GetType() == TauFirstExplorer<Ctx>::typeid)
293 {
294 TauFirstExplorer<Ctx>^ tauFirstExplorer = (TauFirstExplorer<Ctx>^)explorer;
295 action = mwt.Choose_Action(*tauFirstExplorer->Get(), marshal_as<std::string>(unique_key), native_context);
296 }
297 else if (explorer->GetType() == SoftmaxExplorer<Ctx>::typeid)
298 {
299 SoftmaxExplorer<Ctx>^ softmaxExplorer = (SoftmaxExplorer<Ctx>^)explorer;
300 action = mwt.Choose_Action(*softmaxExplorer->Get(), marshal_as<std::string>(unique_key), native_context);
301 }
302 else if (explorer->GetType() == GenericExplorer<Ctx>::typeid)
303 {
304 GenericExplorer<Ctx>^ genericExplorer = (GenericExplorer<Ctx>^)explorer;
305 action = mwt.Choose_Action(*genericExplorer->Get(), marshal_as<std::string>(unique_key), native_context);
306 }
307 else if (explorer->GetType() == BootstrapExplorer<Ctx>::typeid)
308 {
309 BootstrapExplorer<Ctx>^ bootstrapExplorer = (BootstrapExplorer<Ctx>^)explorer;
310 action = mwt.Choose_Action(*bootstrapExplorer->Get(), marshal_as<std::string>(unique_key), native_context);
311 }
312
313 explorerHandle.Free();
314 contextHandle.Free();
315 selfHandle.Free();
316
317 return action;
318 }
319
320 internal:
321 virtual void InvokeRecorderCallback(Ctx context, UInt32 action, float probability, String^ unique_key) override
322 {
323 recorder->Record(context, action, probability, unique_key);
324 }
325
326 private:
327 IRecorder<Ctx>^ recorder;
328 String^ appId;
329 };
330
334 [StructLayout(LayoutKind::Sequential)]
335 public value struct Feature
336 {
337 float Value;
338 UInt32 Id;
339 };
340
345 generic <class Ctx> where Ctx : IStringContext
346 public ref class StringRecorder : public IRecorder<Ctx>, public ToStringCallback<Ctx>
347 {
348 public:
349 StringRecorder()
350 {
351 m_string_recorder = new NativeMultiWorldTesting::StringRecorder<NativeStringContext>();
352 }
353
354 ~StringRecorder()
355 {
356 delete m_string_recorder;
357 }
358
359 virtual void Record(Ctx context, UInt32 action, float probability, String^ uniqueKey)
360 {
361 GCHandle contextHandle = GCHandle::Alloc(context);
362 IntPtr contextPtr = (IntPtr)contextHandle;
363
364 NativeStringContext native_context(contextPtr.ToPointer(), GetCallback());
365 m_string_recorder->Record(native_context, (u32)action, probability, marshal_as<string>(uniqueKey));
366 }
367
374 String^ GetRecording()
375 {
376 // Workaround for C++-CLI bug which does not allow default value for parameter
377 return GetRecording(true);
378 }
379
387 String^ GetRecording(bool flush)
388 {
389 return gcnew String(m_string_recorder->Get_Recording(flush).c_str());
390 }
391
392 private:
393 NativeMultiWorldTesting::StringRecorder<NativeStringContext>* m_string_recorder;
394 };
395
399 public ref class SimpleContext : public IStringContext
400 {
401 public:
402 SimpleContext(cli::array<Feature>^ features)
403 {
404 Features = features;
405
406 // TODO: add another constructor overload for native SimpleContext to avoid copying feature values
407 m_features = new vector<NativeMultiWorldTesting::Feature>();
408 for (int i = 0; i < features->Length; i++)
409 {
410 m_features->push_back({ features[i].Value, features[i].Id });
411 }
412
413 m_native_context = new NativeMultiWorldTesting::SimpleContext(*m_features);
414 }
415
416 String^ ToString() override
417 {
418 return gcnew String(m_native_context->To_String().c_str());
419 }
420
421 ~SimpleContext()
422 {
423 delete m_native_context;
424 }
425
426 public:
427 cli::array<Feature>^ GetFeatures() { return Features; }
428
429 internal:
430 cli::array<Feature>^ Features;
431
432 private:
433 vector<NativeMultiWorldTesting::Feature>* m_features;
434 NativeMultiWorldTesting::SimpleContext* m_native_context;
435 };
436 }
437
MultiWorldTesting::IExplorer
Definition: explore_interface.h:76
MultiWorldTesting::NativeStringContext
Definition: explore_interop.h:68
NativeMultiWorldTesting::SimpleContext
Definition: MWTExplorer.h:230
MultiWorldTesting::NativeContext
Definition: explore_interop.h:37
NativeMultiWorldTesting::StringRecorder
Definition: MWTExplorer.h:178
MultiWorldTesting::TauFirstExplorer
The tau-first exploration class.
Definition: explore_clr_wrapper.h:64
MultiWorldTesting::MwtExplorer::MwtExplorer
MwtExplorer(String^appId, IRecorder< Ctx >^recorder)
Constructor.
Definition: explore_clr_wrapper.h:258
MultiWorldTesting::IStringContext
Definition: explore_interface.h:80
MultiWorldTesting::IScorer
Exposes a method for specifying a score (weight) for each action given a generic context.
Definition: explore_interface.h:64
NativeMultiWorldTesting::SoftmaxExplorer
Definition: MWTExplorer.h:357
NativeMultiWorldTesting::TauFirstExplorer
Definition: MWTExplorer.h:528
MultiWorldTesting::SoftmaxExplorer::SoftmaxExplorer
SoftmaxExplorer(IScorer< Ctx >^defaultScorer, float lambda, UInt32 numActions)
The constructor is the only public member, because this should be used with the MwtExplorer.
Definition: explore_clr_wrapper.h:119
MultiWorldTesting::SimpleContext
A sample context class that stores a vector of Features.
Definition: explore_clr_wrapper.h:399
MultiWorldTesting::BootstrapExplorer::BootstrapExplorer
BootstrapExplorer(cli::array< IPolicy< Ctx >^>^defaultPolicies, UInt32 numActions)
The constructor is the only public member, because this should be used with the MwtExplorer.
Definition: explore_clr_wrapper.h:208
MultiWorldTesting
Definition: explore_clr_wrapper.cpp:16
NativeMultiWorldTesting::EpsilonGreedyExplorer
Definition: MWTExplorer.h:272
NativeMultiWorldTesting::BootstrapExplorer
Definition: MWTExplorer.h:595
MultiWorldTesting::Feature
Represents a feature in a sparse array.
Definition: explore_clr_wrapper.h:335
MultiWorldTesting::SoftmaxExplorer
The epsilon greedy exploration class.
Definition: explore_clr_wrapper.h:110
MultiWorldTesting::BootstrapExplorer
The bootstrap exploration class.
Definition: explore_clr_wrapper.h:200
MultiWorldTesting::IPolicy
Exposes a method for choosing an action given a generic context. IPolicy objects are passed to (and i...
Definition: explore_interface.h:48
MultiWorldTesting::EpsilonGreedyExplorer::EpsilonGreedyExplorer
EpsilonGreedyExplorer(IPolicy< Ctx >^defaultPolicy, float epsilon, UInt32 numActions)
The constructor is the only public member, because this should be used with the MwtExplorer.
Definition: explore_clr_wrapper.h:28
MultiWorldTesting::IRecorder
Represents a recorder that exposes a method to record exploration data based on generic contexts...
Definition: explore_interface.h:29
NativeMultiWorldTesting::GenericExplorer
Definition: MWTExplorer.h:447
MultiWorldTesting::TauFirstExplorer::TauFirstExplorer
TauFirstExplorer(IPolicy< Ctx >^defaultPolicy, UInt32 tau, UInt32 numActions)
The constructor is the only public member, because this should be used with the MwtExplorer.
Definition: explore_clr_wrapper.h:73
NativeMultiWorldTesting::MwtExplorer::Choose_Action
u32 Choose_Action(IExplorer< Ctx > &explorer, string unique_key, Ctx &context)
Definition: MWTExplorer.h:76
NativeMultiWorldTesting::MwtExplorer
Definition: MWTExplorer.h:54
MultiWorldTesting::GenericExplorer
The generic exploration class.
Definition: explore_clr_wrapper.h:155
MultiWorldTesting::EpsilonGreedyExplorer
The epsilon greedy exploration class.
Definition: explore_clr_wrapper.h:19
MultiWorldTesting::MwtExplorer
The top level MwtExplorer class. Using this makes sure that the right bits are recorded and good rand...
Definition: explore_clr_wrapper.h:250
MultiWorldTesting::MwtExplorer::ChooseAction
UInt32 ChooseAction(IExplorer< Ctx >^explorer, String^unique_key, Ctx context)
Choose_Action should be drop-in replacement for any existing policy function.
Definition: explore_clr_wrapper.h:271
MultiWorldTesting::GenericExplorer::GenericExplorer
GenericExplorer(IScorer< Ctx >^defaultScorer, UInt32 numActions)
The constructor is the only public member, because this should be used with the MwtExplorer.
Definition: explore_clr_wrapper.h:163
MultiWorldTesting::Ctx
where Ctx
A sample recorder class that converts the exploration tuple into string format.
Definition: explore_clr_wrapper.h:346
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: clr/explore_interface.h Source File

  • File List
  • clr
explore_interface.h
1 #pragma once
2
3 using namespace System;
4 using namespace System::Collections::Generic;
5
15 namespace MultiWorldTesting {
18
28 generic <class Ctx>
29 public interface class IRecorder
30 {
31 public:
39 virtual void Record(Ctx context, UInt32 action, float probability, String^ uniqueKey) = 0;
40 };
41
47 generic <class Ctx>
48 public interface class IPolicy
49 {
50 public:
56 virtual UInt32 ChooseAction(Ctx context) = 0;
57 };
58
63 generic <class Ctx>
64 public interface class IScorer
65 {
66 public:
72 virtual List<float>^ ScoreActions(Ctx context) = 0;
73 };
74
75 generic <class Ctx>
76 public interface class IExplorer
77 {
78 };
79
80 public interface class IStringContext
81 {
82 public:
83 virtual String^ ToString() = 0;
84 };
85
86 }
87
MultiWorldTesting::IExplorer
Definition: explore_interface.h:76
Generic
MultiWorldTesting::IStringContext
Definition: explore_interface.h:80
MultiWorldTesting::IScorer
Exposes a method for specifying a score (weight) for each action given a generic context.
Definition: explore_interface.h:64
System
MultiWorldTesting
Definition: explore_clr_wrapper.cpp:16
MultiWorldTesting::IPolicy
Exposes a method for choosing an action given a generic context. IPolicy objects are passed to (and i...
Definition: explore_interface.h:48
MultiWorldTesting::IRecorder
Represents a recorder that exposes a method to record exploration data based on generic contexts...
Definition: explore_interface.h:29
MultiWorldTesting::Ctx
where Ctx
A sample recorder class that converts the exploration tuple into string format.
Definition: explore_clr_wrapper.h:346
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: clr/explore_interop.h Source File

  • File List
  • clr
explore_interop.h
1 #pragma once
2
3 #define MANAGED_CODE
4
5 #include "explore_interface.h"
6 #include "MWTExplorer.h"
7
8 #include <msclr\marshal_cppstd.h>
9
10 using namespace System;
11 using namespace System::Collections::Generic;
12 using namespace System::IO;
13 using namespace System::Runtime::InteropServices;
14 using namespace System::Xml::Serialization;
15 using namespace msclr::interop;
16
17 namespace MultiWorldTesting {
18
19 // Policy callback
20 private delegate UInt32 ClrPolicyCallback(IntPtr explorerPtr, IntPtr contextPtr, int index);
21 typedef u32 Native_Policy_Callback(void* explorer, void* context, int index);
22
23 // Scorer callback
24 private delegate void ClrScorerCallback(IntPtr explorerPtr, IntPtr contextPtr, IntPtr scores, IntPtr size);
25 typedef void Native_Scorer_Callback(void* explorer, void* context, float* scores[], u32* size);
26
27 // Recorder callback
28 private delegate void ClrRecorderCallback(IntPtr mwtPtr, IntPtr contextPtr, UInt32 action, float probability, IntPtr uniqueKey);
29 typedef void Native_Recorder_Callback(void* mwt, void* context, u32 action, float probability, void* unique_key);
30
31 // ToString callback
32 private delegate void ClrToStringCallback(IntPtr contextPtr, IntPtr stringValue);
33 typedef void Native_To_String_Callback(void* explorer, void* string_value);
34
35 // NativeContext travels through interop space and contains instances of Mwt, Explorer, Context
36 // used for triggering callback for Policy, Scorer, Recorder
37 class NativeContext
38 {
39 public:
40 NativeContext(void* clr_mwt, void* clr_explorer, void* clr_context)
41 {
42 m_clr_mwt = clr_mwt;
43 m_clr_explorer = clr_explorer;
44 m_clr_context = clr_context;
45 }
46
47 void* Get_Clr_Mwt()
48 {
49 return m_clr_mwt;
50 }
51
52 void* Get_Clr_Context()
53 {
54 return m_clr_context;
55 }
56
57 void* Get_Clr_Explorer()
58 {
59 return m_clr_explorer;
60 }
61
62 private:
63 void* m_clr_mwt;
64 void* m_clr_context;
65 void* m_clr_explorer;
66 };
67
68 class NativeStringContext
69 {
70 public:
71 NativeStringContext(void* clr_context, Native_To_String_Callback* func)
72 {
73 m_clr_context = clr_context;
74 m_func = func;
75 }
76
77 string To_String()
78 {
79 string value;
80 m_func(m_clr_context, &value);
81 return value;
82 }
83 private:
84 void* m_clr_context;
85 Native_To_String_Callback* m_func;
86 };
87
88 // NativeRecorder listens to callback event and reroute it to the managed Recorder instance
89 class NativeRecorder : public NativeMultiWorldTesting::IRecorder<NativeContext>
90 {
91 public:
92 NativeRecorder(Native_Recorder_Callback* native_func)
93 {
94 m_func = native_func;
95 }
96
97 void Record(NativeContext& context, u32 action, float probability, string unique_key)
98 {
99 GCHandle uniqueKeyHandle = GCHandle::Alloc(gcnew String(unique_key.c_str()));
100 IntPtr uniqueKeyPtr = (IntPtr)uniqueKeyHandle;
101
102 m_func(context.Get_Clr_Mwt(), context.Get_Clr_Context(), action, probability, uniqueKeyPtr.ToPointer());
103
104 uniqueKeyHandle.Free();
105 }
106 private:
107 Native_Recorder_Callback* m_func;
108 };
109
110 // NativePolicy listens to callback event and reroute it to the managed Policy instance
111 class NativePolicy : public NativeMultiWorldTesting::IPolicy<NativeContext>
112 {
113 public:
114 NativePolicy(Native_Policy_Callback* func, int index = -1)
115 {
116 m_func = func;
117 m_index = index;
118 }
119
120 u32 Choose_Action(NativeContext& context)
121 {
122 return m_func(context.Get_Clr_Explorer(), context.Get_Clr_Context(), m_index);
123 }
124
125 private:
126 Native_Policy_Callback* m_func;
127 int m_index;
128 };
129
130 class NativeScorer : public NativeMultiWorldTesting::IScorer<NativeContext>
131 {
132 public:
133 NativeScorer(Native_Scorer_Callback* func)
134 {
135 m_func = func;
136 }
137
138 vector<float> Score_Actions(NativeContext& context)
139 {
140 float* scores = nullptr;
141 u32 num_scores = 0;
142 m_func(context.Get_Clr_Explorer(), context.Get_Clr_Context(), &scores, &num_scores);
143
144 // It's ok if scores is null, vector will be empty
145 vector<float> scores_vector(scores, scores + num_scores);
146 delete[] scores;
147
148 return scores_vector;
149 }
150 private:
151 Native_Scorer_Callback* m_func;
152 };
153
154 // Triggers callback to the Policy instance to choose an action
155 generic <class Ctx>
156 public ref class PolicyCallback abstract
157 {
158 internal:
159 virtual UInt32 InvokePolicyCallback(Ctx context, int index) = 0;
160
161 PolicyCallback()
162 {
163 policyCallback = gcnew ClrPolicyCallback(&PolicyCallback<Ctx>::InteropInvoke);
164 IntPtr policyCallbackPtr = Marshal::GetFunctionPointerForDelegate(policyCallback);
165 m_callback = static_cast<Native_Policy_Callback*>(policyCallbackPtr.ToPointer());
166 m_native_policy = nullptr;
167 m_native_policies = nullptr;
168 }
169
170 ~PolicyCallback()
171 {
172 delete m_native_policy;
173 delete m_native_policies;
174 }
175
176 NativePolicy* GetNativePolicy()
177 {
178 if (m_native_policy == nullptr)
179 {
180 m_native_policy = new NativePolicy(m_callback);
181 }
182 return m_native_policy;
183 }
184
185 vector<unique_ptr<NativeMultiWorldTesting::IPolicy<NativeContext>>>* GetNativePolicies(int count)
186 {
187 if (m_native_policies == nullptr)
188 {
189 m_native_policies = new vector<unique_ptr<NativeMultiWorldTesting::IPolicy<NativeContext>>>();
190 for (int i = 0; i < count; i++)
191 {
192 m_native_policies->push_back(unique_ptr<NativeMultiWorldTesting::IPolicy<NativeContext>>(new NativePolicy(m_callback, i)));
193 }
194 }
195
196 return m_native_policies;
197 }
198
199 static UInt32 InteropInvoke(IntPtr callbackPtr, IntPtr contextPtr, int index)
200 {
201 GCHandle callbackHandle = (GCHandle)callbackPtr;
202 PolicyCallback<Ctx>^ callback = (PolicyCallback<Ctx>^)callbackHandle.Target;
203
204 GCHandle contextHandle = (GCHandle)contextPtr;
205 Ctx context = (Ctx)contextHandle.Target;
206
207 return callback->InvokePolicyCallback(context, index);
208 }
209
210 private:
211 ClrPolicyCallback^ policyCallback;
212
213 private:
214 NativePolicy* m_native_policy;
215 vector<unique_ptr<NativeMultiWorldTesting::IPolicy<NativeContext>>>* m_native_policies;
216 Native_Policy_Callback* m_callback;
217 };
218
219 // Triggers callback to the Recorder instance to record interaction data
220 generic <class Ctx>
221 public ref class RecorderCallback abstract
222 {
223 internal:
224 virtual void InvokeRecorderCallback(Ctx context, UInt32 action, float probability, String^ unique_key) = 0;
225
226 RecorderCallback()
227 {
228 recorderCallback = gcnew ClrRecorderCallback(&RecorderCallback<Ctx>::InteropInvoke);
229 IntPtr recorderCallbackPtr = Marshal::GetFunctionPointerForDelegate(recorderCallback);
230 Native_Recorder_Callback* callback = static_cast<Native_Recorder_Callback*>(recorderCallbackPtr.ToPointer());
231 m_native_recorder = new NativeRecorder(callback);
232 }
233
234 ~RecorderCallback()
235 {
236 delete m_native_recorder;
237 }
238
239 NativeRecorder* GetNativeRecorder()
240 {
241 return m_native_recorder;
242 }
243
244 static void InteropInvoke(IntPtr mwtPtr, IntPtr contextPtr, UInt32 action, float probability, IntPtr uniqueKeyPtr)
245 {
246 GCHandle mwtHandle = (GCHandle)mwtPtr;
247 RecorderCallback<Ctx>^ callback = (RecorderCallback<Ctx>^)mwtHandle.Target;
248
249 GCHandle contextHandle = (GCHandle)contextPtr;
250 Ctx context = (Ctx)contextHandle.Target;
251
252 GCHandle uniqueKeyHandle = (GCHandle)uniqueKeyPtr;
253 String^ uniqueKey = (String^)uniqueKeyHandle.Target;
254
255 callback->InvokeRecorderCallback(context, action, probability, uniqueKey);
256 }
257
258 private:
259 ClrRecorderCallback^ recorderCallback;
260
261 private:
262 NativeRecorder* m_native_recorder;
263 };
264
265 // Triggers callback to the Recorder instance to record interaction data
266 generic <class Ctx>
267 public ref class ScorerCallback abstract
268 {
269 internal:
270 virtual List<float>^ InvokeScorerCallback(Ctx context) = 0;
271
272 ScorerCallback()
273 {
274 scorerCallback = gcnew ClrScorerCallback(&ScorerCallback<Ctx>::InteropInvoke);
275 IntPtr scorerCallbackPtr = Marshal::GetFunctionPointerForDelegate(scorerCallback);
276 Native_Scorer_Callback* callback = static_cast<Native_Scorer_Callback*>(scorerCallbackPtr.ToPointer());
277 m_native_scorer = new NativeScorer(callback);
278 }
279
280 ~ScorerCallback()
281 {
282 delete m_native_scorer;
283 }
284
285 NativeScorer* GetNativeScorer()
286 {
287 return m_native_scorer;
288 }
289
290 static void InteropInvoke(IntPtr callbackPtr, IntPtr contextPtr, IntPtr scoresPtr, IntPtr sizePtr)
291 {
292 GCHandle callbackHandle = (GCHandle)callbackPtr;
293 ScorerCallback<Ctx>^ callback = (ScorerCallback<Ctx>^)callbackHandle.Target;
294
295 GCHandle contextHandle = (GCHandle)contextPtr;
296 Ctx context = (Ctx)contextHandle.Target;
297
298 List<float>^ scoreList = callback->InvokeScorerCallback(context);
299
300 if (scoreList == nullptr || scoreList->Count == 0)
301 {
302 return;
303 }
304
305 u32* num_scores = (u32*)sizePtr.ToPointer();
306 *num_scores = (u32)scoreList->Count;
307
308 float* scores = new float[*num_scores];
309 for (u32 i = 0; i < *num_scores; i++)
310 {
311 scores[i] = scoreList[i];
312 }
313
314 float** native_scores = (float**)scoresPtr.ToPointer();
315 *native_scores = scores;
316 }
317
318 private:
319 ClrScorerCallback^ scorerCallback;
320
321 private:
322 NativeScorer* m_native_scorer;
323 };
324
325 // Triggers callback to the Context instance to perform ToString() operation
326 generic <class Ctx> where Ctx : IStringContext
327 public ref class ToStringCallback
328 {
329 internal:
330 ToStringCallback()
331 {
332 toStringCallback = gcnew ClrToStringCallback(&ToStringCallback<Ctx>::InteropInvoke);
333 IntPtr toStringCallbackPtr = Marshal::GetFunctionPointerForDelegate(toStringCallback);
334 m_callback = static_cast<Native_To_String_Callback*>(toStringCallbackPtr.ToPointer());
335 }
336
337 Native_To_String_Callback* GetCallback()
338 {
339 return m_callback;
340 }
341
342 static void InteropInvoke(IntPtr contextPtr, IntPtr stringPtr)
343 {
344 GCHandle contextHandle = (GCHandle)contextPtr;
345 Ctx context = (Ctx)contextHandle.Target;
346
347 string* out_string = (string*)stringPtr.ToPointer();
348 *out_string = marshal_as<string>(context->ToString());
349 }
350
351 private:
352 ClrToStringCallback^ toStringCallback;
353
354 private:
355 Native_To_String_Callback* m_callback;
356 };
357
358 }
MultiWorldTesting::NativeStringContext
Definition: explore_interop.h:68
MultiWorldTesting::NativeContext
Definition: explore_interop.h:37
Generic
InteropServices
Serialization
MultiWorldTesting::NativePolicy
Definition: explore_interop.h:111
System
NativeMultiWorldTesting::IRecorder
Definition: MWTExplorer.h:45
NativeMultiWorldTesting::IPolicy
Definition: MWTExplorer.h:146
MultiWorldTesting
Definition: explore_clr_wrapper.cpp:16
interop
IO
MultiWorldTesting::NativeRecorder
Definition: explore_interop.h:89
NativeMultiWorldTesting::IScorer
Definition: MWTExplorer.h:162
MultiWorldTesting::NativeScorer
Definition: explore_interop.h:130
MultiWorldTesting::Ctx
where Ctx
A sample recorder class that converts the exploration tuple into string format.
Definition: explore_clr_wrapper.h:346
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: static Directory Reference

  • static
static Directory Reference

Files

file | explore.cpp
file | MWTExplorer.h [code]
file | utility.h [code]


Multiworld Testing Exploration Library: static/MWTExplorer.h Source File

  • File List
  • static
MWTExplorer.h
1 //
2 // Main interface for clients of the Multiworld testing (MWT) service.
3 //
4
5 #pragma once
6
7 #include <stdexcept>
8 #include <float.h>
9 #include <math.h>
10 #include <stdio.h>
11 #include <string.h>
12 #include <vector>
13 #include <utility>
14 #include <memory>
15 #include <limits.h>
16 #include <tuple>
17
18 #ifdef MANAGED_CODE
19 #define PORTING_INTERFACE public
20 #define MWT_NAMESPACE namespace NativeMultiWorldTesting
21 #else
22 #define PORTING_INTERFACE private
23 #define MWT_NAMESPACE namespace MultiWorldTesting
24 #endif
25
26 using namespace std;
27
28 #include "utility.h"
29
39 namespace NativeMultiWorldTesting {
42
43 // Forward declarations
44 template <class Ctx>
45 class IRecorder;
46 template <class Ctx>
47 class IExplorer;
48
53 template <class Ctx>
54 class MwtExplorer
55 {
56 public:
63 MwtExplorer(std::string app_id, IRecorder<Ctx>& recorder) : m_recorder(recorder)
64 {
65 m_app_id = HashUtils::Compute_Id_Hash(app_id);
66 }
67
76 u32 Choose_Action(IExplorer<Ctx>& explorer, string unique_key, Ctx& context)
77 {
78 u64 seed = HashUtils::Compute_Id_Hash(unique_key);
79
80 std::tuple<u32, float, bool> action_probability_log_tuple = explorer.Choose_Action(seed + m_app_id, context);
81
82 u32 action = std::get<0>(action_probability_log_tuple);
83 float prob = std::get<1>(action_probability_log_tuple);
84
85 if (std::get<2>(action_probability_log_tuple))
86 {
87 m_recorder.Record(context, action, prob, unique_key);
88 }
89
90 return action;
91 }
92
93 private:
94 u64 m_app_id;
95 IRecorder<Ctx>& m_recorder;
96 };
97
104 template <class Ctx>
105 class IRecorder
106 {
107 public:
116 virtual void Record(Ctx& context, u32 action, float probability, string unique_key) = 0;
117 };
118
125 template <class Ctx>
126 class IExplorer
127 {
128 public:
138 virtual std::tuple<u32, float, bool> Choose_Action(u64 salted_seed, Ctx& context) = 0;
139 };
140
145 template <class Ctx>
146 class IPolicy
147 {
148 public:
155 virtual u32 Choose_Action(Ctx& context) = 0;
156 };
157
161 template <class Ctx>
162 class IScorer
163 {
164 public:
171 virtual vector<float> Score_Actions(Ctx& context) = 0;
172 };
173
177 template <class Ctx>
178 struct StringRecorder : public IRecorder<Ctx>
179 {
180 void Record(Ctx& context, u32 action, float probability, string unique_key)
181 {
182 // Implicitly enforce To_String() API on the context
183 m_recording.append(to_string(action));
184 m_recording.append(" ", 1);
185 m_recording.append(unique_key);
186 m_recording.append(" ", 1);
187
188 char prob_str[10] = { 0 };
189 NumberUtils::Float_To_String(probability, prob_str);
190 m_recording.append(prob_str);
191
192 m_recording.append(" | ", 3);
193 m_recording.append(context.To_String());
194 m_recording.append("\n");
195 }
196
197 // Gets the content of the recording so far as a string and optionally clears internal content.
198 string Get_Recording(bool flush = true)
199 {
200 if (!flush)
201 {
202 return m_recording;
203 }
204 string recording = m_recording;
205 m_recording.clear();
206 return recording;
207 }
208
209 private:
210 string m_recording;
211 };
212
216 struct Feature
217 {
218 float Value;
219 u32 Id;
220
221 bool operator==(Feature other_feature)
222 {
223 return Id == other_feature.Id;
224 }
225 };
226
230 class SimpleContext
231 {
232 public:
233 SimpleContext(vector<Feature>& features) :
234 m_features(features)
235 { }
236
237 vector<Feature>& Get_Features()
238 {
239 return m_features;
240 }
241
242 string To_String()
243 {
244 string out_string;
245 char feature_str[35] = { 0 };
246 for (size_t i = 0; i < m_features.size(); i++)
247 {
248 int chars;
249 if (i == 0)
250 {
251 chars = sprintf(feature_str, "%d:", m_features[i].Id);
252 }
253 else
254 {
255 chars = sprintf(feature_str, " %d:", m_features[i].Id);
256 }
257 NumberUtils::print_float(feature_str + chars, m_features[i].Value);
258 out_string.append(feature_str);
259 }
260 return out_string;
261 }
262
263 private:
264 vector<Feature>& m_features;
265 };
266
271 template <class Ctx>
272 class EpsilonGreedyExplorer : public IExplorer<Ctx>
273 {
274 public:
282 EpsilonGreedyExplorer(IPolicy<Ctx>& default_policy, float epsilon, u32 num_actions) :
283 m_default_policy(default_policy), m_epsilon(epsilon), m_num_actions(num_actions)
284 {
285 if (m_num_actions < 1)
286 {
287 throw std::invalid_argument("Number of actions must be at least 1.");
288 }
289
290 if (m_epsilon < 0 || m_epsilon > 1)
291 {
292 throw std::invalid_argument("Epsilon must be between 0 and 1.");
293 }
294 }
295
296 ~EpsilonGreedyExplorer()
297 {
298 }
299
300 private:
301 std::tuple<u32, float, bool> Choose_Action(u64 salted_seed, Ctx& context)
302 {
303 PRG::prg random_generator(salted_seed);
304
305 // Invoke the default policy function to get the action
306 u32 chosen_action = m_default_policy.Choose_Action(context);
307
308 if (chosen_action == 0 || chosen_action > m_num_actions)
309 {
310 throw std::invalid_argument("Action chosen by default policy is not within valid range.");
311 }
312
313 float action_probability = 0.f;
314 float base_probability = m_epsilon / m_num_actions; // uniform probability
315
316 // TODO: check this random generation
317 if (random_generator.Uniform_Unit_Interval() < 1.f - m_epsilon)
318 {
319 action_probability = 1.f - m_epsilon + base_probability;
320 }
321 else
322 {
323 // Get uniform random action ID
324 u32 actionId = random_generator.Uniform_Int(1, m_num_actions);
325
326 if (actionId == chosen_action)
327 {
328 // IF it matches the one chosen by the default policy
329 // then increase the probability
330 action_probability = 1.f - m_epsilon + base_probability;
331 }
332 else
333 {
334 // Otherwise it's just the uniform probability
335 action_probability = base_probability;
336 }
337 chosen_action = actionId;
338 }
339
340 return std::tuple<u32, float, bool>(chosen_action, action_probability, true);
341 }
342
343 private:
344 IPolicy<Ctx>& m_default_policy;
345 float m_epsilon;
346 u32 m_num_actions;
347
348 private:
349 friend class MwtExplorer<Ctx>;
350 };
351
356 template <class Ctx>
357 class SoftmaxExplorer : public IExplorer<Ctx>
358 {
359 public:
367 SoftmaxExplorer(IScorer<Ctx>& default_scorer, float lambda, u32 num_actions) :
368 m_default_scorer(default_scorer), m_lambda(lambda), m_num_actions(num_actions)
369 {
370 if (m_num_actions < 1)
371 {
372 throw std::invalid_argument("Number of actions must be at least 1.");
373 }
374 }
375
376 private:
377 std::tuple<u32, float, bool> Choose_Action(u64 salted_seed, Ctx& context)
378 {
379 PRG::prg random_generator(salted_seed);
380
381 // Invoke the default scorer function
382 vector<float> scores = m_default_scorer.Score_Actions(context);
383 u32 num_scores = (u32)scores.size();
384 if (num_scores != m_num_actions)
385 {
386 throw std::invalid_argument("The number of scores returned by the scorer must equal number of actions");
387 }
388
389 u32 i = 0;
390
391 float max_score = -FLT_MAX;
392 for (i = 0; i < num_scores; i++)
393 {
394 if (max_score < scores[i])
395 {
396 max_score = scores[i];
397 }
398 }
399
400 // Create a normalized exponential distribution based on the returned scores
401 for (i = 0; i < num_scores; i++)
402 {
403 scores[i] = exp(m_lambda * (scores[i] - max_score));
404 }
405
406 // Create a discrete_distribution based on the returned weights. This class handles the
407 // case where the sum of the weights is < or > 1, by normalizing agains the sum.
408 float total = 0.f;
409 for (size_t i = 0; i < num_scores; i++)
410 total += scores[i];
411
412 float draw = random_generator.Uniform_Unit_Interval();
413
414 float sum = 0.f;
415 float action_probability = 0.f;
416 u32 action_index = num_scores - 1;
417 for (u32 i = 0; i < num_scores; i++)
418 {
419 scores[i] = scores[i] / total;
420 sum += scores[i];
421 if (sum > draw)
422 {
423 action_index = i;
424 action_probability = scores[i];
425 break;
426 }
427 }
428
429 // action id is one-based
430 return std::tuple<u32, float, bool>(action_index + 1, action_probability, true);
431 }
432
433 private:
434 IScorer<Ctx>& m_default_scorer;
435 float m_lambda;
436 u32 m_num_actions;
437
438 private:
439 friend class MwtExplorer<Ctx>;
440 };
441
446 template <class Ctx>
447 class GenericExplorer : public IExplorer<Ctx>
448 {
449 public:
456 GenericExplorer(IScorer<Ctx>& default_scorer, u32 num_actions) :
457 m_default_scorer(default_scorer), m_num_actions(num_actions)
458 {
459 if (m_num_actions < 1)
460 {
461 throw std::invalid_argument("Number of actions must be at least 1.");
462 }
463 }
464
465 private:
466 std::tuple<u32, float, bool> Choose_Action(u64 salted_seed, Ctx& context)
467 {
468 PRG::prg random_generator(salted_seed);
469
470 // Invoke the default scorer function
471 vector<float> weights = m_default_scorer.Score_Actions(context);
472 u32 num_weights = (u32)weights.size();
473 if (num_weights != m_num_actions)
474 {
475 throw std::invalid_argument("The number of weights returned by the scorer must equal number of actions");
476 }
477
478 // Create a discrete_distribution based on the returned weights. This class handles the
479 // case where the sum of the weights is < or > 1, by normalizing agains the sum.
480 float total = 0.f;
481 for (size_t i = 0; i < num_weights; i++)
482 {
483 if (weights[i] < 0)
484 {
485 throw std::invalid_argument("Scores must be non-negative.");
486 }
487 total += weights[i];
488 }
489 if (total == 0)
490 {
491 throw std::invalid_argument("At least one score must be positive.");
492 }
493
494 float draw = random_generator.Uniform_Unit_Interval();
495
496 float sum = 0.f;
497 float action_probability = 0.f;
498 u32 action_index = num_weights - 1;
499 for (u32 i = 0; i < num_weights; i++)
500 {
501 weights[i] = weights[i] / total;
502 sum += weights[i];
503 if (sum > draw)
504 {
505 action_index = i;
506 action_probability = weights[i];
507 break;
508 }
509 }
510
511 // action id is one-based
512 return std::tuple<u32, float, bool>(action_index + 1, action_probability, true);
513 }
514
515 private:
516 IScorer<Ctx>& m_default_scorer;
517 u32 m_num_actions;
518
519 private:
520 friend class MwtExplorer<Ctx>;
521 };
522
527 template <class Ctx>
528 class TauFirstExplorer : public IExplorer<Ctx>
529 {
530 public:
531
539 TauFirstExplorer(IPolicy<Ctx>& default_policy, u32 tau, u32 num_actions) :
540 m_default_policy(default_policy), m_tau(tau), m_num_actions(num_actions)
541 {
542 if (m_num_actions < 1)
543 {
544 throw std::invalid_argument("Number of actions must be at least 1.");
545 }
546 }
547
548 private:
549 std::tuple<u32, float, bool> Choose_Action(u64 salted_seed, Ctx& context)
550 {
551 PRG::prg random_generator(salted_seed);
552
553 u32 chosen_action = 0;
554 float action_probability = 0.f;
555 bool log_action;
556 if (m_tau)
557 {
558 m_tau--;
559 u32 actionId = random_generator.Uniform_Int(1, m_num_actions);
560 action_probability = 1.f / m_num_actions;
561 chosen_action = actionId;
562 log_action = true;
563 }
564 else
565 {
566 // Invoke the default policy function to get the action
567 chosen_action = m_default_policy.Choose_Action(context);
568
569 if (chosen_action == 0 || chosen_action > m_num_actions)
570 {
571 throw std::invalid_argument("Action chosen by default policy is not within valid range.");
572 }
573
574 action_probability = 1.f;
575 log_action = false;
576 }
577
578 return std::tuple<u32, float, bool>(chosen_action, action_probability, log_action);
579 }
580
581 private:
582 IPolicy<Ctx>& m_default_policy;
583 u32 m_tau;
584 u32 m_num_actions;
585
586 private:
587 friend class MwtExplorer<Ctx>;
588 };
589
594 template <class Ctx>
595 class BootstrapExplorer : public IExplorer<Ctx>
596 {
597 public:
604 BootstrapExplorer(vector<unique_ptr<IPolicy<Ctx>>>& default_policy_functions, u32 num_actions) :
605 m_default_policy_functions(default_policy_functions),
606 m_num_actions(num_actions)
607 {
608 m_bags = (u32)default_policy_functions.size();
609 if (m_num_actions < 1)
610 {
611 throw std::invalid_argument("Number of actions must be at least 1.");
612 }
613
614 if (m_bags < 1)
615 {
616 throw std::invalid_argument("Number of bags must be at least 1.");
617 }
618 }
619
620 private:
621 std::tuple<u32, float, bool> Choose_Action(u64 salted_seed, Ctx& context)
622 {
623 PRG::prg random_generator(salted_seed);
624
625 // Select bag
626 u32 chosen_bag = random_generator.Uniform_Int(0, m_bags - 1);
627
628 // Invoke the default policy function to get the action
629 u32 chosen_action = 0;
630 u32 action_from_bag = 0;
631 vector<u32> actions_selected;
632 for (size_t i = 0; i < m_num_actions; i++)
633 {
634 actions_selected.push_back(0);
635 }
636
637 // Invoke the default policy function to get the action
638 for (u32 current_bag = 0; current_bag < m_bags; current_bag++)
639 {
640 action_from_bag = m_default_policy_functions[current_bag]->Choose_Action(context);
641
642 if (action_from_bag == 0 || action_from_bag > m_num_actions)
643 {
644 throw std::invalid_argument("Action chosen by default policy is not within valid range.");
645 }
646
647 if (current_bag == chosen_bag)
648 {
649 chosen_action = action_from_bag;
650 }
651 //this won't work if actions aren't 0 to Count
652 actions_selected[action_from_bag - 1]++; // action id is one-based
653 }
654 float action_probability = (float)actions_selected[chosen_action - 1] / m_bags; // action id is one-based
655
656 return std::tuple<u32, float, bool>(chosen_action, action_probability, true);
657 }
658
659 private:
660 vector<unique_ptr<IPolicy<Ctx>>>& m_default_policy_functions;
661 u32 m_bags;
662 u32 m_num_actions;
663
664 private:
665 friend class MwtExplorer<Ctx>;
666 };
667 } // End namespace MultiWorldTestingCpp
NativeMultiWorldTesting::SimpleContext
Definition: MWTExplorer.h:230
NativeMultiWorldTesting::StringRecorder
Definition: MWTExplorer.h:178
NativeMultiWorldTesting::IExplorer
Definition: MWTExplorer.h:47
NativeMultiWorldTesting::SoftmaxExplorer
Definition: MWTExplorer.h:357
NativeMultiWorldTesting::TauFirstExplorer
Definition: MWTExplorer.h:528
NativeMultiWorldTesting::IRecorder
Definition: MWTExplorer.h:45
NativeMultiWorldTesting::Feature
Definition: MWTExplorer.h:216
std
NativeMultiWorldTesting::GenericExplorer::GenericExplorer
GenericExplorer(IScorer< Ctx > &default_scorer, u32 num_actions)
Definition: MWTExplorer.h:456
NativeMultiWorldTesting::BootstrapExplorer::BootstrapExplorer
BootstrapExplorer(vector< unique_ptr< IPolicy< Ctx >>> &default_policy_functions, u32 num_actions)
Definition: MWTExplorer.h:604
NativeMultiWorldTesting::IPolicy
Definition: MWTExplorer.h:146
NativeMultiWorldTesting::SoftmaxExplorer::SoftmaxExplorer
SoftmaxExplorer(IScorer< Ctx > &default_scorer, float lambda, u32 num_actions)
Definition: MWTExplorer.h:367
NativeMultiWorldTesting::EpsilonGreedyExplorer
Definition: MWTExplorer.h:272
NativeMultiWorldTesting::BootstrapExplorer
Definition: MWTExplorer.h:595
NativeMultiWorldTesting::EpsilonGreedyExplorer::EpsilonGreedyExplorer
EpsilonGreedyExplorer(IPolicy< Ctx > &default_policy, float epsilon, u32 num_actions)
Definition: MWTExplorer.h:282
NativeMultiWorldTesting::PRG::prg
Definition: utility.h:257
NativeMultiWorldTesting::StringRecorder::Record
void Record(Ctx &context, u32 action, float probability, string unique_key)
Definition: MWTExplorer.h:180
NativeMultiWorldTesting::GenericExplorer
Definition: MWTExplorer.h:447
NativeMultiWorldTesting::IExplorer::Choose_Action
virtual std::tuple< u32, float, bool > Choose_Action(u64 salted_seed, Ctx &context)=0
NativeMultiWorldTesting::MwtExplorer::Choose_Action
u32 Choose_Action(IExplorer< Ctx > &explorer, string unique_key, Ctx &context)
Definition: MWTExplorer.h:76
NativeMultiWorldTesting::MwtExplorer
Definition: MWTExplorer.h:54
NativeMultiWorldTesting::IScorer
Definition: MWTExplorer.h:162
NativeMultiWorldTesting::TauFirstExplorer::TauFirstExplorer
TauFirstExplorer(IPolicy< Ctx > &default_policy, u32 tau, u32 num_actions)
Definition: MWTExplorer.h:539
NativeMultiWorldTesting::MwtExplorer::MwtExplorer
MwtExplorer(std::string app_id, IRecorder< Ctx > &recorder)
Definition: MWTExplorer.h:63
NativeMultiWorldTesting
Definition: MWTExplorer.h:41
MultiWorldTesting::Ctx
where Ctx
A sample recorder class that converts the exploration tuple into string format.
Definition: explore_clr_wrapper.h:346
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: static/utility.h Source File

  • File List
  • static
utility.h
1 /*******************************************************************/
2 // Classes declared in this file are intended for internal use only.
3 /*******************************************************************/
4
5 #pragma once
6 #include <stdint.h>
7 #include <sys/types.h> /* defines size_t */
8
9 #ifdef WIN32
10 typedef unsigned __int64 u64;
11 typedef unsigned __int32 u32;
12 typedef unsigned __int16 u16;
13 typedef unsigned __int8 u8;
14 typedef signed __int64 i64;
15 typedef signed __int32 i32;
16 typedef signed __int16 i16;
17 typedef signed __int8 i8;
18 #else
19 typedef uint64_t u64;
20 typedef uint32_t u32;
21 typedef uint16_t u16;
22 typedef uint8_t u8;
23 typedef int64_t i64;
24 typedef int32_t i32;
25 typedef int16_t i16;
26 typedef int8_t i8;
27 #endif
28
29 typedef unsigned char byte;
30
31 #include <string>
32 #include <stdint.h>
33 #include <math.h>
34
40 namespace NativeMultiWorldTesting {
41
42 //
43 // MurmurHash3, by Austin Appleby
44 //
45 // Originals at:
46 // http://code.google.com/p/smhasher/source/browse/trunk/MurmurHash3.cpp
47 // http://code.google.com/p/smhasher/source/browse/trunk/MurmurHash3.h
48 //
49 // Notes:
50 // 1) this code assumes we can read a 4-byte value from any address
51 // without crashing (i.e non aligned access is supported). This is
52 // not a problem on Intel/x86/AMD64 machines (including new Macs)
53 // 2) It produces different results on little-endian and big-endian machines.
54 //
55 //-----------------------------------------------------------------------------
56 // MurmurHash3 was written by Austin Appleby, and is placed in the public
57 // domain. The author hereby disclaims copyright to this source code.
58
59 // Note - The x86 and x64 versions do _not_ produce the same results, as the
60 // algorithms are optimized for their respective platforms. You can still
61 // compile and run any of them on any platform, but your performance with the
62 // non-native version will be less than optimal.
63 //-----------------------------------------------------------------------------
64
65 // Platform-specific functions and macros
66 #if defined(_MSC_VER) // Microsoft Visual Studio
67 # include <stdint.h>
68
69 # include <stdlib.h>
70 # define ROTL32(x,y) _rotl(x,y)
71 # define BIG_CONSTANT(x) (x)
72
73 #else // Other compilers
74 # include <stdint.h> /* defines uint32_t etc */
75
76 inline uint32_t rotl32(uint32_t x, int8_t r)
77 {
78 return (x << r) | (x >> (32 - r));
79 }
80
81 # define ROTL32(x,y) rotl32(x,y)
82 # define BIG_CONSTANT(x) (x##LLU)
83
84 #endif // !defined(_MSC_VER)
85
86 struct murmur_hash {
87
88 //-----------------------------------------------------------------------------
89 // Block read - if your platform needs to do endian-swapping or can only
90 // handle aligned reads, do the conversion here
91 private:
92 static inline uint32_t getblock(const uint32_t * p, int i)
93 {
94 return p[i];
95 }
96
97 //-----------------------------------------------------------------------------
98 // Finalization mix - force all bits of a hash block to avalanche
99
100 static inline uint32_t fmix(uint32_t h)
101 {
102 h ^= h >> 16;
103 h *= 0x85ebca6b;
104 h ^= h >> 13;
105 h *= 0xc2b2ae35;
106 h ^= h >> 16;
107
108 return h;
109 }
110
111 //-----------------------------------------------------------------------------
112 public:
113 uint32_t uniform_hash(const void * key, size_t len, uint32_t seed)
114 {
115 const uint8_t * data = (const uint8_t*)key;
116 const int nblocks = (int)len / 4;
117
118 uint32_t h1 = seed;
119
120 const uint32_t c1 = 0xcc9e2d51;
121 const uint32_t c2 = 0x1b873593;
122
123 // --- body
124 const uint32_t * blocks = (const uint32_t *)(data + nblocks * 4);
125
126 for (int i = -nblocks; i; i++) {
127 uint32_t k1 = getblock(blocks, i);
128
129 k1 *= c1;
130 k1 = ROTL32(k1, 15);
131 k1 *= c2;
132
133 h1 ^= k1;
134 h1 = ROTL32(h1, 13);
135 h1 = h1 * 5 + 0xe6546b64;
136 }
137
138 // --- tail
139 const uint8_t * tail = (const uint8_t*)(data + nblocks * 4);
140
141 uint32_t k1 = 0;
142
143 switch (len & 3) {
144 case 3: k1 ^= tail[2] << 16;
145 case 2: k1 ^= tail[1] << 8;
146 case 1: k1 ^= tail[0];
147 k1 *= c1; k1 = ROTL32(k1, 15); k1 *= c2; h1 ^= k1;
148 }
149
150 // --- finalization
151 h1 ^= len;
152
153 return fmix(h1);
154 }
155 };
156
157 class HashUtils
158 {
159 public:
160 static u64 Compute_Id_Hash(const std::string& unique_id)
161 {
162 size_t ret = 0;
163 const char *p = unique_id.c_str();
164 while (*p != '\0')
165 if (*p >= '0' && *p <= '9')
166 ret = 10 * ret + *(p++) - '0';
167 else
168 {
169 murmur_hash foo;
170 return foo.uniform_hash(unique_id.c_str(), unique_id.size(), 0);
171 }
172 return ret;
173 }
174 };
175
176 const size_t max_int = 100000;
177 const float max_float = max_int;
178 const float min_float = 0.00001f;
179 const size_t max_digits = (size_t) roundf((float) (-log(min_float) / log(10.)));
180
181 class NumberUtils
182 {
183 public:
184 static void Float_To_String(float f, char* str)
185 {
186 int x = (int)f;
187 int d = (int)(fabs(f - x) * 100000);
188 sprintf(str, "%d.%05d", x, d);
189 }
190
191 template<bool trailing_zeros>
192 static void print_mantissa(char*& begin, float f)
193 { // helper for print_float
194 char values[10];
195 size_t v = (size_t)f;
196 size_t digit = 0;
197 size_t first_nonzero = 0;
198 for (size_t max = 1; max <= v; ++digit)
199 {
200 size_t max_next = max * 10;
201 char v_mod = (char) (v % max_next / max);
202 if (!trailing_zeros && v_mod != '\0' && first_nonzero == 0)
203 first_nonzero = digit;
204 values[digit] = '0' + v_mod;
205 max = max_next;
206 }
207 if (!trailing_zeros)
208 for (size_t i = max_digits; i > digit; i--)
209 *begin++ = '0';
210 while (digit > first_nonzero)
211 *begin++ = values[--digit];
212 }
213
214 static void print_float(char* begin, float f)
215 {
216 bool sign = false;
217 if (f < 0.f)
218 sign = true;
219 float unsigned_f = fabsf(f);
220 if (unsigned_f < max_float && unsigned_f > min_float)
221 {
222 if (sign)
223 *begin++ = '-';
224 print_mantissa<true>(begin, unsigned_f);
225 unsigned_f -= (size_t)unsigned_f;
226 unsigned_f *= max_int;
227 if (unsigned_f >= 1.f)
228 {
229 *begin++ = '.';
230 print_mantissa<false>(begin, unsigned_f);
231 }
232 }
233 else if (unsigned_f == 0.)
234 *begin++ = '0';
235 else
236 {
237 sprintf(begin, "%g", f);
238 return;
239 }
240 *begin = '\0';
241 return;
242 }
243 };
244
245 //A quick implementation similar to drand48 for cross-platform compatibility
246 namespace PRG {
247 const uint64_t a = 0xeece66d5deece66dULL;
248 const uint64_t c = 2147483647;
249
250 const int bias = 127 << 23;
251
252 union int_float {
253 int32_t i;
254 float f;
255 };
256
257 struct prg {
258 private:
259 uint64_t v;
260 public:
261 prg() { v = c; }
262 prg(uint64_t initial) { v = initial; }
263
264 float merand48(uint64_t& initial)
265 {
266 initial = a * initial + c;
267 int_float temp;
268 temp.i = ((initial >> 25) & 0x7FFFFF) | bias;
269 return temp.f - 1;
270 }
271
272 float Uniform_Unit_Interval()
273 {
274 return merand48(v);
275 }
276
277 uint32_t Uniform_Int(uint32_t low, uint32_t high)
278 {
279 merand48(v);
280 uint32_t ret = low + ((v >> 25) % (high - low + 1));
281 return ret;
282 }
283 };
284 }
285 }
NativeMultiWorldTesting::murmur_hash
Definition: utility.h:86
NativeMultiWorldTesting::NumberUtils
Definition: utility.h:181
NativeMultiWorldTesting::HashUtils
Definition: utility.h:157
NativeMultiWorldTesting::PRG::prg
Definition: utility.h:257
NativeMultiWorldTesting::PRG::int_float
Definition: utility.h:252
NativeMultiWorldTesting
Definition: MWTExplorer.h:41
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • BootstrapExplorer
MultiWorldTesting::BootstrapExplorer< Ctx > Member List

This is the complete list of members for MultiWorldTesting::BootstrapExplorer< Ctx >, including all inherited members.

__pad0__ (defined in MultiWorldTesting::BootstrapExplorer< Ctx >) | MultiWorldTesting::BootstrapExplorer< Ctx > |
BootstrapExplorer(cli::array< IPolicy< Ctx >^>^defaultPolicies, UInt32 numActions) | MultiWorldTesting::BootstrapExplorer< Ctx > | inline
Get() (defined in MultiWorldTesting::BootstrapExplorer< Ctx >) | MultiWorldTesting::BootstrapExplorer< Ctx > | inline
override (defined in MultiWorldTesting::BootstrapExplorer< Ctx >) | MultiWorldTesting::BootstrapExplorer< Ctx > |
~BootstrapExplorer() (defined in MultiWorldTesting::BootstrapExplorer< Ctx >) | MultiWorldTesting::BootstrapExplorer< Ctx > | inline
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • EpsilonGreedyExplorer
MultiWorldTesting::EpsilonGreedyExplorer< Ctx > Member List

This is the complete list of members for MultiWorldTesting::EpsilonGreedyExplorer< Ctx >, including all inherited members.

__pad0__ (defined in MultiWorldTesting::EpsilonGreedyExplorer< Ctx >) | MultiWorldTesting::EpsilonGreedyExplorer< Ctx > |
EpsilonGreedyExplorer(IPolicy< Ctx >^defaultPolicy, float epsilon, UInt32 numActions) | MultiWorldTesting::EpsilonGreedyExplorer< Ctx > | inline
Get() (defined in MultiWorldTesting::EpsilonGreedyExplorer< Ctx >) | MultiWorldTesting::EpsilonGreedyExplorer< Ctx > | inline
override (defined in MultiWorldTesting::EpsilonGreedyExplorer< Ctx >) | MultiWorldTesting::EpsilonGreedyExplorer< Ctx > |
~EpsilonGreedyExplorer() (defined in MultiWorldTesting::EpsilonGreedyExplorer< Ctx >) | MultiWorldTesting::EpsilonGreedyExplorer< Ctx > | inline
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • GenericExplorer
MultiWorldTesting::GenericExplorer< Ctx > Member List

This is the complete list of members for MultiWorldTesting::GenericExplorer< Ctx >, including all inherited members.

__pad0__ (defined in MultiWorldTesting::GenericExplorer< Ctx >) | MultiWorldTesting::GenericExplorer< Ctx > |
GenericExplorer(IScorer< Ctx >^defaultScorer, UInt32 numActions) | MultiWorldTesting::GenericExplorer< Ctx > | inline
Get() (defined in MultiWorldTesting::GenericExplorer< Ctx >) | MultiWorldTesting::GenericExplorer< Ctx > | inline
~GenericExplorer() (defined in MultiWorldTesting::GenericExplorer< Ctx >) | MultiWorldTesting::GenericExplorer< Ctx > | inline
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • MwtExplorer
MultiWorldTesting::MwtExplorer< Ctx > Member List

This is the complete list of members for MultiWorldTesting::MwtExplorer< Ctx >, including all inherited members.

__pad0__ (defined in MultiWorldTesting::MwtExplorer< Ctx >) | MultiWorldTesting::MwtExplorer< Ctx > |
action (defined in MultiWorldTesting::MwtExplorer< Ctx >) | MultiWorldTesting::MwtExplorer< Ctx > |
ChooseAction(IExplorer< Ctx >^explorer, String^unique_key, Ctx context) | MultiWorldTesting::MwtExplorer< Ctx > | inline
MwtExplorer(String^appId, IRecorder< Ctx >^recorder) | MultiWorldTesting::MwtExplorer< Ctx > | inline
override (defined in MultiWorldTesting::MwtExplorer< Ctx >) | MultiWorldTesting::MwtExplorer< Ctx > |
probability (defined in MultiWorldTesting::MwtExplorer< Ctx >) | MultiWorldTesting::MwtExplorer< Ctx > |


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • NativeContext
MultiWorldTesting::NativeContext Member List

This is the complete list of members for MultiWorldTesting::NativeContext, including all inherited members.

Get_Clr_Context() (defined in MultiWorldTesting::NativeContext) | MultiWorldTesting::NativeContext | inline
Get_Clr_Explorer() (defined in MultiWorldTesting::NativeContext) | MultiWorldTesting::NativeContext | inline
Get_Clr_Mwt() (defined in MultiWorldTesting::NativeContext) | MultiWorldTesting::NativeContext | inline
NativeContext(void *clr_mwt, void *clr_explorer, void *clr_context) (defined in MultiWorldTesting::NativeContext) | MultiWorldTesting::NativeContext | inline


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • NativePolicy
MultiWorldTesting::NativePolicy Member List

This is the complete list of members for MultiWorldTesting::NativePolicy, including all inherited members.

Choose_Action(NativeContext &context) (defined in MultiWorldTesting::NativePolicy) | MultiWorldTesting::NativePolicy | inline
IPolicy< NativeContext >::Choose_Action(NativeContext &context)=0 | NativeMultiWorldTesting::IPolicy< NativeContext > | pure virtual
NativePolicy(Native_Policy_Callback *func, int index=-1) (defined in MultiWorldTesting::NativePolicy) | MultiWorldTesting::NativePolicy | inline


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • NativeRecorder
MultiWorldTesting::NativeRecorder Member List

This is the complete list of members for MultiWorldTesting::NativeRecorder, including all inherited members.

NativeRecorder(Native_Recorder_Callback *native_func) (defined in MultiWorldTesting::NativeRecorder) | MultiWorldTesting::NativeRecorder | inline
Record(NativeContext &context, u32 action, float probability, string unique_key) (defined in MultiWorldTesting::NativeRecorder) | MultiWorldTesting::NativeRecorder | inline
IRecorder< NativeContext >::Record(NativeContext &context, u32 action, float probability, string unique_key)=0 | NativeMultiWorldTesting::IRecorder< NativeContext > | pure virtual


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • NativeScorer
MultiWorldTesting::NativeScorer Member List

This is the complete list of members for MultiWorldTesting::NativeScorer, including all inherited members.

NativeScorer(Native_Scorer_Callback *func) (defined in MultiWorldTesting::NativeScorer) | MultiWorldTesting::NativeScorer | inline
Score_Actions(NativeContext &context) (defined in MultiWorldTesting::NativeScorer) | MultiWorldTesting::NativeScorer | inline
IScorer< NativeContext >::Score_Actions(NativeContext &context)=0 | NativeMultiWorldTesting::IScorer< NativeContext > | pure virtual


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • NativeStringContext
MultiWorldTesting::NativeStringContext Member List

This is the complete list of members for MultiWorldTesting::NativeStringContext, including all inherited members.

NativeStringContext(void *clr_context, Native_To_String_Callback *func) (defined in MultiWorldTesting::NativeStringContext) | MultiWorldTesting::NativeStringContext | inline
To_String() (defined in MultiWorldTesting::NativeStringContext) | MultiWorldTesting::NativeStringContext | inline


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • SimpleContext
MultiWorldTesting::SimpleContext Member List

This is the complete list of members for MultiWorldTesting::SimpleContext, including all inherited members.

__pad0__ (defined in MultiWorldTesting::SimpleContext) | MultiWorldTesting::SimpleContext |
GetFeatures() (defined in MultiWorldTesting::SimpleContext) | MultiWorldTesting::SimpleContext | inline
SimpleContext(cli::array< Feature >^features) (defined in MultiWorldTesting::SimpleContext) | MultiWorldTesting::SimpleContext | inline
ToString() override (defined in MultiWorldTesting::SimpleContext) | MultiWorldTesting::SimpleContext | inlinevirtual
~SimpleContext() (defined in MultiWorldTesting::SimpleContext) | MultiWorldTesting::SimpleContext | inline


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • SoftmaxExplorer
MultiWorldTesting::SoftmaxExplorer< Ctx > Member List

This is the complete list of members for MultiWorldTesting::SoftmaxExplorer< Ctx >, including all inherited members.

__pad0__ (defined in MultiWorldTesting::SoftmaxExplorer< Ctx >) | MultiWorldTesting::SoftmaxExplorer< Ctx > |
Get() (defined in MultiWorldTesting::SoftmaxExplorer< Ctx >) | MultiWorldTesting::SoftmaxExplorer< Ctx > | inline
SoftmaxExplorer(IScorer< Ctx >^defaultScorer, float lambda, UInt32 numActions) | MultiWorldTesting::SoftmaxExplorer< Ctx > | inline
~SoftmaxExplorer() (defined in MultiWorldTesting::SoftmaxExplorer< Ctx >) | MultiWorldTesting::SoftmaxExplorer< Ctx > | inline


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • TauFirstExplorer
MultiWorldTesting::TauFirstExplorer< Ctx > Member List

This is the complete list of members for MultiWorldTesting::TauFirstExplorer< Ctx >, including all inherited members.

__pad0__ (defined in MultiWorldTesting::TauFirstExplorer< Ctx >) | MultiWorldTesting::TauFirstExplorer< Ctx > |
Get() (defined in MultiWorldTesting::TauFirstExplorer< Ctx >) | MultiWorldTesting::TauFirstExplorer< Ctx > | inline
override (defined in MultiWorldTesting::TauFirstExplorer< Ctx >) | MultiWorldTesting::TauFirstExplorer< Ctx > |
TauFirstExplorer(IPolicy< Ctx >^defaultPolicy, UInt32 tau, UInt32 numActions) | MultiWorldTesting::TauFirstExplorer< Ctx > | inline
~TauFirstExplorer() (defined in MultiWorldTesting::TauFirstExplorer< Ctx >) | MultiWorldTesting::TauFirstExplorer< Ctx > | inline


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • abstract
MultiWorldTesting::abstract< Ctx > Member List

This is the complete list of members for MultiWorldTesting::abstract< Ctx >, including all inherited members.

__pad0__ (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > |
action (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > |
GetNativePolicies(int count) (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inline
GetNativePolicy() (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inline
GetNativeRecorder() (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inline
GetNativeScorer() (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inline
index (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > |
InteropInvoke(IntPtr callbackPtr, IntPtr contextPtr, int index) (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inlinestatic
InteropInvoke(IntPtr mwtPtr, IntPtr contextPtr, UInt32 action, float probability, IntPtr uniqueKeyPtr) (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inlinestatic
InteropInvoke(IntPtr callbackPtr, IntPtr contextPtr, IntPtr scoresPtr, IntPtr sizePtr) (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inlinestatic
PolicyCallback() (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inline
probability (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > |
RecorderCallback() (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inline
ScorerCallback() (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inline
unique_key (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > |
~PolicyCallback() (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inline
~RecorderCallback() (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inline
~ScorerCallback() (defined in MultiWorldTesting::abstract< Ctx >) | MultiWorldTesting::abstract< Ctx > | inline
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

MyPolicy Member List

This is the complete list of members for MyPolicy, including all inherited members.

Choose_Action(MyContext &context) (defined in MyPolicy) | MyPolicy | inline
ChooseAction(MyContextcontext)=0 | MultiWorldTesting::IPolicy< MyContext > | pure virtual
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

MyRecorder Member List

This is the complete list of members for MyRecorder, including all inherited members.

Record(MyContext &context, u32 action, float probability, string unique_key) (defined in MyRecorder) | MyRecorder | inlinevirtual
IRecorder< MyContext >::Record(MyContextcontext, UInt32 action, float probability, String^uniqueKey)=0 | MultiWorldTesting::IRecorder< MyContext > | pure virtual
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

MyScorer Member List

This is the complete list of members for MyScorer, including all inherited members.

MyScorer(u32 num_actions) (defined in MyScorer) | MyScorer | inline
Score_Actions(MyContext &context) (defined in MyScorer) | MyScorer | inline
ScoreActions(MyContextcontext)=0 | MultiWorldTesting::IScorer< MyContext > | pure virtual
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

MySimplePolicy Member List

This is the complete list of members for MySimplePolicy, including all inherited members.

Choose_Action(SimpleContext &context) (defined in MySimplePolicy) | MySimplePolicy | inline
Choose_Action(SimpleContext &context) (defined in MySimplePolicy) | MySimplePolicy | inline
ChooseAction(SimpleContextcontext)=0 | MultiWorldTesting::IPolicy< SimpleContext > | pure virtual
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • BootstrapExplorer
NativeMultiWorldTesting::BootstrapExplorer< Ctx > Member List

This is the complete list of members for NativeMultiWorldTesting::BootstrapExplorer< Ctx >, including all inherited members.

BootstrapExplorer(vector< unique_ptr< IPolicy< Ctx >>> &default_policy_functions, u32 num_actions) | NativeMultiWorldTesting::BootstrapExplorer< Ctx > | inline
MwtExplorer< Ctx > (defined in NativeMultiWorldTesting::BootstrapExplorer< Ctx >) | NativeMultiWorldTesting::BootstrapExplorer< Ctx > | friend


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • EpsilonGreedyExplorer
NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx > Member List

This is the complete list of members for NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx >, including all inherited members.

EpsilonGreedyExplorer(IPolicy< Ctx > &default_policy, float epsilon, u32 num_actions) | NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx > | inline
MwtExplorer< Ctx > (defined in NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx >) | NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx > | friend
~EpsilonGreedyExplorer() (defined in NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx >) | NativeMultiWorldTesting::EpsilonGreedyExplorer< Ctx > | inline


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • GenericExplorer
NativeMultiWorldTesting::GenericExplorer< Ctx > Member List

This is the complete list of members for NativeMultiWorldTesting::GenericExplorer< Ctx >, including all inherited members.

GenericExplorer(IScorer< Ctx > &default_scorer, u32 num_actions) | NativeMultiWorldTesting::GenericExplorer< Ctx > | inline
MwtExplorer< Ctx > (defined in NativeMultiWorldTesting::GenericExplorer< Ctx >) | NativeMultiWorldTesting::GenericExplorer< Ctx > | friend


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • HashUtils
NativeMultiWorldTesting::HashUtils Member List

This is the complete list of members for NativeMultiWorldTesting::HashUtils, including all inherited members.

Compute_Id_Hash(const std::string &unique_id) (defined in NativeMultiWorldTesting::HashUtils) NativeMultiWorldTesting::HashUtils inlinestatic


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • IExplorer
NativeMultiWorldTesting::IExplorer< Ctx > Member List

This is the complete list of members for NativeMultiWorldTesting::IExplorer< Ctx >, including all inherited members.

Choose_Action(u64 salted_seed, Ctx &context)=0 NativeMultiWorldTesting::IExplorer< Ctx > pure virtual


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • IPolicy
NativeMultiWorldTesting::IPolicy< Ctx > Member List

This is the complete list of members for NativeMultiWorldTesting::IPolicy< Ctx >, including all inherited members.

Choose_Action(Ctx &context)=0 NativeMultiWorldTesting::IPolicy< Ctx > pure virtual


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • IRecorder
NativeMultiWorldTesting::IRecorder< Ctx > Member List

This is the complete list of members for NativeMultiWorldTesting::IRecorder< Ctx >, including all inherited members.

Record(Ctx &context, u32 action, float probability, string unique_key)=0 NativeMultiWorldTesting::IRecorder< Ctx > pure virtual


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • IScorer
NativeMultiWorldTesting::IScorer< Ctx > Member List

This is the complete list of members for NativeMultiWorldTesting::IScorer< Ctx >, including all inherited members.

Score_Actions(Ctx &context)=0 NativeMultiWorldTesting::IScorer< Ctx > pure virtual


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • MwtExplorer
NativeMultiWorldTesting::MwtExplorer< Ctx > Member List

This is the complete list of members for NativeMultiWorldTesting::MwtExplorer< Ctx >, including all inherited members.

Choose_Action(IExplorer< Ctx > &explorer, string unique_key, Ctx &context) | NativeMultiWorldTesting::MwtExplorer< Ctx > | inline
MwtExplorer(std::string app_id, IRecorder< Ctx > &recorder) | NativeMultiWorldTesting::MwtExplorer< Ctx > | inline


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • NumberUtils
NativeMultiWorldTesting::NumberUtils Member List

This is the complete list of members for NativeMultiWorldTesting::NumberUtils, including all inherited members.

Float_To_String(float f, char *str) (defined in NativeMultiWorldTesting::NumberUtils) | NativeMultiWorldTesting::NumberUtils | inlinestatic
print_float(char *begin, float f) (defined in NativeMultiWorldTesting::NumberUtils) | NativeMultiWorldTesting::NumberUtils | inlinestatic
print_mantissa(char *&begin, float f) (defined in NativeMultiWorldTesting::NumberUtils) | NativeMultiWorldTesting::NumberUtils | inlinestatic


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • SimpleContext
NativeMultiWorldTesting::SimpleContext Member List

This is the complete list of members for NativeMultiWorldTesting::SimpleContext, including all inherited members.

Get_Features() (defined in NativeMultiWorldTesting::SimpleContext) | NativeMultiWorldTesting::SimpleContext | inline
SimpleContext(vector< Feature > &features) (defined in NativeMultiWorldTesting::SimpleContext) | NativeMultiWorldTesting::SimpleContext | inline
To_String() (defined in NativeMultiWorldTesting::SimpleContext) | NativeMultiWorldTesting::SimpleContext | inline


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • SoftmaxExplorer
NativeMultiWorldTesting::SoftmaxExplorer< Ctx > Member List

This is the complete list of members for NativeMultiWorldTesting::SoftmaxExplorer< Ctx >, including all inherited members.

MwtExplorer< Ctx > (defined in NativeMultiWorldTesting::SoftmaxExplorer< Ctx >) | NativeMultiWorldTesting::SoftmaxExplorer< Ctx > | friend
SoftmaxExplorer(IScorer< Ctx > &default_scorer, float lambda, u32 num_actions) | NativeMultiWorldTesting::SoftmaxExplorer< Ctx > | inline


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • TauFirstExplorer
NativeMultiWorldTesting::TauFirstExplorer< Ctx > Member List

This is the complete list of members for NativeMultiWorldTesting::TauFirstExplorer< Ctx >, including all inherited members.

MwtExplorer< Ctx > (defined in NativeMultiWorldTesting::TauFirstExplorer< Ctx >) | NativeMultiWorldTesting::TauFirstExplorer< Ctx > | friend
TauFirstExplorer(IPolicy< Ctx > &default_policy, u32 tau, u32 num_actions) | NativeMultiWorldTesting::TauFirstExplorer< Ctx > | inline


Multiworld Testing Exploration Library: Main Page

Multiworld Testing Exploration Library Documentation


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • IPolicy
MultiWorldTesting::IPolicy< Ctx > Member List

This is the complete list of members for MultiWorldTesting::IPolicy< Ctx >, including all inherited members.

ChooseAction(Ctx context)=0 MultiWorldTesting::IPolicy< Ctx > pure virtual
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • IRecorder
MultiWorldTesting::IRecorder< Ctx > Member List

This is the complete list of members for MultiWorldTesting::IRecorder< Ctx >, including all inherited members.

Record(Ctx context, UInt32 action, float probability, String^uniqueKey)=0 MultiWorldTesting::IRecorder< Ctx > pure virtual
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • IScorer
MultiWorldTesting::IScorer< Ctx > Member List

This is the complete list of members for MultiWorldTesting::IScorer< Ctx >, including all inherited members.

ScoreActions(Ctx context)=0 MultiWorldTesting::IScorer< Ctx > pure virtual


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • IStringContext
MultiWorldTesting::IStringContext Member List

This is the complete list of members for MultiWorldTesting::IStringContext, including all inherited members.

ToString()=0 (defined in MultiWorldTesting::IStringContext) MultiWorldTesting::IStringContext pure virtual


Multiworld Testing Exploration Library: Member List

  • MultiWorldTesting
  • Feature
MultiWorldTesting::Feature Member List

This is the complete list of members for MultiWorldTesting::Feature, including all inherited members.

Id (defined in MultiWorldTesting::Feature) | MultiWorldTesting::Feature
Value (defined in MultiWorldTesting::Feature) | MultiWorldTesting::Feature
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

MyInteraction< Ctx > Member List

This is the complete list of members for MyInteraction< Ctx >, including all inherited members.

Action (defined in MyInteraction< Ctx >) | MyInteraction< Ctx >
Context (defined in MyInteraction< Ctx >) | MyInteraction< Ctx >
Probability (defined in MyInteraction< Ctx >) | MyInteraction< Ctx >
Unique_Key (defined in MyInteraction< Ctx >) | MyInteraction< Ctx >
Generated on Wed Nov 12 2014 23:28:14 for Multiworld Testing Exploration Library by doxygen 1.8.8


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • Feature
NativeMultiWorldTesting::Feature Member List

This is the complete list of members for NativeMultiWorldTesting::Feature, including all inherited members.

Id (defined in NativeMultiWorldTesting::Feature) | NativeMultiWorldTesting::Feature |
operator==(Feature other_feature) (defined in NativeMultiWorldTesting::Feature) | NativeMultiWorldTesting::Feature | inline
Value (defined in NativeMultiWorldTesting::Feature) | NativeMultiWorldTesting::Feature |


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • PRG
  • prg
NativeMultiWorldTesting::PRG::prg Member List

This is the complete list of members for NativeMultiWorldTesting::PRG::prg, including all inherited members.

merand48(uint64_t &initial) (defined in NativeMultiWorldTesting::PRG::prg) | NativeMultiWorldTesting::PRG::prg | inline
prg() (defined in NativeMultiWorldTesting::PRG::prg) | NativeMultiWorldTesting::PRG::prg | inline
prg(uint64_t initial) (defined in NativeMultiWorldTesting::PRG::prg) | NativeMultiWorldTesting::PRG::prg | inline
Uniform_Int(uint32_t low, uint32_t high) (defined in NativeMultiWorldTesting::PRG::prg) | NativeMultiWorldTesting::PRG::prg | inline
Uniform_Unit_Interval() (defined in NativeMultiWorldTesting::PRG::prg) | NativeMultiWorldTesting::PRG::prg | inline


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • StringRecorder
NativeMultiWorldTesting::StringRecorder< Ctx > Member List

This is the complete list of members for NativeMultiWorldTesting::StringRecorder< Ctx >, including all inherited members.

Get_Recording(bool flush=true) (defined in NativeMultiWorldTesting::StringRecorder< Ctx >) | NativeMultiWorldTesting::StringRecorder< Ctx > | inline
Record(Ctx &context, u32 action, float probability, string unique_key) | NativeMultiWorldTesting::StringRecorder< Ctx > | inlinevirtual


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • murmur_hash
NativeMultiWorldTesting::murmur_hash Member List

This is the complete list of members for NativeMultiWorldTesting::murmur_hash, including all inherited members.

uniform_hash(const void *key, size_t len, uint32_t seed) (defined in NativeMultiWorldTesting::murmur_hash) NativeMultiWorldTesting::murmur_hash inline


Multiworld Testing Exploration Library: Member List

  • NativeMultiWorldTesting
  • PRG
  • int_float
NativeMultiWorldTesting::PRG::int_float Member List

This is the complete list of members for NativeMultiWorldTesting::PRG::int_float, including all inherited members.

f (defined in NativeMultiWorldTesting::PRG::int_float) | NativeMultiWorldTesting::PRG::int_float
i (defined in NativeMultiWorldTesting::PRG::int_float) | NativeMultiWorldTesting::PRG::int_float