Contextual Bandits

Usage with Contextual Multi-Armed Bandits

Eppo supports contextual multi-armed bandits. You configure bandit flags in the Eppo application, where you can set up:

• Flag keys
• Status quo variations
• Bandit variations
• Targeting rules

When you query the bandit in your code, you provide the available actions to the SDK.

To leverage bandits using the .NET SDK, there are two additional steps over regular feature flags:

1. Add a bandit action logger to the SDK client instance
2. Query the bandit for an action

Define a bandit assignment logger

In order for the bandit to learn an optimized policy, we need to capture and log the bandit's actions. This requires defining a bandit logger in addition to an assignment logger.

internal class BanditLogger : IAssignmentLogger
{
    public void LogAssignment(AssignmentLogData assignmentLogData)
    {
        Console.WriteLine("TODO: save assignment information to data warehouse", assignmentLogData);
    }

    public void LogBanditAction(BanditLogEvent banditLogEvent)
    {
        Console.WriteLine("TODO: save bandit action information to the data warehouse", banditLogEvent);
    }
}

// Initialize the SDK with both loggers provided
var eppoClientConfig = new EppoClientConfig("<SDK_KEY>", new BanditLogger());
var eppoClient = EppoClient.Init(eppoClientConfig);

The SDK will invoke the LogBanditAction() method with a BanditLogEvent object that contains the following fields:

Field	Description	Example
Timestamp (DateTime)	The time when the action is taken in UTC	"2024-03-22T14:26:55.000Z"
FlagKey (string)	The key of the feature flag corresponding to the bandit	"bandit-test-allocation-4"
BanditKey (string)	The key (unique identifier) of the bandit	"ad-bandit-1"
SubjectKey (string)	An identifier of the subject or user assigned to the experiment variation	"ed6f85019080"
SubjectNumericAttributes (IDictionary		ContextAttributes)
SubjectCategoricalAttributes (IDictionary		ContextAttributes)
Action (string)	The action assigned by the bandit	"promo-20%-off"
ActionNumericAttributes (IDictionary)	Metadata about numeric attributes of the assigned action. Map of the name of attributes their provided values	{"brandAffinity": 0.2}
ActionCategoricalAttributes (IDictionary)	Metadata about non-numeric attributes of the assigned action. Map of the name of attributes their provided values	{"previouslyPurchased": false}
ActionProbability (double)	The weight between 0 and 1 the bandit valued the assigned action	0.25
OptimalityGap (double)	The difference between the score of the selected action and the highest-scored action	456
ModelVersion (string)	Unique identifier for the version (iteration) of the bandit parameters used to determine the action probability	"v123"
MetaData (IDictionary)	Any additional freeform meta data, such as the version of the SDK	{ "sdkLibVersion": "3.5.1" }

Query the bandit for an action

To query the bandit for an action, use the GetBanditAction() method. The most specific implementation takes the following parameters:

• flagKey (string): The key of the feature flag corresponding to the bandit
• subjectKey (string): The key of the subject or user assigned to the experiment variation
• subjectAttributes (IDictionary): The subject's attributes
• actions (IDictionary): Map of actions (by name) to their categorical and numeric attributes
• defaultValue (string): The default variation to return if the flag is not successfully evaluated

var subjectAttributes = new ContextAttributes("user123")
{
    ["age"] = 30,
    ["country"] = "uk",
    ["pricingTier"] = "1"
};

var actions = new Dictionary<string, ContextAttributes>()
{
    ["nike"] = new ContextAttributes("nike")
    {
        ["brandLoyalty"] = 0.4,
        ["from"] = "usa"
    },
    ["adidas"] = new ContextAttributes("adidas")
    {
        ["brandLoyalty"] = 2,
        ["from"] = "germany"
    }
};

var result = eppoClient.GetBanditAction(
    "flag-with-shoe-bandit",
    "user123",
    subjectAttributes,
    actions,
    "default");

if (result.Action != null)
{
    // Follow the Bandit action
    RenderShoeAd(result.Action);
} else {
    // User was not selected for a Bandit.
    // A variation is still assigned.
    RenderDefaultShoeAd(result.Variation);
}

GetBanditAction Overloads

There are several overloads of the GetBanditAction() method available:

For a simple list of actions without attributes:

public BanditResult GetBanditAction(s
    string flagKey,
    string subjectKey,
    IDictionary<string, object?> subjectAttributes,
    string[] actions,
    string defaultValue)

For a simple list of actions without attributes, using a ContextAttributes subject:

public BanditResult GetBanditAction(
    string flagKey,
    ContextAttributes subject,
    string[] actions,
    string defaultValue)

Using ContextAttributes objects for subject and actions:

public BanditResult GetBanditAction(
    string flagKey,
    ContextAttributes subject,
    IDictionary<string, ContextAttributes> actions,
    string defaultValue)

Subject Context

The subject context contains contextual information about the subject that is independent of bandit actions. For example, the subject's age or country.

The subject context can be provided as IDictionary<string, object?>, which will then assume anything that is a number is a numeric attribute, and everything else is a categorical attribute.

You can also explicitly bucket the attribute types by providing the context as ContextAttributes. For example, you may have an attribute named priority, with possible values 0, 1, and 2 that you want to be treated categorically rather than numeric. ContextAttributes have two nested sets of attributes:

• NumericAttributes (IDictionary): A mapping of attribute names to their numeric values (e.g., age: 30)
• CategoricalAttributes (IDictionary): A mapping of attribute names to their categorical values (e.g., country: "US")

// Using ContextAttributes for explicit categorization
var subjectContext = new ContextAttributes("user123")
{
    NumericAttributes = new Dictionary<string, double>
    {
        ["age"] = 30,
        ["daysActive"] = 45
    },
    CategoricalAttributes = new Dictionary<string, string>
    {
        ["country"] = "US",
        ["priority"] = "1"  // Treating numeric value as categorical
    }
};

Action Context

The action context contains contextual information about each action. They can be provided as a mapping of attribute names to their contexts.

Similar to subject context, action contexts can be provided as:

• IDictionary<string, object?> - which will then assume anything that is a number is a numeric attribute, and everything else is a categorical attribute
• ContextAttributes, which have explicit bucketing into NumericAttributes and CategoricalAttributes.

If there is no action context, an array of strings comprising only the actions names can also be passed in.

// Simple array of action names
string[] actions = new[] { "nike", "adidas", "puma" };

// Or with context
var actionsWithContext = new Dictionary<string, ContextAttributes>
{
    ["nike"] = new ContextAttributes("nike")
    {
        NumericAttributes = new Dictionary<string, double>
        {
            ["brandAffinity"] = 0.8,
            ["pricePoint"] = 120
        },
        CategoricalAttributes = new Dictionary<string, string>
        {
            ["category"] = "athletic",
            ["inStock"] = "true"
        }
    }
};

Debugging

You may encounter a situation where a flag assignment produces a value that you did not expect. There are functions detailed here to help you understand how flags are assigned, which will allow you to take corrective action on potential configuration issues.