Skip to content

Commit fdbaa1d

Browse files
Mat001Mat001
authored
[FSSDK-11587] Implement CMAB config (#439)
* add cmab config * remove enabled config, remove 3 ENV vars for cmab, only use a single one * cleanup readme * add cmab logic to agent * add cmab config * remove enabled config, remove 3 ENV vars for cmab, only use a single one * cleanup readme * add cmab logic to agent * Use go-sdk branch mpirnovar-cmab-gosdk-agent-fssdk-11589 for CMAB support * fix formatting * surface cmabUUID in Decide API response * Add support for returning the cmabUUID field in Decide API responses and tests. * remove duplicate CmabUUID * Add configurable CMAB prediction endpoint for FSC testing * Force GitHub refresh * add prediction endpoint handling to main.go * Update agent to use CMAB configuration approach * fix tests * Force GitHub refresh * configure ENV var OPTIMIZELY_CMAB_PREDICTIONENDPOINT to allow fsc tests to run * remove %s, already in the endpoint in fsc * Add client reset functionality for FSC CMAB test isolation * Trigger PR check * fix formatting issues * Refactored CMAB configuration from unstructured map[string]interface{} to structured types for better type safety and maintainability. * restore retry config * fix Prisma crypto dependency alert * clean up go-mod to point to go-sdk master * point go.sum to the latet go-sdk master which is ahead of current 2.1.0 release
1 parent f0973df commit fdbaa1d

File tree

24 files changed

+1047
-49
lines changed

24 files changed

+1047
-49
lines changed

‎Makefile‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ GOBIN=bin
1212
GOPATH:=$(shell $(GOCMD) env GOPATH 2> /dev/null)
1313
GOBUILD=$(GOCMD) build
1414
GOCLEAN=$(GOCMD) clean
15-
GOTEST=$(GOCMD) test -race
15+
GOTEST=$(GOCMD) test -race -mod=mod
1616
GOGET=$(GOCMD) get
1717
GOLINT=$(GOPATH)/bin/golangci-lint
1818
BINARY_UNIX=$(TARGET)_unix
@@ -35,9 +35,7 @@ build: $(TARGET) check-go ## builds and installs binary in bin/
3535
@true
3636

3737
check-go:
38-
ifndef GOPATH
39-
$(error "go is not available please install golang version 1.24.0+, https://golang.org/dl/")
40-
endif
38+
@which go > /dev/null || (echo "go is not available please install golang version 1.21.0+, https://golang.org/dl/" && exit 1)
4139

4240
clean: check-go ## runs `go clean` and removes the bin/ dir
4341
$(GOCLEAN) --modcache
@@ -75,7 +73,7 @@ static: check-go
7573
$(GOPATH)/bin/statik -src=web/static -f
7674

7775
test: check-go static ## recursively tests all .go files
78-
$(GOTEST) ./...
76+
$(GOTEST) -count=1 ./...
7977

8078
include scripts/Makefile.ci
8179

‎README.md‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,10 @@ Below is a comprehensive list of available configuration properties.
124124
| log.level | OPTIMIZELY_LOG_LEVEL | The log [level](https://github.com/rs/zerolog#leveled-logging) for the agent. Default: info |
125125
| log.pretty | OPTIMIZELY_LOG_PRETTY | Flag used to set colorized console output as opposed to structured json logs. Default: false |
126126
| name | OPTIMIZELY_NAME | Agent name. Default: optimizely |
127-
| sdkKeys | OPTIMIZELY_SDKKEYS | Comma delimited list of SDK keys used to initialize on startup |
127+
| sdkKeys | OPTIMIZELY_SDKKEYS | Comma delimited list of SDK keys used to initialize on startup |
128+
| cmab | OPTIMIZELY_CMAB | Complete JSON configuration for CMAB. Format: see example below |
129+
| cmab.cache | OPTIMIZELY_CMAB_CACHE | JSON configuration for just the CMAB cache section. Format: see example below |
130+
| cmab.retryConfig | OPTIMIZELY_CMAB_RETRYCONFIG | JSON configuration for just the CMAB retry settings. Format: see example below |
128131
| server.allowedHosts | OPTIMIZELY_SERVER_ALLOWEDHOSTS | List of allowed request host values. Requests whose host value does not match either the configured server.host, or one of these, will be rejected with a 404 response. To match all subdomains, you can use a leading dot (for example `.example.com` matches `my.example.com`, `hello.world.example.com`, etc.). You can use the value `.` to disable allowed host checking, allowing requests with any host. Request host is determined in the following priority order: 1. X-Forwarded-Host header value, 2. Forwarded header host= directive value, 3. Host property of request (see Host under https://pkg.go.dev/net/http#Request). Note: don't include port in these hosts values - port is stripped from the request host before comparing against these. |
129132
| server.batchRequests.maxConcurrency | OPTIMIZELY_SERVER_BATCHREQUESTS_MAXCONCURRENCY | Number of requests running in parallel. Default: 10 |
130133
| server.batchRequests.operationsLimit | OPTIMIZELY_SERVER_BATCHREQUESTS_OPERATIONSLIMIT | Number of allowed operations. ( will flag an error if the number of operations exeeds this parameter) Default: 500 |
@@ -142,6 +145,25 @@ Below is a comprehensive list of available configuration properties.
142145
| webhook.projects.<_projectId_>.secret | N/A | Webhook secret used to validate webhook requests originating from the respective projectId |
143146
| webhook.projects.<_projectId_>.skipSignatureCheck | N/A | Boolean to indicate whether the signature should be validated. TODO remove in favor of empty secret. |
144147

148+
### CMAB Configuration Example
149+
150+
```json
151+
{
152+
"requestTimeout": "5s",
153+
"cache": {
154+
"type": "memory",
155+
"size": 2000,
156+
"ttl": "45m"
157+
},
158+
"retryConfig": {
159+
"maxRetries": 3,
160+
"initialBackoff": "100ms",
161+
"maxBackoff": "10s",
162+
"backoffMultiplier": 2.0
163+
}
164+
}
165+
```
166+
145167
More information about configuring Agent can be found in the [Advanced Configuration Notes](https://docs.developers.optimizely.com/experimentation/v4.0.0-full-stack/docs/advanced-configuration).
146168

147169
### API

‎cmd/optimizely/main.go‎

Lines changed: 82 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ import (
2525
"runtime"
2626
"strings"
2727
"syscall"
28+
"time"
2829

2930
"github.com/rs/zerolog"
3031
"github.com/rs/zerolog/log"
@@ -98,15 +99,94 @@ func loadConfig(v *viper.Viper) *config.AgentConfig {
9899
}
99100

100101
// Check if JSON string was set using OPTIMIZELY_CLIENT_USERPROFILESERVICE environment variable
101-
if userProfileService := v.GetStringMap("client.userprofileservice"); userProfileService != nil {
102+
if userProfileService := v.GetStringMap("client.userprofileservice"); len(userProfileService) > 0 {
102103
conf.Client.UserProfileService = userProfileService
103104
}
104105

105106
// Check if JSON string was set using OPTIMIZELY_CLIENT_ODP_SEGMENTSCACHE environment variable
106-
if odpSegmentsCache := v.GetStringMap("client.odp.segmentsCache"); odpSegmentsCache != nil {
107+
if odpSegmentsCache := v.GetStringMap("client.odp.segmentsCache"); len(odpSegmentsCache) > 0 {
107108
conf.Client.ODP.SegmentsCache = odpSegmentsCache
108109
}
109110

111+
// Handle CMAB configuration using the same approach as UserProfileService
112+
// Check for complete CMAB configuration first
113+
if cmab := v.GetStringMap("cmab"); len(cmab) > 0 {
114+
if timeout, ok := cmab["requestTimeout"].(string); ok {
115+
if duration, err := time.ParseDuration(timeout); err == nil {
116+
conf.CMAB.RequestTimeout = duration
117+
}
118+
}
119+
if cache, ok := cmab["cache"].(map[string]interface{}); ok {
120+
if cacheType, ok := cache["type"].(string); ok {
121+
conf.CMAB.Cache.Type = cacheType
122+
}
123+
if cacheSize, ok := cache["size"].(float64); ok {
124+
conf.CMAB.Cache.Size = int(cacheSize)
125+
}
126+
if cacheTTL, ok := cache["ttl"].(string); ok {
127+
if duration, err := time.ParseDuration(cacheTTL); err == nil {
128+
conf.CMAB.Cache.TTL = duration
129+
}
130+
}
131+
}
132+
if retryConfig, ok := cmab["retryConfig"].(map[string]interface{}); ok {
133+
if maxRetries, ok := retryConfig["maxRetries"].(float64); ok {
134+
conf.CMAB.RetryConfig.MaxRetries = int(maxRetries)
135+
}
136+
if initialBackoff, ok := retryConfig["initialBackoff"].(string); ok {
137+
if duration, err := time.ParseDuration(initialBackoff); err == nil {
138+
conf.CMAB.RetryConfig.InitialBackoff = duration
139+
}
140+
}
141+
if maxBackoff, ok := retryConfig["maxBackoff"].(string); ok {
142+
if duration, err := time.ParseDuration(maxBackoff); err == nil {
143+
conf.CMAB.RetryConfig.MaxBackoff = duration
144+
}
145+
}
146+
if backoffMultiplier, ok := retryConfig["backoffMultiplier"].(float64); ok {
147+
conf.CMAB.RetryConfig.BackoffMultiplier = backoffMultiplier
148+
}
149+
}
150+
}
151+
152+
// Check for individual map sections
153+
if cmabCache := v.GetStringMap("cmab.cache"); len(cmabCache) > 0 {
154+
if cacheType, ok := cmabCache["type"].(string); ok {
155+
conf.CMAB.Cache.Type = cacheType
156+
}
157+
if cacheSize, ok := cmabCache["size"].(int); ok {
158+
conf.CMAB.Cache.Size = cacheSize
159+
} else if cacheSize, ok := cmabCache["size"].(float64); ok {
160+
conf.CMAB.Cache.Size = int(cacheSize)
161+
}
162+
if cacheTTL, ok := cmabCache["ttl"].(string); ok {
163+
if duration, err := time.ParseDuration(cacheTTL); err == nil {
164+
conf.CMAB.Cache.TTL = duration
165+
}
166+
}
167+
}
168+
169+
if cmabRetryConfig := v.GetStringMap("cmab.retryConfig"); len(cmabRetryConfig) > 0 {
170+
if maxRetries, ok := cmabRetryConfig["maxRetries"].(int); ok {
171+
conf.CMAB.RetryConfig.MaxRetries = maxRetries
172+
} else if maxRetries, ok := cmabRetryConfig["maxRetries"].(float64); ok {
173+
conf.CMAB.RetryConfig.MaxRetries = int(maxRetries)
174+
}
175+
if initialBackoff, ok := cmabRetryConfig["initialBackoff"].(string); ok {
176+
if duration, err := time.ParseDuration(initialBackoff); err == nil {
177+
conf.CMAB.RetryConfig.InitialBackoff = duration
178+
}
179+
}
180+
if maxBackoff, ok := cmabRetryConfig["maxBackoff"].(string); ok {
181+
if duration, err := time.ParseDuration(maxBackoff); err == nil {
182+
conf.CMAB.RetryConfig.MaxBackoff = duration
183+
}
184+
}
185+
if backoffMultiplier, ok := cmabRetryConfig["backoffMultiplier"].(float64); ok {
186+
conf.CMAB.RetryConfig.BackoffMultiplier = backoffMultiplier
187+
}
188+
}
189+
110190
return conf
111191
}
112192

‎cmd/optimizely/main_test.go‎

Lines changed: 134 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/****************************************************************************
2-
* Copyright 2019-2020,2022-2023, Optimizely, Inc. and contributors *
2+
* Copyright 2019-2020,2022-2025, Optimizely, Inc. and contributors *
33
* *
44
* Licensed under the Apache License, Version 2.0 (the "License"); *
55
* you may not use this file except in compliance with the License. *
@@ -17,7 +17,9 @@
1717
package main
1818

1919
import (
20+
"fmt"
2021
"os"
22+
"strings"
2123
"testing"
2224
"time"
2325

@@ -178,6 +180,93 @@ func assertWebhook(t *testing.T, actual config.WebhookConfig) {
178180
assert.False(t, actual.Projects[20000].SkipSignatureCheck)
179181
}
180182

183+
func assertCMAB(t *testing.T, cmab config.CMABConfig) {
184+
fmt.Println("In assertCMAB, received CMAB config:")
185+
fmt.Printf(" RequestTimeout: %v\n", cmab.RequestTimeout)
186+
fmt.Printf(" Cache: %#v\n", cmab.Cache)
187+
fmt.Printf(" RetryConfig: %#v\n", cmab.RetryConfig)
188+
189+
// Base assertions
190+
assert.Equal(t, 15*time.Second, cmab.RequestTimeout)
191+
192+
// Check cache configuration
193+
cache := cmab.Cache
194+
assert.Equal(t, "redis", cache.Type)
195+
assert.Equal(t, 2000, cache.Size)
196+
assert.Equal(t, 45*time.Minute, cache.TTL)
197+
198+
// Check retry configuration
199+
retry := cmab.RetryConfig
200+
assert.Equal(t, 5, retry.MaxRetries)
201+
assert.Equal(t, 200*time.Millisecond, retry.InitialBackoff)
202+
assert.Equal(t, 30*time.Second, retry.MaxBackoff)
203+
assert.Equal(t, 3.0, retry.BackoffMultiplier)
204+
}
205+
206+
func TestCMABEnvDebug(t *testing.T) {
207+
_ = os.Setenv("OPTIMIZELY_CMAB", `{
208+
"requestTimeout": "15s",
209+
"cache": {
210+
"type": "redis",
211+
"size": 2000,
212+
"ttl": "45m"
213+
},
214+
"retryConfig": {
215+
"maxRetries": 5,
216+
"initialBackoff": "200ms",
217+
"maxBackoff": "30s",
218+
"backoffMultiplier": 3.0
219+
}
220+
}`)
221+
222+
// Load config using Viper
223+
v := viper.New()
224+
v.SetEnvPrefix("optimizely")
225+
v.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))
226+
v.AutomaticEnv()
227+
228+
// Create config
229+
assert.NoError(t, initConfig(v))
230+
conf := loadConfig(v)
231+
232+
// Debug: Print the parsed config
233+
fmt.Println("Parsed CMAB config from JSON env var:")
234+
fmt.Printf(" RequestTimeout: %v\n", conf.CMAB.RequestTimeout)
235+
fmt.Printf(" Cache: %+v\n", conf.CMAB.Cache)
236+
fmt.Printf(" RetryConfig: %+v\n", conf.CMAB.RetryConfig)
237+
238+
// Call assertCMAB
239+
assertCMAB(t, conf.CMAB)
240+
}
241+
242+
func TestCMABPartialConfig(t *testing.T) {
243+
// Clean any existing environment variables
244+
os.Unsetenv("OPTIMIZELY_CMAB")
245+
os.Unsetenv("OPTIMIZELY_CMAB_CACHE")
246+
os.Unsetenv("OPTIMIZELY_CMAB_RETRYCONFIG")
247+
248+
// Set partial configuration through CMAB_CACHE and CMAB_RETRYCONFIG
249+
_ = os.Setenv("OPTIMIZELY_CMAB_CACHE", `{"type": "redis", "size": 3000}`)
250+
_ = os.Setenv("OPTIMIZELY_CMAB_RETRYCONFIG", `{"maxRetries": 10}`)
251+
252+
// Load config
253+
v := viper.New()
254+
assert.NoError(t, initConfig(v))
255+
conf := loadConfig(v)
256+
257+
// Cache assertions
258+
assert.Equal(t, "redis", conf.CMAB.Cache.Type)
259+
assert.Equal(t, 3000, conf.CMAB.Cache.Size)
260+
261+
// RetryConfig assertions
262+
assert.Equal(t, 10, conf.CMAB.RetryConfig.MaxRetries)
263+
264+
// Clean up
265+
os.Unsetenv("OPTIMIZELY_CMAB")
266+
os.Unsetenv("OPTIMIZELY_CMAB_CACHE")
267+
os.Unsetenv("OPTIMIZELY_CMAB_RETRYCONFIG")
268+
}
269+
181270
func TestViperYaml(t *testing.T) {
182271
v := viper.New()
183272
v.Set("config.filename", "./testdata/default.yaml")
@@ -392,6 +481,21 @@ func TestViperEnv(t *testing.T) {
392481
_ = os.Setenv("OPTIMIZELY_WEBHOOK_PROJECTS_20000_SDKKEYS", "xxx,yyy,zzz")
393482
_ = os.Setenv("OPTIMIZELY_WEBHOOK_PROJECTS_20000_SKIPSIGNATURECHECK", "false")
394483

484+
_ = os.Setenv("OPTIMIZELY_CMAB", `{
485+
"requestTimeout": "15s",
486+
"cache": {
487+
"type": "redis",
488+
"size": 2000,
489+
"ttl": "45m"
490+
},
491+
"retryConfig": {
492+
"maxRetries": 5,
493+
"initialBackoff": "200ms",
494+
"maxBackoff": "30s",
495+
"backoffMultiplier": 3.0
496+
}
497+
}`)
498+
395499
_ = os.Setenv("OPTIMIZELY_RUNTIME_BLOCKPROFILERATE", "1")
396500
_ = os.Setenv("OPTIMIZELY_RUNTIME_MUTEXPROFILEFRACTION", "2")
397501

@@ -407,6 +511,7 @@ func TestViperEnv(t *testing.T) {
407511
assertAPI(t, actual.API)
408512
//assertWebhook(t, actual.Webhook) // Maps don't appear to be supported
409513
assertRuntime(t, actual.Runtime)
514+
assertCMAB(t, actual.CMAB)
410515
}
411516

412517
func TestLoggingWithIncludeSdkKey(t *testing.T) {
@@ -507,3 +612,31 @@ func Test_initTracing(t *testing.T) {
507612
})
508613
}
509614
}
615+
616+
func TestCMABComplexJSON(t *testing.T) {
617+
// Clean any existing environment variables for CMAB
618+
os.Unsetenv("OPTIMIZELY_CMAB_CACHE_TYPE")
619+
os.Unsetenv("OPTIMIZELY_CMAB_CACHE_SIZE")
620+
os.Unsetenv("OPTIMIZELY_CMAB_CACHE_TTL")
621+
os.Unsetenv("OPTIMIZELY_CMAB_CACHE_REDIS_HOST")
622+
os.Unsetenv("OPTIMIZELY_CMAB_CACHE_REDIS_PASSWORD")
623+
os.Unsetenv("OPTIMIZELY_CMAB_CACHE_REDIS_DATABASE")
624+
625+
// Set complex JSON environment variable for CMAB cache
626+
_ = os.Setenv("OPTIMIZELY_CMAB_CACHE", `{"type":"redis","size":5000,"ttl":"3h"}`)
627+
628+
defer func() {
629+
// Clean up
630+
os.Unsetenv("OPTIMIZELY_CMAB_CACHE")
631+
}()
632+
633+
v := viper.New()
634+
assert.NoError(t, initConfig(v))
635+
actual := loadConfig(v)
636+
637+
// Test cache settings from JSON environment variable
638+
cache := actual.CMAB.Cache
639+
assert.Equal(t, "redis", cache.Type)
640+
assert.Equal(t, 5000, cache.Size)
641+
assert.Equal(t, 3*time.Hour, cache.TTL)
642+
}

‎config.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,3 +262,28 @@ synchronization:
262262
datafile:
263263
enable: false
264264
default: "redis"
265+
266+
##
267+
## cmab: Contextual Multi-Armed Bandit configuration
268+
##
269+
cmab:
270+
## timeout for CMAB API requests
271+
requestTimeout: 10s
272+
## CMAB cache configuration
273+
cache:
274+
## cache type (memory or redis)
275+
type: "memory"
276+
## maximum number of entries for in-memory cache
277+
size: 1000
278+
## time-to-live for cached decisions
279+
ttl: 30m
280+
## retry configuration for CMAB API requests
281+
retryConfig:
282+
## maximum number of retry attempts
283+
maxRetries: 3
284+
## initial backoff duration
285+
initialBackoff: 100ms
286+
## maximum backoff duration
287+
maxBackoff: 10s
288+
## multiplier for exponential backoff
289+
backoffMultiplier: 2.0

0 commit comments

Comments
 (0)