It is possible to set slot settings to be sticky which will keep it in the deployment slot after swapping deployment/prod.

In ARM templates it is done using slotConfigNames of Microsoft.Web/sites/config resource. Documentation

Proposed solution:

let myWebApp = webApp {
    name "test"
    add_slot deploymentSlot

    settings ["my_setting", "test value"
                  "my_slot_setting", "sticky value"]

    slot_setting_names ["my_slot_setting" ]
}