Update rollouts-and-experiments.mdx (#624)
This commit is contained in:
Родитель
77aed0f4a0
Коммит
2ad52e0144
|
@ -18,24 +18,26 @@ You can run a 100% rollout and a 5% experiment and both will fill up, but everyo
|
|||
|
||||
----
|
||||
|
||||
### Question
|
||||
If there is a concurrent experiment and rollout on the same featureId, once the experiment ends, assuming the rollout is at 100%, a given client will immediately be put into the rollout. Correct?
|
||||
### Question: What happens when an experiment and rollout are on the same feature, and the experiment ends?
|
||||
|
||||
### Answer
|
||||
The short answer is **“yes”**, in most situations.
|
||||
Typically users will see the experiment applied, and then get the rollout treatment when that ends with a small delay to re-evaluate.
|
||||
|
||||
The long answer is: while both are running, an experiment client with actually be enrolled in both and the treatment they’ll receive will be the combination of both. This enables clients in subsequent experiments to easily get rolled out winners of prior experiments.
|
||||
If both the experiment and the rollout are focused on the same feature, **the experiment will take precedence** and in this case the behavior will be seeing the experiment, when that ends the client will unenroll, then upon next recipe evaluation will join the rollout.
|
||||
|
||||
If both the experiment and the rollout are focused on the same fields of the treatment JSON, **the experiment will take precedence** and in this case the behavior will be as you state: effectively, the client immediately joins the rollout.
|
||||
|
||||
If they’re focused on different fields of the JSON, the behavior the clients gets will go from “experiment+rollout” to “rollout only”.
|
||||
UNLESS the feature is mobile messaging, in that case while both are running, an experiment client with actually be enrolled in both and the treatment they’ll receive will be the combination of both. This enables clients in subsequent experiments to easily get rolled out winners of prior experiments. If they’re focused on different fields of the JSON, the behavior the clients gets will go from “experiment+rollout” to “rollout only”.
|
||||
|
||||
----
|
||||
|
||||
### Question
|
||||
I have a feature that is now on by default in v115, and I want to wait until it has saturated the user base before shutting off the rollout. Do we have a standard around how long we should keep a rollout going in a situation like this?
|
||||
### Question - How do I decide when to end a rollout?
|
||||
|
||||
ex: I have a feature that is now on by default in v115, and I want to wait until it has saturated the user base before shutting off the rollout. Do we have a standard around how long we should keep a rollout going in a situation like this?
|
||||
|
||||
### Answer
|
||||
We typically wait 1-3 releases for the upgrade tail to catch up. You'll probably have the vast majority, so it just depends on the ramifications if people have it enabled on 114 and then lose it.
|
||||
|
||||
If it's **not** a big deal, turn off the rollout in 116 and most people will have it and the stragglers will get it when they upgrade. If it **is** a big deal - turn off in 117 or 118 and you'll have reached a few percent more with each release.
|
||||
If it's **not** a big deal if the feature goes away, turn off the rollout in the next release. People who haven't yet upgraded will lose the feature until they upgrade.
|
||||
|
||||
If it **is** a big deal for the feature to go away (think UX changes or bug fixes) - turn off in 117 or 118 and you'll have reached a few percent more with each release.
|
||||
|
||||
If it is a very big deal - we sometimes leave those running for 6 months or so to hit maximum reach. Please remember to set a reminder appointment to people on your team so someone remembers and doesn't leave it on forever.
|
||||
|
|
Загрузка…
Ссылка в новой задаче