Skip to main content

Experiments

A/B tests. A deck declares its experiments in castle.json; Castle gives each player one variant of each, the same one every time they play, and the deck reads it.

"experiments": {
"tutorial_length": { "variants": ["short", "long"] },
"shop_price": { "variants": ["low", "mid", "high"], "weights": [50, 25, 25] }
}
  • An experiment's name and its variants' names are 1 to 32 characters of a-z, 0-9 and _. A deck declares at most 8 experiments, each with 2 to 8 variants.
  • weights are whole percentages, one per variant, summing to 100. Leave them out for an equal split.
  • winner (one of the variants) ends the test: every player gets that variant.
  • save-deck sends the block to Castle, and castle-web check and save-deck both refuse one that breaks a rule. Removing the block ends every experiment at the next save.
import { experiment, experimentsReady } from "castle-web-sdk";

await experimentsReady();
const tutorial = experiment("tutorial_length") === "long" ? longTutorial : shortTutorial;

experiment(name): string | nullโ€‹

The variant this player has in name. Synchronous. null means the player is in no such experiment: the deck does not declare it, it was removed, or the host is an older one that knows nothing of experiments. Always give null a behavior, normally the one the deck had before the test.

The first read of each name that returns a variant also records an exposure, the moment the player was shown that arm, for the deck's analytics. Read it where the variant changes what the player sees, not up front.

experiments(): Record<string, string>โ€‹

Every experiment this player is in on this deck, as { name: variant }. It records no exposure.

experimentsReady(): Promise<Record<string, string>>โ€‹

Resolves with that same map once the host has answered, and never rejects. Await it once at boot, before the first read: some hosts hand the answer over with the page, where it is already resolved, and the rest answer one request. A read made before then returns null and logs a warning.

A multiplayer deck's server reads the same answer with session.experiment(player, name).

On castle-web serve the experiments come from the folder's castle.json, assigned to you exactly as they will be once the deck is saved. Add ?experiment.<name>=<variant> to the serve's URL to force an arm (a declared experiment and one of its variants); it holds for the whole serve, client and local server alike, until ?experiment.<name>= clears it or the serve stops. An open deck page keeps the arm it loaded with until it is reloaded. On a published deck the same parameter on the deck's page forces an arm for the deck's owner only, for a day. devExperiments in castle.json, a { "name": "variant" } map, answers for experiments the deck does not declare itself (ones Castle runs across decks), on the serve only.