ListRandomizer

class alfred3.randomizer.ListRandomizer(*args, **kwargs)

Bases: alfred3.quota.SessionQuota

Offers list randomization.

Parameters

• *conditions – A variable number of tuples, defining the experiment conditions. The tuples have the form ("str", int), where str is a string giving the condition’s name, and int is the target number of observations for this condition.

• exp (alfred3.ExperimentSession) – The alfred3 experiment session.

• session_ids (list) – A list of experiment session ids that should be treated as a group, i.e. share a single condition slot. That means, they will be allocated to the same condition and, as a group, take up only one space in the count of sessions that the randomizer keeps. Most useful for group experiments. Defaults to [exp.session_id], i.e. a list of length 1 with the session id of the current experiment instance as its only element.

• respect_version (bool) – If True, randomization will start anew for each experiment version. This is especially important, if you make changes to the condition setup of an ongoing experiment. This might cause the randomizer to fail, causing SlotInconsistency errors. Setting respect_version to True can fix such issues. Defaults to True.

• inclusive (bool) –

  If False (default), the randomizer will only assign a condition slot, if there are no pending sessions for that slot. It will not assign a condition slot, if a session in that slot is finished, or if there is an ongoing session in that slot that has not yet timed out. You will end up with exactly as many participants in each condition as specified in the target size.

  If True, the randomizer will assign a condition slot, if there is no finished session in that slot. That means, there might be two ongoing sessions for the same slot, and both might end up to finish.

  While inclusive=*False* may lead to participants being turned away before the experiment is really complete, inclusive=True may lead to more data being collected than necessary.

  Defaults to False.

• name (str) – An identifier for the randomizer. If you set this to a custom value, you can use multiple randomizers in the same experiment. Defaults to ‘randomizer’.

• random_seed – The random seed used for reproducible pseudo-random behavior. This seed will be used for shuffling the condition list. Defaults to the current system time. Valid seeds are all values that are accepted by random.seed()

• abort_page (alfred3.page.Page) – You can reference a custom page to be displayed to new participants, if the experiment is full.

• id (str) – A unique identifier for the condition slot assigned by this instance of ListRandomizer. If None, the current ExperimentSession.session_id will be used. This argument enables you to assign, for example, a group id that connects several sessions, to the randomizer. Defaults to None. Deprecated in version 2.1.7: Please use parameter session_ids instead. If you use the ‘id’ parameter, the ListRandomizer will start in compatibility mode.

• mode (str) – Deprecated in favor of inclusive. Please use the argument inclusive instead.

The ListRandomizer is used by initializing it (either directly or via the convenience method balanced()) and using the method get_condition() to receive a condition. By default, the ListRandomizer will automatically abort sessions if the experiment is full when get_condition is called and display an information page for participants. This behavior can be customized (see get_condition()).

The ListRandomizer will not count experiment sessions that have expired due to being inactive for a long time. You can control this timeout via ExperimentSession.session_timeout.

Warning

Be mindful of the argument respect_version! With the default setting (True), randomization starts from scratch for every experiment version. If you set it to False, you will run into an error, if you change anything about the conditions.

Changed in version 2.2.0: - Deprecated the parameter session_ids without replacement. The ListRandomizer is now aimed exclusively at allocating one session at a time. - Removed the method abort_if_full. Instead, you can check the randomizer’s status with the attributes full, allfinished, nopen, npending, and nfinished and call ExperimentSession.abort() directly.

Changed in version 2.1.7: New parameters session_ids and name, new alternative constructor factors(). Deprecated the parameter id.

Notes

Why use list randomization?

In “naive” randomization, you might end up with a very unbalanced design. For example, if you recruit 300 participants and randomize them into two conditions, you might end up with 100 participants in the first and 200 in the second condition. If you aim for a balanced design with 150 participants in either condition, you might have to recruit more than 300 participants and throw away lots of observations in the condition that turned out to be larger.

List randomization solves this problem, which is why it is commonly used in offline studies. Let’s take an easy example. We might have two conditions, a and b, each of which should be completed by three participants. The order in which participants are assigned to a condition should be random. To achieve this, we create a list that contains a condition slot for each participant:

["a", "a", "a", "b", "b", "b"]

We then shuffle this list, which might lead to something like the following:

["b", "a", "a", "b", "a", "b"]

Then, the conditions are assigned to participants based on the shuffled list in the order that they start the experiment.

Using the ListRandomizer offline

You can use the ListRandomizer offline, i.e. without using a mongoDB for data saving. In this case, you must keep in mind, that the shuffled list will only be shared on one machine. Offline, randomization cannot be synchronized across multiple machines.

Examples

A minimal experiment with two conditions. If the experiment is full, the experiment will immediately abort new sessions and display an abort page to new participants:

import alfred3 as al
exp = al.Experiment()

@exp.setup
def setup(exp):
    randomizer = al.ListRandomizer(("cond1", 10), ("cond2", 10), exp=exp)
    exp.condition = randomizer.get_condition()

@exp.member
class DemoPage(al.Page):

    def on_exp_access(self):

        if self.exp.condition == "cond1":
            lab = "label in condition 1"

        elif self.exp.condition == "cond2":
            lab = "label in condition 2"

        self += al.TextEntry(leftlab=lab, name="t1")

You can use the alternative constructor balanced() for simplified initialization, if you use the same sample size for all experiment conditions:

import alfred3 as al
exp = al.Experiment()

@exp.setup
def setup(exp):
    randomizer = al.ListRandomizer.balanced("cond1", "cond2", n=10, exp=exp)
    exp.condition = randomizer.get_condition()

@exp.member
class DemoPage(al.Page):

    def on_exp_access(self):

        if self.exp.condition == "cond1":
            lab = "label in condition 1"

        elif self.exp.condition == "cond2":
            lab = "label in condition 2"

        self += al.TextEntry(leftlab=lab, name="t1")

Methods

balanced	Alternative constructor, creates a ListRandomizer where all conditions have the same size.
count	Counts the experiment session associated with the quota.
factors	Alternative constructor, creates a balanced ListRandomizer where the conditions are combinations of several factors.
get_condition	Returns a condition.
next	Returns the next open slot.

Attributes

DATA_TYPE
allfinished	Indicates, whether all slots in the quota are finished.
full	True, if the randomizer has allocated all available slots.
nfinished	Number of finished slots.
nopen	Number of open slots.
npending	Number of slots in which a session is still ongoing.
nslots