Example of an A-efficient RUM design with ChoiceDesign

This notebook illustrates how to use ChoiceDesign to generate a simple A-efficient experimental design for a Random Utility Maximisation (RUM) model. Instead of minimising the D-error (the determinant of the inverse Fisher information matrix), the A-criterion minimises the A-error:

\[A\text{-error} = \frac{\operatorname{trace}(I^{-1})}{K}\]

where \(K\) is the number of non-ASC parameters. The A-error is the average variance across all parameter estimates, making it more sensitive to individual parameters that are poorly estimated.

Step 1: Load modules, define design parameters and set attributes

The following lines load:

EffDesign: the class of efficient designs,
Attribute and Parameter: the classes of attributes and parameters, respectively.

[1]:

from choicedesign.design import EffDesign
from choicedesign.expressions import Attribute, Parameter

The following lines define 2 alternatives, named alt1 and alt2, and 4 attributes named from \(A\) to \(D\):

[2]:

alt1_A = Attribute('alt1_A',[1,2,3])
alt1_B = Attribute('alt1_B',[10,15,15.5])
alt1_C = Attribute('alt1_C',[0,3,5])
alt1_D = Attribute('alt1_D',[0,1,2])

alt2_A = Attribute('alt2_A',[1,2,3])
alt2_B = Attribute('alt2_B',[10,15,15.5])
alt2_C = Attribute('alt2_C',[0,3,5])
alt2_D = Attribute('alt2_D',[0,1,2])

Step 2: Construct efficient design object and generate initial design matrix

The design object is constructed with EffDesign, passing the list of attributes and the number of choice situations:

[3]:

design = EffDesign(
    X = [alt1_A,alt1_B,alt1_C,alt1_D,
         alt2_A,alt2_B,alt2_C,alt2_D],
    ncs=18)

[4]:

init_design = design.gen_initdesign()
init_design

[4]:

	alt1_A	alt1_B	alt1_C	alt1_D	alt2_A	alt2_B	alt2_C	alt2_D
0	3.0	15.0	5.0	1.0	2.0	15.5	0.0	0.0
1	2.0	15.0	5.0	1.0	1.0	15.5	0.0	2.0
2	2.0	15.5	3.0	0.0	1.0	10.0	0.0	0.0
3	2.0	15.0	0.0	2.0	2.0	15.0	5.0	2.0
4	3.0	15.5	3.0	0.0	2.0	10.0	3.0	1.0
5	2.0	10.0	3.0	1.0	3.0	15.0	3.0	0.0
6	1.0	15.0	0.0	2.0	3.0	15.5	0.0	0.0
7	1.0	10.0	0.0	1.0	3.0	15.5	5.0	0.0
8	1.0	10.0	5.0	1.0	1.0	15.0	3.0	2.0
9	2.0	15.5	3.0	0.0	2.0	15.0	3.0	2.0
10	1.0	15.5	0.0	1.0	1.0	15.5	5.0	1.0
11	2.0	10.0	3.0	2.0	1.0	10.0	5.0	2.0
12	3.0	15.0	0.0	0.0	3.0	10.0	5.0	2.0
13	3.0	10.0	3.0	0.0	1.0	10.0	3.0	1.0
14	1.0	15.5	0.0	2.0	2.0	15.5	5.0	1.0
15	3.0	15.5	5.0	2.0	3.0	15.0	0.0	0.0
16	3.0	10.0	5.0	2.0	2.0	15.0	3.0	1.0
17	1.0	15.0	5.0	0.0	3.0	10.0	0.0	1.0

Step 3: Set the utility functions

Parameters are defined with the Parameter class. The arguments are:

name: The parameter name
prior: The prior value

The following lines define four parameters:

[5]:

beta_A = Parameter('beta_A',-0.1)
beta_B = Parameter('beta_B',-0.02)
beta_C = Parameter('beta_C',0.1)
beta_D = Parameter('beta_D',0.15)

[6]:

V1 = beta_A * alt1_A + beta_B * alt1_B + beta_C * alt1_C + beta_D * alt1_D
V2 = beta_A * alt2_A + beta_B * alt2_B + beta_C * alt2_C + beta_D * alt2_D

V = {1: V1, 2: V2}

Step 4: Optimise the design using the A-error criterion

The optimise() method accepts a criterion argument that selects the optimality criterion:

'd' (default): minimise the D-error — \(\det(I^{-1})^{1/K}\)
'a': minimise the A-error — \(\operatorname{trace}(I^{-1})/K\)
'c': minimise the C-error (WTP variance sum)

Setting criterion='a' yields an A-efficient design. All other arguments remain the same as in the D-efficient case:

[7]:

optimal_design, init_perf, final_perf, final_iter, ubalance_ratio = design.optimise(
    init_design=init_design,
    V=V,
    model='mnl',
    criterion='a',
    time_lim=1,
    verbose=True
)

Evaluating initial design
Optimization complete 0:00:59 / A-error: 0.050462
Elapsed time: 0:01:00
A-error of initial design:  0.118812
A-error of last stored design:  0.050462
Utility Balance ratio:  94.56 %
Algorithm iterations:  183454

Blocking the design

The optimal design can be blocked using the method gen_blocks(). The following line creates 3 blocks:

[10]:

optimal_design_blocked, corr_history = design.gen_blocks(optimal_design, n_blocks=3)
optimal_design_blocked

[10]:

	CS	alt1_A	alt1_B	alt1_C	alt1_D	alt2_A	alt2_B	alt2_C	alt2_D	Block
0	1.0	3.0	15.5	0.0	2.0	1.0	10.0	3.0	0.0	1
1	2.0	2.0	15.0	5.0	1.0	2.0	15.0	0.0	1.0	2
2	3.0	2.0	15.0	5.0	2.0	2.0	15.0	0.0	0.0	1
3	4.0	2.0	15.0	0.0	0.0	2.0	15.0	5.0	2.0	3
4	5.0	3.0	10.0	3.0	0.0	1.0	15.5	3.0	2.0	3
5	6.0	1.0	10.0	0.0	2.0	3.0	15.5	5.0	0.0	1
6	7.0	1.0	15.5	5.0	1.0	3.0	10.0	0.0	1.0	3
7	8.0	1.0	10.0	0.0	2.0	3.0	15.5	3.0	0.0	3
8	9.0	1.0	15.0	3.0	0.0	3.0	15.0	5.0	2.0	2
9	10.0	3.0	15.0	5.0	0.0	1.0	15.0	0.0	2.0	1
10	11.0	2.0	15.0	0.0	1.0	2.0	15.0	5.0	1.0	2
11	12.0	2.0	15.5	3.0	2.0	2.0	10.0	3.0	0.0	3
12	13.0	2.0	15.5	3.0	0.0	2.0	10.0	5.0	2.0	1
13	14.0	3.0	10.0	5.0	1.0	1.0	15.5	0.0	1.0	3
14	15.0	1.0	15.5	3.0	0.0	3.0	10.0	3.0	2.0	2
15	16.0	3.0	15.5	3.0	2.0	1.0	10.0	3.0	0.0	2
16	17.0	3.0	10.0	0.0	1.0	1.0	15.5	5.0	1.0	2
17	18.0	1.0	10.0	5.0	1.0	3.0	15.5	0.0	1.0	1

(optional) Evaluate the design

Pass criterion='a' to evaluate() to compute the A-error of a stored design:

[11]:

perf, ubalance = design.evaluate(optimal_design, V, model='mnl', criterion='a')

print('A-error:', perf)
print('Utility balance:', ubalance)

A-error: 0.050462107185962744
Utility balance: 94.56297363503425

Export the design

Export the A-efficient design to Excel.

[ ]:

attr_names = {
    'alt1_A': 'Attribute A', 'alt2_A': 'Attribute A',
    'alt1_B': 'Attribute B', 'alt2_B': 'Attribute B',
    'alt1_C': 'Attribute C', 'alt2_C': 'Attribute C',
    'alt1_D': 'Attribute D', 'alt2_D': 'Attribute D',
}
design.export_design(optimal_design, attr_names, 'rum_a_efficient_design.xlsx')

Save the optimisation summary

After calling optimise(), the method export_output() writes a plain-text summary of the optimisation run — design configuration, stopping criteria, criterion values, utility balance, elapsed time, and iteration count — to a file.

[ ]:

design.export_output('rum_a_efficient_output.txt')

References

[1] Quan, W., Rose, J. M., Collins, A. T., & Bliemer, M. C. (2011). A comparison of algorithms for generating efficient choice experiments.