{ "cells": [ { "cell_type": "markdown", "id": "cell-0", "metadata": {}, "source": [ "# Example of an A-efficient RUM design with ChoiceDesign\n", "\n", "This notebook illustrates how to use **ChoiceDesign** to generate a simple A-efficient experimental design for a Random Utility Maximisation (RUM) model. Instead of minimising the D-error (the determinant of the inverse Fisher information matrix), the A-criterion minimises the **A-error**:\n", "\n", "$$A\\text{-error} = \\frac{\\operatorname{trace}(I^{-1})}{K}$$\n", "\n", "where $K$ is the number of non-ASC parameters. The A-error is the average variance across all parameter estimates, making it more sensitive to individual parameters that are poorly estimated.\n", "\n", "## Step 1: Load modules, define design parameters and set attributes\n", "\n", "The following lines load:\n", "- `EffDesign`: the class of efficient designs,\n", "- `Attribute` and `Parameter`: the classes of attributes and parameters, respectively." ] }, { "cell_type": "code", "execution_count": 1, "id": "cell-1", "metadata": {}, "outputs": [], "source": [ "from choicedesign.design import EffDesign\n", "from choicedesign.expressions import Attribute, Parameter" ] }, { "cell_type": "markdown", "id": "cell-2", "metadata": {}, "source": [ "The following lines define 2 alternatives, named `alt1` and `alt2`, and 4 attributes named from $A$ to $D$:" ] }, { "cell_type": "code", "execution_count": 2, "id": "cell-3", "metadata": {}, "outputs": [], "source": [ "alt1_A = Attribute('alt1_A',[1,2,3])\n", "alt1_B = Attribute('alt1_B',[10,15,15.5])\n", "alt1_C = Attribute('alt1_C',[0,3,5])\n", "alt1_D = Attribute('alt1_D',[0,1,2])\n", "\n", "alt2_A = Attribute('alt2_A',[1,2,3])\n", "alt2_B = Attribute('alt2_B',[10,15,15.5])\n", "alt2_C = Attribute('alt2_C',[0,3,5])\n", "alt2_D = Attribute('alt2_D',[0,1,2])" ] }, { "cell_type": "markdown", "id": "cell-4", "metadata": {}, "source": [ "## Step 2: Construct efficient design object and generate initial design matrix\n", "\n", "The design object is constructed with `EffDesign`, passing the list of attributes and the number of choice situations:" ] }, { "cell_type": "code", "execution_count": 3, "id": "cell-5", "metadata": {}, "outputs": [], "source": [ "design = EffDesign(\n", " X = [alt1_A,alt1_B,alt1_C,alt1_D,\n", " alt2_A,alt2_B,alt2_C,alt2_D],\n", " ncs=18)" ] }, { "cell_type": "code", "execution_count": 4, "id": "cell-6", "metadata": {}, "outputs": [ { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "alt1_A", "rawType": "float64", "type": "float" }, { "name": "alt1_B", "rawType": "float64", "type": "float" }, { "name": "alt1_C", "rawType": "float64", "type": "float" }, { "name": "alt1_D", "rawType": "float64", "type": "float" }, { "name": "alt2_A", "rawType": "float64", "type": "float" }, { "name": "alt2_B", "rawType": "float64", "type": "float" }, { "name": "alt2_C", "rawType": "float64", "type": "float" }, { "name": "alt2_D", "rawType": "float64", "type": "float" } ], "ref": "c5cf8a77-ee43-4540-8558-80bce220e4c8", "rows": [ [ "0", "3.0", "15.0", "5.0", "1.0", "2.0", "15.5", "0.0", "0.0" ], [ "1", "2.0", "15.0", "5.0", "1.0", "1.0", "15.5", "0.0", "2.0" ], [ "2", "2.0", "15.5", "3.0", "0.0", "1.0", "10.0", "0.0", "0.0" ], [ "3", "2.0", "15.0", "0.0", "2.0", "2.0", "15.0", "5.0", "2.0" ], [ "4", "3.0", "15.5", "3.0", "0.0", "2.0", "10.0", "3.0", "1.0" ], [ "5", "2.0", "10.0", "3.0", "1.0", "3.0", "15.0", "3.0", "0.0" ], [ "6", "1.0", "15.0", "0.0", "2.0", "3.0", "15.5", "0.0", "0.0" ], [ "7", "1.0", "10.0", "0.0", "1.0", "3.0", "15.5", "5.0", "0.0" ], [ "8", "1.0", "10.0", "5.0", "1.0", "1.0", "15.0", "3.0", "2.0" ], [ "9", "2.0", "15.5", "3.0", "0.0", "2.0", "15.0", "3.0", "2.0" ], [ "10", "1.0", "15.5", "0.0", "1.0", "1.0", "15.5", "5.0", "1.0" ], [ "11", "2.0", "10.0", "3.0", "2.0", "1.0", "10.0", "5.0", "2.0" ], [ "12", "3.0", "15.0", "0.0", "0.0", "3.0", "10.0", "5.0", "2.0" ], [ "13", "3.0", "10.0", "3.0", "0.0", "1.0", "10.0", "3.0", "1.0" ], [ "14", "1.0", "15.5", "0.0", "2.0", "2.0", "15.5", "5.0", "1.0" ], [ "15", "3.0", "15.5", "5.0", "2.0", "3.0", "15.0", "0.0", "0.0" ], [ "16", "3.0", "10.0", "5.0", "2.0", "2.0", "15.0", "3.0", "1.0" ], [ "17", "1.0", "15.0", "5.0", "0.0", "3.0", "10.0", "0.0", "1.0" ] ], "shape": { "columns": 8, "rows": 18 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
alt1_Aalt1_Balt1_Calt1_Dalt2_Aalt2_Balt2_Calt2_D
03.015.05.01.02.015.50.00.0
12.015.05.01.01.015.50.02.0
22.015.53.00.01.010.00.00.0
32.015.00.02.02.015.05.02.0
43.015.53.00.02.010.03.01.0
52.010.03.01.03.015.03.00.0
61.015.00.02.03.015.50.00.0
71.010.00.01.03.015.55.00.0
81.010.05.01.01.015.03.02.0
92.015.53.00.02.015.03.02.0
101.015.50.01.01.015.55.01.0
112.010.03.02.01.010.05.02.0
123.015.00.00.03.010.05.02.0
133.010.03.00.01.010.03.01.0
141.015.50.02.02.015.55.01.0
153.015.55.02.03.015.00.00.0
163.010.05.02.02.015.03.01.0
171.015.05.00.03.010.00.01.0
\n", "
" ], "text/plain": [ " alt1_A alt1_B alt1_C alt1_D alt2_A alt2_B alt2_C alt2_D\n", "0 3.0 15.0 5.0 1.0 2.0 15.5 0.0 0.0\n", "1 2.0 15.0 5.0 1.0 1.0 15.5 0.0 2.0\n", "2 2.0 15.5 3.0 0.0 1.0 10.0 0.0 0.0\n", "3 2.0 15.0 0.0 2.0 2.0 15.0 5.0 2.0\n", "4 3.0 15.5 3.0 0.0 2.0 10.0 3.0 1.0\n", "5 2.0 10.0 3.0 1.0 3.0 15.0 3.0 0.0\n", "6 1.0 15.0 0.0 2.0 3.0 15.5 0.0 0.0\n", "7 1.0 10.0 0.0 1.0 3.0 15.5 5.0 0.0\n", "8 1.0 10.0 5.0 1.0 1.0 15.0 3.0 2.0\n", "9 2.0 15.5 3.0 0.0 2.0 15.0 3.0 2.0\n", "10 1.0 15.5 0.0 1.0 1.0 15.5 5.0 1.0\n", "11 2.0 10.0 3.0 2.0 1.0 10.0 5.0 2.0\n", "12 3.0 15.0 0.0 0.0 3.0 10.0 5.0 2.0\n", "13 3.0 10.0 3.0 0.0 1.0 10.0 3.0 1.0\n", "14 1.0 15.5 0.0 2.0 2.0 15.5 5.0 1.0\n", "15 3.0 15.5 5.0 2.0 3.0 15.0 0.0 0.0\n", "16 3.0 10.0 5.0 2.0 2.0 15.0 3.0 1.0\n", "17 1.0 15.0 5.0 0.0 3.0 10.0 0.0 1.0" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "init_design = design.gen_initdesign()\n", "init_design" ] }, { "cell_type": "markdown", "id": "cell-7", "metadata": {}, "source": [ "## Step 3: Set the utility functions\n", "\n", "Parameters are defined with the `Parameter` class. The arguments are:\n", "\n", "* `name`: The parameter name\n", "* `prior`: The prior value\n", "\n", "The following lines define four parameters:" ] }, { "cell_type": "code", "execution_count": 5, "id": "cell-8", "metadata": {}, "outputs": [], "source": [ "beta_A = Parameter('beta_A',-0.1)\n", "beta_B = Parameter('beta_B',-0.02)\n", "beta_C = Parameter('beta_C',0.1)\n", "beta_D = Parameter('beta_D',0.15)" ] }, { "cell_type": "code", "execution_count": 6, "id": "cell-9", "metadata": {}, "outputs": [], "source": [ "V1 = beta_A * alt1_A + beta_B * alt1_B + beta_C * alt1_C + beta_D * alt1_D\n", "V2 = beta_A * alt2_A + beta_B * alt2_B + beta_C * alt2_C + beta_D * alt2_D\n", "\n", "V = {1: V1, 2: V2}" ] }, { "cell_type": "markdown", "id": "cell-10", "metadata": {}, "source": [ "## Step 4: Optimise the design using the A-error criterion\n", "\n", "The `optimise()` method accepts a `criterion` argument that selects the optimality criterion:\n", "\n", "* `'d'` (default): minimise the D-error \u2014 $\\det(I^{-1})^{1/K}$\n", "* `'a'`: minimise the A-error \u2014 $\\operatorname{trace}(I^{-1})/K$\n", "* `'c'`: minimise the C-error (WTP variance sum)\n", "\n", "Setting `criterion='a'` yields an A-efficient design. All other arguments remain the same as in the D-efficient case:" ] }, { "cell_type": "code", "execution_count": 7, "id": "cell-11", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Evaluating initial design\n", "Optimization complete 0:00:59 / A-error: 0.050462\n", "Elapsed time: 0:01:00\n", "A-error of initial design: 0.118812\n", "A-error of last stored design: 0.050462\n", "Utility Balance ratio: 94.56 %\n", "Algorithm iterations: 183454\n", "\n" ] } ], "source": [ "optimal_design, init_perf, final_perf, final_iter, ubalance_ratio = design.optimise(\n", " init_design=init_design,\n", " V=V,\n", " model='mnl',\n", " criterion='a',\n", " time_lim=1,\n", " verbose=True\n", ")" ] }, { "cell_type": "markdown", "id": "cell-12", "metadata": {}, "source": [ "## Blocking the design\n", "\n", "The optimal design can be blocked using the method `gen_blocks()`. The following line creates 3 blocks:" ] }, { "cell_type": "code", "execution_count": 10, "id": "cell-13", "metadata": {}, "outputs": [ { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "CS", "rawType": "float64", "type": "float" }, { "name": "alt1_A", "rawType": "float64", "type": "float" }, { "name": "alt1_B", "rawType": "float64", "type": "float" }, { "name": "alt1_C", "rawType": "float64", "type": "float" }, { "name": "alt1_D", "rawType": "float64", "type": "float" }, { "name": "alt2_A", "rawType": "float64", "type": "float" }, { "name": "alt2_B", "rawType": "float64", "type": "float" }, { "name": "alt2_C", "rawType": "float64", "type": "float" }, { "name": "alt2_D", "rawType": "float64", "type": "float" }, { "name": "Block", "rawType": "int64", "type": "integer" } ], "ref": "15dac499-841d-46cc-b6e2-51b2f787997c", "rows": [ [ "0", "1.0", "3.0", "15.5", "0.0", "2.0", "1.0", "10.0", "3.0", "0.0", "1" ], [ "1", "2.0", "2.0", "15.0", "5.0", "1.0", "2.0", "15.0", "0.0", "1.0", "2" ], [ "2", "3.0", "2.0", "15.0", "5.0", "2.0", "2.0", "15.0", "0.0", "0.0", "1" ], [ "3", "4.0", "2.0", "15.0", "0.0", "0.0", "2.0", "15.0", "5.0", "2.0", "3" ], [ "4", "5.0", "3.0", "10.0", "3.0", "0.0", "1.0", "15.5", "3.0", "2.0", "3" ], [ "5", "6.0", "1.0", "10.0", "0.0", "2.0", "3.0", "15.5", "5.0", "0.0", "1" ], [ "6", "7.0", "1.0", "15.5", "5.0", "1.0", "3.0", "10.0", "0.0", "1.0", "3" ], [ "7", "8.0", "1.0", "10.0", "0.0", "2.0", "3.0", "15.5", "3.0", "0.0", "3" ], [ "8", "9.0", "1.0", "15.0", "3.0", "0.0", "3.0", "15.0", "5.0", "2.0", "2" ], [ "9", "10.0", "3.0", "15.0", "5.0", "0.0", "1.0", "15.0", "0.0", "2.0", "1" ], [ "10", "11.0", "2.0", "15.0", "0.0", "1.0", "2.0", "15.0", "5.0", "1.0", "2" ], [ "11", "12.0", "2.0", "15.5", "3.0", "2.0", "2.0", "10.0", "3.0", "0.0", "3" ], [ "12", "13.0", "2.0", "15.5", "3.0", "0.0", "2.0", "10.0", "5.0", "2.0", "1" ], [ "13", "14.0", "3.0", "10.0", "5.0", "1.0", "1.0", "15.5", "0.0", "1.0", "3" ], [ "14", "15.0", "1.0", "15.5", "3.0", "0.0", "3.0", "10.0", "3.0", "2.0", "2" ], [ "15", "16.0", "3.0", "15.5", "3.0", "2.0", "1.0", "10.0", "3.0", "0.0", "2" ], [ "16", "17.0", "3.0", "10.0", "0.0", "1.0", "1.0", "15.5", "5.0", "1.0", "2" ], [ "17", "18.0", "1.0", "10.0", "5.0", "1.0", "3.0", "15.5", "0.0", "1.0", "1" ] ], "shape": { "columns": 10, "rows": 18 } }, "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
CSalt1_Aalt1_Balt1_Calt1_Dalt2_Aalt2_Balt2_Calt2_DBlock
01.03.015.50.02.01.010.03.00.01
12.02.015.05.01.02.015.00.01.02
23.02.015.05.02.02.015.00.00.01
34.02.015.00.00.02.015.05.02.03
45.03.010.03.00.01.015.53.02.03
56.01.010.00.02.03.015.55.00.01
67.01.015.55.01.03.010.00.01.03
78.01.010.00.02.03.015.53.00.03
89.01.015.03.00.03.015.05.02.02
910.03.015.05.00.01.015.00.02.01
1011.02.015.00.01.02.015.05.01.02
1112.02.015.53.02.02.010.03.00.03
1213.02.015.53.00.02.010.05.02.01
1314.03.010.05.01.01.015.50.01.03
1415.01.015.53.00.03.010.03.02.02
1516.03.015.53.02.01.010.03.00.02
1617.03.010.00.01.01.015.55.01.02
1718.01.010.05.01.03.015.50.01.01
\n", "
" ], "text/plain": [ " CS alt1_A alt1_B alt1_C alt1_D alt2_A alt2_B alt2_C alt2_D \\\n", "0 1.0 3.0 15.5 0.0 2.0 1.0 10.0 3.0 0.0 \n", "1 2.0 2.0 15.0 5.0 1.0 2.0 15.0 0.0 1.0 \n", "2 3.0 2.0 15.0 5.0 2.0 2.0 15.0 0.0 0.0 \n", "3 4.0 2.0 15.0 0.0 0.0 2.0 15.0 5.0 2.0 \n", "4 5.0 3.0 10.0 3.0 0.0 1.0 15.5 3.0 2.0 \n", "5 6.0 1.0 10.0 0.0 2.0 3.0 15.5 5.0 0.0 \n", "6 7.0 1.0 15.5 5.0 1.0 3.0 10.0 0.0 1.0 \n", "7 8.0 1.0 10.0 0.0 2.0 3.0 15.5 3.0 0.0 \n", "8 9.0 1.0 15.0 3.0 0.0 3.0 15.0 5.0 2.0 \n", "9 10.0 3.0 15.0 5.0 0.0 1.0 15.0 0.0 2.0 \n", "10 11.0 2.0 15.0 0.0 1.0 2.0 15.0 5.0 1.0 \n", "11 12.0 2.0 15.5 3.0 2.0 2.0 10.0 3.0 0.0 \n", "12 13.0 2.0 15.5 3.0 0.0 2.0 10.0 5.0 2.0 \n", "13 14.0 3.0 10.0 5.0 1.0 1.0 15.5 0.0 1.0 \n", "14 15.0 1.0 15.5 3.0 0.0 3.0 10.0 3.0 2.0 \n", "15 16.0 3.0 15.5 3.0 2.0 1.0 10.0 3.0 0.0 \n", "16 17.0 3.0 10.0 0.0 1.0 1.0 15.5 5.0 1.0 \n", "17 18.0 1.0 10.0 5.0 1.0 3.0 15.5 0.0 1.0 \n", "\n", " Block \n", "0 1 \n", "1 2 \n", "2 1 \n", "3 3 \n", "4 3 \n", "5 1 \n", "6 3 \n", "7 3 \n", "8 2 \n", "9 1 \n", "10 2 \n", "11 3 \n", "12 1 \n", "13 3 \n", "14 2 \n", "15 2 \n", "16 2 \n", "17 1 " ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "optimal_design_blocked, corr_history = design.gen_blocks(optimal_design, n_blocks=3)\n", "optimal_design_blocked" ] }, { "cell_type": "markdown", "id": "cell-14", "metadata": {}, "source": [ "## (optional) Evaluate the design\n", "\n", "Pass `criterion='a'` to `evaluate()` to compute the A-error of a stored design:" ] }, { "cell_type": "code", "execution_count": 11, "id": "cell-15", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "A-error: 0.050462107185962744\n", "Utility balance: 94.56297363503425\n" ] } ], "source": [ "perf, ubalance = design.evaluate(optimal_design, V, model='mnl', criterion='a')\n", "\n", "print('A-error:', perf)\n", "print('Utility balance:', ubalance)" ] }, { "cell_type": "markdown", "id": "cell-16", "metadata": {}, "source": [ "## References\n\n[1] Quan, W., Rose, J. M., Collins, A. T., & Bliemer, M. C. (2011). A comparison of algorithms for generating efficient choice experiments.\n" ] } ], "metadata": { "kernelspec": { "display_name": "choicedesign-oSBhddzi-py3.13", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.1" } }, "nbformat": 4, "nbformat_minor": 5 }