The Optsee Prioritizer™

Imagine that you could poll 100,000 experts like yourself to help select your best choices, and then get a complete statistical analyses of the results. The Optsee Prioritizer™ works by simulating that kind of study. It analyzes your choices in up to 100,000 different decision models generated using a Monte Carlo simulation, and then displays the results in easy-to-understand charts and tables. The Optsee Prioritizer™ is a powerful technology for rigorously determining your best choices.

The Monte Carlo simulation methodology was named for Monte Carlo, Monaco; a city that is famous for its casinos and games of chance such as roulette wheels, dice, cards, and slot machines. Games of chance exhibit random behavior within the context of the game equipment and rules. For example, a shuffled deck of cards will contain 52 cards, but the card order is random. An Optsee® Monte Carlo simulation involves creating thousands of random decision models and/or portfolios within a set of defined parameters. Your portfolios are tested in the models, and the average rank, standard deviation, and cumulative percentage ranking for each choice is calculated.

Example Prioritization Statistical Results Chart

Three primary types of prioritizations can be performed:

1. Variable Weights: Two types of Variable Weights prioritizations can be executed. In the first type, the attribute rank order of preference (the left column of the decision model) is kept constant, but the relative weights of the attributes (in the right column of the decision model) are randomly varied. This is called a Fixed Sequential Order prioritization. In the second type, both the attribute rank order and the weights of the attributes are varied. This is called a Random Order prioritization. A Ranked Attribute Order prioritization will provide a statistical ranking of the choices based on how the choices were ranked in up to 100,000 decision models that maintained the same attribute rank order as the original decision model. This allows you to see which choices are statistically most attractive with a particular rank order. A Random Attribute Order prioritization will provide a statistical ranking of the choices in up to 100,000 decision models that have randomly ranked the order of attribute weights. This allows you to see which choices are statistically most attractive when the attributes are randomly ranked (e.g. no fixed order).

2. Variable Choices: A Variable Choices prioritization involves generating multiple portfolios using the parent decision model where the choice attribute values are modified within the % uncertainty range of each individual choice. For example, if a choice has an attribute value of "500" and a % uncertainty of "5" for that attribute, the Variable Portfolio prioritization would create portfolios where that choice attribute would randomly vary between "475" and "525" or between 95% and 105% of the attribute value. The random variation can be distributed over a normal Gaussian distribution (bell curve) or a uniform distribution (equally distributed between the maximum and minimum values). Maximum and minimum values outside of the decision model best and worst attribute outcome constraints are not used in the prioritization. Also note that when a choice attribute value is "0" and the % uncertainty for that value is not "0," Optsee® uses a value of 0.001 in the prioritization.

3. Combined: A Combined prioritization combines the Variable Decision Model prioritization with the Variable Choices prioritization such that the decision models weights and the portfolio choice attribute values are varied simultaneously within the constraints described above.

In the example below, we'll demonstrate how to use Variable Weights prioritizations.

An Example of using the Optsee Prioritizer™:

In this example, our goal is to select a candidate for a managerial position in a technical field. The following 15 attributes have been selected for evaluating the candidates:

  • Management Experience: Previous experience, level of responsibility, and demonstrated competency
  • Technical Experience: Knowledge of field, specific expertise, publications
  • Education: University degree achieved
  • Cost to Hire: Relocation and/or recruiter fees
  • Salary Requirements: $80,000 to $125,000
  • References: Quality of references
  • Presentation: Presents professional image
  • Industry Knowledge: Knowledge of Industry and overall business
  • Time Management: Organization/Planning, setting goals and determining priorities. Time management skills.
  • Communication Skills: Oral skills during interview. Listening skills and non-verbal sensitivity.
  • Motivation: Self-motivated Can motivate others to achieve goals
  • Interpersonal Skills: Gets along with peer groups and superiors. Team consciousness. Gains understanding and cooperation from others.
  • Leadership: Realistic a view of her/his strengths/weaknesses. Good career growth potential
  • Problem Solving: Analyze situations and data. Uses sound judgement in arriving at final decision.
  • Creativity/Innovation: Ability to develop methods to improve results.

Figure 1 displays the decision model. Note that the attributes have been weighted and ranked in simple numerical order with Management Experience having the highest weight (15) and Cost to Hire having the lowest weight (1).

Figure 1 [View Larger Image]

Figures 2a, 2a and 2c show the six candidates under consideration in a Portfolio form. Figure 2a displays the first five attributes, Figure 2b displays the next five attributes, and Figure 2c displays the next five attributes. Note that the top three candidates are closely ranked with overall attractiveness values ranging from 76.8 to 72.5.

Figure 2a [View Larger Image]

Figure 2b [View Larger Image]

Figure 2c [View Larger Image]

We can run our first prioritization on this portfolio by clicking on the <Prioritization> button. This opens the Create Optsee Prioritizer Model Form (Figure 3). We'll set the number of Test Models to 10,000 and the Weight Range to 5000. These settings mean that the portfolio will be evaluated in 10,000 models, and that the highest possible attribute weight is 5,000 and the lowest possible attribute weight is 1

Figure 3

Clicking Run Prioritization displays the "Performing Prioritization Testing" dialog form that lets you monitor the progress of the testing or stop it.

Figure 4

After the decision models are created, the portfolio choices are evaluated and then the results are displayed in the Optsee Prioritizer Summary List form (Figure 5)

Figure 5 [View Larger Image]

Click on Statistics to open the Prioritization Statistics Chart (Figure 6). The rankings show that the candidates have retained the same order as in the portfolio, but that the top two candidates (Levitt and Mathews) were ranked significantly higher than the other four candidates. Similarly, the bottom candidate (Smith) was ranked well below the other five.

Figure 6 [View Larger Image]

Figure 7 displays the same chart with the Maximum and Minimum represented by the vertical lines. In his representation, we can see that top candidate were ranked first in at least one model and third in at least one model. The third candidate (Abreu) was ranked second in at least one model and sixth in at least one model.

Figure 7 [View Larger Image]

You can scroll through the ranking distribution histograms by clicking on the <Distributions> button on the Optsee Prioritizer Summary List form to open the Distributions List form (Figure 7a)

Figure 7a

You can open an individual ranking distribution histogram in the Distribution form (Figure 7b).

Figure 7b

However, the minimum and maximum range does not necessarily reflect the strength of a particular candidate. For example, a candidate that was ranked first in only 1 out of 10,000 models should not be considered as strong as a candidate as a candidate that was ranked first in 5,000 models out of 10,000. Therefore, another way to ascertain the strength of individual candidates is to look at the Cumulative Percentage Chart. The Cumulative Percentage Chart (opened by clicking on the Cumulative % button on the Statistics Chart form) is shown in Figure 8a.

In this chart, we can see that candidate Levitt was ranked first in 82% of the models (Point A) and candidate Mathews was ranked first in 20% of the models (Point B). They were both ranked second or higher in 99% of the models (Point C), and were ranked third or higher in all of the models. The candidate ranked third (Abreu) was ranked second or higher in 2% of the models (Point E), and third or higher in 62% of the models (Point D). Therefore, based on both the Statistical Ranking and the Cumulative Percentage analysis for this decision model, it would appear that Susan Levitt is the preferred candidate.

Figure 8a [View Larger Image]

The Cumulative Percentage Bar Chart is shown in Figure 9. This chart summarizes the information from the Cumulative Percentage Line Chart based on calculating the area under the curve (AUC) for each choice, and then normalizing them relative to the choice with the greatest area under the curve. Thus, in the example above, Susan Levitt, who has the greatest area under the curve, has a normalized cumulative percent ranking of 100, and the other choices have relative values below it. For example, Julie Mathews has an AUC of 93.5 and Jose Abreu has an AUC of 57. Therefore, the Cumulative % Rank column of the Statistics Summary List form and the corresponding bar chart are very useful tools to rank the project choices based on cumulative percentage analyses, particularly for portfolios containing large numbers of choices.

Figure 8b [View Larger Image]

Now lets run the prioritization again using 10,000 models and a Random Order of attribute weights (Figure 9).

Figure 9

After the decision models are created, the portfolio choices are evaluated and then the results are displayed in the Optsee Prioritizer Summary List form (Figure 10)

Figure 10 [View Larger Image]

Figure 11 displays the statistics chart resulting from this prioritization. Note that the rank order of the candidates has changed entirely from the Fixed Sequential Order prioritization, and the standard deviations have grown significantly. This is because each random decision model can change the candidate rankings based on the strengths and weaknesses of each candidate because the ranking of the attribute weights change from one decision model to the next. For example, Education could have a weight of 5000 (top rank) in one model, and a weight of 1 in the next.

Figure 11 [View Larger Image]

Figure 12 displays the same chart with the Maximum and Minimum represented by the vertical lines. In his representation, we can see that all of the candidates were ranked as high as first and as low as sixth. Again, this is a result of the random order of the attribute weights in the decision models.

Figure 12 [View Larger Image]

The associated Cumulative Percentage Chart is shown in Figure 13. This chart shows that candidates Adams and Abreu have virtually identical profiles, suggesting they are identical in this prioritization. However, based on the analysis of this Random Order Prioritization (both the statistical and cumulative percentage charts), no single candidate is strongly distinguished from the other candidates.

Figure 13 [View Larger Image]

The associated Cumulative Percentage Bar Chart is shown in Figure 13a. This chart shows how the candidates are split into three tiers when evaluated using the random order decision model prioritization.

Figure 13a [View Larger Image]

In both the fixed-order and random order prioritizations, Mary Smith and Joe Jones were the lowest ranked candidates. In the random prioritization, onlly Susan Levitt emerged as the clearly preferred candidate as in the fixed-order prioritization. Therefore, based on these analyses, Levitt should be selected as the primary candidate because she is strongest overall in the fixed-order prioritization, which has prioritized the criteria most important for the position.

headimg