Sensory Computer Systems Forums Index Sensory Computer Systems Forums
Welcome to the Sensory Computer Systems Online Forums!
 
 FAQFAQ   SearchSearch   RegisterRegister 
 ProfileProfile   Log inLog in   
   Sensory Computer Systems Home   

Difference-From-Control Test DOD - Dunnett's T Test

 
Post new topic   Reply to topic    Sensory Computer Systems Forums Index -> Test Design
View previous topic :: View next topic  
Author Message
Mary



Joined: Dec 21 2005
Posts: 8
Location: North America

PostPosted: Fri Feb 03, 2006 8:33 am    Post subject: Difference-From-Control Test DOD - Dunnett's T Test Reply with quote



I would like some information on know to how set up a Difference-From-Control Test in SIMS Sensory Quality Software System. Also the Dunnett's T Test.

Thanks in advance,
Mary
Back to top
View user's profile
PAL



Joined: Jun 01 2005
Posts: 24

PostPosted: Fri Feb 03, 2006 6:01 pm    Post subject: Difference-From-Control Test Reply with quote



Here's an example of the Dunnett's Statistics in your SIMS System:




Updated Summer 2022



Set up:

This document describes how to set up a Difference-From-Control Test in SIMS software.
This test is also known as a Degree of Difference Test (DOD).
To jump right to Dunnett's T Test see #6) below.

Use this test when the project objectives include the following:

  • Determine whether there is a difference between a control (reference) sample and one or more other samples.
  • Estimate the degree of such difference(s).

Questionnaire example showing both an hedonic scale and a line scale.



Overview
    Present to each panelist the control (reference) sample plus one or more additional samples. Panelist will rate the degree of difference between each sample and the control. Explain to panelists that some of the test samples may be identical to the control.

    While it is acceptable to ask the panelists other questions about each sample, for the purpose of this discussion assume that the difference-from-control question is the only one on the test.


1) Questionnaire

  1. Create a questionnaire, Affective type, perhaps named DIFF_CTRL (reusable for future studies).
    (Note: As discussed below, the degree of difference attribute can be either a Line Scale or Hedonic Scale. While it is possible to use a Descriptive questionnaire for a Line Scale attribute, setting the questionnaire type as Affective provides more flexibility to add other attribute types at a later time.)

  2. Create a Hedonic or Line Scale attribute which will be used to record the extent of the difference of each sample from the Control. For a Hedonic Scale the categories will look something like this:
    Code:


                      Choice Description         Return Value
                 o    No Difference                       0
                 o    Very Small Difference               1
                 o    Small Difference                    2
                 o    Small/Moderate Difference           3
                 o    Moderate Difference                 4
                 o    Moderate/Large Difference           5
                 o    Large Difference                    6
                 o    Very Large Difference               7



    For a Line Scale the Minimum Value should be 0 associated with a description of "No Difference." The Maximum Value should correspond to a description like "Very Large Difference." The scale can be either continuous or incremental. Reference Points associated with mid-level degrees of difference may be included to aid the respondents.

  3. Question Example: Please rate the level of difference for Sample nnn relative to the control sample.

  4. Set the "Attribute Seen With" option to "All Samples."


    SIMS software express package file: DIFF-CTRL.SIMS2000ExpressPackage.sql.ZIP



Note: This forum topic and the notes below are about presentations which include Control vs Blind Control.
    Control vs Control
    Control vs Test Sample #1
    Control vs Test Sample #2


In contrast, if your study only includes presentations of Control vs Test(s), and does not incorporate presentations for Control vs Blind Control, then we suggest the following:
- For a simple Control vs Test we recommend a simple 1 sample study, a DOD type question, and after the test, review your raw data, means, standard deviations, and other statistical methods of your choosing.
- For a simple Control vs Test1 + Control vs Test2 we recommend a simple 2 sample study, a DOD type question, and after the test, review your raw data, means, standard deviations, and other statistical methods of your choosing.
- skip all of the notes below.



Read on for:
For Control vs Blind Control , Control vs Test1 , Control vs Test2. This is a 3C3 design.
and very similar if just Control vs Blind Control , Control vs Test1. This is a 2C2 design.
and very similar Control & 3 test samples. This is a 4C4 design.
Etc., etc.



2) Experimental Design
    The Experimental Design must be a Complete Block design such as 3 Samples see 3 in order to employ the Dunnett Test when the statistics are generated. It is recommended that Sample Order be randomized within each block to eliminate positional bias, which is called a "Balanced Complete Block".

    The Control or Reference Sample is USUALLY designated as Sample 1 in the Experimental Design.

    Control
    Test Sample #1
    Test Sample #2

    Each Sample Set presented to a panelist will always include a sample labeled as the "Control" or "Reference." In addition the panelist will also be presented with each of the samples labeled with a three-digit Blinding Code.

    The panelist will first taste the Control sample, then they will taste the 3 blind coded samples, one at a time, rating the difference from Control.

    Effectively the panelist will be evaluating, in random order:
    Control vs Control
    Control vs Test Sample #1
    Control vs Test Sample #2


    In this example, the Experimental Plan should be Permutations 3 choose 3 (blocks 1 2 3 randomized).

    So the panelist will compare the Control with each of the blind ordered samples. Exactly one of the ordered samples will be the same as Control, but panelist will not know which one.


3) Test Definition

  1. Create a new Test Definition by selecting your new Questionnaire and Experimental Design. Use whatever Panel Type option works for you.

  2. Print the Rotation Plan for the technician to prepare the ballots properly.

  3. In addition, have the technician also prepare a Control sample to include with every ballot.

4) Test Result Master / Execution

  1. Create a new Test Result Code, using the Test Definition above.
  2. Start the Test.

5) Test Administration
    You may need to explain to the panelists that the SIMS Sample Verification Prompt will not include the Control or Reference sample. However, each panelist must have this Control sample to complete the test properly. This explanation may be important to those panelists used to the presentation of SIMS Duo-Trio tests, which explicitly include the "REF" sample in the Sample Verification Prompt.


6) After the Test is Completed - Dunnett's T Test

  1. Run Statistical Analysis.

  2. Select the "Dunnett's T Test" option

    Used for Multiple Comparison Analysis of a Difference from Control test.

    The Dunnett's test is similar to the Tukey test, but Dunnett's only compares your Control Sample with each of the other test samples, i.e., a Difference from Control test.
    1 vs. All. Sample #1 is generally the default Control Sample for Dunnett's statistics. The Default for your Control Sample is "1 vs."
    Optionally you can specify a different Control Sample for Dunnett's statistics via the list box shown, "1 vs.", "2 vs.", "3 vs." etc.


    SIMS OUTPUT REPORTING NOTES: SIMS Dunnett's output.

    P-value: The P-value on the main TABLE is NOT the reliable indicator of significance for Dunnetts Test.
    This is because the TABLE's P-value shows significant difference amoung ALL the Test Samples, ANOVA.

    For your Dunnett's T-Tests you should be looking at the individual significant differences between 1 vs 2, 1 vs 3, 1 vs 4, etc.

    See your Dunnett's P-values in the TABLE's additional columns labeled 'P-Value Dunnett 1 vs 2', 'P-Value Dunnett 1 vs 3', etc.

    To further interpret your Dunnett's statistical results, go to 'View Analysis Files', and click on 'Statistics Output File'.

    See examples below.




    Code:

      R Dunnett's Output File Example
      -------------------------------

        Go to the R Section labeled '---  Dunnett's Results  DVn  ---' to determine your individual pairs significance.

          P-values, your individual pairs P-values will be shown under column 'Pr(>|t|)'
          Your Individual pairs significance levels will be shown to the right of each p-value.

          See the value for 'Minimum Significant Difference' (MSD), compare this to the individual values for 'Difference Between Means' (DBM).
              Locate the MSD right near the top of the R Dunnett's Results output;  See 'Minimum Significant Difference:  n.nnnn'
              Locate the DBM a little further down;  See 'Linear Hypotheses:',  the DBM is the 'Estimate' column.
            If your DBM is greater than MSD (ie. DBM > MSD) then that Control vs test sample pair combination is YES significantly different.

          Also see the confidence limits section, two value columns, one row for each sample comparison:
            If 0 (zero) is not in the interval value range, then the pair is NOT significantly different.
               Most obvious when lower number is negative and upper number is positive, 0 (zero) is in this intraval range,
               then the pair is NOT significantly different.
            If 0 (zero) is in the interval value range, then the pair is YES significantly different.
               Most obvious when lower number & upper number are both above or below 0, 0 (zero) is NOT in this intraval range,
               then the pair is YES significantly different.


        R Example:

            ---   Begin:  DV1   Overall Liking (Hedonic Scale)  ---

            --------------------------------
            ---  Dunnett's Results  DV1  ---
            --------------------------------

            Alpha:  0.05
            Error Degrees of Freedom:  8
            Error Mean Square:  1.1833
            Critical Value of Dunnett's t:  2.6731
            Minimum Significant Difference:  1.8391


                 Simultaneous Tests for General Linear Hypotheses

            Multiple Comparisons of Means: User-defined Contrasts

            Linear Hypotheses:
                        Estimate Std. Error t value  Pr(>|t|)
            2 - 1 == 0 3.0000000  0.6879922 4.36051 0.0044195 **
            3 - 1 == 0 3.6000000  0.6879922 5.23262 0.0014598 **

            Signif. codes:  0 '***'   0.001 '**'   0.01 '*'   0.05 '.'   0.1 ' '   1
            (Adjusted p values reported -- single-step method)


                 Simultaneous Confidence Intervals

            Multiple Comparisons of Means: User-defined Contrasts

            Quantile = 2.6731265
            95% family-wise confidence level


            Linear Hypotheses:
                       Estimate  lwr       upr
            2 - 1 == 0 3.0000000 1.1609097 4.8390903
            3 - 1 == 0 3.6000000 1.7609097 5.4390903


         In this example, both sample comparisons 1 vs 3 and 1 vs 2 are significantly different.



    Code:

      SAS Dunnett's Listing File Example
      ----------------------------------

        Go to the SAS GLM Section labeled 'Dunnett's t Tests for Xn' to determine your individual pairs significance.

          See the value for 'Minimum Significant Difference' (MSD), compare this to the individual values for 'Difference Between Means' (DBM).
            If your DBM is greater than MSD (ie. DBM > MSD) then that Control vs test sample pair combination is YES significantly different.

          Also see the confidence limits section, two value columns, one row for each sample comparison:
            If 0 (zero) is not in the interval value range, then the pair is NOT significantly different.
               Most obvious when lower number is negative and upper number is positive, 0 (zero) is in this intraval range,
               then the pair is NOT significantly different.
            If 0 (zero) is in the interval value range, then the pair is YES significantly different.
               Most obvious when lower number & upper number are both above or below 0, 0 (zero) is NOT in this intraval range,
               then the pair is YES significantly different.


        SAS Example:

                       Dependent Variable:  X1  Overall Liking

                                       The GLM Procedure

                                    Dunnett's t Tests for X1

                            alpha 0.05
                            Error Degrees of Freedom              8
                            Error Mean Square              1.183333
                            Critical Value of Dunnett's t   2.67281
                            Minimum Significant Difference   1.8389

                  Comparisons significant at the 0.05 level are indicated by ***.

                                      Difference
                          SAMPLE         Between     Simultaneous 95%
                        Comparison         Means    Confidence Limits

                          3 - 1           3.6000      1.7611   5.4389  ***
                          2 - 1           3.0000      1.1611   4.8389  ***


                                   The GLM Procedure
                                  Least Squares Means
                      Adjustment for Multiple Comparisons: Dunnett

                                                    H0:LSMean=
                                                      Control
                          SAMPLE       X1 LSMEAN      Pr > |t|

                          1           1.60000000             
                          2           4.60000000        0.0044
                          3           5.20000000        0.0015


         In this example, both sample comparisons 1 vs 3 and 1 vs 2 are significantly different.






See the Dunnett's T Test Notes in your SIMS system for more information.


Reference: Sensory Evaluation Techniques, Civille, 4th Ed. (pg. 92), 3rd Ed. (pg.86), 2nd Ed. (pg. 81)

Back to top
View user's profile
Display posts from previous:   
Post new topic   Reply to topic    Sensory Computer Systems Forums Index -> Test Design All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group