Sensory Computer Systems Forums Index Sensory Computer Systems Forums
Welcome to the Sensory Computer Systems Online Forums!
 
 FAQFAQ   SearchSearch   RegisterRegister 
 ProfileProfile   Log inLog in   
   Sensory Computer Systems Home   

Same/Different Test

 
Post new topic   Reply to topic    Sensory Computer Systems Forums Index -> Test Design
View previous topic :: View next topic  
Author Message
Mary



Joined: Dec 21 2005
Posts: 8
Location: North America

PostPosted: Fri Jan 27, 2006 9:29 am    Post subject: Same/Different Test Reply with quote


How can I setup a Same/Different Test in our SIMS System?

And Chi-Square Analysis.

And Fisher's Exact Test Analysis.



Back to top
View user's profile
John



Joined: Jun 13 2005
Posts: 23

PostPosted: Fri Jan 27, 2006 2:55 pm    Post subject: SAME / DIFFERENT Tests in SIMS Reply with quote


Here's our detailed Same/Different Test webpage, from your SIMS test set-up and thru your statistical analysis.


SAME / DIFFERENT Tests in SIMS

Updated Summer 2017

SIMS software express package of the following example: SAMEDIFF.SIMS2000ExpressPackage.sql.ZIP

The Same/Different Test is also commonly referred to as the Simple Difference Test. The Same/Different Test method is sometimes used when your two samples are unsuitable for triple or multiple test presentations, e.g. when the Triangle and Duo-trio tests can not be used.


The following describes how to set up a Same/Different Test in SIMS. Although SIMS does not have a specifically named Same/Different Test option, it is fairly easy to set up and prepare the Chi-Square Analysis.


This description assumes that you are not asking individual questions about each sample.


1) Questionnaire:

    Questionnaire Example:



  1. Create a questionnaire, Affective type, maybe named SAMEDIFF  (reusable for future studies)
      
  2. Create a single Hedonic attribute question with two choices:
              ( )  The two samples are the SAME.  (Set Return Value = 1)
              ( )  The two samples are DIFFERENT. (Set Return Value = 0)
    - This is an important step, please double check your Return Values.
    If you use different return values, then after the test if you use our Chi-Square Analysis Excel XLS file discussed below, you'll need to adjust the all formulas in the result column G.


  3. Set the Hedonic 'Attribute Seen With' option to 'Relative Sample Order 1.'
      
  4. Set any/all other Questionnaire elements, such as Instruction boxes, same 'Seen With' as 1c. above.
    - This is an important step, please double check your selected 'Seen With' settings.



  5. Optional Feedback to Panelist:
    - Use an Instruction Box (any page following the above Hedonic).
    - Add in the special Tag <SAME/DIFFERENT>
    - Which will transform to text SAME or DIFFERENT depending on current Panelist's Sample Set.
    - Example: Your samples were <SAME/DIFFERENT>.



2) Experimental Design:

    You want to present samples to the panelists in one of these four combinations:
       AA  Two Sample A (SAME)
       AB  Sample A followed by Sample B (DIFFERENT)
       BA  Sample B followed by Sample A (DIFFERENT)
       BB  Two Sample B (SAME)
  1. First, create a new Experimental Plan, named SAMEDIFF  (reusable for future studies)
         Plan Type:  Affective/Descriptive
         Number of Samples(T): 2 
         Number Presented (K): 2
         Number of Blocks: 4  (each associated with each of the AB combinations above)
         Number of Reps: 1
         Save, then Go to 2nd Tab, 'Blocks'
         Edit Block  1 -> 1 1  /  2 -> 1 2  /  3 -> 2 1  /  4 -> 2 2
    If you use a Block sample pairings are not exactly 1 1, 1 2, 2 1, and 2 2, then after the test if you use our Chi-Square Analysis Excel XLS file discussed below, you'll need to adjust the Lookup Table shown in Excel columns H&I, and J&K
      
  2. Create a new Experimental Design, named SAMEDIFF  (reusable for future studies)
         Design Type:  Affective
         Number of Respondents:  Usually divisible by 4 (such as 48, 60, 120, etc.)
         Number of Samples: 2
         Number of Reps: 1
         Experimental Plan:  Use SAMEDIFF, the one you created above in 2a.
         Block and Sample Presentation 'As in Plan', 'As in Plan'. ---- Do *not* select Randomization.
         Save, then Go to 2nd Tab, 'Sample Information'
         Optionally change Blinding codes and Sample Descriptions.

  3. See Print Screen Example Experimental Design at the end of this Forum Post below.

3) Test Definition:
  1. Create a new Test Definition, select your new Questionnaire and Design.
      
  2. Print the Rotation Plan for technician to prepare the ballots properly.

4) Test Result Master / Execution:
  1. Create a new Test Result Code, using the Test Definition created above.
      
  2. VERY IMPORTANT: for Sample Set Selection,
               please select  'Force Panelist to Register Sample Set as Ballot Number'
      
  3. Start the Test. 
      
  4. Test Execution Parameters:
               Check 'Force All Panelists to Exit After Every:  1 Sample'
               Check 'Bypass Sample Set Verification Prompt' (optional)
               Check 'Bypass Sample Number Verification Prompt'
               Check 'Remove Sample Code & Count from Navigation Bar'

5) Test Administration:
  1. Each tray should have a Ballot Number and two samples with Blinding Codes.
    Ballot Numbers will be 1 to the number of respondents (N).
    Properly match the 2 samples and Ballot numbers on each tray.
    See the Rotation Plan, Ballot Number = Sample Set.
      
  2. The panelist will be asked to enter the Ballot Number.

6) Final Pre-Test Step:
    Test your Test, as you should do with any new type of Test.

7) Data Results.
    Tallying the Same/Different question results is more complicated than usual since you have to consider a) did the panelist get a Matched or Unmatched pair, and b) did the panelist respond Same or Different.

    You need four numbers:
      Count of Matched samples presented and panelist responded SAME
      Count of UnMatched samples presented and panelist responded SAME
      Count of Matched samples presented and panelist responded DIFFERENT
      Count of UnMatched samples presented and panelist responded DIFFERENT


    - Option 1 recommended: Run the SIMS Test Data Frequency Report, use the 3rd sort option: By Attribute - Aggregated Over All Samples. New November 2013

      SIMS Test Data Frequency Report example:



    - Option 2 manually: The Excel file has an older method on Excel worksheet tab #1, involving using the SIMS raw data export report to excel.

    - Option 3 manually: Print out the SIMS 2000 Test Definition Rotation Plan and align with the Test Raw Data Excel Export Report. Manually match up the Panelist Sample Set Matched or Unmatched pair to the answer in the data 1=Same and 0=Different.

    - Then utilize the XLS Excel file worksheet tab #2 below to plug in your four numbers for your Same/Different data. See the next section about Excel.




References: Sensory Evaluation Techniques, Civille, 4th Ed.(pg 84), 3rd Ed.(pg 79), 2nd Ed.(pg 74).
ASTM E2139-05 Standard Test Method for Same-Different Test


SIMS software express package of the above example: SAMEDIFF.SIMS2000ExpressPackage.sql.ZIP

Back to top
View user's profile
ForumAdmin
Site Admin


Joined: May 20 2005
Posts: 120
Location: New Jersey

PostPosted: Thu Nov 14, 2013 10:13 am    Post subject: After your Same/Different Test is completed Reply with quote


After your Same/Different Test is completed:

Here's how to do the Chi-Square Analysis in Microsoft Excel. Chi-Square Analysis is also known as χ2-Analysis.

Sensory Computer Systems Excel File Available:
Same-Different_Test_Analysis_Using_Chi-Square_Using_Fishers_Exact_Test.xls (or right click and 'Save Target As' or 'Save Link As' )

Example, Chi-Square Analysis and Fisher's Exact Test Analysis:





Last edited by ForumAdmin on Thu May 18, 2023 3:39 pm; edited 9 times in total

Back to top
View user's profile
ForumAdmin
Site Admin


Joined: May 20 2005
Posts: 120
Location: New Jersey

PostPosted: Thu Nov 14, 2013 10:22 am    Post subject: Miscellaneous Reply with quote


Miscellaneous


Print Screen Example of Experimental Design:






**********************************************************************
**********************************************************************


Advanced

Multiple Reps (each panelist doing same/diff test more than once).
  1. Use the same Ballot SAMEDIFF.
  2. Copy Experimental Design SAMEDIFF to SAMEDIFF2, change it to 2 or more Reps. Same Experimental Plan is ok.
  3. New Test Definition SAMEDIFF2. Review and understand the Rotation Plan.
  4. Test Execution, same instructions as written above, plus
      Select [X] Allow Continuous Presentation Over Reps, panelists will signin once, do both reps in one sitting.
      Do NOT Check 'Force All Panelists to Exit After Every: 1 Sample' as stated above.
      Check 'Bypass Sample Set Verification Prompt', because only first sample set pair will be shown.
  5. Final Step: Test your Test, as you should do with any new type of Test.



Advanced

Fisher's Exact Test (FET) statistical test of the equality of two independent binomial proportions.

References: Sensory Evaluation Techniques, Civille, 4th Ed.(pg 84), 3rd Ed.(pg 79), 2nd Ed.(pg 74).
ASTM E2139-05 Standard Test Method for Same-Different Test


Updated Summer 2017

Back to top
View user's profile
Display posts from previous:   
Post new topic   Reply to topic    Sensory Computer Systems Forums Index -> Test Design All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group