Sensory Computer Systems Forums Index Sensory Computer Systems Forums
Welcome to the Sensory Computer Systems Online Forums!
 
 FAQFAQ   SearchSearch   RegisterRegister 
 ProfileProfile   Log inLog in   
   Sensory Computer Systems Home   

ALPHA=0.1 vs ALPHA=0.05

 
Post new topic   Reply to topic    Sensory Computer Systems Forums Index -> Reporting and Analysis
View previous topic :: View next topic  
Author Message
ForumAdmin
Site Admin


Joined: May 20 2005
Posts: 120
Location: New Jersey

PostPosted: Fri May 30, 2014 10:04 am    Post subject: ALPHA=0.1 vs ALPHA=0.05 Reply with quote

Interesting Q&A recently:

TUKEY ALPHA=0.1 vs TUKEY ALPHA=0.05, same exact data set, complete & balanced block of data.

Is there a valid explanation as to why the mean separation letters are different? I think it’s because ALPHA @ 90 (lower confidence level) will detect more significance amongst the individual samples. Can you confirm?
Thanks!

Code:

PROC GLM;
    CLASS JUDGE SAMPLE;
    MODEL X1-X58 = JUDGE SAMPLE;
    MEANS SAMPLE / LINES TUKEY ALPHA=0.1;
RUN;

                               The GLM Procedure

                  Tukey's Studentized Range (HSD) Test for X13

NOTE: This test controls the Type I experimentwise error rate, but it generally
                  has a higher Type II error rate than REGWQ.


                  Alpha                                    0.1
                  Error Degrees of Freedom                 210
                  Error Mean Square                   1.826325
                  Critical Value of Studentized Range  2.91834
                  Minimum Significant Difference        0.3831


           Means with the same letter are not significantly different.

 
           Tukey Grouping          Mean      N    SAMPLE

                        A        5.0283    106    1     
                        A                               
                        A        4.9811    106    2     
                                                       
                        B        4.5755    106    3   




PROC GLM;
    CLASS JUDGE SAMPLE;
    MODEL X1-X58 = JUDGE SAMPLE;
    MEANS SAMPLE / LINES TUKEY ALPHA=0.05;
RUN;


                               The GLM Procedure

                  Tukey's Studentized Range (HSD) Test for X13

NOTE: This test controls the Type I experimentwise error rate, but it generally
                  has a higher Type II error rate than REGWQ.


                  Alpha                                   0.05
                  Error Degrees of Freedom                 210
                  Error Mean Square                   1.826325
                  Critical Value of Studentized Range  3.33818
                  Minimum Significant Difference        0.4382


          Means with the same letter are not significantly different.

 
             Tukey Grouping          Mean      N    SAMPLE

                          A        5.0283    106    1     
                          A                               
                     B    A        4.9811    106    2     
                     B                                   
                     B             4.5755    106    3   



Last edited by ForumAdmin on Mon Jun 02, 2014 10:57 am; edited 2 times in total
Back to top
View user's profile
ForumAdmin
Site Admin


Joined: May 20 2005
Posts: 120
Location: New Jersey

PostPosted: Fri May 30, 2014 10:07 am    Post subject: Answer Reply with quote


Below is the answer from a leading Statistician in our industry.


You are exactly correct. As you can see in the output, the Minimum Significant Difference at 90% confidence is 0.3831 and the Minimum Significant Difference at 95% confidence is 0.4382. So the sample means need to be farther apart to conclude that they are significantly different at the 95% confidence level than they need to be at the 90% confidence level.

However, your client might be asking a slightly different question. When you round the samples means to a reasonable number of decimal places (1), you get:

Code:
                                90%             95%
Sample 1:            5.0          a              A
Sample 2:            5.0          a              AB
Sample 3:            4.6          b              B


How can Sample 1 be significantly higher than Sample 3 at the 95% level but Sample 2 (with the same average rating) not be? If you round the Minimum Significant Difference values for 90% and 95%, you will see that they both round to 0.4, so both sets of annotations should be A, A, B – that is, Samples 1 and 2 are significantly higher than Sample 3 at both the 90% and the 95% levels.

Back to top
View user's profile
Display posts from previous:   
Post new topic   Reply to topic    Sensory Computer Systems Forums Index -> Reporting and Analysis All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group