2*2 Table Simulation

Questions to be answered before the simulation are not yet implemented in this test version.


Begin by answering the questions, even if you have to guess. The first time you answer the questions you will not be told whether you are correct or not.


Once you have answered all the questions, answer them again using the simulation to help you. This time you will get feedback about each individual answer.


General Instructions


The Chi Square Test is an approximate not an exact test. This simulation allows you to examine the accuracy and power of the Chi Square Test in a variety of situations. The significance of the difference between the proportion who succeed in Condition 1 and the proportion who succeed in Condition 2 is tested.


With the default parameters, the probability of success is the same (0.60) in both conditions so the null hypothesis is true. The default sample size is 10 per condition and the default significance level is 0.05.


Some authors have suggested that a correction called the “Yates Correction” be done whenever the expected frequency of any cell is below five. A test without this correction and one with the correction (if a cell has an expected frequency bellow five) is conducted for each simulated experiment.


If you click the “Simulate 1″ button, one simulated experiment will be conducted. The observed abd expected frequencies are presented as well as the Chi Square Test with and without the Yates correction. A tally of the number of times the tests were significant is shown below.


If you click the “Simulate 1000″ (or 5000) button then 1000 (or 5000) simulated experiments are conducted and the numbers of significant and non-significant tests are shown.


Step By Step Instructions


1. Using the default values, click on the “Simulate 1″ button. Note the number of successes and failures in each of the two conditions. Note the value of the Chi Square with and without the Yate’s correction. Was either significant? Check the lower sectioon that shows the count of the number significant. Most likely, it will show one nonsignificant.


2. Test whether the Type I error rate is close to the nominal significance level of 0.05. Do this by clicking the “Simulate 5000″ button several times. Compare the proportion significant to 0.05. Look at the results both when the Yates correction is never used and when it is used when an expected cell frequency is less than 5. Is the test conservative (proportion significant < 0.05) or is it liberal (proportion significant > 0.05)


3. Redo the previous simulations when the probability of success is .50 for each condition. Are the results similar? Try making one of the sample sizes 10 and the other one 6.


4. Try to find a set of parameters such that the proportion significant is greater than 0.06. (Make sure the null hypothesis is always true — that the probability of success is the same for both conditions.) Could you find such a set of parameters? Are there many circumstances in which the test is that liberal? Do you think the Yates correction is a good procedure?


5. Now consider cases in which the probability of success is different for the two conditions. Here the null hypothesis is false so the higher the proportion significant the better. What is the effect of using the Yates correction on rejecting a false null hypothesis?


Summary


The Chi Square test is generally a conservative test. The probability of a Type I error is generally below the significance level. There are situations when it is higher but it is rarely above 0.06 when the 0.05 level is used. The Fisher Exact Test tends to be very conservative and is generally not a good idea.