In more detail
The statistic is χ² = Σ (O − E)² / E, summed over every category, where O is the observed count and E the count expected under the hypothesis. It is compared with a chi-square distribution whose degrees of freedom are k − 1 for a goodness-of-fit test with k categories, or (rows − 1) × (columns − 1) for a test of independence in a table. A large χ² gives a small p-value. The usual rule of thumb is an expected count of at least 5 in every cell.
Why it matters
It answers questions like "is this die fair?" or "does preference depend on age group?". The test works on raw counts: feeding it percentages gives a meaningless result, and doubling the sample with the same proportions doubles χ². A significant result says the counts differ from the hypothesis, not where; look at each cell's contribution to find out.
Example
Sixty rolls of a die, 10 expected per face:
Face 1 2 3 4 5 6
Observed 5 8 9 8 10 20
Expected 10 10 10 10 10 10
(O−E)²/E 2.5 0.4 0.1 0.4 0.0 10.0
χ² = 13.4 with 5 degrees of freedom, p = 0.0199. The 5% critical value for 5 degrees of freedom is 11.07, so the die looks unfair, and almost all of the evidence comes from face 6.
Common confusion
A t-test compares the means of measurements such as weights or times; a chi-square test compares counts in categories.