In more detail
The sample rate is how many times per second a digital audio signal is measured, in Hertz (Hz). CDs use 44,100 Hz and video usually 48,000 Hz. The highest frequency that can be captured is half the sample rate (the Nyquist limit), 22,050 Hz at 44.1 kHz. Frequencies above that limit are not just lost: they fold back down as false lower tones (aliasing), which is why converters filter them out first. A file whose content stops far below the limit may have been converted from a lower-quality source, a useful forensic clue.
Why it matters
Mixing rates without conversion makes audio play too fast or too slow, and converting between them is lossy in practice. Human hearing reaches roughly 20 kHz, so 44.1 kHz covers it with a little room. For analysis, the rate decides which frequencies exist at all: a "high-resolution" file showing nothing above 16 kHz is probably an upsampled lossy file.
Example
CD audio is 44,100 samples per second, 16 bits (2 bytes) and 2 channels, so one second takes 44,100 x 2 x 2 = 176,400 bytes, and three minutes take 31,752,000 bytes, about 31.8 MB, uncompressed. A 30 kHz tone sampled at 44.1 kHz exceeds the 22,050 Hz limit and aliases to 44,100 - 30,000 = 14,100 Hz. At 48 kHz the limit is 24,000 Hz. See the spectrogram view in the audio forensics tool.
Common confusion
Sample rate is not bit rate. The sample rate counts measurements per second, the bit depth sizes each one, and the bit rate of a compressed file (for example 128 kbit/s) is a separate figure.