In more detail
Firewalls run in hardware at the edge of a network or as software on a single computer. Modern firewalls are stateful: once an outgoing connection is allowed, its replies are let back in automatically, so rules usually only need to describe new connections. A good starting policy is to deny everything by default and open only the ports a service needs, ideally only to the addresses that need them. Web application firewalls go further and inspect HTTP requests themselves.
Why it matters
A database or admin panel left reachable from the internet is a common cause of breaches. The symptoms also differ: a port silently dropped by a firewall makes a client wait until "connection timed out", while a rejected or closed port fails at once with "connection refused".
Example
A server that accepts HTTPS from anyone and SSH only from an office network, with ufw on Linux:
ufw default deny incoming
ufw default allow outgoing
ufw allow 443/tcp
ufw allow from 198.51.100.0/24 to any port 22 proto tcp
See Common Network Ports Cheat Sheet for which service uses which port.