In more detail
Latency depends mostly on distance (light in fibre travels roughly 200 km per millisecond), the number of network hops and how busy the equipment is. It is usually quoted as round-trip time (RTT), there and back, which is what ping measures. Each new HTTPS connection needs several round trips for the TCP and TLS handshakes before the first byte of content arrives, so latency adds up quickly. Variation in latency from one packet to the next is called jitter.
Why it matters
Latency matters most for calls, games and anything interactive. A server on another continent adds tens of milliseconds to every request, which a CDN or a nearer hosting region removes. Latency that jumps when someone starts a big upload or download points to an overloaded router queue (bufferbloat), not a slow line.
Example
London to New York is roughly 5,600 km. Light in fibre needs about 5,600 ÷ 200 = 28 ms one way, so no connection between them can have a round trip below about 56 ms; real cable routes are longer, so measured pings are higher.
Common confusion
Latency is not Bandwidth: a satellite link can have high bandwidth and high latency, and a faster plan does not shorten the distance.