What is Context Window in an LLM?

One of the biggest misconceptions about Large Language Models (LLMs) is that they have memory like humans.