A hole is defined as a chunk of memory not in current use. Holes develop as jobs are loaded, run and then finish. Reclaimed memory from dead jobs often gives rise to many small holes. Operating systems try to merge adjacent holes in order to find the largest chunk of available memory possible for new jobs. By the way, memory itself doesn't look any different between holes and jobs. This is only a fiction in the "mind" of the operating system, which keeps track of which regions of memory belong to its jobs.

When there are many holes and not enough memory in any one place to load a new job that wants to come in, we say that memory is fragmented, as if the chunk of available memory is broken into small, unusable pieces. Some systems perform compaction, which is copying jobs around so that all the active jobs are put at one end of the memory and all the holes coalesce at the other end, making one big hole that is usable.