Chicago sunset/sunrise from http://www.sciquill.com/analemma/ 
This is due to the changing speed of the orbit of Earth. Earth's orbits is slightly oblong. Therefore on the longer parts the planet picks up speed. Which means that although Earth made a full rotation, since it is further in its orbit it is still seeing the sun.
This happens twice, on both longer stretches of the orbit. However the effect on one is more pronounced than the other. This is because in one leg Earth is speeding away from the sun, which slows it a bit, and in the other it is coming toward the sun.
The longest and shortest days are on both points of the oblong circle but the later sunset is before that point.
It's because we measure time by a clock rather than by a sundial. That the latest sunrise and earliest sunset don't occur on the winter solstice pretty much (but not completely) vanishes when one looks at time as measured by a sundial.
From early December to mid January, a solar day (the time from one local noon to the next) is 24 or more seconds longer than is a day as measured by a clock, with the discrepancy peaking at about 30 seconds on December 25 or so. While the timing of sunrise and sunset is more or less symmetrical about local solar noon, this discrepancy between the length of a day as measured by a sundial versus the 24 hour day as measured a clock means that in the northern hemisphere, the earliest sunset (as measured by a clock) occurs sometime before the winter solstice and that the latest sunrise occurs sometime after the winter solstice. (How much depends on latitude.)
This discrepancy between solar time and clock time is a consequence of the Earth's axial tilt and the eccentricity of the Earth's orbit. How these two two effects combine is called the equation of time. The axial tilt results in a sine wave with a period of six months and zero crossings at the solstices and equinoxes. The eccentricity results in a sine wave with a period of a year (plus a tiny bit) with zero crossings at perihelion (when the Earth is closest to the Sun) and aphelion (furthest from the Sun).
Right now, perihelion occurs a couple of weeks after the northern hemisphere winter solstice. (This is just coincidental. In 5000 years or so, perihelion will occur around the time of the spring equinox, and in 10000 years, around the time of the summer solstice.) The difference between length of day as measured by a sundial vs a clock currently reaches its peak in late December because of that coincidence.