What is a Bypass Capacitor?

Bypass capacitor : A capacitor employed to conduct an alternating current around a component or group of components. 
Often the AC is removed from an AC/DC mixture, the DC being free to pass through the bypassed component.

Essentially they remove any power fluctuations from an AC circuit.

Generally a 0.1uF monolithic ceramic capacitor is used. For more information, see The Basics - Bypass Capacitors (The Encoder - The Newsletter of the Seattle Robotics Society)