My Avent breast pump leaks where it connects to the bottle

Make sure you hold the pump upright; as if you bend it milk will leak from the joints where the pump and bottle connect.