Comets are thought to orbit the sun in either the Oort cloud or Kuiper belt. When another star passes by the solar system, its gravity pushes the Oort cloud and/or Kuiper belt and causes comets to descend toward the sun in a highly elliptical orbit with the sun at one focus of the ellipse. Comets can have short period orbits (less than 200 years such as Halley's comet) or long period orbits (greater than 200 years such as comet Hale-Bopp).
As the comet passes within six AUs of the sun, the ice begins to go directly from the solid to the gas state (sublimation) much like the way fog is formed. When the ice sublimes, the gas and dust particles flow away from the sun to form the comet's tail.