Frederik Cabana scored the only goal for the Phantoms (15-24-2-3) as they lost their fourth straight.
Goalie Martin Houle allowed six goals on 21 shots before being replaced by Michael Leighton at the start the third period.
Tomas Fleischmann had a goal and an assist for Hershey (29-8-3-5). Goaltender Davis Parley made 21 saves to win his AHL debut. Matt Hendricks and Dave Steckel also scored for the Bears.