When a quantum system evolves so that it returns to its initial state, it will acquire a geometric phase acting as a memory of the transformation of a physical system, which has been experimentally measured in a variety of physical systems. In optics, the most prominent example is the Pancharatnam-Berry (PB) phase. Recent technological advances in phase and polarization structure have led to the discovery of high-order PB phases with structured light fields. The study on the high-order PB phase is limited in the context of elementary quantum states of light, especially in the case of photon number states. Here, we experimentally investigate the differences of high-order PB phases between single-photon and N00N states. Our results show that the PB phase, like the dynamic phase, can also be doubled under two-photon states, which can greatly improve the phase sensitivity for greater $N$ in N00N states and high-order structured photons. This may show some implications for quantum precision measurement and quantum state engineering based on geometric phase.