Within-frequency coupling (WFC) and cross-frequency coupling (CFC) in brain networks reflect neural synchronization within the same frequency band and cross-band oscillatory interactions, respectively. Their synergy provides a comprehensive understanding of neural mechanisms underlying cognitive states such as emotion. However, existing multi-channel EEG studies often analyze WFC or CFC separately, failing to fully leverage their complementary properties. This study proposes a dual-branch graph neural network (DB-GNN) to jointly identify within- and cross-frequency coupled brain networks. Firstly, DBGNN leverages its unique dual-branch learning architecture to efficiently mine global collaborative information and local cross-frequency and within-frequency coupling information. Secondly, to more fully perceive the global information of cross-frequency and within-frequency coupling, the global perception branch of DB-GNN adopts a Transformer architecture. To prevent overfitting of the Transformer architecture, this study integrates prior within- and cross-frequency coupling information into the Transformer inference process, thereby enhancing the generalization capability of DB-GNN. Finally, a multi-scale graph contrastive learning regularization term is introduced to constrain the global and local perception branches of DB-GNN at both graph-level and node-level, enhancing its joint perception ability and further improving its generalization performance. Experimental validation on the emotion recognition dataset shows that DB-GNN achieves a testing accuracy of 97.88% and an F1- score of 97.87%, reaching the state-of-the-art performance.