Graph network analysis (GNA) showed a remarkable role for understanding brain functions, but its application is mainly narrowed to fMRI research. Connectivity analysis (CA) is introduced as a signal-to-graph mapping in a time-causality framework. In this paper, we investigate the application of GNA/CA in fNIRS. To solve the inherent challenges of using CA, we also propose a novel metric: a maximum cross-lag magnitude (MCLM) that efficiently extracts major causality information. We tested MCLM in four types of cognitive activities (mental arithmetic, motor imagery, word generation, and brain workload) from 55 participants. CA/MCLM showed a compelling modeling capacity and revealed unexpected cross-subject network patterns. We found that motion imagery and mental arithmetic share a background network structure, and that the right prefrontal cortex, in AFp8, is an invariable destination for information flows in every stimuli and participant. Therefore, CA/MCLM-fNIRS showed potential for its use along with fMRI in clinical studies.