Physical non-invasive security has become crucial for cryptographic modules, which are widely used in pervasive computing. International security evaluation standards, such as U.S. Federal Information Processing Standard (FIPS) 140-3 and Common Criteria (CC) part 3 have added special requirements addressing physical non-invasive security. However, these evaluation standards lack of quantitative metrics to explicitly guide the design and measurement. This paper proposes practice-oriented quantitative evaluation metrics, in which the distinguishability between the key predictions is measured under statistical significance tests. Significant distinguishability between the most possible two key candidates suggests high success rates of the right key prediction, thus indicates a low security degree. The quantitative evaluation results provide high accountability of security performance. The accordance with FIPS 140-3 makes the proposed evaluation metrics a valuable complement to these widely adopted standards. Case studies on various smart cards demonstrate that the proposed evaluation metrics are accurate and feasible.