Is It Reasonable for Saaty's Consistency Test in the Pairwise Comparison Method?

In the pairwise comparison method, Saaty's consistency test is performed to ensure that the decision maker is being neither random nor illogical in his or her pairwise comparisons. Saaty suggested that the consistency ratio (CR) should be less than or equal to 0.1. Some researchers argued that the cut-off rule should be a function of the matrix size. In this paper, by simulation analysis, we find that, as the matrix size increases, the percent of the matrices with acceptable consistency (CR les 0.1) decreases dramatically, however, on the other hand, there will be more and more contradictory judgements in these sufficiently consistent matrices. This paradox shows that it is impossible to find some proper critical values of CR for different matrix sizes. Thus we argue that Saaty's consistency test could be unreasonable.