Research Article  Open Access
Paul Thaddeus Kazibudzki, "The Quality of Ranking during Simulated Pairwise Judgments for Examined Approximation Procedures", Modelling and Simulation in Engineering, vol. 2019, Article ID 1683143, 13 pages, 2019. https://doi.org/10.1155/2019/1683143
The Quality of Ranking during Simulated Pairwise Judgments for Examined Approximation Procedures
Abstract
An overview of current debates and contemporary research devoted to the modeling of decisionmaking processes and their facilitation directs attention to the quality of priority ratios estimation through pairwise comparisons. At the core of the process are various approximation procedures for a pairwise comparison matrix which, in a sense, reflects preferences of decisionmakers. Certainly, when judgments regarding these preferences are perfectly consistent (cardinally transitive), all approximation procedures coincide and the quality of the prioritization process is exemplary. However, human judgments are very rarely consistent, and thus the quality of priority ratios estimation may significantly vary. Obviously, the range of these variations depends on the applied approximation procedure for a pairwise comparison matrix. Although there are many approximation procedures which can be applied in the prioritization process, it has been promoted for many decades that only one should be applied and no others qualify. This paper suggests this opinion is a fallacy. Research results argue that a genuine, commonly applied approximation procedure for a pairwise comparison matrix may deteriorate the quality of priority ratios estimation. Thus, a number of solutions are also proposed which can improve the process of priority ratios estimation. In order to provide credible and high quality results, the problem is studied via a properly designed and coded seminal simulation algorithm, executed in Wolfram Mathematica 8.0.
1. Introduction
The world is a complex system of interacting elements. Thus, people must be supported in expressing their preferences, ordering their priorities, and making tradeoffs to be able to serve the greatest common interest [1, 2]. That is why many support methodologies that have been elaborated to make this process efficient and more credible. Indeed, numerous psychological experiments [3], including the wellknown Miller study [4], put forth the notion that humans are not capable of accurately dealing with more than about seven (±2) items at a time.
Humans learn about anything by two means. The first involves examining and studying some phenomenon from the perspective of its various properties and then synthesizing findings and drawing conclusions. The second entails studying some phenomenon in relation to other similar phenomena and correlating them by making comparisons [5]. The latter method leads directly to the essence of the matter, i.e., judgments regarding a phenomenon.
Judgments can be relative and absolute. A relative judgment deals with the identification of some relation between two stimuli both present to the observer [6]. It is stated that humans can make much better relative than absolute judgments [7].
For detailed knowledge, the mind structures complex reality into its constituent parts and in turn into their elements. The number of parts usually ranges between five and nine. By breaking down reality into homogeneous clusters and subdividing those into smaller units, humans can integrate large amounts of information into the structure of a problem and form a more comprehensive picture of a whole system. Abstractly, this process entails structuring a system into a hierarchy which is a complex reality model. Thus, a hierarchy constitutes a structure of multiple levels where the first level is the objective followed successively by levels of factors, criteria, subcriteria, and so on, down to a bottom level of alternatives. The goal of this hierarchy is to evaluate the influence of higher level elements on those of a lower level or alternatively and the contribution of elements in the lower level to the importance or fulfillment of the elements in the levels above. In this context, these latter elements serve as criteria and are called properties.
Generally, a hierarchy can be functional or structural. The latter closely relates to the way a human brain analyzes complexity by breaking down the objects perceived by the senses into clusters, subclusters, and so on. Thus, in structural hierarchies, complex systems are structured into their constituent parts in descending order according to their structural properties. In contrast, in functional hierarchies, complex systems are reduced to their constituent parts according to their essential relationships.
A large number of hierarchies in application are available in literature [8]. It has been argued that the hierarchical classification is the most powerful method applied by the human mind during intellectual reasoning and ordering of information and/or observations.
There are multiple criteria, decisionmaking support methodology which applies the hierarchical classification described above, i.e., the analytic hierarchy process (AHP). It was developed by Thomas Saaty [9], and although it is a very popular and widely implemented theory of choice, it is also controversial. It should be mentioned that the AHP—as a multicriteria decisionmaking support system—has received criticism [10–12] usually concerning one of its most controversial aspects, i.e., the phenomenon of alternative rank reversal which will be addressed in this paper.
The conventional procedure of priority ranking in AHP is grounded on the welldefined mathematical structure of consistent matrices and their associated right principal eigenvector’s (REV) ability to generate true or approximate values of weights. However, the REV method which the AHP applies constitutes the crucial reason of its criticism which is the main point of reference for this research paper.
A number of authors have noticed that the REV does not optimize any objective function; what entails its results cannot be interpreted in statistical or optimization fashions [13–15]. In particular, solutions obtained with the application of REV cannot be compared to others received from the application of many commonly known methods [15–17]. Besides, unlike many optimization models, REV does not allow decisionmakers to introduce additional constraints that may be deemed necessarily according to particular points of view [10, 18–20]. Moreover, REV is supposed to operate only with reciprocal pairwise comparison matrix (PCM)—at least as long as Saaty’s consistency index is applied—which entails a limited range of applications and an increase of estimation errors. As a result, in practice, the reciprocity of PCM is a very popular requirement, although many authors argue that it is an artificial condition which impoverish the PCM about information concerning the unknown priority vector that otherwise could have been revealed [14, 21–24]. Furthermore, it was proved that the REV does not satisfy the condition of order preservation (COP) [10] which constituted, for a very long time, the barrier for acceptance of other existing methods, in particular those applying optimization procedures, which were rejected for producing nonunique answers.
Facing the above problems, it seems that debates concerning AHP issues will continue, maintaining the rather high popularity of the AHP, in this way making it prone to validation and valuation from the perspective of its applicability.
Undeniably, sustainable growth of the AHP applications’ number (e.g., [2, 25–30]) entails a necessity for research dedicated to problematic issues of the AHP. Certainly, they should be examined, and the questions they provoke should be addressed.
The examination in this paper considers two very crucial issues within the AHP as follows:(1)Is the right principal eigenvector, as the primary approximation procedure within the AHP, necessary for the priorities ratios estimation during the pairwise comparisons process?(2)Is the reciprocity of the pairwise comparison matrix a condition which leads to improvement or deterioration of the priority ratios estimation quality?
2. Background
The AHP seems to be the most widely used multiple criteria decisionmaking approach in the world today. Actual applications in which the AHP results were accepted and used by competent decisionmakers and can be found in [1, 5, 7, 22, 31, 32]. Regardless of AHP popularity, the genuine methodology is also undeniably the most valuated, developed, and perfected contemporary methodology, for example, [33–38].
The AHP allows decisionmakers to set priorities and make choices on the basis of their objectives, knowledge, and experience in a way that is consistent with their intuitive thought process. AHP has substantial theoretical and empirical support encompassing the study of the human judgmental process by cognitive psychologists. It uses the hierarchical structure of the decision problem, pairwise relative comparisons of the elements in the hierarchy, and a series of redundant judgments. This approach reduces errors and provides a measure of judgment consistency. The process permits accurate priorities to be derived from verbal judgments even though the words themselves may not be very explicit. Thus, it is possible to use words for comparing qualitative factors and then deriving ratioscale priorities that can be combined with quantitative factors.
To make a proposed solution possible, i.e., derive ratioscale priorities on the basis of verbal judgments, a scale is utilized to evaluate the preferences for each pair of items. Apparently, the most popular is Saaty’s numerical scale which comprises the integers from one (equivalent to a verbal judgment—“equally preferred”) to nine (equivalent to a verbal judgment—“extremely preferred”) and their reciprocals. However, in conventional AHP applications, it may be desirable to utilize other scales also, i.e., a geometric or any other numerical scale. The former usually consists of the numbers computed in accordance with the formula 2^{n/2}, where n comprises the integers from minus eight to eight. The latter may, for example, involve arbitrary integers from one to n and their reciprocals.
The first step in using AHP is to develop a hierarchy by breaking a problem down into its primary components. The basic AHP model includes a goal (a statement of the overall objective), criteria (the factors to be considered in reaching the ultimate decision), and alternatives (feasible alternatives that are available to achieve the said ultimate goal). Although the most common and basic AHP structure consists of a goalcriteriaalternatives sequence (Figure 1), AHP can easily support more complex hierarchies. A variety of basic hierarchical structures include the following:(i)Goal, criteria, subcriteria, scenarios, and alternatives(ii)Goal, players, criteria, subcriteria, and alternatives(iii)Goal, criteria, levels of intensities, and many alternatives
The conventional procedure of priority ranking in AHP is grounded on the welldefined mathematical structure of consistent matrices and their associated righteigenvector’s ability to generate true or approximate values of weights.
Oskar Perron, a German mathematician, proved in 1907 that, if A = (), > 0, where i, j = 1, … , n, and then A has a simple positive eigenvalue λ_{max} called the principal eigenvalue of A and λ_{max} > λ_{k} for the remaining eigenvalues of A. Furthermore, the principal eigenvector that is a solution of has . Thus, the conventional concept of AHP can be presented as follows:
If the relative weights of a set of activities are known, they can be expressed as a pairwise comparison matrix (PCM) as shown above . Now, knowing but not w (vector of priority ratios), Perron’s theorem can be applied to solve this problem for w. The solution leads to n unique values for λ, with an associated vector w for each of the n values.
PCMs in the AHP reflect relative weights of considered activities (criteria, scenarios, players, and alternatives), so the matrix has a special form. Each subsequent row of that matrix is a constant multiple of its first row. In this case, a matrix has only one nonzero eigenvalue, and since the sum of the eigenvalues of a positive matrix is equal to the sum of its diagonal elements, the only nonzero eigenvalue in such a case equals the size of the matrix and can be denoted as λ_{max} = n.
The norm of the vector w can be written as w = e^{T}w where e = [1, 1, … , 1]^{T} and w can be normalized by dividing it by its norm. For uniqueness, w is referred to in its normalized form.
Theorem 1. A positive n by n matrix has the ratio form , i, j = 1, … , n, if, and only if, it is consistent.
Theorem 2. The matrix of ratios is consistent if and only if n is its principal eigenvalue and = n. Further, w > 0 is unique to within a multiplicative constant.
Definition 1. If the elements of a matrix satisfy the condition for all i, j = 1,…, n then the matrix is called reciprocal.
Definition 2. The matrix is called ordinal transitive if the following conditions hold: (A) if for any i = 1, … , n, an element a_{ij} is not less than an element a_{ik} then for i = 1, … , n, and (B) if for any i = 1, … , n, an element a_{ji} is not less than an element a_{ki} then for i = 1, … , n.
Definition 3. If the elements of a matrix satisfy the condition for all , and the matrix is reciprocal, then it is called consistent or cardinal transitive.
Certainly, in real conditions when AHP is utilized, there is not an which would reflect weights given by the vector of priority ratios. As was stated earlier, the human mind is not a reliable measurement device. Assignments, such as “Compare—applying a given ratio scale—your feelings concerning alternative 1 versus alternative 2”, do not produce accurate outcomes. Thus, is not established but only its estimate A(x) contains intuitive judgments, more or less close to in accordance with experience, skills, specific knowledge, personal taste, and even temporary mood or overall disposition. In such cases, consistency property does not hold, and the relation between elements of and can be expressed as follows:where e_{ij} is a perturbation factor fluctuating near unity. In the statistical approach, e_{ij} reflects a realization of a random variable with a given probability distribution.
It has been shown that, for any matrix, small perturbations in the entries imply similar perturbations in the eigenvalues which is why in the estimation of a true priority vector w, conventional AHP utilizes Perron’s theorem. The solution of the matrix equation Aw = λ_{max}w gives us w as the right principal eigenvector (REV) associated with λ_{max}. In practice, the REV solution is obtained by raising the matrix A(x) to a sufficiently large power; then, the rows of A(x) are totaled, and the resulting vector is normalized to produce w.
The genuine concept of the AHP undeniably attracts attention, and thus, it evolves (see, e.g., [33–38]). At the same time, it is argued that so long as inconsistency in pairwise judgments is tolerated, the REV is the basic theoretical concept for deriving priority ratios and no other approximation procedure (AP) for priority ratios derivation qualifies. Concurrently, during the last three decades, numerous alternative APs have been proposed: beginning with the most popular, the geometric mean procedure [15, 39, 40], and other methods based on constrained optimization models, e.g., [20, 41, 42], including least squares method [24, 43], and various versions of goal programming methods, e.g., [13, 44–46], through methods based on statistical concepts, e.g., [21, 23, 47], ending with methods based on fuzzy preference description, e.g., [48], and heuristic algorithms, e.g., [49].
3. Facet of the Problem
It has been promoted that the REV approximation procedure (AP) is necessary and sufficient to uniquely establish the ratioscale rank order inherent in inconsistent pairwise comparison judgments [40]. However, there are alternative APs devised to cope with this problem. Many of them are optimization based and seek a vector , as a solution of the minimization problem given by the formulasubject to some assigned constraints such as positive coefficients and normalization condition. Because the distance function D measures an interval between matrices and , differing definitions lead to various prioritization concepts and prioritization results. As an example, Choo and Wedley [16] describes and compares eighteen APs for ranking purposes although some authors suggest there are only fifteen of them that are distinct. Furthermore, since the publication of the above article, a few additional procedures have been introduced to the literature, e.g., [50].
Certainly, when the PCM is consistent, all known procedures coincide. However, in reallife situations, as was discussed earlier, human judgments produce inconsistent PCMs. The inconsistency is a natural consequence of human brain dynamics described earlier and also a consequence of the questioning methodology, mistaken entering of judgment values, and scaling procedure, i.e., rounding errors. It seems crucial to emphasize here that even perfectly consistent PCMs are not errorfree only because of rounding errors what can be illustrated on the basis of the following hypothetic example (Example 1).
Example 1. The genuine priority vector w = [7/20, 1/4, 1/4, 3/20] is considered and derived from it, and can be presented as follows:Now it is considered that A(x) produced by a hypothetic decisionmaker (DM), whose judgments are perfectly consistent. Even if it is assumed that the selected DM is very trustworthy and can express judgments very precisely, DM is still somehow limited by the necessity of expressing judgments on a scale (the example utilizes Saaty’s scale). As such, the DM will produce the PCM (A(x)) which is not errorfree because the entries must be, in this case, rounded to the closest values of Saaty’s scale. Since A(x) must be reciprocal (the fundamental requirement of the AHP), the PCM appears as follows:It may be noticed that the above PCM is perfectly consistent, so this construct seems to be exemplary. However, the hypothetical DM, despite best intentions, is burdened with inescapable estimation errors. In the above situation, the priority vector (PV) derived from A(x) by any AP provides the following priority ratios (PRs): x = [2/7, 2/7, 2/7, 1/7] which are not equal to those considered exemplary and w = [7/20, 1/4, 1/4, 3/20]. Obviously, the deviation between those PVs can also be expressed by their mean absolute error (MAE), for instance, established by the following formula:where n is the number of elements within the particular PV. Noticeably, in the above example, MAE equals 1/28. Certainly, this error concerns any AP applied in this case, so it also concerns the REV.
Another exemplary scenario can be considered with the hypothetic case provided earlier (Example 1). Thus, the same genuine PV is reconsidered, w = [7/20, 1/4, 1/4, 3/20] and is derived from that PV as before.
Example 2. However, two PCMs are now considered, i.e., R(x) and A(x) produced by a hypothetical DM, whose judgments are rounded to Saaty’s scale. In the first scenario, entries of are rounded to Saaty’s scale and the entries are made reciprocal (a principal condition for a PCM in the AHP) producingIn the second scenario, only entries of are rounded to Saaty’s scale (nonreciprocal case) producingIt should be noted that R(x) is perfectly consistent and A(x) is not. Tables 1 and 2 present that PVs derived from R(x) and A(x) with application of the REV and two other arbitrarily selected APs, mean absolute errors (MAEs) (Formula (6)), and Spearman rank correlation coefficients (SRCs) among w ∗ (AP) and the genuine w for the case.
 
REV: Right Principal Eigenvector; LUA: Logarithmic Utility Approach; LLSM: Logarithmic Least Squares Method. 
 
REV: Right Principal Eigenvector; LUA: Logarithmic Utility Approach; LLSM: Logarithmic Least Squares Method. 
Surprisingly, a very interesting phenomenon can be noted on the basis of information provided in Tables 1 and 2. Apparent are smaller values of MAEs and perfect correlation of ranks between estimated and genuine PV for nonreciprocal versions of an analyzed PCM. Certainly, this conclusion concerns all analyzed approximation procedures.
Regardless of these facts, when only reciprocal PCMs are taken into consideration, Saaty and Hu [40] provide an example of a situation where variability in ranks does not occur for each individual judgment matrix, but it occurs in the overall ranking of the final alternatives due to the application of different APs and the multicriteria process itself. They argue that only the REV possesses a sound mathematical background directly dealing with the question of inconsistency. Furthermore, as they state, only the REV captures the rank order inherent in the inconsistent data in a unique manner. Thus, these statements will be verified with the application of the Monte Carlo simulations.
For that purpose, apart from the REV, four different APs have been arbitrarily selected which were ranked as the best within AHP methodology [16, 42, 43] (Table 3).

4. Methodology and Research Results
Taking into account the exemplary study of Saaty and Hu [40], it seems that the best way to evaluate the problem is to examine whether different APs are in fact inferior in the estimation of true PVs whose intent is accurate estimation. From that perspective, only computer simulations can illuminate the question, for it is possible to elaborate an algorithm which enables simulation of different kinds of errors which may occur during the process of judgment, and enable assessment which one from the selected APs delivers the better estimates (from a given perspective) of the genuine PV. Firstly, the process will be depicted in Example 3.
Example 3. It is assumed that the hierarchy consists of three levels: goal, criteria, and alternatives. Then, in order to compare the accuracy of estimations obtained by selected APs, different situations related to various sources of the PCM inconsistency are simulated [42, 50]. Primarily, an inconsistency usually results from errors caused by the nature of human judgments—that can be represented as a realization of some random process in accordance with Formula (2)—and technical errors connected with a realization of the comparison procedure, i.e., rounding errors and errors resulting from the forced reciprocity requirement.
In the example provided, only rounding errors (Saaty’s scale) and forced reciprocity are taken into consideration. Let us consider the following ideal model of the AHP framework adopted from [42] with three levels (four criteria and four alternatives):
(1) With respect to the goal:(2) With respect to criteria c1c2:(3) With respect to criteria c3c4:where “dPVc” and “dPVa” denote, respectively, designated priority vector of criteria and designated priority vector of alternatives.
After application of a standard AHP synthesis process, the following result is obtained: dCPV = [0.25, 0.21, 0.23, 0.31]^{T}, where dCPV stands for the designated cumulative priority vector. For the purpose of scenario illustration, each element of the designated PCM is rounded consecutively to Saaty’s numerical scale, and its reciprocity is imposed. Then, on the basis of such PCMs, their respective priority vectors are computed, in the example, with application of the REV, and the CPV (cumulative priority vector) is calculated as earlier.
After all these transformations, the model presents itself as follows:
(1) With respect to the goal:(2) With respect to criteria c1c2:(3) With respect to criteria c3c4:After synthesis, the following result is obtained: CPV_{REV} = [0.2034, 0.2336, 0.2316, 0.3315]^{T}, which do not coincide with dCPV = [0.25, 0.21, 0.23, 0.31]^{T}. Moreover, a rank reversal situation can be noticed: dCPV = {2, 4, 3, 1} versus CPV_{REV} = {4, 2, 3, 1}. Thus, even from that singular perspective, very reasonable and desired is a search for other APs that could at least partially eliminate discrepancies depicted here during this illustrative prioritization process.
Thus, the following simulation algorithm was constructed. Assuming that the decisional problem can be presented in the form of a threelevel hierarchy (goal, criteria, and alternatives; Figure 1). In order to emulate the problem presented in [40], the hypothetical hierarchy is also designed as a four criteria and four alternatives structure, i.e., n = 4 and m = 4. In agreement with these assumptions, it is possible to elaborate and execute the simulation algorithm SA 1 comprising the following steps:
Step 1. Randomly generate a priority vector k = [k_{1}, …, k_{n}]^{T} of assigned size [n × 1] for criteria and related perfect PCM(k) = K(k).
Step 2. Randomly generate n priority vectors a_{n} = [a_{n,1}, …, a_{n,m}] of the assigned size [m × 1] for alternatives under each criterion and related perfect PCMs(a) = A_{n}(a).
Step 3. Compute a total priority vector w of the size [m × 1] applying the following procedure: , where z ∈ {1, 2, 3, … , m – 1, m}.
Step 4. Randomly choose a number e from the assigned interval [α; β] on the basis of assigned probability distribution π.
Step 5. Apply separately Step 5A and Step 5B.
Step 5A. The case of PCM forced reciprocity implementation; replace all elements a_{ij} for i < j of all A_{n}(a) with ea_{ij} and all elements k_{ij} for i < j of K(k) with ek_{ij}.
Step 5B. The case of arbitrary PCM acceptance; replace all elements a_{ij} for i ≠ j of all A_{n}(a) with ea_{ij}, and all elements k_{ij} for i ≠ j of K(k) with ek_{ij}.
Step 6. Apply separately Step 6A and Step 6B.
Step 6A. When Step 5A is performed, round all values of elements a_{ij} for i < j of all A_{n}(a) and all values of elements k_{ij} for i < j of K(k) to the closest values from a considered scale, and then replace all elements a_{ij} for i > j of all A_{n}(a) with 1/a_{ij} and all elements k_{ij} for i > j of K(k) with 1/k_{ij}.
Step 6B. When Step 5B is performed, round all values of elements a_{ij} for i ≠ j of all A_{n}(a), and all values of elements k_{ij} for i ≠ j of K(k) to the closest values from a considered scale.
Step 7. On the basis of all perturbed A_{n}(a) denoted as A_{n}(a)^{∗} and perturbed K(k) denoted as K(k)^{∗}, compute their respective priorities vectors a_{n}^{∗} and k^{∗} with application of assigned approximation procedure (AP), i.e., REV, LUA, SRDM, LLSM, and SNCS.
Step 8. Compute a total priority vector w ∗ (AP) of the size [m × 1] applying the following procedure: , where: z ∈ {1, 2, 3, …, m – 1, m}.
Step 9. Calculate Spearman rank correlation coefficients—SRC_{γ,χ} (w ∗ (AP), w) between all w ∗ (AP) and w, as welldesignated estimation precision characteristics, i.e., mean relative errors:along with mean relative ratios:
Step 10. Repeat Steps 4 to 9, χ times, where χ denotes a size of the sample.
Step 11. Repeat Steps 1 to 9, γ times, where γ denotes a number of considered AHP models.
Step 12. Return arithmetic average values of all SRC_{γ,χ}(w ∗ (AP), w), RE_{γ,χ}(w ∗ (AP), w), and RR_{γ,χ}(w ∗ (AP), w) computed during all runs in Steps 10 and 11, i.e.,where MSRC(w ∗ (AP), w), MRE(w ∗ (AP), w), and MRR(w ∗ (AP), w) denotes mean Spearman rank correlation coefficient, average mean relative error, and average mean relative ratio, respectively.
In the first experiment, the probability distribution π attributed in Step 4 to the perturbation factor e is selected arbitrarily to be the gamma or uniform distribution. These are two of the distribution types most frequently considered in literature for various implementation purposes [24, 43, 50, 51]. Usually recommended are types such as gamma, lognormal, truncated normal, or uniform. Apart from these most popular π, one can find applications of the Couchy, Laplace, or triangle or beta probability distributions (e.g., [52]).
The first simulation scenario also assumes that the perturbation factor e will be drawn from the interval e ∈ [0.01; 1.99] with its expected value EV(e) = 1. The latter condition comes from a reasonable assumption about human judgments, which undeniably intend to be perfect, although they are not perfect permanently.
Furthermore, the number of alternatives and criteria in a single AHP model will be assigned randomly. “Randomly” without any other explicit specification is meant hereafter “uniformly distributed.” All simulation scenarios also assume application of the rounding procedure which in the first scenario operates according to the geometric scale described earlier in this paper.
Finally, the first scenario also takes into account the obligatory assumption in conventional AHP applications, i.e., the PCM reciprocity condition. In such cases, only judgments from the upper triangle of a given PCM are taken into account and those from the lower triangle are replaced by the inverses of the former.
The outcomes—mean characteristics—for 30,000 cases (χ = 15 and γ = 2,000) of the first simulation scenario are presented in Table 4.
It may be noticed from Table 4, that the REV can be undeniably classified as the worst AP from the perspective of PRs derived from ranks established on the basis of three different prioritization quality measures, i.e., MRE, MSRC, and MRR. The best two APs, from the viewpoint of this classification, are LLSM (known also as geometric mean procedure (GM)), and LUA. Certainly, the first scenario experiment was designed only to contrast the results presented in [40]. It is the intention to establish wider and more fundamental relationships among the selected APs.
The second simulation scenario was designed to encompass new assumptions not yet taken into account in literature. First of all, taken into consideration were results obtained not only on the basis of reciprocal PCM but also the simulation outcomes of nonreciprocal PCM. Secondly, it was decided to implement into the simulations new intervals for random errors and apply their new probability distribution. As is known, many simulation analyses presented in literature assume very nonsymmetric intervals for a perturbation factor (considering its influence on the particular element of PCM). For example, consider the interval for the perturbation factor applied in the first simulation scenario, i.e., e ∈ [0.01; 1.99]. Under this assumption, it becomes apparent that if some entry of PCM is modified in plus by the perturbation factor from that particular interval, it is multiplied maximally by 1.99, so if the original entry is 3, the modified value will be around 6. However, if some entry of PCM is modified in minus by the perturbation factor from that particular interval, it may result that some entry will be multiplied by the number 0.01, so in fact the entry will be divided by 100. Thus, in the situation where the original entry is 9, the modified value will be 0.09, which can be rounded to 1/9 on Saaty’s scale. It may be noticed that this modification practically reverses the preference of DM from, for example, extremely preferred A over B, to extremely preferred B over A (applying the Saaty scale).
It is obvious that this very common assumption is imposed by another very crucial and logical assumption which states that the expected value of e in every particular simulation scenario should equal one, i.e., EV(e) = 1. It is quite easy to fulfill that requirement on the basis of an asymmetric interval for the perturbation factor (from the perspective of its influence on a particular element of PCM). However, it is rather a challenge to have this assumption implemented with a symmetric interval for the perturbation factor. That is why commonly applied simulation scenarios minimize the range for the perturbation factor in order to achieve at least the delusion of symmetry for e ∈ [0.5; 1.5]. Nevertheless, that objective has been attained with the present research, yet to be achieved by other researchers. Presently, it seems reasonable to apply symmetric intervals to simulations for the perturbation factor because they better reflect true life situations. Thus, different kinds of probability distributions (PDs) were experimented with, and it was discovered that Fisher–Snedecor PD possesses the feature that can be useful in the present analysis. It occurs that, for n_{1} = 14 and n_{2} = 40 degrees of freedom for one thousand randomly generated numbers on the basis of this PD, their mean equals 1.03617, so it is very close to unity, and these numbers fluctuate from 0.174526 to 5.57826. So, with these assumptions, we have e ∈ [0.174526; 5.57826], which gives a very symmetric distribution for the perturbation factor, and EV(e) ≈ 1. The results of prioritization quality for different selected APs and assumed prioritization quality measures, i.e., MSRC, MRE, and MRR, obtained on the basis of the earlier described simulation scenario, are presented in Table 5.
As can be noticed from Table 5, the REV again is not the dominant AP from the perspective of all simulation scenarios under prescribed frameworks (it takes the third place in the total classification order). Certainly, apparent differences in the PV estimation quality in relation to the selected AP are noticeable for nonreciprocal PCMs. Then, the LUA and SRDM or LLSM dominate over the rest of the selected APs, especially from the perspective of rank correlations which are the crucial issue from the viewpoint of rank preservation phenomena.
This phenomenon will now be addressed further from the perspective of this study’s research objectives (preservation of preferences intensity during the prioritization process). Bana e Costa and Vansnick [10] provide the following definition: for all alternatives A1, A2, A3, and A4, A1 dominates A2 and A3 dominates A4, and the extent to which A1 dominates A2 is greater than the extent of which A3 dominates A4, we have not only and but also for the derived PV.
Thus, the following scenario, provided in [10], is considered. When the PCM is given asfollowing a common linguistic interpretation for AHP, A1 strongly dominates A4 (A1/A4 = 5), and A4 very strongly dominates A5 (A4/A5 = 7). That implies A1/A4 < A4/A5. However, the PV derived from the REV provides [0.4262, 0.2809, 0.1652, 0.1008, 0.0269]^{T} and yields the ratios A1/A4 = 4.218 > A4/A5 = 3.741, which violates the condition of order preservation (COP). On the contrary, the PV derived, e.g., from LUA provides [0.434659, 0.282449, 0.163602, 0.097671, 0.021620]^{T} and yields the ratios A1/A4 = 4.450245 < A4/A5 = 4.517668 which, contrary to REV, satisfy the COP.
 
Note: FRPCM denotes forced reciprocity applied to PCM during simulations. 
 
Note: AHP models drawn randomly (uniformly) for the assigned set of criteria and alternatives. The scenario assumes application of both the perturbation factor drawn with FSnedecor probability for n_{1} = 14 and n_{2} = 40 degrees of freedom and rounding errors associated with a given scale (geometric or Saaty’s). It assumes calculation of performance measures either for reciprocal PCMs (FRPCMs) or nonreciprocal PCMs (NRPCMs). 
5. Research Breakthrough
As was said by the creator of AHP: … “there is a wellknown principle in mathematics that is widely practiced, but seldom enunciated with sufficient forcefulness to impress its importance. A necessary condition that a procedure for solving a problem be a good one is that if it produces desired results, and we perturb the variables of the problem in some small sense, it gives us results that are ‘close’ to the original ones. (…) An extension of this philosophy in problems where order relations between the variables are important is that on small perturbations of the variables, the procedure produces close, order preserving results [[53], p.18].
With the said notion in mind, an effort was undertaken to verify the statement of followers of the REV, boldly spreading the idea that so long as inconsistency is accepted, the REV is the paramount theoretical basis for deriving a scale and no other concepts qualify.
It is a fact that in order to support some theory, one must verify it through many experiments to validate its reliability. On the contrary, one needs only one example showing it does not work in order to abolish its credibility. Thus, numerous examples were provided indicating that the REV concedes to other devised APs to determine alternative rankings. Although data obtained during experiment simulations are unequivocal, scientific verification of their meaning on the basis of the statistical hypothesis testing theory (HTT) will now be carried out.
If MSRC_{AP} and MSRC_{REV,} respectively, are denoted as mean SRC for a selected AP and mean SRC for the REV, their difference significance can be tested using “t” statistics defined by the following formula:where R is the difference between particular MSRCs.
This statistic has a distribution of tstudent with n minus 2 degrees of freedom (df), where n equals the size of the sample.
The following hypothesis was tested.
In order to conform to the example presented in [40], the data gathered in Table 4 were considered. The simulation framework of that case is df = 29,998. Thus, for assumed levels of significance α = 0.01, α = 0.02, or α = 0.03, the critical values of t–student statistics equal consecutively t_{0.01} = 2.326472, t_{0.02} = 2.053838, or t_{0.03} = 1.880865.
In the situation when a level of tested tstudent statistics is higher than its critical value for the assumed level of significance, the hypothesis H_{0} must be rejected in favor of alternative hypothesis H_{1}. In the opposite situation, there are no foundations to reject H_{0}. The selected statistics and their values for the problem evaluation are presented in Table 6.
 
Note: the closest significance level providing the ground to reject a tested hypothesis. 
Clearly, the results of the simulation scenario, designed in accordance with the framework presented in [40], indicate two APs which on the basis of statistical HTT always perform better than the REV, regardless of the preselected PD. It remains to be noted that the performance of selected APs is examined from the perspective of rank preservation phenomena which is reflected in this research by the MSRC between genuine and perturbed PV. It should be evident that the above conclusions, unlike any other before, are the effect of sound statistical reasoning (rigorous significance level) based on the seminal approach toward AHP methodology evaluation grounded on precisely planned and performed simulation study.
In order to develop the concept further, it was decided to expand the simulation program. The results of this endeavor are presented in Table 5. They should be considered as surprising, especially when one realizes that the AP embedded in the AHP is merely placed third in the overall performance ranking. The ranking takes into account not only MSRC, but also MRE and MRR, and the latter is never taken into consideration in previous simulation research. The MRR will now be examined to expand its concept and highlight its innovation.
Lets consider a vector k of values to be estimated, k = [3, 3, 3, 3], and three of its estimates, k_{1} = [2, 4, 2, 4], k_{2} = [2, 2, 2, 2], and k_{3} = [4, 4, 4, 4]. It may be noted that the MREs of all the estimates (given by formula (18)) are the same and equal 1/3. However, MRRs of the estimates (given by formula (19)) are not the same and equal, respectively, MRR_{1}(k, k_{1}) = 1, MRR_{2}(k, k_{2}) = 2/3, and MRR_{3}(k, k_{3}) = 4/3. Obviously, the goal of estimation is both, i.e., to minimize MREs and maintain the MRRs close to unity. This prerequisite is of great importance when one deals with PVs, i.e., vectors normalized to unity, as in the case of AHP. Certainly, one can encounter the following three estimation scenarios.
Scenario 1. Consider a vector w of genuine PRs trying to estimate w = [0.25, 0.25, 0.25, 0.25] and its estimate w_{1} = [0.01, 0.49, 0.05, 0.45]. This scenario gives a rather high MRE of 0.88, which indicates a mean of 88% volatility for estimated PRs in relation to their primary values and MRR = 1.
Scenarios 2 and 3. Consider a vector p of genuine PRs trying to estimate p = [0.1, 0.2, 0.3, 0.4] and its two estimates p_{1} = [0.15, 0.3, 0.25, 0.3] and p_{2} = [0.05, 0.1, 0.35, 0.5]. This situation entails a moderate MRE of 0.35425 for both estimates and two MRRs, i.e., MRR_{1}(p, p_{1}) = 1.145 and RR_{2}(p, p_{2}) = 0.85425, for the second and third scenario, respectively.
Obviously, during the PRs estimation process, it is desirable to avoid situations exemplified by the first and second scenario. Noticeably, they both have something in common. Apart from estimation discrepancies, they lead to rank reversal of the initial priorities (emphasis added).
Turning back to Table 5, having in mind the imposed simulation scenario, with an FSnedecor PD mean value of a perturbation factor EV(e) = 1.03617, we can conclude as follows:(1)The applied measures (MRE, MSRC, and MRR) reflecting the PRs estimation process quality within the simulation framework are always better for nonreciprocal PCMs in relation to their reciprocal equivalents(2)The applied measures of the PRs estimation quality within the simulation framework indicate better estimation results for a relatively higher number of alternatives(3)Both MRE and MRR values indicate that the PRs estimation quality within the simulation framework is better when a geometric scale is implemented instead of Saaty’s scale for preferences expression of DMs (MRR is then more often less than 1.03617, which indicates less risk of rank reversal)(4)Last, but not least, the REV procedure is not a dominating procedure during PRs estimation in the simulated framework of the AHP
6. Conclusions
The objective of the research was to generate answers to the following questions:(1)Is the right principal eigenvector, as the primary approximation procedure within the AHP, necessary for the priorities ratios estimation during the pairwise comparisons process?(2)Is the reciprocity of the pairwise comparison matrix a condition which leads to improvement or deterioration of the priority ratios estimation quality?
The examination involved Monte Carlo simulations which were designed and executed in Wolfram Mathematica Software because only computer simulations provide the opportunity to analyze and evaluate research problems as in this examination. Indeed, only simulations make possible to apply an algorithm which enables evaluation of different kinds of errors which may occur during a process of judgment and enable assessment of selected APs from a given perspective, i.e., which of them delivers better estimates of the genuine PV.
The problem of this research was initially approached from the perspective of the case study presented in [40] and then expanded to other scenarios. The second simulation scenario was designed to encompass new assumptions not yet taken into account by the current literature. First of all, taken into consideration were results obtained not only on the basis of reciprocal PCM but also nonreciprocal PCM simulation outcomes. Secondly, it was decided to implement new intervals for random errors into the simulations and apply their new probability distribution. Commonly, many simulation analyses presented in current literature assume significant nonsymmetric intervals for a perturbation factor (considering its influence on the particular element of PCM). It is obvious that this very common assumption is imposed by another very crucial and logical assumption which states that the expected value of a perturbation factor (e) in every particular simulation scenario should equal one, i.e., EV(e) = 1. It is quite easy to fulfill that requirement on the basis of an asymmetric interval for the perturbation factor (from the perspective of its influence on a particular element of PCM). However, it is rather a challenge to have this assumption implemented with a symmetric interval for the perturbation factor. That is why commonly applied simulation scenarios minimize the range for the perturbation factor in order to achieve at least the illusion of symmetry for e ∈ [0.5; 1.5]. Nevertheless, that objective has been attained with the present research, yet to be achieved by other researchers. Contemporarily, it seems reasonable to apply symmetric intervals to simulations for the perturbation factor because they better reflect real conditions.
Concluding the thorough and seminal investigation which significantly upgrades the methodology of the prioritization process provides the following answers to questions stated in this examination:(1)The REV as the approximation procedure is neither necessary nor sufficient within the priority ratios estimation process through pairwise comparisons. Moreover, this research reveals two approximation procedures which outperform the REV from a perspective of several conditions, including the condition of order preservation.(2)The reciprocity of PCM in the prioritization process is the artificial condition which directly leads to the deterioration of the priority ratios estimation quality.
6.1. Further Research
Certainly, there is a necessity for further research in this area; e.g., other approximation procedures can be taken into consideration for the simulation scenario provided in this paper, perhaps other probability distributions for perturbation factors could be considered and studied, and last but not least, the characteristics of the approximation procedures tested herein may also be examined during research concerning the real condition experiments regarding human behavior dynamics in decisionmaking.
6.2. Proposed
Withhold the PCM reciprocity requirement within the priority ratios estimation process through pairwise comparisons, and consider the replacement of the REV as the approximation procedure within the priority ratios estimation process through pairwise comparisons in favor of the approximation procedure which prevailed over the REV.
To recapitulate in conjunction with other contemporary and seminal research papers, e.g., [2, 34–38, 50, 54, 55], the results of this scientific research enriches the state of knowledge regarding the true value of the priority ratios estimation process through pairwise comparisons which is widely recognized and applied in many MCDM support systems. Hopefully, the results of this authentic, freshly finished examination will improve the quality of human’s prospective choices.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The author declares that they have no conflicts of interest.
References
 N. Bhushan and K. Ria, Strategic Decision Making: Applying the Analytic Hierarchy Process, SpringerVerlag London Limited, London, UK, 2004.
 M. GarcíaMelón, B. PérezGladish, T. GómezNavarro, and P. MendezRodriguez, “Assessing mutual funds’ corporate social responsibility: a multistakeholderAHP based methodology,” Annals of Operations Research, vol. 244, no. 2, pp. 475–503, 2016. View at: Publisher Site  Google Scholar
 J. Martin, Design of ManComputer Dialogues, PrenticeHall, Englewood Cliffs, New York, NY, USA, 1973.
 G. A. Miller, “The magical number seven, plus or minus two: some limits on our capacity for information processing,” Psychological Review, vol. 63, pp. 81–97, 1956. View at: Publisher Site  Google Scholar
 T. L. Saaty, “Decision making with the analytic hierarchy process,” International Journal of Services Sciences, vol. 1, no. 1, pp. 83–98, 2008. View at: Publisher Site  Google Scholar
 A. L. Blumenthal, The Process of Cognition, PrenticeHall, Englewood Cliffs, New York, NY, USA, 1977.
 T. L. Saaty and L. G. Vargas, Decision Making with the Analytic Network Process: Economic, Political, Social and Technological Applications with Benefits, Opportunities, Cost and Risks, Springer, New York, NY, USA, 2006.
 T. L. Saaty, The Hierarchon, RWS Publication, Pittsburgh, PA, USA, 1993.
 T. L. Saaty, “A scaling method for priorities in hierarchical structures,” Journal of Mathematical Psychology, vol. 15, pp. 234–281, 1977. View at: Publisher Site  Google Scholar
 C. A. Bana e Costa and J. C. Vansnick, “A critical analysis of the eigenvalue method used to derive priorities in AHP,” European Journal of Operational Research, vol. 187, pp. 1422–1428, 2008. View at: Publisher Site  Google Scholar
 A. Farkas, “The analysis of the principal eigenvector of pairwise comparison matrices,” Acta Polytechnica Hungarica, vol. 4, no. 2, 2007. View at: Google Scholar
 J. Barzilai, “Measurement and preference function modeling,” International Transactions in Operational Research, vol. 12, pp. 173–183, 2005. View at: Publisher Site  Google Scholar
 N. Bryson, “A goal programming method for generating priority vectors,” Journal of the Operational Research Society, vol. 46, pp. 641–648, 1995. View at: Publisher Site  Google Scholar
 D. V. Budescu, R. Zwick, and A. Rapoport, “Comparison of the analytic hierarchy process and the geometric mean procedure for ratio scaling,” Applied Psychological Measurement, vol. 10, pp. 69–78, 1986. View at: Google Scholar
 G. Crawford and C. A. Williams, “A note on the analysis of subjective judgment matrices,” Journal of Mathematical Psychology, vol. 29, pp. 387–405, 1985. View at: Publisher Site  Google Scholar
 E. U. Choo and W. C. Wedley, “A common framework for deriving preference values from pairwise comparison matrices,” Computers & Operations Research, vol. 31, no. 6, pp. 893–908, 2004. View at: Publisher Site  Google Scholar
 G. B. Crawford, “The geometric mean procedure for estimating the scale of a judgment matrix,” Mathematical Modeling, vol. 9, no. 3–5, pp. 327–334, 1987. View at: Publisher Site  Google Scholar
 V. Belton and A. E. Gear, “On a shortcoming of Saaty’s method of analytic hierarchies,” Omega, vol. 11, no. 3, pp. 228–230, 1983. View at: Publisher Site  Google Scholar
 S. Lipovetsky and M. M. Conklin, “Robust estimation of priorities in the AHP,” European Journal of Operational Research, vol. 137, no. 1, pp. 110–122, 2002. View at: Publisher Site  Google Scholar
 K. F. Lam and E. U. Choo, “Goal programming in preference decomposition,” Journal of the Operational Research Society, vol. 46, no. 2, pp. 205–213, 1995. View at: Publisher Site  Google Scholar
 I. Basak, “Comparison of statistical procedures in analytic hierarchy process using a ranking test,” Mathematical and Computer Modelling, vol. 28, no. 12, pp. 105–118, 1998. View at: Publisher Site  Google Scholar
 W. Ho, “Integrated analytic hierarchy process and its applications–a literature review,” European Journal of Operational Research, vol. 186, no. 1, pp. 211–228, 2008. View at: Publisher Site  Google Scholar
 S. Lipovetsky and A. Tishler, “Interval estimation of priorities in the AHP,” European Journal of Operational Research, vol. 114, no. 1, pp. 153–164, 1997. View at: Publisher Site  Google Scholar
 F. Zahedi, “A simulation study of estimation methods in the analytic hierarchy process,” SocioEconomic Planning Sciences, vol. 20, no. 6, pp. 347–354, 1986. View at: Publisher Site  Google Scholar
 L. Lidinska and J. Jablonsky, “AHP model for performance evaluation of employees in a Czech management consulting company,” Central European Journal of Operations Research, vol. 26, no. 1, pp. 239–258, 2018. View at: Publisher Site  Google Scholar
 R. K. Goyal and S. Kaushal, “Deriving crisp and consistent priorities for fuzzy AHPbased multicriteria systems using nonlinear constrained optimisation,” Fuzzy Optimization and Decision Making, vol. 17, no. 2, pp. 195–209, 2017. View at: Publisher Site  Google Scholar
 J. Krejčí, O. Pavlačka, and Talašová, “A fuzzy extension of Analytic Hierarchy Process based on the constrained fuzzy arithmetic,” Fuzzy Optimization and Decision Making, vol. 16, no. 1, pp. 89–110, 2017. View at: Publisher Site  Google Scholar
 J. Kramulová and J. Jablonský, “AHP model for competitiveness analysis of selected countries,” Central European Journal of Operations Research, vol. 24, no. 2, pp. 335–351, 2016. View at: Publisher Site  Google Scholar
 T. F. Abdelmaguid and W. Elrashidy, “Halting decisions for gas pipeline construction projects using AHP: a case study,” Operational Research, pp. 1–21, 2016. View at: Publisher Site  Google Scholar
 S. T. Ponis, S. P. Gayialis, I. P. Tatsiopoulos, N. A. Panayiotou, D.R. I. Stamatiou, and A. C. Ntalla, “An application of AHP in the development process of a supply chain reference model focusing on demand variability,” Operational Research, vol. 15, no. 3, pp. 337–357, 2015. View at: Publisher Site  Google Scholar
 A. Ishizaka and A. Labib, “Review of the main developments in the analytic hierarchy process,” Expert Systems with Applications, vol. 11, no. 38, pp. 14336–14345, 2011. View at: Publisher Site  Google Scholar
 O. S. Vaidya and S. Kumar, “Analytic hierarchy process: an overview of applications,” European Journal of Operational Research, vol. 169, no. 1, pp. 1–29, 2006. View at: Publisher Site  Google Scholar
 P. T. Kazibudzki, “An examination of performance relations among selected consistency measures for simulated pairwise judgments,” Annals of Operations Research, vol. 244, no. 2, pp. 525–544, 2016. View at: Publisher Site  Google Scholar
 K. Chen, G. Kou, J. M. Tarn, and Y. Song, “Bridging the gap between missing and inconsistent values in eliciting preference from pairwise comparison matrices,” Annals of Operations Research, vol. 235, no. 1, pp. 155–175, 2015. View at: Publisher Site  Google Scholar
 V. Pereira and H. G. Costa, “Nonlinear programming applied to the reduction of inconsistency in the AHP method,” Annals of Operations Research, vol. 229, no. 1, pp. 635–655, 2015. View at: Publisher Site  Google Scholar
 P. Linares, S. Lumbreras, A. Santamaría, and A. Veiga, “How relevant is the lack of reciprocity in pairwise comparisons? An experiment with AHP,” Annals of Operations Research, vol. 245, no. 12, pp. 227–244, 2014. View at: Publisher Site  Google Scholar
 J. M. MorenoJiménez, M. Salvador, P. Gargallo, and A. Altuzarra, “Systemic decision making in AHP: a Bayesian approach,” Annals of Operations Research, vol. 245, no. 12, pp. 261–284, 2014. View at: Publisher Site  Google Scholar
 J. Aguarón, M. T. Escobar, and J. M. MorenoJiménez, “The precise consistency consensus matrix in a local AHPgroup decision making context,” Annals of Operations Research, vol. 245, no. 12, pp. 245–259, 2014. View at: Publisher Site  Google Scholar
 W. D. Cook and M. Kress, “Deriving weights from pairwise comparison ratio matrices: an axiomatic approach,” European Journal of Operational Research, vol. 37, no. 3, pp. 355–362, 1988. View at: Publisher Site  Google Scholar
 T. L. Saaty and G. Hu, “Ranking by eigenvector versus other methods in the analytic hierarchy process,” Applied Mathematics Letters, vol. 11, no. 4, pp. 121–125, 1998. View at: Publisher Site  Google Scholar
 N. V. Hovanov, J. W. Kolari, and M. V. Sokolov, “Deriving weights from general pairwise comparison matrices,” Mathematical Social Sciences, vol. 55, no. 2, pp. 205–220, 2008. View at: Publisher Site  Google Scholar
 P. T. Kazibudzki and A. Z. Grzybowski, “On some advancements within certain multicriteria decision making support methodology,” American Journal of Business and Management, vol. 2, no. 2, pp. 143–154, 2013. View at: Publisher Site  Google Scholar
 C. Lin, “A revised framework for deriving preference values from pairwise comparison matrices,” European Journal of Operational Research, vol. 176, pp. 1145–1150, 2007. View at: Publisher Site  Google Scholar
 A. Z. Grzybowski, “Goal programming approach for deriving priority vectors–some new ideas,” Scientific Research of the Institute of Mathematics and Computer Science, vol. 1, no. 9, pp. 17–27, 2010. View at: Google Scholar
 C.C. Lin, “An enhanced goal programming method for generating priority vectors,” Journal of the Operational Research Society, vol. 57, no. 12, pp. 1491–1496, 2006. View at: Publisher Site  Google Scholar
 J. W. Moy, K. F. Lam, and E. U. Choo, “Deriving the partial values in MCDM by goal programming,” Annals of Operations Research, vol. 74, pp. 277–288, 1997. View at: Publisher Site  Google Scholar
 Y. Dong, Y. Xu, H. Li, and M. Dai, “A comparative study of the numerical scales and the prioritization methods in AHP,” European Journal of Operational Research, vol. 186, no. 1, pp. 229–242, 2008. View at: Publisher Site  Google Scholar
 Z. S. Xu, “Goal programming models for obtaining the priority vector of incomplete fuzzy preference relation,” International Journal of Approximate Reasoning, vol. 36, no. 3, pp. 261–270, 2004. View at: Publisher Site  Google Scholar
 K. Kułakowski, “A heuristic rating estimation algorithm for the pairwise comparisons method,” Central European Journal of Operations Research, vol. 23, no. 1, pp. 187–203, 2015. View at: Publisher Site  Google Scholar
 A. Z. Grzybowski, “Note on a new optimization based approach for estimating priority weights and related consistency index,” Expert Systems with Applications, vol. 39, no. 14, pp. 11699–11708, 2012. View at: Publisher Site  Google Scholar
 A. Z. Grzybowski, “New results on inconsistency indices and their relationship with the quality of priority vector estimation,” Expert Systems with Applications, vol. 43, pp. 197–212, 2016. View at: Publisher Site  Google Scholar
 T. K. Dijkstra, “On the extraction of weights from pairwise comparison matrices,” Central European Journal of Operations Research, vol. 21, no. 1, pp. 103–123, 2013. View at: Publisher Site  Google Scholar
 T. L. Saaty, “How to make a decision: the analytic hierarchy process,” European Journal of Operational Research, vol. 48, no. 1, pp. 9–26, 1990. View at: Publisher Site  Google Scholar
 M. Brunelli, L. Canal, and M. Fedrizzi, “Inconsistency indices for pairwise comparison matrices: a numerical study,” Annals of Operations Research, vol. 211, no. 1, pp. 493–509, 2013. View at: Publisher Site  Google Scholar
 C. Lin, G. Kou, and D. Ergu, “An improved statistical approach for consistency test in AHP,” Annals of Operations Research, vol. 211, no. 1, pp. 289–299, 2013. View at: Publisher Site  Google Scholar
Copyright
Copyright © 2019 Paul Thaddeus Kazibudzki. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.