In order to further assess the aspects of the product that stakeholders value throughout the product development, production, sales, and usage processes and determine the order of optimization for our project, we established an evaluation system of indicators using the Analytic Hierarchy Process (AHP).
Analytic hierarchy process (AHP) is defined as follows. Before we analyze a phenomenon or problem, we decompose it into related factors according to their properties, and classify them according to their relationship to form a multi-level structural model. After that, through experience or comprehensive stakeholders’ opinions, the relative importance of low-level factors to high-level factors is judged and measured. The weighted ranking is obtained according to the degree of importance, and finally, the quantitative analysis and comparison can be made. The core of the hierarchical analysis method is to stratify and datalize the influencing factors, which decomposes an abstract phenomenon or problem from difficult to easy, and makes it easy to make intuitive judgments on complex problems and make decisions. AHP has the advantages of simplifying complex problems and simple calculations. We decided to use this method to establish our evaluation system of indicators.
Our evaluation system consists of target layer, system layer and indicator layer. At the target level, the overall target should be defined at the beginning. At the systemic level, various factors that could potentially influence stakeholders' perceptions of the product were identified as criteria. At the indicator level, specific measurable aspects were defined for each criterion. For indicators within the same level, pairwise comparisons were made using the criteria from the higher level, resulting in the construction of pairwise comparison judgment matrices. Consistency tests were performed to ensure the reliability of these comparisons. Subsequently, indicator weights were determined. Using the weights of all subordinate indicators within the same level and the weights of all criteria from the higher level, weighted calculations were conducted to derive the weights of all indicators at this level concerning their importance in relation to the highest level. Finally, a comprehensive weight was obtained, allowing us to prioritize and rank various requirements related to the project based on their weighted importance.
Based on our project, we considered the opinions of multiple stakeholders and constructed an evaluation system as shown below:
Target Layer |
System Layer |
Indicator Layer |
Prioritize the Requirements Related to The Project |
Quality |
Detection accuracy; Shelf life of reagent kit and test paper; Storage conditions of reagent kit and test paper; Error control; Quality control between different batches; The science of the principle of the project; Specificity of test objects; Detection sensitivity; Accuracy of detection method |
Safety |
Waste disposal during production; Waste disposal during use; Sample handling; Safety in use |
Ethics |
Sampling methods; Feedback format for test results; Protection of privacy data |
Respect |
Personalized needs of cardiovascular disease patients |
Public Health |
Sample contamination; Risk of cross-infection |
Innovation |
Technical innovation; Business model innovation; Product or service innovation; Marketing innovation; Organizational management |
Efficiency |
Production cycle; Detection time; Level of detail in the instructions; Ease of use; variability in data interpretation and understanding among different users |
Costs |
Environmental impact; Stability; Reagent production and packaging; User storage conditions; Transportation conditions; Medical waste disposal |
In relation to the various requirements associated with the project, we conduct comprehensive assessments from eight aspects: quality, safety, ethics, respect, public health, innovation, efficiency, and cost. Each system level is further subdivided into various evaluation indicators. We use this as our evaluation system to comprehensively assess the priorities of various indicators that need to be considered, with the aim of scientifically and effectively optimizing our project to the maximum extent possible within the limited time frame.
AHP Hierarchical Analysis Results for System Layer
Items |
Feature Vector |
Weight Value(%) |
Maximum Eigenvalue |
CI Value |
Quality |
2.337 |
29.21 |
8.622 |
0.089 |
Safety |
1.144 |
14.294 |
Ethics |
1.256 |
15.699 |
Respect |
0.31 |
3.879 |
Public Health |
1.06 |
13.253 |
Innovation |
0.42 |
5.249 |
Efficiency |
0.838 |
10.476 |
Costs |
0.635 |
7.94 |
According to the system-level analysis results, it is evident that quality is the most critical factor, accounting for 29.21% of the total weight and should be given the highest priority. Ethics, safety, public health, and efficiency follow in descending order of importance. Respect and innovation, on the other hand, have lower weights, each below 10%. Therefore, in the future development and sales processes, special attention should be paid to the quality of the product to ensure the accuracy and precision of the testing results, providing consumers with reliable outcomes. While ensuring quality, medical ethics should also be considered, appropriate sampling methods chosen, and consumers' privacy data protected. Additionally, there is a need to further enhance product safety and testing efficiency.
According to this approach, for details on how we specifically carried out our work, please refer to our IHP page. We genuinely hope that our analytical method can serve as a reference for those who will be involved in testing or related fields in the future. This can help them pinpoint the most crucial factors during project development. By focusing their efforts more precisely, they can ensure responsibility to users, society, the environment, and other aspects.