A Critical Perspective on Safeguard Selection Processes

Safeguards have become an essential part of every IT environment. As companies become more reliant on modern technology, they also have to face more vulnerabilities that must be handled efficiently. However, the selection of an appropriate safeguard can be challenging.

The various attributes and as much preliminary information as possible should be considered in the selection process. A systematic process of review and decision-making techniques helps to avoid inopportune justifications that are based on hastiness and poor preparation. Stakeholders should be aware of possible problems that can occur during the selection process, including any general shortcomings in the decision-making techniques employed. The selection process can also be affected by unforeseen cost, time and quality issues.

During the preparation, execution and quality assurance of the safeguard selection, a critical perspective should be used for pointing out possible problems.

Motives

Sometimes, the selection of a safeguard is based on faulty justifications. The causes of those justifications are often fear, uncertainty and doubt. Some unethical product representatives may even enhance these conditions to increase their sales through the use of comprehensive and often subtle and subliminal disinformation. Neutral sources are often referred to, but the significance of the reference is greatly exaggerated or presented in the wrong context. The result can be an expensive and inadequate investment in a suboptimal safeguard.

On the other hand, hastiness and poor preparation can also lead to faulty justification. If managers must react quickly because of severe threats, they may not have the opportunity to consult with appropriate experts prior to the investment decision. In cases such as this, poor decisions are very common. Subsequently, the expenses are high without generating the expected security enhancement.

The best way to avoid poor justification is a well-thought-out safeguard selection process that involves experts. To find the most appropriate safeguard, the decision maker must be able to evaluate several safeguards and identify and select the best one. The selection of a safeguard is based on the evaluation of multiple qualitative or quantitative attributes. Therefore, a process should cover the comparison of these attributes and the evaluation of the existing safeguards. The attributes are weighted so that the evaluation of the safeguards can be performed considering the specific situation and characteristics of the company.

The advantages of using a structured process over an unsystematic process include the following:

• The problem to be addressed must be defined before actually starting the evaluation.
• The identification and use of attributes facilitate the consideration of different perspectives within the evaluation.
• The selection process is organized with transparent steps and it is divided into aggregated subparts.

How the safeguard selection process is composed in detail depends on the company that develops and uses the process. In general, a structured process includes the preparation of the selection, the narrowing of the decision making and the activities for quality assurance (figure 1).

Each phase in the selection process can be further described as follows:

1. The initiation should be based on a solid reason for a specific safeguard selection with consideration of the stakeholders’ perception of information security.
2. The sponsoring phase is used to get acceptance and support from executive management and select an appropriate sponsor.
3. The first step of decision making—defining the problem to be solved via the decision—aims to gain a thorough understanding of important elements (e.g., strategy, scope, assets, risk, protection, stakeholders) that will factor into the decision-making process.
4. During attribute identification, the decision maker must consider all relevant attributes that will be used for evaluating the safeguard alternatives.
5. Afterward, the decision maker must also determine how important each attribute is by performing an attribute evaluation.
6. Alternative identification is a crucial step and its outcome depends on the information that can be gathered about available alternatives via research of external knowledge.
7. The alternative evaluation is needed to evaluate the alternatives with regard to the relevant attributes and to create a subsequent ranking.
8. Based on the ranking, the best alternative (the alternative with the highest rank) can be identified.
9. Documentation ensures that the decision-making process can be understood by third parties, who can then use it to gain insights into the substeps and to gather indications about the substeps’ correctness and completeness.
10. A separate approval, mostly by senior management, allows an additional quality check so that the results are not used thoughtlessly.

Decision-making Techniques

When faced with selecting among alternatives that have multiple attributes, the most common decision-making techniques are simple additive weighting (SAW)¹ and the analytic hierarchy process (AHP).² Both techniques are based on the same general sequence:

• Define the decision problem.
• Identify and evaluate the attributes.
• Identify and evaluate the alternatives.
• Select the best alternative.

The differences lie in the calculations of the evaluations. SAW uses calculations that are based on independent evaluations of separate attributes and alternatives, while AHP uses pairwise comparisons of two attributes or alternatives at a time. Since these different techniques include different calculation methods, the same alternatives can lead to dissimilar results, e.g., an alternative could be the most appropriate when using SAW, but only a second choice when using AHP. Evaluation results with alternative scores that are close to each other can lead to this situation.

There are also other decision-making techniques in the scientific field, e.g., the analytic network process (ANP), the technique for order preference by similarity to ideal solution (TOPSIS) and data envelopment analysis (DEA). However, SAW and AHP provide the best balance between an ease of understanding and thoughtful application for practical use in the enterprise sector. Like most multi-attribute decision-making techniques, SAW and AHP also come with potential disadvantages that should be known and, if possible, avoided including:

• The techniques can be manipulated in many possible ways. Because precise figures and calculation methods are used, objectivity can be faked. Due to the general subjectivity in the weighting and evaluation of attributes, the result can be significantly affected by undetected manipulation. Although the subjectivity can be reduced by including experts, it cannot be eliminated. In addition, the decision maker has the freedom of choice regarding the attribute selection.
• The addition of the subscores implies the independence of the attributes. However, dependencies between attributes, such as competitive or complementary relationships, often cannot be completely avoided. For example, the protection level and vendor support of a safeguard are often closely related. Consequently, there is a risk that strongly dependent attributes lead to an unintentional over- or undervaluation of alternatives.
• The aforementioned addition also leads to a possible substitutability of the subscores. In particular, subscores that are derived from very bad characteristics of an alternative can be substituted with subscores from very good characteristics. Therefore, single attributes might be neglected. Even if the attributes are divided into exclusion and comparison attributes, this problem can be only partially eliminated. The comparison attributes are still affected by a possible substitutability.
• The overall result can be subject to leveling. In this case, it is likely that the weaknesses or strengths of the best alternative are no longer recognizable in the result. The more attributes considered, the more likely the results are positioned in the middle region of the range of the possible overall scores.
• Due to the assessment of alternatives with individual attributes, the overall problem will be broken down into many single problems. This decomposition is questionable because, first, the overall problem is no longer clear and, second, there is a risk that the assessments of many single problems lead to an undesirable overall assessment. If the attributes are in a competitive relationship to each other, the improvement of a subscore regarding a single attribute can lead to the evaluation of a competing attribute resulting in a lower subscore. For example, the reduction of false positive events in biometric access control systems often leads to an increase of false-negative events.

In addition to the disadvantages of the particular decision-making technique, general difficulties can occur during the selection of a safeguard. The company can be influenced by cost, time and quality aspects while making the decision. For example, the company might focus on the safeguard costs and neglect the costs of the selection process. Various events or activities might slow down the selection process for unforeseen reasons and delay the planned selection. Errors and overlooked information might cause quality issues in the selection results.

Problem Identification

The result from the selection process should be checked critically for any indication that implies potential problems in the process. Only results that are free of obvious errors and doubts should be approved and used for acquiring and implementing the selected safeguard. Among other things, the following scenarios can indicate problems in the selection process:

• The decision problem, including the strategy and protection requirements, has changed during the process or it was not analyzed sufficiently from the beginning. Therefore, senior management has to assume that the result does not completely meet the underlying problem.
• Internal conditions (e.g., resources and schedules) have been overlooked or changed so that the recommended solution would actually not be the best solution. Unfavorable internal conditions can also lead to major problems in implementation.
• External conditions (e.g., laws, standards, market conditions) have changed so that the initial decision-making process is not accurate anymore. Certain environmental factors can lead to new requirements in the planning or potential problems in the acquisition, implementation and operation of the safeguard.
• The decision-making process was incorrect or incomplete, resulting in errors that can significantly influence the evaluation results. If an error happens to be related to a critical attribute or evaluation, the ranking of the alternatives can even be changed. In this case, the selected alternative would not be the best alternative.

• The documentation of the decision-making process is not sufficient in regard to scope and quality. If the documentation is insufficient or missing, senior management can reject the result of the selection process.
• Abuse of power might have influenced the decision-making process. Often, this abuse can legally be categorized as fraud, which is generally caused by motivation, justification and opportunity, as described in the fraud triangle (figure 2).³ The attribute evaluation is one of a number of activities that might have been exploited with abuse. Senior management should be aware of potential weaknesses or missing control measures in the safeguard selection process. Indications for abuse should be taken seriously. If abuse seems to have impacted the result, senior management should reject it. Common indications for abuse are discrepancies in records, conflicting or missing evidence, and problematic or unusual relationships between involved parties.⁴

Conclusion

Every company is interested in finding the most appropriate safeguards to protect the company appropriately and cost-effectively. Therefore, the safeguard selection process is crucial and should not be based on faulty justifications. A structured process that allows handling multiple attributes is preferable to unsystematic activities.

This process should also be supported with a decision-making technique that is manageable and delivers transparent and understandable results. However, common disadvantages such as substitutability, leveling, and over- or undervaluation, should be considered. The overall selection process can be characterized by various problems, which cannot always be avoided and, therefore, should be considered continually, especially when checking the result of the process. Indications of these problems are, among other things, major changes in the decision problem as well as changed internal or external conditions, errors, insufficient documentation, and possible fraudulent activities. Mostly, the safeguard selection of a company can be greatly improved by focusing not only on evaluating the safeguard alternatives, but also on critically examining the selection process itself.

Endnotes

¹ Fishburn, P. C.; “Additive Utilities With Incomplete Product Set: Applications to Priorities and Assignments,” Operations Research, vol. 15, iss. 3, April 1967, p. 537–542
² Saaty, T. L.; “How to Make a Decision: The Analytic Hierarchy Process,” Interfaces, vol. 24, December 1994, p. 19–43
³ Nimwegen, S.; Prevention and Identification of Fraud: Possibilities of Internal Corporate Governance Elements, dissertation, University of Münster, Westfalen, Germany, 2009
⁴ American Institute of Certified Public Accountants, “AU Section 316—Consideration of Fraud in a Financial Statement Audit,” USA, 2002, www.aicpa.org/Research/Standards/AuditAttest/DownloadableDocuments/AU-00316.pdf