Assuring compliance with policies and regulations is a major stake for organizations that invest significant time and money to avoid the consequences and penalties of noncompliance. Yet answering the simple question of compliance with a policy is more complex than it looks. Often, people are puzzled and find themselves unable to objectively determine compliance. Sometimes, this leads to heated debates between policy writers, auditors, managers and staff who may have diverging perspectives on how to determine compliance. But a simple decision tree with four precise questions can help address this challenge.
Setting Up the Scene
The compliance question should be considered from the distinct perspectives of the following personas:
- The staff who are responsible for learning and following the rules prescribed by the policy—They report compliance on their scope of work (daily work or a project) to their managers.
- The manager who is responsible for implementing and assuring compliance with the policy—The manager reports compliance on her or his area of responsibility to the authority.
- The auditor who is responsible for controlling compliance with the policy—This person reports compliance on an audit scope to the authority.
- The authority who issues, communicates and oversees compliance—The authority is responsible for overseeing the policy in its domain of legitimacy.
The final consideration is the policy. It has goals, which often consist of managing risk. It is also bound by scope. It prescribes rules that may consist of mandatory or discretionary prescriptions or interdictions. Finally, rules are triggered by conditions.
This provides the conceptual model depicted in figure 1.
The Building Blocks
Before going into the details of the decision tree, foundational principles that should continuously guide compliance decisions should be explored.
Obtaining an Adequate Level of Assurance
To know something means to reach an adequate level of assurance that the information is true. Direct observation (Gemba walks1), metrics, corroboration, tests (by sampling methods or systematic), controls, audits, self-assessments and, above all, critical thinking with tough questions are all methods to raise one’s level of assurance.
When it comes to reporting compliance with a policy, what is an adequate level of assurance? The precise answer to this question is unique to every organization and policy. If the level of inherent risk is low, the organization may simply rely on the self-assessment of the manager in charge. Or people may wish to personally inspect every single piece of work that goes through the process. Setting the bar too low exposes an organization to consequences of noncompliance. Setting it too high may be counterproductive. It is the duty of professional staff, managers and auditors to apply their intelligence and efficiently raise assurance to an optimal level. Once that level has been reached, the next step is to make a compliance statement and be held accountable for that process.
Overcoming Cognitive Bias
It is a truism that absolute objectivity is incompatible with human nature. “If you are human, you are biased.”2 The staff, the manager, the auditor and the authority are all consciously and unconsciously biased by the very way heuristics are wired in their brains. Cognitive biases—such as confirmation bias (which describes the tendency to seek or interpret information in a way that confirms preconceptions), attribution error (which describes the tendency to over-emphasize personality over circumstances to explain behaviors) or hindsight bias (which describes the tendency to filter memories of past events to make present events look more predictable)3 to name just a few examples—have been researched for decades, and their study is far beyond the scope of this discussion. Nevertheless, while unbiased thinking is biologically impossible, the desire to make optimal decisions should motivate practitioners to strive to reduce the negative effects of biases.
Learning about biases, acknowledging their existence and developing a habit of self-observation are some of the methods through which self-awareness can be raised, which helps people curb their biases and mitigate the negative effects of biases on their decisions.
Auditor Independence and Groupthink
Faced with the uncertainties linked to reporting noncompliance, both the staff and manager may tend to embellish reality. At the end of the day, it is the compliance of their own work they are assessing, and rare are those who genuinely seek critique. This may lead them to groupthink, where critical thinking and controversy are shunned.
This is where the auditor’s independence comes into play. Independence is the freedom from conditions that threaten unbiased judgment. It is subtly distinct from objectivity: While objectivity is a property of the person, independence is a property of the person’s function.4 And the auditor’s independence is precisely what is needed to protect the staff and manager from fooling themselves.
Pursuing Objectivity
To determine a level of compliance, it is necessary to provide objective answers to precise questions. Objectivity, by definition, requires that if these questions were asked to different personas having the same information, each would come to the same conclusion.5 Thus, pursuing objectivity requires:
- Sharing the information basis
- Making decisions on the same grounds
TO DETERMINE A LEVEL OF COMPLIANCE, IT IS NECESSARY TO PROVIDE OBJECTIVE ANSWERS TO PRECISE QUESTIONS.
Once enough information to reach an adequate level of assurance is collected, it is paramount to assure that all other personas have that same information with which to make their decisions. Openly sharing this information is called transparency, and it is a prerequisite to objectivity.
If it is felt that some information should not be disclosed (or actively reported) because it may lead others to different conclusions, one is led astray. If people cross that line, they no longer work for the organization’s best interests but are building an illusion of compliance. This is doomed to fail, and the remaining question is when will the house of cards collapse.
Once all personas have the same information basis, it is important to assure that they are making decisions on the same grounds. There is a simple way to accomplish this. For every question in the decision tree, everyone should ask themselves a supplementary question: Knowing what I know, would the other three personas come to the same conclusion?
If the answer is a clear yes, this is not far from objectivity. If there is a doubt, or any stakeholder thinks the other personas would come to different conclusions, that person should reach out to them and engage in an open dialog to clarify the issue.
Compensating for Noncompliance
No policy is perfect, and there are circumstances where complying with a policy would be detrimental to the organization. For instance, complying with a policy may incur costs that are disproportionate to the risk the policy intends to manage in the first place.
But not complying with policy is a serious matter, and making exceptions increases moral hazard.6 The key piece of advice here is no one should unilaterally make the decision that it is OK to not follow the rule.
An organization must have an efficient exception management process in place to manage these situations at the right level of authority. This process must ensure that the justification for the exception is grounded and that the risk posed by noncompliance is identified, analyzed and managed through mitigation, transfer, acceptance and/or avoidance.
To “compensate noncompliance” means to follow this process. When this is done properly, it is acceptable to still claim compliance with the policy because it is implicit (if not explicit) that exceptions are always possible. Coming back to the preceding section, transparent disclosure of exceptions is necessary to reach objectivity.
One limitation of compensation is law. In effect, compliance with the law is not optional.
Giving Feedback to Policy Authorities
There is no such thing as a perfectly univocal policy.7 But the implementation of policies incurs costs, and policies may be improved to optimize the allocation of resources and the management of risk. Because policy writers are not omniscient, if nobody dares to provide them with critiques on the challenges posed by the rules they prescribe, policies will not improve. Staff, managers and auditors should engage in a constructive dialog with the policy authority and propose amendments.
The Decision Tree
The decision tree, illustrated in figure 2, takes a scope of interest and a policy as inputs and returns a compliance level as output. Compliance is measured along a nominal scale consisting of the following exclusive categories:
- Best-in-class compliance
- Satisfactory compliance
- Unknown compliance
- Noncompliance
- Irrelevant policy
Figure 2 depicts the four steps of the decision tree. Starting from the upper left, it depicts a structured approach to determining compliance. Every decision point (rectangle) and state are described in the following sections.
Relevance Check
Is this policy relevant? Compare the scope of the policy with the scope of interest. Do they intersect? This may look like a trivial question, but many times, staff and managers are investing efforts and money implementing irrelevant policies. Spending a few hours studying and clarifying policies and a responsible, accountable, consulted and informed (RACI) matrix may spare the organization an expensive implementation project.
If the answer to this question is no, stakeholders can move on to the Irrelevant Policy section. If the answer is yes, move on to the Relevant Policy section.
Irrelevant Policy
In this case, the scope is compliant because the policy is not relevant, so the compliance level should be reported as irrelevant and valuable time can be spent on other issues. Managers have the legitimacy to adopt policies, so if implementing this irrelevant policy would create value in any given area of responsibility, managers may voluntarily promote it to the rank of relevant policies.
Relevant Policy
In this case, the policy applies to the scope of interest and must be complied with so stakeholders can move on to the Noncompliance Check section.
Noncompliance Check
Based on what is known, stakeholders should ask if there is any single rule in the policy that meets all the following criteria:
- Are the rule conditions realized?
- Is it a mandatory prescription or interdiction?
- Is the rule not complied with?
- Has this rule noncompliance not been compensated?
If the answer to any of these questions is yes, the process can move on to the Noncompliance section. If the answer is no, then the process moves on to the No Known Noncompliance section.
Noncompliance
In this section, the compliance level should be reported as noncompliant. This may sound like bad news, but it is not. Something was learned, so now it can be acted upon.
Managers or staff may feel the instinct to rationalize, but falling into the trap of misleading anyone should be avoided. Any progress should be reported, and the execution of the remediation plan should be pursued.
If it is estimated that complying with the rule would be detrimental to the organization, the Compensating for Noncompliance section should be referred to next.
No Known Noncompliance
At this point, there is more to do with the decision tree, but to the best of anyone’s knowledge, there is no mandatory policy rule that can be categorized as noncompliant. Stakeholders can move on to the Assurance Level Check section.
Assurance Level Check
The question here is has an adequate level of assurance regarding the compliance status of all mandatory policy rules been reached? If the answer is yes, the process can move on to the Known Compliance section. If the answer is no, stakeholders should move on to the Unknown Compliance section.
Unknown Compliance
The compliance level must now be reported as unknown. There is not enough substantiated information to make a statement.
In the report, it is possible to put forward a hypothesis regarding the expected compliance level, but care must be taken not to fool the audience: The organization is below the threshold of an adequate level of assurance and any assumption may be proven wrong. It is now time to collect the information needed to obtain an adequate level of assurance. Once done, the process should move back to the Noncompliance Check section. Auditors who run out of budget must wrap up their mission and transparently report this unknown compliance level. This part of the audit report should not be murky.
Known Compliance
When the scope of interest is compliant with the policy, it can be reported as compliant, but the organization should go the extra mile and move on to the Best-in-Class Compliance Check section.
Best-in-Class Compliance Check
To qualify for best-in-class compliance, the Noncompliance Check and Assurance Level Check should be applied to discretionary policy rules. Discretionary rules show where the organization is heading.
If the scope of interest fails these tests, the process should move on to the Satisfactory Compliance section. If it succeeds, stakeholders should move on to the Best-in-Class Compliance section.
Satisfactory Compliance
If the scope of interest complies with the mandatory rules of the policy, the compliance level can be reported as satisfactory.
Staff or managers must now maintain and continuously improve the compliance level. If they compensated some noncompliance with risk remediation plans, any risk should be monitored and actively managed.
Best-in-Class Compliance
If the scope of interest complies with all policy rules, the compliance level can be reported as best in class.
It is important to note that the journey does not end here. Compliance is something that must be maintained and continuously improved. If the organization compensated some noncompliance with risk remediation plans, it must monitor and actively manage the risk. It may also be beneficial to proactively work with the policy authority to become a contributor and let others benefit from that experience.
COMPLIANCE IS SOMETHING THAT MUST BE MAINTAINED AND CONTINUOUSLY IMPROVED.
Conclusion
It is important to remember that all models are a simplification of an elusive reality and only as useful as they have predictive power. The decision tree presented here, with its underlying conceptual model, is no exception to this rule. Organizational and personal interactions are infinitely complex and chaotic, and the goal of the decision tree presented is to marginally enhance the understanding of compliance and strengthen the collaboration of those pursuing it.
Endnotes
1 In Japanese, gemba means “the actual place.” In the field of lean management, “gemba walk” denotes the action of physically going to look at the actual process, asking questions to those who do the work and gaining an effective understanding of the process.
2 Ross, H. J.; Everyday Bias: Identifying and Navigating Unconscious Judgments in Our Daily Lives, Rowman & Littlefield Publishers, USA, 2014
3 Ibid.
4 Anderson, U. L.; Internal Auditing: Assurance and Advisory Services, 4th Edition, The Internal Audit Foundation, USA, 2017
5 Franceschini, F.; M. Galetto; D. Maisano; Designing Performance Measurement Systems: Theory and Practice of Key Performance Indicators, Management for Professionals, Springer, USA, 2019
6 Rowell, D.; L. B. Connelly; “A History of the Term ‘Moral Hazard,’” Journal of Risk and Insurance, vol. 79, 2012, p. 1051–1075
7 Op cit Franceschini et al
David Doret, CISSP, GRCP, ISO 27001 LA, Lean Six Sigma Green Belt, PMP
Is an identity and access management (IAM) manager at BNP Paribas. He is a veteran IAM and cybersecurity professional and twice held the position of chief information security officer at financial institutions. He founded and manages the nonprofit Open-Measure wiki project for IAM professionals. He spent several years working in cybersecurity advisory services, helping organizations of all sizes in diverse industries.