IAM Cost Measurement Methodology

Title	Measuring the Cost of IAM - Methodology
Version	`1.1` Draft
Summary	This research note discusses how IAM costs may be measured in such a way as to yield meaningful, comparable and reproducible results, form the basis for the development of performance indicators and enable benchmarking between organizations.
Out-of-scope	The key question of the value created by IAM is not discussed in this article and will be discussed in separate research notes.
See Also	OM-IND-0002: IAM TCO (Indicator) OM-IND-0003: Identity Average TCO (Indicator) IAM Cost Components

1 The Business Case for IAM Cost Measurement
2 Barriers and Negative Effects
3 Benchmarking and Measurement Scales
4 An Iterative Approach to Standardize the Measurement of IAM TCO
5 Distinguishing IAM TCO from IAM Program Cost
6 Accounting Periods
7 Direct versus Indirect Cost
8 IAM Investments and Depreciation
9 Listing IAM Cost Components
10 Bibliography

The Business Case for IAM Cost Measurement

Surprisingly, a 2009 survey in Europe found that 49% of its respondents didn’t know nor measure their IAM service delivery costs (KPMG and Everett, 2009, p. 30).

To improve their competitiveness, organizations must simultaneously minimize their costs and maximize their added value. Hence, the cost of IAM considered as an overall service to the organization is a fundamental component of IAM performance measurement. Put differently, the reason why organizations allocate resources to IAM is because they expect a favorable RoI (Return on Investment) in view of the other initiatives the organization must sustain, in particular its core business.

Key goals of IAM are compliance assurance (Compliance Assurance (IAM Goal)) and information security resilience (Information Security Resilience (IAM Goal)). Nevertheless, the above RoI argument is still valid if we consider the costs incurred by organizations as consequences and penalties linked to non-compliance or the realization of operational risks.

TCO (Total Cost of Ownership) is a measurement of the overall cost of an item or service, from conception to acquisition to operations and maintenance (cf. Total Cost of Ownership (Dictionary Entry)). In consequence, the TCO of IAM is a natural measurement of IAM overall cost.

Organizations use centralization, standardization and automation to reduce TCO (David et al., 2002). The evolution of IAM TCO over time should thus reflect the performance of these efforts.

Finally, top management should have a strong interest in benchmarking IAM costs. In effect, without any point of comparison, top management is blind and does not know if they allocate a low, average or high level of resources to IAM relative to their competitors. Without comparable cost measures, top management is left with FUD and guts feeling.

Barriers and Negative Effects

There are barriers to cost measurement and the implementation of TCO in particular. These include: the difficulty to gather the data and cultural issues such as resistance to change. Top management support is understood as a key element to overcome these barriers. (Hurkens et al., 2006)

TCO outweighs these barriers by providing benefits associated to performance measurement, decision making, communication, understanding and continuous improvement (Ellram, 1993).

TCO is thought as being directly proportional to service levels, hence organizations should strive to simultaneously reduce their TCO while maintaining or improving their service levels. This indicator may induce a focus on costs to the detriment of service levels. For instance, centralization, standardization and automation strategies are not enough to reduce costs without impacting service levels - careful examination of service levels, careful planning and how well these strategies are implemented are key factors. (David et al., 2002)

Cost analysis is not a straightforward task, it is complex and there are different ways to do it. Thus, comparing costs may be prone to errors. (KPMG and Everett, 2009, p. 3, Hurkens et al., 2006)

Even if cost analysis is exempt of mistake, a sound cost analysis may be subject to misinterpretations or misuse to support political agendas. (Foussier, 2006, chapter 2, Cost Measurement)

Benchmarking and Measurement Scales

The absolute TCO of IAM (OM-IND-0002: IAM TCO (Indicator)) is not a comparable indicator. For instance, if organization A manages 150’000 core IAM active identities while organization B manages 3’000, absolute costs are incomparable and thus meaningless. But absolute costs may be used as a base metric to compute a comparable indicator, OM-IND-0003: Identity Average TCO (Indicator), that is expressed as average cost per active identity being managed by the organization.

An Iterative Approach to Standardize the Measurement of IAM TCO

To obtain valid figures that are adequate for performance measurement and benchmarking purposes, it is necessary to develop guidelines and standardize the method. At the same time, it is obviously out of the scope of IAM cost measurement to align the detailed accounting practices of organizations throughout the world. Thus a balance need to be found and we should ask ourselves what is an appropriate method of measurement function of its intended usage and the level of reliability that is needed. (Foussier, 2006, chapter 2, Cost Measurement)

In view of these challenges, we propose here an iterative approach to develop the guidelines and standard methodology. As illustrated below, a rough version 1.0 will be distributed to the community of IAM professionals. In turn, IAM professionals will experiment in the field and provide feedback. Consolidation will be made on the Open-Measure wiki. The methodology will be edited and issued with a new version number. And we pursue with a new iteration from there.

Figure: an iterative approach to develop the guidelines and standard methodology for IAM TCO measurement

Distinguishing IAM TCO from IAM Program Cost

When considering IAM costs from the perspective of the overall organizational efficiency, we take an enterprise-wide perspective, use the TCO approach and do not consider who is responsible for what.

Let’s make the following thought experiment and consider the Failed Acme company. This organization does not run an IAM program, has not appointed an IAM manager, has no documented IAM processes. Still, this organization is doing IAM in the sense that identities are somehow being provisioned and some people somewhere grant access to systems. The organization does not incur the costs associated with a traditional IAM organization and infrastructure but does incur the costs associated with dysfunctional processes, slow staff onboarding, failed audits and security incidents. For this organization, measuring the IAM TCO indicator makes perfect sense.

In contrast, let’s consider Winning Acme company. This organization runs an IAM program, has a competent IAM manager in place and runs efficient IAM processes. This organization incurs the cost of its IAM organization and infrastructure but does not (hopefully) incur costs of dysfunctional processes, slow staff onboarding, failed audits and security incidents. For this organization, measuring the IAM TCO indicator makes perfect sense.

Hence, the IAM TCO should capture the overall cost of IAM independently of the organizational structure that supports it or the maturity of its processes.

Definition: IAM TCO is the total cost of the organization’s IAM services, independently of the organizational structures supporting them.

Large and mature organizations have IAM programs in place, run by IAM managers with clearly defined IAM processes. IAM managers do have a genuine need to measure the costs of the program that is under their responsibility. But the scope of IAM programs differ from organization to organization and may change in time. For instance, some organizations consider PAM as a sub-component of IAM. Others consider CIAM as a sub-component of IAM. And it is legitimate for organizations to define their IAM programs in the way that makes sense in view of their unique constraints.

Definition: IAM Program Cost is the total cost of a program designated as IAM that has a defined scope. That scope may be distinct from the organization’s overall IAM scope, that is some activities traditionally linked to IAM may be out-scoped and other activities traditionally not linked to IAM may be in-scoped.

Accounting Periods

For the sake of simplicity, we skip the complexities linked to accounting periods.

Direct versus Indirect Cost

For cost measurement purposes, the distinction between direct costs and indirect costs is important because the measurement methods are distinct (Foussier, 2006, chapter 2, Cost Measurement).

Definition: IAM Direct Cost is defined as expenditures that are fully dedicated to IAM (e.g. IAM dedicated personnel, IAM software licenses, etc.).

Measuring direct costs is straightforward: it is the sum of all the individual costs.

Definition: IAM Indirect Cost is defined as expenditures that are not fully dedicated to IAM (e.g. general administration, general IT infrastructure costs required to support IAM systems, etc.).

Indirect costs are more complex to measure and methods may vary between organizations. For instance, some organizations will rely on roughly estimated allocation keys while others will use fine-grained accounting schemes. It is presumably outside the scope of IAM cost measurement to redefine the accounting methods used by organizations, hence we should accept a level of inconsistency when comparing these costs between organizations and keep this in mind when interpreting measurements. To enable proper interpretation of results between organizations engaging in benchmarking, organizations should transparently disclose their high-level accounting methods.

It should be noted that the measurement of indirect costs may also vary between activities. For instance, some activities may have varying costs and may thus be charged (e.g.: the consumption of a workload in an IT infrastructure or cloud) while others may have fixed costs (e.g.: general administration).

IAM Investments and Depreciation

The implementation of Enterprise IAM is known as a complex, demanding and expensive undertaking (e.g.: Royer, 2013, p. 4). Hence, measuring the cost of IAM must not only factor in operating costs but also capital expenses such as IAM projects (deployment of IAM processes and/or systems).

For instance, let’s make a thought experiment where we compare the IAM costs of two organizations: A and B. Let’s assume organization A invested a large amount of money in IAM in year 2 and no more during the following 4 years while organization B invested average amounts of money during these years. We observe that:

For organization A, if the IAM project yields significant productivity gains, operational IAM costs should decrease in year 3 as compared to year 1. But this should not be seen in isolation as the cost of the large IAM project may offset these cost reductions.
If we did not take into account depreciation, comparing organizations A and B in year 2 would be meaningless. The costs of organization A would be significantly higher because of the project and would suddenly drop the year after.

In consequence, investments, or capital expenses in IAM, should be factored in the measurement of IAM costs using depreciation. But depreciation methods vary considerably between countries. While IAM projects have a much broader scope than deploying technology (and often fail when considered only from a technological perspective), IAM projects are often intensive in technology usage. If we review (EY, 2018), we see that depreciation methods for computer software vary considerably around the globe between 2, 3, 5, 8, 10 or more years and that the majority uses the straight line method.

In conclusion, for the sake of simplicity, we suggest that when possible and for the sake of making IAM cost measures comparable between organizations, for the sake of performance measurement (that is distinct from tax issues), IAM investments be factored in IAM cost measurement using depreciation over a period of 5 years with the straight line approach. This looks like an acceptable average.

When this is not feasible, we recommend that organizations engaged in IAM benchmarking disclose their high-level depreciation methods to facilitate the interpretation of their measures by peer organizations.

Listing IAM Cost Components

Inventorying IAM cost components is key to make TCO measurement “complete” and comparable between organizations. Once established, this list may be documented with cost estimation methods and used as a checklist by IAM practitioners to measure their IAM TCO.

Ellram, 1993 provides guidelines on how to determine which cost components are significant enough to warrant tracking: that is use Pareto’s law coupled with common sense.

To help us with this, we have already established an initial list of known IAM activities. Cf. https://open-measure.atlassian.net/wiki/spaces/PROC/pages/91815945 and https://open-measure.atlassian.net/wiki/spaces/ART/pages/62259252. Cost components will be naturally linked to processes.

At this early stage of development, we should start surveying IAM practitioners for cost components and populate our list. The list of IAM cost components will be maintained on the following wiki page:

IAM Cost Components

Articles