Will We Pass the IRAP? Understanding What IRAP Actually Measures

Author: Anna Ko (Principal Assessor @ Cyberly)

As an IRAP assessor, I am often asked the same question before an assessment begins.

“Do you think we’ll pass or fail the IRAP?”

The question is understandable. Organisations may have spent months preparing. Engineering teams have been implementing controls. Compliance teams have been gathering evidence. Executives have committed budgets. Sometimes a government customer is waiting on the outcome before proceeding with procurement.

By the time an assessment begins, people naturally want reassurance that all of that effort is leading somewhere.

Yet every time I am asked the question, I find myself having to explain the same thing:

There is no such thing as passing or failing an IRAP assessment.

There Is No Such Thing as “Passing” an IRAP

That answer often catches people off guard. After all, most assurance activities in the technology industry have conditioned us to think in binary terms. You pass the audit. You obtain the certification. You achieve compliance. If you don’t, you fix the findings and try again.

IRAP does not work that way.

This is not simply a matter of terminology. IRAP is an independent security assessment, not a certification program.

The Australian Cyber Security Centre (ACSC) is explicit on this point. A completed IRAP assessment does not imply that a system is compliant, endorsed, approved, authorised, or secure. Likewise, IRAP assessors do not certify, accredit, or approve systems on behalf of the Australian Government. Their role is to assess security controls and report their findings.

For organisations encountering IRAP for the first time, particularly vendors familiar with SOC 2, ISO 27001, or FedRAMP, this can feel strange. If there is no pass mark, then what exactly is the assessment trying to achieve?

Trust Through Transparency

Many assurance frameworks are ultimately designed to answer a compliance question. Has a prescribed set of requirements been met? Has sufficient evidence been gathered? Has an auditor determined that the organisation conforms to the standard?

IRAP begins from a different premise. The objective is not to award a pass mark, but to create trust through transparency. Government agencies remain responsible for making their own Authorisation to Operate decisions, and they rely on IRAP assessments to understand how a system aligns with the ISM, where its strengths and weaknesses lie, and what operational or business realities influence its security posture. The value of an IRAP assessment is therefore not in producing a binary outcome, but in providing decision-makers with a sufficiently detailed and independent view of the system to make informed risk decisions of their own.

Decisions regarding risk acceptance are ultimately management decisions. They involve judgement. They involve trade-offs. They involve determining whether residual risks are understood and acceptable within a particular context.

An IRAP assessment exists to inform those decisions.

The Assessment Is Not the Destination

Viewed through this lens, the assessor’s role becomes clearer. The assessor is not a gatekeeper standing at the entrance with a stamp marked “Pass” or “Fail.” The assessor is closer to an investigator, gathering evidence, examining control effectiveness, identifying areas of concern, and presenting a picture of the system’s security posture as accurately as possible.

The final decision belongs elsewhere.

This often surprises vendors because they assume the assessment itself is the destination. In reality, the assessment is only one input into a broader authorisation process. The report helps decision-makers understand how effectively controls are operating, and where further treatment may be required. What happens next depends on the risk appetite and responsibilities of the authorising authority.

Consider two agencies reviewing the same IRAP report. One intends to use the system for a relatively low-risk business function and is comfortable accepting a particular control weakness. Another intends to use the same system to support critical government services or process highly sensitive information and requires remediation before proceeding. The IRAP findings have not changed. The system has not changed. The difference lies in how the findings are interpreted within each agency’s operating context.

This is why two organisations can review the same findings and reach different conclusions. Neither outcome means the assessor was more or less satisfied with the environment. Government agencies do not all operate under the same risk profile. An agency responsible for highly sensitive information, national security outcomes, or critical services may reasonably adopt a lower tolerance for residual risk than an agency operating in a less sensitive context. The purpose of an IRAP assessment is not to make that decision on their behalf, but to provide the information needed for them to make it themselves.

The Better Question

The question vendors should be asking is usually not whether they will pass.

The more useful question is whether they are ready to be assessed.

These are not the same thing.

In my experience, organisations rarely struggle with IRAP because they lack security controls. More often, they struggle because they underestimate the effort required to demonstrate those controls. Evidence is fragmented across teams. Architectural decisions exist only in people’s heads. Asset inventories are incomplete. Policies describe one process while operational reality reflects another.

The challenge is not always security. Frequently, it is visibility.

Many first-time vendors assume that implementing controls is the hard part. In reality, some of the most challenging aspects of an IRAP assessment involve demonstrating how those controls operate in practice, tracing responsibilities across teams, producing evidence that supports implementation claims, and clearly explaining the architecture and security model of the system. Strong security controls are important, but they must also be observable, repeatable, and capable of being independently assessed.

This is why readiness activities are often more valuable than organisations expect. Before an assessor begins examining controls, the organisation must first be capable of explaining its environment, defending its decisions, and demonstrating how security is actually implemented in practice. A well-prepared system with a handful of known gaps is often easier to assess than a poorly documented system with excellent security controls.

Seen from this perspective, the obsession with passing or failing starts to look like a distraction.

It encourages organisations to think of security as a hurdle to clear rather than a process of building trust through transparency. Some of the most mature organisations I assess are not those with flawless environments. They are the organisations that can clearly articulate their architecture, demonstrate the effectiveness of their controls, acknowledge limitations where they exist, and provide decision-makers with an accurate picture of their security posture. In many cases, that transparency is more valuable than the pursuit of a perfect assessment outcome.

The ACSC’s guidance reflects this reality. An IRAP assessment is intended to provide an informed view of a system’s security posture. It is a tool for decision-making, not a certificate of perfection.

So when organisations ask me whether they will pass or fail, I usually tell them that they are asking the wrong question.

The better question is whether they understand their environment, their controls, and the evidence required to demonstrate them.

Because that, ultimately, is what the assessment is trying to discover.

Need help with IRAP?

At Cyberly, we help vendors approach IRAP with clearer process expectations and defensible assessment outcomes.

If you are a US vendor preparing for Australian Government customers, we can support you through the IRAP assessment process or help you understand whether your product is IRAP-ready.

Visit Cyberly to learn more or contact us for a readiness discussion.