What Is NSF’s Quality Management Maturity (QMM) Assessment Model? Find Out More

It is well recognized that the Quality Management System (QMS) - whether documentation, deviation, and event management or CAPA – is the focus of Regulatory Inspections and indeed the cause of many findings. Given the requirements have been broadly the same for so long, why is this still an area of non-compliance?

Historically, and perhaps a harsh and simplistic assertion, is that the QMS has been managed as a function with a comprehensive set of instructions. However, more recently regulators, industry and quality professionals are looking at the QMS slightly differently. The QMS must be the heart and lungs of an organization – it needs to breathe life into the organization and must ensure that each operational part – from individuals, technology, equipment, to functions, teams, and the leadership – operate in unison. This shift in mindset recognizes the need to understand the impact and risk of people and culture in the successful deployment of a mature QMS.

So, how does the QMS feed every part of the organization with oxygen and that those within the organization recognize the importance of the QMS? How do regulators assess the impact of these intangible elements on the effectiveness and robustness of a QMS, and therefore compliance?

The NSF QMM Assessment Model (developed and introduced in 2022, following the publication of the US FDA Drug Shortages Task Force report in 2019¹) responds directly to this challenge. It is a disruptive innovation tool to assess organizational quality maturity across the regulated landscape. It requires a different mindset, taking a holistic view of the QMS, in parallel to the traditional considerations of compliance.

The NSF model has been designed to align and respond to the FDA’s findings, to the assessment process considered by the Center for Drug Evaluation and Research (CDER), but importantly also takes into consideration other research on the impact of culture on the (quality) organization, e.g., ISPE Cultural Excellence.

The NSF QMM model has gained interest and credibility among some of the world's largest and most dynamic pharmaceutical companies who want to ensure they have a positive quality culture, and that their quality functions are both effective and robust. Ultimately, it is also recognized that proactive action on maturing a QMS, supports a more robust supply chain and minimizes supply disruption.

A 2019 US Federal Drug Shortages Task Force report concluded that, 62% of shortages between 2013 and 2017 were attributed to manufacturing or product quality problems.

The report concluded that manufacturers are not incentivized for enhancing the maturity of their Quality Systems beyond the bare minimum mandate.

As a result, the CDER (Center for Drug Evaluation and Research) recommended a rating system to assess Quality Management Maturity at Manufacturer’s facilities. This system could potentially inform regulators, and also customers, about the robustness and effectiveness of a supplier’s quality management system.

The potential benefits are illustrated alongside:

The FDA has run two QMM pilots (Domestic and International) to assess the feasibility of implementing the QMM rating system. This pilot was followed by an Industry Stakeholder discussion on the way forward.

For many years regulators have been publishing data and metrics on their inspection findings, with perhaps the hope that it informs the regulated sector of the deficiencies being identified and provides areas of focus for the sector to channel its efforts. Ultimately, the aim is to improve the level of regulatory compliance across the sector and hence the quality and safety of products in patients’ hands.

For years however, deficiencies have been identified within organization quality systems. For example, since at least 2015² Quality Systems have been the subject of the highest number of deficiencies for the Medicines and Healthcare products Regulatory Agency (MHRA), with Documentation coming in second (for the last two year of data publication, in 2018 and 2019). Meanwhile the US Food and Drug Administration (FDA) have within their top ten Warning Letter citations³, deficiencies such as production and process controls or investigations and OOS. A significant number (525 between 2018-2022) of the citations⁴ identified the need for “responsibilities and procedures applicable to the quality control unit shall be in writing; such written procedures shall be followed”.

Regulators, quality professionals, and consultants alike all speak to the need to have procedures documented and “in place” and evidence to demonstrate the procedures are “in use”. Organizations spend significant time and effort training staff on procedures and monitoring training effectiveness, via supervision and audit, to confirm that the procedures are being used appropriately and accurately followed. Yet still regulators around the globe continually identify deficiencies with what is “in place” and “in use”. Perhaps when regulations, guidance, or regulatory expectation changes then the reason for such findings might be understandable and perhaps due to a slowness to respond to the changing regulatory landscape or perhaps to misunderstanding or lack of awareness. However, the GxPs are not new. So why do we still see challenges with Quality Systems?

Simply put, the most likely cause is the human factor.

Indeed, many now agree that the QMS is no longer seen as a Quality Management team, distinct from the operational organization and that manage a comprehensive set of functional procedures. The QMS is now considered as a more holistic function – constituted by both the visible elements and invisible attributes.

In short, it is not just what is “in place” and “in use”, but what is also happening “in reality. It is essential to consider the QMS as the heart and lungs of the organization; the critical function that breathes life into the organization. Procedures being deployed to every inch of the organization, like oxygenated blood reaching every cell within a human body. From individuals, technology, equipment, to functions, teams, and leadership alike – operating in harmony, efficiently and robustly. When a deficiency is identified, it is investigated and resolved quickly, any impact managed and future risks mitigated – much like the treatment of a cancerous growth. Importantly, it is not hidden or ignored, but proactively and openly managed, and an assessment made to ensure that similar situations don’t arise elsewhere within the organization.

Very rarely do we see this level of alignment and flow in an organization. To achieve this level of cohesion requires an enhanced state of Quality Culture which is embedded throughout the organization; driven from the top and positively managed. It requires a different approach - looking at the organization through a different lens, understanding the employees within, creating alignment to drive behavioral changes, motivating and incentivizing individuals toward quality deliverables, and assessing effectiveness and robustness, alongside compliance – all leading to a positive Quality Culture. At its optimal, it is a desired state of culture that requires high degree of alignment and trust, leading to higher performance, improved risk management and enhanced compliance.

Much work⁵ has been undertaken not just to consider Quality Culture, but the “Quality Climate” of an organization where dimensions such as “safety climate, innovation climate, learning climate, ethical climate and inclusion climate” are explored to determine the impact and influence of behaviors on an organization’s culture. Understanding both the culture and the climate enables an organization to better shape the future culture.

However, culture change is hard and even harder to measure. It is therefore imperative that an organization pins down exactly what it needs and wants to change.

So how can an organization understand its Quality Culture and Quality Climate and determine what needs to be changed to drive high performance and ensure QMS compliance?

According to the US FDA⁶, “Quality management maturity (QMM) is the state attained when drug manufacturers have consistent, reliable, and robust business processes to achieve quality objectives and promote continual improvement”. The US FDA has also recently⁷ alluded to the fact that QMM considers “manufacturers and those with oversight and controls over manufacturing, take ownership for quality” and this includes “the tone of commitment to quality” and that “quality systems shape culture”.

Historically the dichotomy between quality and business objectives has shaped the operational activities. As a result, the performance and capability of a QMS are primarily assessed through quantitative metrics (often derived for business performance) and audits. While effective, these often do not provide a holistic assessment. To achieve a sustained and effective quality management trajectory, one must understand their QMS’ current state of maturity or capability and then plot out an action plan to achieve a desirable future state.

QMM however ensures a cohesive approach molding together a mature quality culture, technical processes, and corporate-wide business approaches. Business and quality objectives and integrated and embedded, a delivery plan or roadmap of improvements is developed, and corporate goals developed to focus on business and patient outcomes, success driven by the benefits of a mature quality management system in operation.

Observing the need for integration, NSF has developed a QMM Assessment Model for a “Quality Organization” that provides a qualitative and quantitative measure of the various constituent parts of the QMS.

The Quality Organization is assessed across three pillars – Strategic, Tactical and Operational - each containing several assessment areas. These can be customized to meet the needs for a specific assessment.

Each of the elements assessed have an associated quantitative score, which is calibrated to their respective criticality. The findings are based on information obtained through a blended assessment model of interviews and interactions, documentation review and walkthroughs. The assessment consists of obtaining information from various sources at the site and collecting and reviewing (onsite and remotely) the evidence to support the assessment ratings.

Based on the findings for each element, the maturity of each area is rated on an ascending scale – Undefined, Defined, Managed, Improved and Optimized, through a quantitative mechanism.

Organizations are provided with a quantitative assessment of the organization’s quality maturity level and qualitative narrative for each area reviewed, along with recommendations and a mechanism to prioritize actions, to achieve a higher level of quality maturity.

NSF have observed that organizations with lower levels of Quality Management Maturity tend to exhibit certain traits or characteristics, such as out of date procedures, procedures not being adequately or correctly followed, little to no proactive work on continuous improvement, minimal innovation, low level of creativity and higher than average levels of unplanned work resulting from deviations and excursions. Importantly, these organizations tend also to have fragmented culture (e.g., hierarchical fractures), lower levels of staff morale and higher levels of attrition.

NSF’s QMM Assessment process includes detailed analysis of the effectiveness of the systems in place, delves deeper into the root causes of these systemic challenges and seeks out opportunities. It considers evidence across horizontal and vertical organizational systems, hierarchies, and boundaries, connecting the findings and identifying strategic level options for greater levels of system performance and proficiency. If required, operational level metrics and measurement processes can be proposed for demonstration of improvement effectiveness and maturity sustainability.

At a high level of Quality Management Maturity, organizations are operating in an optimal state. These organizations have low levels of re-work and unplanned activities, high levels of continuous improvement, application of best practice and workforce efficiency, higher levels of trust and autonomy leading to enhanced engagement and retention, and better risk and investment decisions. Importantly, there are clear signals of alignment. A golden thread runs through the organization from the vision and mission, through to the organization and quality strategies, into team and personal objectives. Quality and risk management feature high in the organization’s objectives and values, along with financial business integrity. The Quality Climate and Culture are clear to see – trust, openness, creativity, inclusiveness, and learning are all paramount – the ethos is positive, understood, and visible.

Maintaining this state of maturity requires effort and attention. The right structures and governance must be in place and the impact of change must be carefully considered – particularly in relation to the impact on Quality Culture and Quality Climate.

The organization might also require a gap analysis to identify improvements to achieve the further advanced state of Quality Risk Management Maturity – a system where Quality Management, Quality Culture/Climate and Risk Management are all operating in harmony. Equally, further consideration might also be needed on the more recent recommendations for Outcome Based Co-operative Regulation (OBCR)⁸ where “higher levels of integrity are correlated with commercial success”, “decisions are made by empowered staff, rather than by a limited number of managers” and “the culture of an organization has a powerful moderating (regulating) effect on behavior and outcomes”. “OBCR aims to create a framework that supports cooperation rather than conflict in regulation”. NSF’s QMM Model will pave the way and lay the foundations for these advanced and optimized maturity states and further support can be provided by NSF’s experts to develop and implement a plan to shape the future state of the organization.

The real benefits of QMM come to the forefront when we consider business objectives in a competitive market – i.e., to maintain reliable supply of high quality, safe and compliant products and to minimize supply disruption. To achieve this, senior leaders must be open to the benefits of assessment tools such as NSF’s QMM model. Recognizing the opportunity and taking the first step, could result in a cascade of benefits longer term and a more resilient organization delivering sustainable supply.

In summary, QMM:

• Is a more mature system - resulting in enhanced risk management, greater levels of resilience, and resulting in a higher performing and sustainable organization. This in turn could enhance reputation and business success.
• Considers the organizational culture, business processes and corporate objectives and the impact on the effectiveness of the QMS.
• Ensures and understanding of the current state of the Quality System beyond just metrics and audits. The model aligns to the latest thinking and strategies of global regulatory authorities. It is an important element of oversight and control.
• Future-proofs the organization - supporting proactive continuous improvement leading toward sustainable compliance management of an organization’s QMS. This could also lead to quality system efficiencies, cost savings and regulatory flexibility or less frequent regulatory oversight.
• Supports a positive culture climate that encourages openness, trust, and learning, leading to fewer deficiencies or excursions, resulting in fewer non-compliances or recalls.
• Examines the robustness and effectiveness of the QMS. It looks beyond the immediate environment (i.e., what is in place/in use) and considers the wider strategic influencing factors. It can helps consider the implications of change and supports best practice in execution of robust change management initiatives.
• Is flexible - The model can be used to undertake an organizational/site health check, or to consider QMS maturity improvements following a regulatory inspection, for example.
• Can be tailored to suit your need and focus in on key areas of concern/risk. The subsequent report can provide recommendations for those highlighted areas.

NSF’s QMM Assessment responds directly to the FDA’s Report by the Drug Shortages Task Force in 2019ⁱ and provides a unique solution for organizations to get a step ahead.

It supports the evolution of the pharmaceutical manufacturing sector to be more resilient, more agile, and more open. It shifts perspectives from siloed and burdensome to cohesive and co-operative.

It encourages organizations to embrace and enhance their Quality Management Maturity and is a logical step in the evolving regulatory picture.

Sources

¹Center for Drug Evaluation and Research (2019). Report: Drug Shortages: Root Causes and Potential Solutions. U.S. Food and Drug Administration: www.fda.gov/drugs/drug-shortages/report-drug-shortages-root-causes-and-potential-solutions

²Medicines and Healthcare products Regulatory Agency (2014). Good manufacturing practice inspection deficiencies. GOV.UK: www.gov.uk/government/statistics/good-manufacturing-practice-inspection-deficiencies

³Chapman, J. (2020). Top 10 Pharma Inspection Findings from FDA, MHRA, and the Russian Drug Regulator. Redica: www.redica.com/pharma-top-10-pharma-inspection-findings-from-fda-mhra-and-the-russian-drug-regulator/

⁴www.thefdagroup.com. (n.d.). 2020 FDA Warning Letter & Inspection Observation Trends: www.thefdagroup.com/blog/2019-fda-warning-letter-inspection-observation-trends

⁵CIPD (2022). Organisational Culture and Cultural Change | Factsheets. CIPD: www.cipd.co.uk/knowledge/culture/working-environment/organisation-culture-change-factsheet#gref

⁶Research, C. for D.E. and (2022). CDER Quality Management Maturity. FDA: www.fda.gov/drugs/pharmaceutical-quality-resources/cder-quality-management-maturity

⁷Cder, F., Opq and Oqs (n.d.). CDER’s Quality Management Maturity Program: www.fda.gov/media/143742/download

⁸INDR site. (n.d.). INDR: International Network for Delivery of Regulation: www.indr.org.uk