Designing a learning tool to assess the competencies of public health managers

Our work provides insights on specific tools and approaches to strengthen the capacities of public health officials in low- and middle-income country (LMIC) contexts.

Our project was designed to develop a Learning Measurement Tool to support public health managers to assess the competencies required by them to perform jobs effectively. The tool was designed as a capacity development instrument during the Covid-19 pandemic and responds to the dynamic demands placed on public health managers during such crises. It seeks to guide public health managers to understand their roles and responsibilities in a comprehensive manner and to assess knowledge, skills and behaviors vis-à-vis the role they undertake.

The tool provides a step-by-step guiding framework that helps public health managers understand what ideal performance is for their role. It enables them to reflect on competencies, and on what worked well or could be improved, in relational terms. This tool was developed in the specific context of public health managers in the state of Bihar, India, but a competency-based approach and assessment can be applied more widely to public officials across government departments in any geography.  

Challenges

Health managers increasingly operate in challenging environments, as evidenced during the Covid-19 pandemic, and they need appropriate competencies to deliver their roles. The decentralised system in LMICs often involves a wide range of managerial and administrative stakeholders across geographical and administrative levels; it also involves professionals drawn from both medical and non-medical backgrounds. These managers have limited capacity development support and lack adequate opportunities for professional growth. Without an explicit understanding of competencies required for a role, it is difficult to design and facilitate relevant capacity development initiatives for managers.

As a result, our studies showed that very limited attention has been given to capacity development for developing both managerial and clinical competencies of staff across most public health systems in LMICs. In addition, gender-inclusive leadership development is currently missing despite a significant female workforce. For leaders and managers to be able to set a vision for their healthcare centre and implement it, they require corresponding opportunities; something that has been a systemic challenge and reality in most LMICs. 

We developed a competency framework for two public health managerial cadres—Medical officers in-charge (MOICs) and Block Health Manager (BHMs)—to help understand the responsibilities that each of these public health officials was to execute, by providing a consolidated and relevant list of the knowledge, skills and behaviors expected of managers. The preliminary version of this competency framework was produced in early 2020 and updated after the onset of the Covid-19 pandemic to adapt to the changing context. The competency framework was also aligned with the ethos of Mission Karmayogi, a large-scale administrative reform initiative by the Government of India with a pillar focused on competency-led capacity building of civil servants across diverse departments of the Indian bureaucracy.

Our approach

To complement our competency framework, we subsequently developed a Learning Measurement Tool as an objective benchmark for policymakers, practitioners, and other stakeholders to assess the baseline competencies of the managerial cadre.

The Learning Measurement Tool is a combination of a range of measures that specify how proficient a manager is on each competency vis-à-vis the parameters defined in the competency framework and a method or a mix of methods to assess competencies.

The key features of our Learning Measurement Tool are:

  • The measures are defined at the level of competencies to ensure practicality of use.
  • It has been developed using close-ended questions as they are simpler, quicker to respond to, and help in getting measurable and quantifiable data. For example: A manager rating themselves on a specific parameter such as “I manage the line of treatment for primary healthcare; however, I feel the need to do it confidently.”
  • It is categorised as a four-point Likert scale with an additional parameter of ‘Don’t know’, to be able to rank a manager’s proficiency on a particular competency:
    • Aspirational: I want to do this well;
    • Emerging: I am developing this, but need more support;
    • Good: I do this well, but am looking to improve; and
    • Excellent: I do this really well and can support others.
  • In this context, we proposed self-assessment as the choice of measurement method over tests or job observations to build the acceptance in a system where competency-based approaches are still new; also it is likely to be perceived as less disciplinary.

Outcomes

Our research shows that most tools and measures to understand health systems leadership are often developed in high income countries with a dearth of evidence emanating from LMICs that reflects their context adequately. This Learning Measurement Tool then provides a practical and easy-to-use prototype for developing learning aids for public officials in resource-constrained settings.

Ideally, the framework and Learning Measurement Tool should undergo a consensus-building exercise with relevant stakeholders through a Delphi technique for validation. The Delphi technique involves an iterative process of consensus generation, requiring the participants to receive and provide feedback in real time. The objective is to arrive at a consensus on the required competencies for public health managers and relevant ways to measure them. During this process, health experts will assemble to deliberate on the set of competencies and measures identified in the framework.

We propose the following ways in which the tool could be relevant to policymaking in India:

  • It could be piloted in any district of Bihar by the state government, training institutes or civil society organisations to understand the baseline competencies of health managers currently. It would help diagnose significant gaps that might impact on-the-job performance.
  • It could be used as a basis to develop a bespoke training and capacity development roadmap for public health managers in the state in line with the organizational objectives
  • It could be used as a tool for recruitment and succession planning by clarifying the skills, knowledge, and attitude required for the job or role in question
  • It could facilitate the development of similar competency frameworks and learning tools in other sectors and geographies.

Areas of expertise