more impactful decisions
using data and evidence
and analytics expertise to help
you build the future.”
We work with the United Nations, NGOs, Foundations, and governmental agencies to help them solve significant challenges in various areas such as process optimization, needs assessment, and decision evaluations.
We typically help clients on subjects related to:
Focus areas
Strategic and corporate partnerships/sponsorships, fundraising strategy, maximizing pay-to-use services, profitable events strategy, and grant selection
Social impact measurement, website and user experience design, social media strategy, and customer analytics
Go-to-market strategy for new products and services, market expansion, and membership base growth
Competitor analysis, volunteer acquisition and retention programs, funding models, and data collection
Supply chain strategy, cost reduction, business model transformation, and process optimization
A decision-focused evaluation is an experimental (or quasi-experimental) evaluation to estimate the causal impact of a policy, program, or intervention on one or more outcomes of interest. HDI Group designs and implements DFEs to inform client decisions on the construct of an intervention to continue, whether to scale up a particular program, and whether to continue funding one specific implementer or program. The impact can be defined relative to the status quo or an alternate intervention being considered
Process evaluations systematically and empirically describe what happens during and after program implementation, then carefully compare this description to ex-ante expectations in the implementation plan and theory of change. Process evaluations can serve an audit or accountability function by examining whether planned program operations and achievements have occurred. They also serve a learning function by updating the theory of change and informing a client about where a theory of change decisions and actions are warranted and may provide insight into what these decisions should be.
We build or improve upon a monitoring system that enables a client to collect actionable data on program indicators at regular intervals to facilitate performance management, promote learning opportunities throughout an organization (or across multiple organizations, as when working with a funder), inform real-time program improvements.
At its most basic level, machine learning involves creating algorithms that can be trained using existing data to make predictions or classifications on unseen data. In this definition, algorithms refer to various predictive functions, including regression (including regularized regression), decision tree models, clustering techniques, and neural networks. The data used in these projects may differ from other HDI Group projects, going beyond survey and administrative data to include “Big Data” sources such as images, satellite imagery, text data, and mobile phone records.
Data analytics encompasses a broad range of activities beyond the data analysis typically conducted as part of an evaluation. These include, but are not limited to:
- Modeling: Building models to allow clients to compare different scenarios; for example, to explore the relationship between critical assumptions, inputs, and programmatic costs and outcomes. In some cases, a dynamic model (e.g., Excel workbook) is provided as a deliverable so that the client can update the model with their data or assumptions.
- Analysis: Using existing data (which may or may not be collected by HDI Group) to explore operational and contextual questions about a program and quickly test initial hypotheses about its impact.
- Visualization: Using visual techniques to represent, analyze, and allow clients to interact with data, with the ultimate goal of informing decisions. This can include designing dashboards or mapping.
Program Diagnosis & Design Support
This service includes several activities that HDI Group might conduct to support evidence in programmatic decisions.
- Diagnosis: Critical assessment of the status quo that yields hypotheses about the leading causes of poor outcomes. This will always include theory of change development and include gathering qualitative or quantitative data on program implementation, fundamental stakeholder interests, potential beneficiaries’ needs, and evidence review.
- Program design: Helping clients develop and refine service(s) to address a problem of interest among a target population. This will follow a diagnosis and recommend actions the client should take to address causes of poor outcomes. It might involve further stakeholder interviews, observations, data analysis, evidence review, ideation, critical thinking, and rapid prototyping of ideas. It is often followed by an impact evaluation or a process evaluation.
In a rapidly changing environment, the ability to predict trends using real-time data and present it in an easy-to-understand manner becomes even more critical. Our capabilities include:
- Help to forecast the market demand for new products or features and the success of marketing campaigns
- Predict high users of medical services based on their diagnostic and prescription drug use history
- Use social media analytics to evaluate general causation between services and client satisfaction
- Efficiently analyze billions of mutual fund transactions across numerous file formats and platforms
- Create custom analytics and dashboards that help companies develop adequate controls
- Develop interactive platforms and web apps to help clients manage their data-intensive projects and metrics
- Compile and managing large, complex datasets using proprietary client data as well as data from a variety of third-party data sources
Machine learning, broadly defined, involves computer programs and algorithms that automatically improve their performance at specific tasks through experience. The primary objective of these tasks is out-of-sample performance. The algorithm is first trained on a sample of inputs for which the target output responses are known. Training enables the algorithm to learn highly complex and intricate relationships in high-dimensional data, rather than pre-imposing assumptions on how inputs and outcomes are related. The algorithm’s performance is then assessed on a different sample to determine how it would perform in the real world with new data.
One powerful application of machine learning is natural language processing (NLP). NLP can extract useful data elements from unstructured, raw data. Using language- and grammar-specific constructs builds on a unique combination of algorithms and artificial intelligence tools to analyze, extract, and classify human communications from unstructured data such as online reviews, patent claims, physician notes in a medical file, insurance claims, and even audio recordings. As a result, it can be used to develop and implement predictive models across several sectors.
Our capabilities include:
- Sort through reams of filings and online product reviews to reveal which features were considered relevant to consumers in patent infringement and consumer protection cases
- Collect, cluster, and analyze information coming from a variety of unstructured text financial sources to find relationships between specific language and conducts on the financial markets
- Examine detailed information of online booking information, like pricing or other specific characteristics, to measure the impact of constraints (or removing restrictions) of competition on consumer welfare
- Uncover issues that are not captured by traditional patient-reported outcome instruments, including through the use of social media and online patient data on medical conditions and their treatments
- Efficiently conduct literature reviews, organizing large datasets of scientific articles, ranking abstracts by expected relevancy for particular research topics, and highlighting changes in research topics over time
- Develop a method to identify and standardize medical terminologies, such as disease name, from the unstructured medical data
Statistical sampling plays a significant role in any study that you will ever run. Not having a large enough and reliable sample will pose questions regarding your project findings’ rigor, and worse, it could lead to misinformed budgetary decisions. To properly select a sample or evaluate the quality of an existing sample, our analyses often focus on critical questions about sample design and implementation, including:
- What question needs to be answered?
- What is the population of interest?
- What level of accuracy is required, and what size sample is needed to achieve that desired accuracy level?
- Is there a benefit to selecting a stratified sample?
- Is the sample representative? Is it biased?
- Were the sampling plan and data collection adequately implemented?
- Is the selected sample sufficient to answer the question of interest?
- Which method should be used to extrapolate from the sample to the population of interest?
Besides helping you design the right model, HDI Group can help you create the data collection instruments. Our capabilities include:
- Design of Vignettes instruments
- Design of Mystery Shopper
- Design of Standardized patients
- Design of Patient charts
- Design of Observation files
- Design of Patient files
- Administrative data
- Literature review and meta-analysis
- Web scrapping
- Application programming interface
- RPA
Finally, we provide the resources you need to collect the highest quality data rigorously to end the full circle. Our capabilities include:
- Field data collection (F2F)
- Telephone data collection (CATI)
- Web-data collection (CAWI)
- Focus groups
We aim at improving clients’ ability to demand, consume and generate evidence. The range of activities can vary from light-touch support to help clients interpret results to intensive training on evaluation strategy, biostatistics, HEOR training, and hiring efforts to transition a client’s monitoring system.
Engagements
Briefly, HDI Group can help your organization:
- Optimize service delivery, reduce costs, and build local capacity using public health, economics, and behavioral science insights.
- Provide M&E support in designing data collection platforms, indicators or in evaluating the feasibility of existing ones
- Design and evaluate hospital performance, document the accreditation process, evaluate budget impact and sustainability of investments in this direction
- Evaluate the impact, respectively, the success and effectiveness of programs
- Understand the context of disease such as unmet population needs, epidemiological profile (e.g., incidence, prevalence, mortality, survival rates)
- Evaluate the current status of healthcare services in term of access, quality of health care services, addressability
- Audit operational activities using qualitative and quantitative data to understand the main problems with an existing offering
- Offer design support, data collection, and implementation support of national or pilot operational programs
- Analyze pre-existing or newly generated data and redesign services or create new ones based on principles from behavioral science, public health, epidemiology, and economics
- Develop the training syllabus and train your staff on topics related to epidemiology, early childhood development, behavioral change, communication
- Develop apps and behavioral change interventions to increase coverage and retention
What is the impact of mobile applications on user consumption?
Our experts supported a national alcohol awareness healthcare provider to understand the impact of their mobile application on users’ alcohol consumption and determine which features in the app predicted long-term user engagement.
The client is an independent alcohol awareness healthcare provider that focuses on alcohol misuse and health-related harm. The healthcare provider aims to provide digital tools and information to help their user-base make better choices about drinking.
To prove the healthcare provider’s digital app’s effectiveness in supporting users to self-monitor and reduce their alcohol intake. Additionally, the healthcare provider also wished to understand better which users were more likely to engage with different app features and gather more detailed qualitative feedback on usability and acceptability for the healthcare provider to modify and improve future iterations.
In phase one of this research project, data from over 100,000 app users was pulled from digital records and analyzed to determine if self-reported alcohol consumption levels reduced over a 3-month usage period. Further analyses were conducted to understand if usage patterns differed by user demographic details and degree of engagement with different in-app features, such as goal-setting, rewards and achievements, geolocation and real-time preventative alerts, and monitoring of consumption levels. In phase two, qualitative interviews were conducted with a subset of app users to understand better their perceptions of usability and acceptability of the product.
Findings from the initial quantitative phase of this research allowed the healthcare provider to prove that their app was effective at encouraging users to reduce their drinking, with engaged users reporting a significant reduction in alcohol intake (-3.68 units/week) between first use and 12-week follow up. Findings from the qualitative research phase were used to offer recommendations to the healthcare provider on product improvements for future iterations of the app and ongoing data collection requirements to support future impact assessments.
What should be the standards in place to improve National Authorities’ institutional capacity to conduct centralized procurement of emergency medical devices and products?
Our experts supported a national alcohol awareness healthcare provider to understand the impact of their mobile application on users’ alcohol consumption and determine which features in the app predicted long-term user engagement.
The client is an independent alcohol awareness healthcare provider that focuses on alcohol misuse and health-related harm. The healthcare provider aims to provide digital tools and information to help their user-base make better choices about drinking.
To prove the healthcare provider’s digital app’s effectiveness in supporting users to self-monitor and reduce their alcohol intake. Additionally, the healthcare provider also wished to understand better which users were more likely to engage with different app features and gather more detailed qualitative feedback on usability and acceptability for the healthcare provider to modify and improve future iterations.
In phase one of this research project, data from over 100,000 app users was pulled from digital records and analyzed to determine if self-reported alcohol consumption levels reduced over a 3-month usage period. Further analyses were conducted to understand if usage patterns differed by user demographic details and degree of engagement with different in-app features, such as goal-setting, rewards and achievements, geolocation and real-time preventative alerts, and monitoring of consumption levels. In phase two, qualitative interviews were conducted with a subset of app users to understand better their perceptions of usability and acceptability of the product.
Findings from the initial quantitative phase of this research allowed the healthcare provider to prove that their app was effective at encouraging users to reduce their drinking, with engaged users reporting a significant reduction in alcohol intake (-3.68 units/week) between first use and 12-week follow up. Findings from the qualitative research phase were used to offer recommendations to the healthcare provider on product improvements for future iterations of the app and ongoing data collection requirements to support future impact assessments.
What should be the standards in place to improve National Authorities’ institutional capacity to conduct centralized procurement of emergency medical devices and products?
Our experts supported the National Authorities from Romania by developing an adequate list of medical products used in emergency medical products and devices to be centrally procured in compliance with international best practices and local regulations.
A UN institution is supporting the National Authorities to scale-up the centralized procurement for emergency health services
Fiscal space for health is constrained by ineffective procurement practices, which drives overspending on medicines, medical devices, and supplies, contributing to budgetary inefficiency. Although the share of drugs and medical devices in total current health expenditure has decreased from 40% in 2011 to 27% in 2017, it remains significantly higher than in other EU member states, including Central and European Countries. Unfortunately, the responsible state institutions don’t have the legal mandate to procure goods and services in the health sector, while others continue to conduct a limited number of centralized procurement procedures, mainly for medicines and vaccines. This project aims to support the UN to strengthen National Authorities’ capacity to conduct centralized procurement for medical devices and consumables for emergency services.
The project included five activities: (i) review of a list of medical products prepared by National Authorities, to be centrally procured; followed by (ii) provision of support in the preparation of market studies for the medical products to be centrally procured; then (iii) review of the draft technical specifications prepared by National Authorities priorities for the respective medical products and support in the development of the associated terms of reference; and then (iv) support in conducting consultations with sector stakeholders on the draft terms of reference, including National Authorities’ responses to feedback from suppliers and users.
Recommendations were made addressing the issues listed above.
What is the current level of access, quality, and addressability of healthcare services at the primary and secondary level, and how could resources be better allocated to increase institutional capacity?
The UN launched a set of integrated services in 2014 that addressed vulnerable children in several counties with financing received from Norway Grants. In partnership with a foundation, the clients are interested in mapping all the healthcare providers in the regions and understanding the institutional capacity and geographical reach for children (0-18 years) from rural and urban localities where the project has been implemented.
The clients are a foundation carrying work in the healthcare area, the UN, and local public authorities.
The client would like to plan its strategies and programs of the support measures for children aged 0-18 years old who are vulnerable or find themselves at the risk of social exclusion.
For this purpose, we have addressed the level of addressability, quality of medical services, and access of children (0-18 years) from rural and urban localities where the project has been implemented. A secondary objective consisted of understanding the challenges faced by the Roma Health Mediators and Community medical Assistants in enhancing access and addressability to healthcare services. The study follows the patient’s path in the medical health system, evaluating the beneficiaries and the medical staff’s perception, which provides community medical services, family medicine, and specialty. For this purpose, semi-structured interviews, focus groups, observations, and vignettes were used with community health workers, vulnerable households, general practitioners, and hospital managers.
The study was presented to the local authorities informing the allocation of human and financial resources in the targeted countries.
Is the current investment in the medical education reform worthwhile? Shall the UN continue funding the existing reform?
The UN institution, which was also the lending partner, wanted to know how well the Indonesian Government and the local UN branch performed to reform the medical education system. The UN also wanted to know if the project was cost-effective in the short and long-term, hence informing the feasibility of additional funding rounds.
A major UN institution active globally, the Ministry of Education and Ministry of Health.
At the time of project appraisal, the Indonesian health sector faced significant human resources (HRH) challenges. Although the total availability of health personnel was not low, there were issues regarding HRH distribution, skill mix, and healthcare personnel quality. The production of new physicians had grown steadily; in 2010, this was approximately 9,000 new physicians/year. There were imbalances across provinces and rural and urban areas regarding the distribution of health professionals. HRH quality was a growing concern in Indonesia during project preparation as only half of the physicians, midwives, and nurses correctly responded to vignette questions related to antenatal, pediatric, and adult curative care. The trends in the quality of the healthcare professionals’ services were associated with the fast expansion in the number of private schools.
At the time of project preparation, some initiatives placed the foundation for establishing a quality assurance system for health professional education. Nonetheless, these were not sufficient. As a result, a significant lending project was funded, which had its main objective to enhance quality assurance policies governing health professionals’ education in Indonesia.
Our consultants ran an evaluation of a US$77.82 Million project implemented in Indonesia aimed at enhancing the quality of medical education. As part of the assessment, we assessed: the risks and the quality of outcome indicators and framework, the funders, and government performance. Mainly, we evaluated the project costs and financing, performed a cost-benefit analysis, evaluated the implementation support, the M&E systems, framework, and capacity. For this purpose, we used administrative and qualitative data, workshop results, and internal documents.
The results informed the funder (UN institution) about the feasibility of extending funding for the project. The project involved close collaboration with the UN country teams and Government representatives.
What are the most effective strategies to improve immunization uptake?
Our experts supported a major UN client to understand the impact of interpersonal communication and motivational interviewing on timely immunization uptake among children in selected counties and localities.
The client is a major UN institution that works in 190 countries to reach the most disadvantaged children and adolescents, the Ministry of Health and Directorates of Public Health.
Countries worldwide are confronted with low vaccination uptakes. The low vaccination rates can be attributed to a broad range of factors: stockouts, lack of data on unvaccinated children, lack of information, migration of families with children. The decision by caregivers to refuse or delay vaccinations for their children is one-factor affecting vaccination coverage. It is strongly connected to the availability of information about immunization, its benefits, side effects, and interaction between caregivers and health workers.
Motivational interviewing (MI) is an example of a brief behavioral counseling intervention that improves the interpersonal communication (IPC) skills of health care workers (HCWs). IPC skills enable HCW to engage with caregivers to encourage timely uptake of recommended vaccines, with the ultimate goal of changing knowledge, attitude, and practices of caregivers towards immunizations. MI and other IPC techniques have been used successfully to encourage uptake of various health behaviors and improve the interaction between HCWs and caregivers by encouraging active listening and providing feedback to caregivers’ concerns (2-4). The interventions aimed at teaching hospital, primary and community-based health care workers how to initiate and maintain conversations with new mothers about vaccination and teach community-based mobilizers how to approach caregivers who defaulted on childhood vaccination visits. The goal of these conversations will be to deliver information about vaccines, probe for vaccine hesitancy, address concerns, and establish a personal relationship with the caregiver.
The client wanted to know whether the MI training produces any changes in the HCPs, caregivers’ behavior, and ultimately vaccination uptake.
To evaluate the impact of MI training on HCPs, caregivers’ behavior, and ultimately on vaccination uptake, HDI Group designed and implemented a 3-year impact evaluation. Briefly, we considered the difference in difference (DiD) method. In a DiD design, we will be collecting data on all the eligible units of the reference population: those who will enroll in the program (whether finally participating or not). To minimize bias associated with assuming that the treatment group would have experienced any changes in the control group’s outcome in the absence of the MI intervention, we will consider either randomization or propensity score matching. Propensity score matching groups individuals based on their likelihood to participate in the project training. Hence, this will account for extraneous factors that may influence the outcome of interest. For this purpose, registry data, semi-structured surveys on Knowledge, Attitudes, and Practices, in-depth interviews, and vignettes will be used with caregivers and HCWs.
This study is ongoing.