Created on : 23.12.2024 Last modified : 23.12.2024

Identifying Clinical Skill Gaps of Healthcare Workers Using a Decision Support Algorithm in Rwanda

Rwanda, 2021 - 2023

Reference ID

10.16909-dataset-55

Producer(s)

Haykel Karoui

Collections

Department of occupational and environmental health (DSTE)

Metadata

DDI/XML JSON

Identification

Survey ID number

10.16909-dataset-55

Title

Identifying Clinical Skill Gaps of Healthcare Workers Using a Decision Support Algorithm in Rwanda

Abbreviation or Acronym

ICSG-CDSA

Country

Name	Country code
Rwanda	RW

Abstract

Digital clinical decision support algorithms (CDSAs) that guide healthcare workers during consultations can enhance adherence to guidelines and the resulting quality of care. However, this improvement depends on the accuracy of inputs (symptoms and signs) entered by healthcare workers into the digital tool, which relies mainly on their clinical skills, that are often limited, especially in resource-constrained primary care settings. This study aimed to identify and characterize potential clinical skill gaps based on CDSA data patterns and clinical observations. We retrospectively analyzed data from 20,204 pediatric consultations conducted using an IMCI-based CDSA in 16 primary health centers in Rwanda. We focused on clinical signs with numerical values: temperature, mid-upper arm circumference (MUAC), weight, height, z-scores (MUAC for age, weight for age, and weight for height), heart rate, respiratory rate and blood oxygen saturation. Statistical summary measures (frequency of skipped measurements, frequent plausible and implausible values) and their variation in individual health centers compared to the overall average were used to identify 10 health centers with irregular data patterns signaling potential clinical skill gaps. We subsequently observed 188 consultations in these health centers and interviewed healthcare workers to understand potential error causes. Observations indicated basic measurements not being assessed correctly in most children; weight (70%), MUAC (69%), temperature (67%), height (54%). These measures were predominantly conducted by minimally trained non-clinical staff in the registration area. More complex measures, done mostly by healthcare workers in the consultation room, were often skipped: respiratory rate (43%), heart rate (37%), blood oxygen saturation (33%). This was linked to underestimating the importance of these signs in child management, especially in the context of high patient loads typical at primary care level. Addressing clinical skill gaps through in-person training, eLearning and regular personalized mentoring tailored to specific health center needs is imperative to improve quality of care and enhance the benefits of CDSAs.

Kind of Data

Clinical data [cli]

Unit of Analysis

First dataset was collected directly by the ePOCT+ CDSA during 20,204 pediatric consultations across 16 primary health centers in Rwanda. It includes anonymized patient, healthfacility and consultation data with key clinical measurements (temperature, mid-upper arm circumference (MUAC), weight, height, MUAC for age z-score, weight for age z-score, weight for height z-score, heart rate, respiratory rate and blood oxygen saturation (SpO2).)
Second dataset results from structured observations of 188 routine pediatric consultations at a subset of 10 health facilities. Clinicians used a standardized evaluation form to record clinical measurements, mirroring variables in the first dataset. This dataset is used to deepen the analysis from the primary dataset by understanding the reason for the patterns appearing from the quantitative analysis of the first dataset.

Version

Version Description

Version 2.1: Edited, anonymous dataset for public distribution.

Version Date

2024-09-30

Scope

Notes

First dataset :
• medical_case_local_id: Unique identifier for each consultation.
• group: Specifies the group of the consultation based on a specific period of time.
• medical_case_consultation_month: Month during which the consultation took place.
• BC...Age.in.months...7354.categorical: Age of the child in months, categorized.
• BC...Axillary.temperature...7823: Axillary temperature of the child in degrees Celsius.
• PE215...Heart.rate..beats.per.minute....7787: Heart rate measured in beats per minute.
• VS5...Respiratory.rate..breaths.min....8469: Respiratory rate measured in breaths per minute.
• PE214...Blood.oxygen.saturation.......8385: Blood oxygen saturation percentage.
• BC...MUAC.in.cm...7833: Mid-upper arm circumference (MUAC) in centimeters.
• BC61...MUAC.for.age.z.score...7839: MUAC-for-age Z-score.
• BM1...Current.Weight..XX.X.kg....7805: Weight of the child in kilograms.
• BC7...Weight.for.age..z.score....8434: Weight-for-age Z-score.
• BM52...Height..XXX.X.cm....if.length.is.measured.subtract.0.7cm...7435: Height of the child in centimeters (0.7 cm subtracted if length was measured instead).
• BC95...Weight.for.height...7451: Weight-for-height Z-score.
• hf_code: Anonymized code for the health facility where the consultation occurred.

Second dataset :
• T° Assessment: Whether the temperature was assessed (assessed/skipped).
• If skipped, why?: Reason for skipping the temperature assessment (if applicable).
• T° Quality of Assessment: Quality rating of the temperature assessment (sufficient/insufficient).
• If insufficient, why?: Reason for insufficient temperature assessment (if applicable).
• Extra remark about T°: Additional comments on the temperature assessment.
• MUAC Assessment: Whether the MUAC measurement was assessed (assessed/skipped).
• If skipped, why?: Reason for skipping the MUAC assessment (if applicable).
• MUAC Quality of Assessment: Quality rating of the MUAC assessment (sufficient/insufficient).
• If insufficient, why?: Reason for insufficient MUAC assessment (if applicable).
• Extra remark about MUAC: Additional comments on the MUAC assessment.
• Weight Assessment: Whether the weight measurement was assessed (assessed/skipped).
• If skipped, why?: Reason for skipping the weight assessment (if applicable).
• Weight Quality of Assessment: Quality rating of the weight assessment (sufficient/insufficient).
• If insufficient, why?: Reason for insufficient weight assessment (if applicable).
• Extra remark about Weight: Additional comments on the weight assessment.
• Height Assessment: Whether the height measurement was assessed (assessed/skipped).
• If skipped, why?: Reason for skipping the height assessment (if applicable).
• Height Quality of Assessment: Quality rating of the height assessment (sufficient/insufficient).
• If insufficient, why?: Reason for insufficient height assessment (if applicable).
• Extra remark about Height: Additional comments on the height assessment.
• RR Assessment: Whether the respiratory rate was assessed (assessed/skipped).
• If skipped, why?: Reason for skipping the respiratory rate assessment (if applicable).
• RR Quality of Assessment: Quality rating of the respiratory rate assessment (sufficient/insufficient).
• If insufficient, why?: Reason for insufficient respiratory rate assessment (if applicable).
• Extra remark about RR: Additional comments on the respiratory rate assessment.
• Sat Assessment: Whether the blood oxygen saturation was assessed (assessed/skipped).
• If skipped, why?: Reason for skipping the saturation assessment (if applicable).
• Sat Quality of Assessment: Quality rating of the saturation assessment (sufficient/insufficient).
• If insufficient, why?: Reason for insufficient blood oxygen saturation assessment (if applicable).
• Extra remark about Sat: Additional comments on the saturation assessment.
• HR Assessment: Whether the heart rate was assessed (assessed/skipped).
• If skipped, why?: Reason for skipping the heart rate assessment (if applicable).
• HR Quality of Assessment: Quality rating of the heart rate assessment (sufficient/insufficient).
• If insufficient, why?: Reason for insufficient heart rate assessment (if applicable).
• Extra remark about HR: Additional comments on the heart rate assessment.
• Remark about other SS: Any additional remarks about other signs and symptoms assessed during the consultation.

Keywords

Clinical Skill Gaps Clinical Decision Support Algorithm (CDSA) Children Pediatric care

Coverage

Geographic Coverage

16 primary healthcare centers (HCs) of Rusizi and Nyamasheke districts in Rwanda.

Universe

Children aged 1 day to 14 years with an acute condition, in the 16 HCs where the intervention was deployed.

Producers and sponsors

Primary investigators

Name	Affiliation
Haykel Karoui	Center for Primary Care and Public Health (Unisanté), University of Lausanne, Switzerland

Producers

Name	Affiliation	Role
Victor P. Rwandarwacu	Swiss Tropical and Public Health Institute in Rwanda (Swiss TPH RW)	Team manager
Jonathan Niyonzima	Swiss Tropical and Public Health Institute in Rwanda (Swiss TPH RW)	Data collection
Antoinette Makuza	Swiss Tropical and Public Health Institute in Rwanda (Swiss TPH RW)	Data collection
John B. Nkuranga	Swiss Tropical and Public Health Institute in Rwanda (Swiss TPH RW)	Data collection
Valérie D’Acremont	Center for Primary Care and Public Health (Unisanté), University of Lausanne, Switzerland and Swiss Tropical and Public Health Institute (Swiss TPH)	Supervision
Alexandra V. Kulinkina	Swiss Tropical and Public Health Institute (Swiss TPH)	Supervision

Sampling

Sampling Procedure

First dataset: : ePOCT+ stores all the information (date of consultation, anthropometric measures, vitals, presence/absence of specific symptoms and signs prompted by the algorithm, diagnoses, medicines, managements, etc.) entered by the HW in the tablet during consultations. We retrospectively analyzed data from 20,204 outpatient consultations conducted between November 2021 and October 2022 with children aged 1 day to 14 years with an acute condition, in the 16 HCs where the intervention was deployed. Data cleaning, management, and analyses were conducted using R software (version 4.2.1).

Second dataset: Based on the results of the retrospective analysis, we observed 188 routine consultations in a subset of 10 of 16 HCs (approximately 19 observations per HC). The selection of HCs was guided by the retrospective analysis, ensuring that the 10 HCs chosen were those showing the most critical results. The observing study clinician obtained oral consent from the HWs and was instructed not to interfere with the consultation to avoid introducing any additional bias to the observer effect. To ensure a standardized and consistent evaluation, a digital evaluation form (Google sheets) was used. These observations were conducted over 3 days per HC, with efforts made to separate them by a few days in order to have more chance to observe several different HWs and minimize potential bias. At the end of each day of observation in a HC (and not after each consultation to avoid any influence on subsequent consultations), the observing study clinician conducted an interview with the HW to understand why the assessment of some signs was skipped. Data were exported to Microsoft Excel (Version 16.77.1) for further simple descriptive analysis.

Deviations from the Sample Design

Second dataset: Most of the time, there was only one HW attending to children in the HC on a given day. On the rare occasions when two HW were present, each was observed by one of the two study clinicians.

Survey instrument

Questionnaires

The second dataset for this study was derived from structured observations of 188 routine pediatric consultations conducted across a subset of 10 health facilities. Clinicians utilized a standardized evaluation form that included variables aligning with those in the first dataset. This secondary dataset was designed to provide deeper insights into patterns observed in the primary dataset through the quantitative analysis.

The data collection focused on various clinical measurements and observations, categorized as follows:

General Information:
• Date of the consultation.
• Health facility (coded for anonymity).
• Clinical measurements taken at the reception and during the consultation.
• Presence of a conducting line.
• Additional remarks related to the consultation.

Clinical Measurements: For each of the following, the dataset records whether the measurement was assessed or skipped, the quality of assessment (sufficient/insufficient), reasons for skipping or insufficient assessments, and any extra remarks:
• Temperature (T°).
• MUAC (Mid-Upper Arm Circumference).
• Weight.
• Height.
• Respiratory Rate (RR).
• Blood Oxygen Saturation (Sat).
• Heart Rate (HR).

Additional Observations:
Remarks on other signs and symptoms assessed during the consultation.
The structured nature of this dataset ensures consistency in evaluating the reasons behind clinical decisions and the quality of care provided in routine pediatric consultations.

Data collection

Dates of Data Collection

Start	End
2021-11-01	2022-10-31
2023-01-01	2023-03-31

Mode of data collection

Data collected through ePOCT+ CDSA (Clinical Decision Support Algorithm)

Data Collectors

Name	Affiliation	Abbreviation
Haykel Karoui	Center for Primary Care and Public Health (Unisanté), University of Lausanne, Switzerland	HK
Jonathan Niyonzima	Swiss Tropical and Public Health Institute (Swiss TPH)	JN
Antoinette Makuza	Swiss Tropical and Public Health Institute (Swiss TPH)	AM

Data processing

Data Editing

Data editing was conducted as follows:
First data set:
• Data Extraction:
The dataset was extracted from the larger ePOCT+ storage system, which records all consultation-related information entered by healthcare workers (HWs) in tablets during consultations. This includes details such as the date of consultation, anthropometric measures, vital signs, the presence or absence of specific symptoms and signs prompted by the algorithm, diagnoses, medicines, and managements.

• Data Cleaning:
The extracted data were systematically cleaned to focus solely on the variables of interest for this analysis. Irrelevant variables and incomplete records were excluded to ensure a streamlined and accurate dataset.

• Anonymization:
To protect patient and health facilities confidentiality, the data were anonymized prior to analysis. All personal identifiers were removed, and only aggregated or coded information was retained.

• Analysis Preparation:
After cleaning and anonymization, the dataset was reviewed for consistency and coherence. Specific patterns of data were analyzed for the selected variables of interest, ensuring alignment with the study objectives.

• Software Used:
Data cleaning, management, and analyses were conducted using R software (version 4.2.1). All processes, including extraction, cleaning, and anonymization, were documented to maintain transparency and reproducibility.

Second dataset:
• Data Collection:
Data were collected directly from respondents through a Google Forms questionnaire. The structured format ensured standardized responses across all participants, facilitating subsequent data processing and analysis.

• Data Export:
Upon completion of data collection, the dataset was exported from Google Forms to Microsoft Excel (Version 16.77.1). This provided a structured and organized format for further data handling.

• Anonymization:
All personally identifiable information was removed during the data processing phase to protect participant confidentiality. Anonymization measures included replacing personal identifiers with unique codes and omitting any information that could reveal the identity of respondents.

• Data Cleaning and Descriptive Analysis:
The dataset was reviewed in Microsoft Excel to ensure consistency and completeness. Responses were screened for missing or inconsistent data, and necessary corrections were made where appropriate. Simple descriptive analyses were conducted within Excel to summarize key variables and identify initial patterns in the data.

Access policy

Location of Data Collection

Unisanté Data repository

URL for Location of Data Collection

https://doi.org/10.16909/dataset/55

Data Access

Access authority

Name	Affiliation	Email
RACINE, Céline (Repository manager)	Center for Primary Care and Public Health (Unisanté), University of Lausanne, Switzerland	udd.data@unisante.ch
Haykel Karoui	Center for Primary Care and Public Health (Unisanté), University of Lausanne, Switzerland	haykel.karoui@unil.ch

Access conditions

The datasets and R scripts are under embargo until the publication. Reviewers can request the files during the perr reviewing process. After publication, all files will be publish in Open Access under Creative Commons license

Citation requirements

Haykel Karoui, Identifying Clinical Skill Gaps of Healthcare Workers Using a Decision Support Algorithm in Rwanda (ICSG-CDSA-RW), 11/2021-10/2022, Version 2.1.

Availability Status

Public use files, accessible to all. Only for anonymous data

Contacts

Name	Affiliation	Email
Haykel Karoui	Center for Primary Care and Public Health (Unisanté), University of Lausanne, Switzerland	haykel.karoui@unil.ch

Study description in PDF

Back to Catalog