Site risk (risk-based monitoring)
Key risk indicators of each site, thresholds, the risk level and the record of reviews.
Site risk shows which site of the study needs attention now. It is centralised monitoring as ICH E6(R3) describes it: the sponsor manages quality by risk (section 3.10) and uses the data to find sites that differ from the others (3.11.4). ICH E8(R1) names what such signals protect: the patients' safety, eligibility and the reliability of the key data.
The Site risk page is in the menu of every YouEDC Capture study. The numbers are computed again each time from the live data of the study; nothing is copied.
The table of sites
Sites come from the highest risk down. Each cell holds the value of an indicator, and its colour is the zone: normal (green), attention (amber), signal (red), too little data (grey). The last row is the whole study, so you see at once how far a site is from it. The arrow next to the points shows whether they went up or down since the site's last review.
Indicators
| Indicator | What is counted | Attention / signal | Weight | Scored from |
|---|---|---|---|---|
| Overdue visits | share of scheduled visits whose window closed with no data | 10% / 20% | 1 | 5 visits |
| Pages left unfinished | share of pages left "in progress" for more than 7 days | 10% / 20% | 1 | 10 pages |
| Queries per 10 pages | all queries (automatic and manual) for every 10 pages entered | 3 / 5 | 1 | 10 pages |
| Queries waiting too long | queries without the site's answer for more than 14 days, a count | 3 / 10 | 2 | 1 patient |
| Days to answer a query | median days from a query to the investigator's first answer | 5 / 10 | 1 | 3 answers |
| SDV backlog | share of completed pages without SDV for more than 14 days | 30% / 60% | 1 | 10 pages |
| Withdrawals | withdrawn patients out of all patients the site ever enrolled | 10% / 20% | 1 | 5 patients |
| Adverse events reported | AEs per patient at the site as a percentage of the study's rate; here fewer is worse | 50% / 25% | 2 | 5 patients |
| Randomisation problems | wrong stratum, randomised by mistake or not eligible, emergency unblinding, a count | 1 / 3 | 2 | 1 patient |
"Scored from" protects against false signals: at a site with two patients one withdrawal is 50%, which means nothing yet. Until there is enough data, the cell is grey and gives no points.
How the level is set
Attention gives 1 point, a signal 3, multiplied by the weight of the indicator. The sum is the site's points. By default medium risk starts at 2 points and high at 6. One signal with weight 2 (queries waiting or few adverse events) makes the site high risk at once: these indicators concern the patients' safety and data cleaning.
Reviewing a site
Open a site from the table: each indicator is a card with its value, the study's value (the black mark on the bar), the thresholds and how much data it is based on. A monitor, a data manager or the project administrator records a review: the decision (no action, watch, contact the site, a monitoring visit, escalate to the sponsor) and a note on what was looked at and what was decided.
The review saves the values and zones of all indicators at that moment, and the audit trail gets a "Site risk review" record. A review cannot be changed: it is the evidence of oversight for an inspection. At the next review you see whether the points went up or down.
Thresholds
Thresholds, weights and periods (how many days a query may wait, a page stay unfinished, SDV be late) are set in Site risk → Thresholds. Take them from the study's risk assessment and monitoring plan. Weight 0 switches an indicator off, for example the SDV backlog when the monitoring plan does not ask for SDV of every page. The adverse event form code is AE by default; give the code of your form if it differs. Every change of a threshold goes to the audit trail. "Recommended values" puts back the thresholds of the table above.
Who sees what
- The project administrator and the data manager see every site, record reviews and change the thresholds.
- A monitor sees and reviews only their sites. The study's value still counts all sites.
- The sponsor (viewer) sees the table and the cards: they are summary numbers with no patient data.
- An investigator does not see the page.
The home screen has a Site risk widget: the sites with the most points across your studies. Excel for the meeting downloads the coloured table and the thresholds.
Limits of stage 1
- YouEDC Capture has no protocol deviation log yet. It will be the next source of indicators.
- The indicators are signals for people, not a statistical finding. The statistical review of the data itself (outliers, data too smooth, digit preference) is stage 2, central statistical monitoring; its findings not yet reviewed can count here as the indicator “Central monitoring findings” (weight 0 by default).
- Before the study is launched, test patients count too.
- An answer counts as the investigator's when a member with the Investigator role wrote it.