Central statistical monitoring
The data of each site against the other sites: the methods, the threshold for many tests, what each finding means and its limits.
Central statistical monitoring compares the data of each site with the data of the other sites, so a monitor sees where a site's data look different and can ask why. ICH E6(R3) (3.11.4) asks centralised monitoring to look at the range, consistency and variability of data within and across sites; the methods here are the classic ones of the literature (Venet et al., 2012; Kirkwood et al., 2013; the TransCelerate position paper, 2013). Nothing is learned or guessed: the same data always give the same findings, and every finding says how it was found.
Open Central monitoring in the study menu, or the button of the same name on the Site risk page. The tests run again on the live data each time (in a big study they are kept for 10 minutes).
The methods
Each site is compared with all the other sites together. Number fields of every form are looked at, except computed fields (they repeat the fields they come from) and forms for the unblinded team.
| Method | What is compared | Test | A finding also needs |
|---|---|---|---|
| Values far from the other sites | the median of the first value of each patient at the site and at the other sites | robust z-score: the difference divided by its standard error built from the median absolute deviation (MAD) of all patients | a difference of at least half a robust standard deviation; at least 3 sites with enough data |
| Too smooth: small spread | the variance of all the site's values and of the other sites' (values with a modified z-score above 3.5 left out on both sides) | F test, degrees of freedom = patients − 1 (fewer than the values: conservative) | the site's variance at most half the others' |
| Too smooth: the same value in many patients | the value found in the most patients of the site | binomial, against the larger of: the share of the other sites' patients with that value, the share with their own most frequent value; times the number of different values at the site | at least 3 patients and 30% of the site's patients |
| Digit preference | the last digit of measured values (blood pressure, weight, heart rate…), each different value of a patient once | chi-square against ten equal shares, 9 degrees of freedom, from 50 values | the most frequent digit in at least 15% of values; when the other sites prefer the same digits, the site must also differ from them |
| Trend over time | the first value of each patient by the day of enrolment: the slope at the site and at the other sites | difference of slopes divided by its standard error, Student's t with n − 2 degrees of freedom | a change over the site's enrolment of at least half a standard deviation, enrolment over 60 days or more; at least 3 sites with enough data |
| Rates | adverse events per patient, withdrawals, queries per page, missing values | Poisson (adverse events, queries) or binomial (withdrawals, missing values) against the other sites' rate | 1.5 times the others' rate (half or less for adverse events: fewer is worse) |
Digits are tested only in fields whose values span at least 30 steps of their last digit at the site (from the 5th to the 95th percentile): a 0–10 scale is not a measurement, and in a range so narrow the last digits are not spread evenly, so such a site is listed as not judged. The trend is computed for the key fields of the study (marked critical in the eCRF), or for every number field when none is marked. The rates use the same counts as the site risk indicators (withdrawals, queries per 10 pages, adverse events reported) and the completeness rule of the data quality report: here they are tested against the other sites, there they are compared with thresholds.
Many tests and the threshold
A study with 60 number fields and 5 sites makes over a thousand tests. At the usual 0.05, about fifty of them would look "significant" by chance alone. So a result counts as a finding only when p is below 0.05 divided by the number of tests (the Bonferroni correction): with 1,000 tests the threshold is 0.00005. The page shows the number of tests and the threshold, and each finding shows its p. The threshold counts every test of the study, whichever sites the reader may see, so two people never see different verdicts.
The correction is conservative: a finding is rarely chance, but a small real difference may go unseen. That is why a finding also needs a difference large enough to matter (the right column above): in a big study a tiny shift can be "significant" and still mean nothing.
Not enough data
A site with few patients cannot be judged: two patients with the same blood pressure mean nothing. Below 5 patients (the setting), below 50 values for the digit test, or with fewer than 5 patients or 60 days of enrolment for a trend, the site gets not enough data instead of a verdict. The list "Not judged for lack of data" at the bottom of the page says what was missing for each site. The two numbers are set in Site risk → Thresholds (section "Central statistical monitoring"), and their changes go to the audit trail.
Reading a finding
Each finding is a card: the site, the method, the field and its form, then in plain words what was compared, the numbers, n and p against the threshold. The chart shows the site in blue against the other sites together in grey; hover a mark to see its exact numbers.
- Range chart (values far from the others, small spread): the dot is the median, the bar the quartiles, the line the 10th to 90th percentiles.
- Digits: the share of each last digit at the site and at the other sites; the dashed line is the 10% of evenly spread digits.
- Trend: the mean first value of the patients enrolled in each month, with the straight lines of the site (solid) and of the other sites (dashed).
- Bars (the same value, rates): the site's share or rate next to the other sites'.
Marking a finding
A monitor, a data manager or the project administrator opens Mark reviewed or turn into an action under the finding and writes what was looked at and concluded. Mark reviewed closes it: the comment, the author, the time and the numbers of that moment are kept, and the finding leaves the list "Not reviewed yet" (choose "All findings" to see it). Turn into an action also records a review of the site's risk with the decision (watch, contact the site, a monitoring visit, escalate): it appears among the site's reviews on the Site risk page. Both are written to the audit trail ("Central monitoring finding marked") and can never be changed; a new mark can be added later, and the card says when the numbers changed since.
In the site risk
The site risk has an indicator Central monitoring findings: the findings of the site not yet reviewed. Its weight is 0, so by default it does not count. When the monitoring plan uses the statistical review, give it a weight in Site risk → Thresholds: amber from 1 finding, red from 3. The site page of the site risk lists the site's findings with a link to their charts.
Blinding and who sees what
- The randomisation arm is never read. Forms for the unblinded team are always left out, whoever looks, so the review cannot reveal the arms.
- Patients who withdrew the use of their data are left out.
- The page shows only numbers per site, no patient. The sponsor's viewer may read it; an investigator does not see it.
- A monitor limited to some sites sees only their findings; the charts show the other sites only together, never by name.
- Marking needs the right to record a site risk review (manager, data manager, monitor); after the database lock nothing can be marked.
Limits
- The tests assume roughly bell-shaped values. Strongly skewed values (some lab results) can give a finding that the chart explains at once.
- Values of one patient are not independent. The tests count patients, not values, where it matters (the first value, degrees of freedom, distinct values for digits), and so they are on the safe side.
- A site that differs is not a site that cheats. Another population (a specialised centre), another device (a scale that weighs in 0.5 kg steps prefers 0 and 5), another season: look at the source before deciding.
- With two sites a two-sided comparison would flag both alike: each is "the others" of the other. So values far from the others and trends need at least 3 sites with enough data; below that they are listed under "Not judged for lack of data" with the reason. The one-sided tests (small spread, the same value, digits, rates in the direction that is worse) still judge two sites.
- Before the study is launched, test patients count too.