Evidence
What we have measured, on what data, and what it does not yet show. Every pass mark was written down before the test was run.
| Component | Result | Data | What it means |
|---|---|---|---|
| Liver outline | 0.951 average Dice; 1.17% average volume error | 102 of 103 external colorectal cases (1 excluded), MCT-LTDiag (public) | The liver outline closely matches reference outlines, and liver volume is measured to within about 1% on average. |
| Lesion detection | 97.5% of lesions over 20 mm; 78.5% of lesions 10 to 20 mm | 103 external colorectal cases, MCT-LTDiag (public) | Larger lesions are found reliably. Smaller lesions and false positives (2.26 per scan, above our target) are the main work ahead. |
| Lesion matching | 128 / 129 expert-outlined lesions linked correctly | 37 of 43 sealed scan pairs the alignment accepted (6 refused, shown "not available"); public melanoma longitudinal research set (Longitudinal-CT) | Given outlined lesions on two scans, the matching step links the right ones. This tests matching, not detection and matching together. |
These are research results on public data, not clinical validation. They were not measured on NHS patients.
Show the numbers as a table
| Lesion size | Found |
|---|---|
| 10 to 20 mm | 78.5% |
| Over 20 mm | 97.5% |
| False positives per scan | 2.26 |
How we test.
- Pass marks are written down before a result is seen, so a test cannot be tuned to pass.
- Held-out data is kept sealed until the test it was set aside for.
- We report surface and volume error beside overlap, because overlap alone can hide a large volume error.
- Every figure is quoted with the dataset and number of cases it was measured on.
What we will show next.
- Detection and matching working together on real paired follow-up scans (so far tested on research and constructed data).
- Performance on NHS imaging, across scanner vendors and hospitals.
- Agreement between our lesion measurements and radiologists' own measurements. Our first test did not yet reach the agreement we require, and this is a priority.
- A reader study: time saved and agreement between radiologists.
- Results across several datasets, against reference outlines checked independently to a written standard.
These are the studies we are looking for partners to run. Get in touch.
Work with us
We are looking for hospitals and research groups with baseline and follow-up CT of colorectal liver metastases, radiologists for a reader study, and academic and NHS partners for funded validation.