NIST TAC Knowledge Base Population (KBP2017) Entity Discovery and Linking Track
- KBP Entity Discovery and Linking Scorer written by Ben Hachey and Joel Nothman
1. Clustering Performance:
For all mentions, we will ignore their KB IDs, use three measures to compute clustering performance:
(1) CEAF: Official measure because it's technically more sound than B-cubed. We will use CEAF-m (mention level) instead of CEAF-e (entity level) because CEAF-m is more sensitive to cluster size.
(2) B-cubed: in order to compare with results from previous years
2. Linking Performance:
We will use a standard Wikification F-measure to evaluate the performance of linking Non-NIL mentions to KB. A mention's label is correct only if its offsets, type and KB id are all correct.