Special Issue: Integrated System Evaluation: A Consumer/Warfighter Perspective

2006 ◽  
Vol 3 (4) ◽  
pp. 206-206
Author(s):  
Jerry M. Couretas
Keyword(s):  
Integrated System ◽  
System Evaluation ◽  
Special Issue

Introduction to the special issue on evaluating word sense disambiguation systems

2002 ◽  
Vol 8 (4) ◽  
pp. 279-291 ◽  
Author(s):  
PHILIP EDMONDS ◽  
ADAM KILGARRIFF
Keyword(s):  
System Performance ◽  
Word Sense Disambiguation ◽  
Evaluation Methodology ◽  
System Evaluation ◽  
Word Sense ◽  
Special Issue ◽  
Sense Disambiguation ◽  
Automatic Systems ◽  
Better Than ◽  
The Way

Has system performance on Word Sense Disambiguation (WSD) reached a limit? Automatic systems don't perform nearly as well as humans on the task, and from the results of the SENSEVAL exercises, recent improvements in system performance appear negligible or even negative. Still, systems do perform much better than the baselines, so something is being done right. System evaluation is crucial to explain these results and to show the way forward. Indeed, the success of any project in WSD is tied to the evaluation methodology used, and especially to the formalization of the task that the systems perform. The evaluation of WSD has turned out to be as difficult as designing the systems in the first place.

Sign in / Sign up

Export Citation Format

Share Document