13th Annual Conference of the International Speech Communication Association

Portland, OR, USA
September 9-13, 2012

Power Mean Pyramid Scores for Summarization Evaluation

Sameer Maskey (1), Andrew Rosenberg (2)

(1) IBM Research, New York, NY, USA; (2) Queens College, CUNY, Queens, NY, USA

We present Power Mean Pyramid Scores (PMP), an evaluation metric that extends the Pyramid evaluation scheme for summarization by combining Sentence Content Units (SCU) scores using Power Mean. The Pyramid method generates a summarization score by linearly combining component SCU scores. We find that by combining SCU scores using Power Mean, we can optimize a single parameter, α, leading to significantly improved correlation with human judgements. We demonstrate this result through an empirical study based on TAC-08 evaluation.

Full Paper

Bibliographic reference.  Maskey, Sameer / Rosenberg, Andrew (2012): "Power mean pyramid scores for summarization evaluation", In INTERSPEECH-2012, 2378-2381.