Question about the `recall` and `recall area`

[textcunma]

I tried the code in this repo and got the result proposal recall area 41.133031, which is much lower than the Average Recall value (52.95, from Tab. 4) shown in your paper. Meanwhile, the Recall value reported in Average across all tIoUs was 78.6760 which seems to be more reasonable. So I'm wondering which one is the Average Recall?

[josch14]

As how I understand it, the 52.95 from the paper are only for the Event proposal part and not for the captioning results on learned event proposals. You also did the evaluation for this part?

I got a Recall of 87.2372 for the captioning part.

[textcunma]

The following is the output from the trained masked transformer model:

Details

--------------------------------------------------------------------------------
Average across all tIoUs
--------------------------------------------------------------------------------
| Bleu_1: 10.8643
| Bleu_2: 5.1233
| Bleu_3: 2.5911
| Bleu_4: 1.2892
| METEOR: 6.7267
| ROUGE_L: 10.3282
| CIDEr: 9.6086
| Recall: 78.6760
| Precision: 34.1658

[INIT] Loaded annotations from validation subset.
	Number of ground truth instances: 16654
	Number of proposals: 550392
	Fixed threshold for tiou score: [0.5  0.55 0.6  0.65 0.7  0.75 0.8  0.85 0.9  0.95]
[RESULTS] Performance on ActivityNet proposal task.
	Area Under the AR vs AN curve: 41.13303110363877%
proposal recall area: 41.133031

And the following is the output from the trained end-to-end masked transformer model:

Details

--------------------------------------------------------------------------------
Average across all tIoUs
--------------------------------------------------------------------------------
| Bleu_1: 10.1033
| Bleu_2: 4.8944
| Bleu_3: 2.4937
| Bleu_4: 1.2244
| METEOR: 6.0311
| ROUGE_L: 8.6687
| CIDEr: 9.31356
| Recall: 91.4836
| Precision: 25.6057

[INIT] Loaded annotations from validation subset.
	Number of ground truth instances: 16654
	Number of proposals: 1064099
	Fixed threshold for tiou score: [0.5  0.55 0.6  0.65 0.7  0.75 0.8  0.85 0.9  0.95]
[RESULTS] Performance on ActivityNet proposal task.
	Area Under the AR vs AN curve: 45.02161042392217%
proposal recall area: 45.021610

As you can see, there is a Recall (in the Average across all tIoUs section) and a proposal recall area (at the last line) in the output. Which one is the value 87.2372 you talked about?