fix(scraper.py): double sending articles#48
Conversation
|
merging this PR the bot will be updated, let's monitor for 1 month the ersu website and the channel to be sure that it continues to publish the articles |
|
btw, good job! |
|
Thank you! |
I might have found a reason of that behavior. Since the function was looking for |
|
I am afraid that I could have done the mistake, due to the non-intuitive HTML structure. Btw, if the current solution works we can keep as it is, otherwise feel free to open a new PR |
I figured out that the function
soup.find_all('article')in line 178, listed two times some articles, here is the output of that function.I currently have no clue about the reason of this behavior.
I found that
<header class="sow-entry-header">was exclusively used for articles, here is the output ofsoup.find_all('header', class_='sow-entry-header')I tested this with the latest 10 ERSU articles and it works perfectly.
closes #40