-
Notifications
You must be signed in to change notification settings - Fork 65
Scraping periodically returns potentially cached results. #46
Description
Issue: When posting a new video, scraping returns the correct video id's including the new video, the next scrape returns the previous videos without the new video, the third time scraping returns the correct video ids.
How to reproduce:
Every 70 seconds
videos = scrapetube.get_channel(channel_username = your youtube username)
video_ids = [video['videoId'] for video in videos]
print(video_ids)
Run the script and wait for a print to compare with, then post a video on your youtube channel, and wait for 3+ prints, then compare the results.
I'll use numbers instead of youtube video ID's to demonstrate what results i get, think of the number as a youtube video ID.
[5,4,3,2,1] (state of the channel before new video posted)
[6,5,4,3,2] (new video posted)
[5,4,3,2,1] (scrape now returns the old state of the channel, the previous 5 videos, is this from cache?)
[6,5,4,3,2] (from now on, it returns the correct video ID's)
[6,5,4,3,2]
[6,5,4,3,2]
EDIT: I increased the scrape period to 120 seconds and that worked