Proposal: break files up, so there are exposed APIs for unit testing.
Right now, plannedWork/crawler.js has one exposed method that invokes five other methods together in complex nested logic. It's essentially a god class.
The underlying logic is brittle and heavily coupled. Separating the methods that build links from the methods that generate a list of route-date pairs from the page request from the parsing from the formatting for return will be necessary before any useful tests can be added.