The idea is almost same with mine. Since that the request.js package is difficult to maintain and extend. Wow, I should consider the module in my next refactor of crawler. Years ago I published my distributed crawler framework based on middleware called floodesh, and these years I've been thinking about how to construct the single-node version of crawler, the most import idea is composable like my design in floodesh, which is also inspired by koa compose. Glad to see this, I'm watching the great work.
The idea is almost same with mine. Since that the request.js package is difficult to maintain and extend. Wow, I should consider the module in my next refactor of
crawler. Years ago I published my distributed crawler framework based on middleware calledfloodesh, and these years I've been thinking about how to construct the single-node version of crawler, the most import idea is composable like my design infloodesh, which is also inspired bykoacompose. Glad to see this, I'm watching the great work.