Merge pull request #7 from bitmakerla/add-middleware

Add context factory middleware

Author: brenda151295

src/ps_helper/middlewares/README.md

@@ -0,0 +1,18 @@
+# LegacyConnectContextFactory
+
+This middleware provides a custom `ClientContextFactory` for **Scrapy** to fix the SSL error:
+
+`SSL routines: unsafe legacy renegotiation disabled error`
+
+---
+
+## Installation
+
+Place the extension in your Scrapy project and enable it in `settings.py`:
+
+```python
+DOWNLOADER_CLIENTCONTEXTFACTORY = "ps_helper.middlewares.LegacyConnectContextFactory"
+```
+Scrapy will then use the LegacyConnectContextFactory for all HTTPS connections.
+
+--------------------------------------

src/ps_helper/middlewares/__init__.py

@@ -0,0 +1,10 @@
+from scrapy.core.downloader.contextfactory import ScrapyClientContextFactory
+
+
+class LegacyConnectContextFactory(ScrapyClientContextFactory):
+    """Solve SSL routines unsafe legacy renegotiation disabled error."""
+
+    def getContext(self, hostname=None, port=None):
+        ctx = self.getCertificateOptions().getContext()
+        ctx.set_options(0x4)
+        return ctx