Skip to content

Use raw buffer to parse files#97

Open
PaperStrike wants to merge 3 commits intongryman:masterfrom
PaperStrike:master
Open

Use raw buffer to parse files#97
PaperStrike wants to merge 3 commits intongryman:masterfrom
PaperStrike:master

Conversation

@PaperStrike
Copy link

The main cause of #94 is unpkg (or the whole cloudflare, not sure) doesn't reply Content-Length anymore. Considering Content-Length isn't a reliable header nowadays, this PR enables badge-size to actually download almost all required files, which may significantly increase the bandwidth usage. Thus I implemented #16 at the same time.

The download threshold temporarily being set to 5 megabytes.

We have to find the best threshold before this is merged. 🤔


Demo: https://badge-size-teal.vercel.app/

- Fix the version number of target URL in 'broken absolute' test case
- Support download size limit
- Temporary set the download limit to 5 MB
- Update got to 11.8.2
@ngryman
Copy link
Owner

ngryman commented Sep 10, 2021

Hey @PaperStrike, thanks for the PR 🙌

I'm currently moving house, hence the delay in my answer, sorry about that.

I would need to think about the implications of dropping the HEAD HTTP method. Currently, the service already costs me quite a bit of 💰 and I'm concerned that downloading every file's payload would drastically increase my bills 😬

I'll probably dive into this during the course of this month. I might also consider an alternative of Vercel for hosting this project to reduce the costs.

@PaperStrike
Copy link
Author

It is worth to mention that if the raw sizes of files in npm packages matter significantly (I have no idea the amount of these users), we can try to read them from unpkg's official ?meta, or/and jsDelivr's official data.jsdelivr.com service. It's more reliable and faster.

@ngryman
Copy link
Owner

ngryman commented Oct 16, 2021

👋 Sorry for the lack of news.

Just a heads up that I've started a Rust implementation in the #rust branch.
It should improve a bunch of things. It's not relying on shields.io anymore so it should be more performant as well 🚀

I would like to switch to it during the next month of so 🤞

@ngryman ngryman mentioned this pull request Nov 10, 2021
@ngryman
Copy link
Owner

ngryman commented Nov 10, 2021

Alright, I have the Rust version running.
Could you take a look at #100? If everything goes well, I expect to push it to production by EOW. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Absolute url no longer supported? Limit file size

2 participants