My IP appears to have been completely blocked by the domain. Multiple browsers, devices, confirm it.

If anyone has any suggestions for other options, I’m listening.

If we could target different byte ranges, having 10-20 different people spaced through the expected range could cover a lot of ground!

I had the script crash at line 324: BadStatusLine: HTTP/1.1 0 Init

EDIT: It’s worth noting that about every time I (re) start it after seemingly been blocked a bit, I get about 1gb more before it slows WAY down (no server response).

EDIT: It looks to me, that if I’m getting only FAILED: No server response, stopping the script for a minute or two and restarting immediately garners a lot more results. I think having a longer pause with many failures might be worth looking at. – I’ll play around a bit.

Gonna grab a some tea, then get back at it. Will update when I have something.

Thanks for this!

EDIT: This works quite well. Getting chunks right off the bat. About 1 per second, just guessing.

Can also confirm, receiving more chunks again.

EDIT: Someone should play around with the retry and backoff settings to see if a certain configuration can avoid being blocked for a longer period of time. IP rotating is too much trouble.

I also was getting the same error. Going to the link successfully downloads.

Updating the cookies fixed the issue.

Nor I. I got a single chunk back before never getting anything again.

I saw this too; yesterday I tried manually accessing the page to explore just how many there are. Seems like some of the pages are duplicates (I was simply comparing the last listed file name and content between some of the first 10 pages, and even had 1-2 duplications.)

Far as maximum page number goes, if you use the query parameter ?page=200000000 it will still resolve a list of files. — actually crazy.

https://www.justice.gov/epstein/doj-disclosures/data-set-9-files?page=200000000