Skip to content

FIX for PremProxy and FreeProxy #75

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

alsrua7222
Copy link

http_request_randomizer.requests.errors.ProxyListException.ProxyListException: list is empty
-> FreeProxyParser.py: html parsing is list error. (now solved.)

reqeusts.exceptions.InvalidURL: Failed to parse: http://(ip):(port) | FreeProxy
-> UnPack.py: headers add
-> PremProxyParser.py: headers add

@pgaref
Copy link
Owner

pgaref commented Jan 9, 2022

Thanks for the PR @alsrua7222 !
Tested locally and fixes both PremProxyParser by adding headers on the requests as #74 and FreeProxyParser by updating the attributes.
Can you please fix test_FreeProxyParser so we can merge this? I believe a simple update on free_proxy_mock is needed.

@urlmatch(netloc=r'(.*\.)?free-proxy-list\.net$') def free_proxy_mock(url, request): return """<table border="0" cellpadding="0" cellspacing="0" class="table table-striped table-bordered"

@pgaref pgaref changed the title solution work for FreeProxy FIX PremProxy and FreeProxy Jan 9, 2022
@pgaref pgaref changed the title FIX PremProxy and FreeProxy FIX for PremProxy and FreeProxy Jan 9, 2022
@alsrua7222
Copy link
Author

I didn't know if it was updated or not.
It's already been updated.
I wasted my time.
However, libraries help a lot with proxies. thank you.
After an error, when I traced back, the html parsing structure was changed.
Also, I wanted to point out that "headers" should be added to the library.

As you said, we simply updated the html table structure of free_proxy_list.net.
Is it correct to do this?

@pgaref
Copy link
Owner

pgaref commented Jan 12, 2022

I didn't know if it was updated or not. It's already been updated. I wasted my time. However, libraries help a lot with proxies. thank you. After an error, when I traced back, the html parsing structure was changed. Also, I wanted to point out that "headers" should be added to the library.

As you said, we simply updated the html table structure of free_proxy_list.net. Is it correct to do this?

Hey @alsrua7222 thanks for updating the PR!
The new html table structure is fine but since you used different proxy exmamples the assertion of the Test should also be fixed.

Please take a look at:

  self.assertEqual(proxy_list_addr, free_proxy_expected)

E AssertionError: Lists differ: ['58.234.116.197:8193', '20.122.24.225:80',[40 chars]080'] != ['138.197.136.46:3128', '177.207.75.227:8080']
E
E First differing element 0:
E '58.234.116.197:8193'
E '138.197.136.46:3128'
E
E First list contains 2 additional elements.
E First extra element 2:
E '154.236.177.100:1981'
E
E + ['138.197.136.46:3128', '177.207.75.227:8080']
E - ['58.234.116.197:8193',
E - '20.122.24.225:80',
E - '154.236.177.100:1981',
E - '54.37.160.92:1080']

@alsrua7222 alsrua7222 closed this Jan 12, 2022
@pgaref pgaref reopened this Jan 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants