爬虫使用相关问题


伪装浏览器

from fake_useragent import UserAgent
import requests


ua = UserAgent()
print(ua.chrome)
header = {'User-Agent':str(ua.chrome)}
print(header)
url = "https://www.hybrid-analysis.com/recent-submissions?filter=file&sort=^timestamp"
htmlContent = requests.get(url, headers=header)
print(htmlContent)

fake-useragent

Requests.get in Python using “User-Agent” not simulating a browser request

How to use Python requests to fake a browser visit?


bs4使用技巧

bs4查询对象

How to find tags with only certain attributes - BeautifulSoup

BeautifulSoup :一些常用功能的使用和测试

How to find elements by class Ask Question

获取href

BeautifulSoup getting href [duplicate] Ask