↓Skip to main content

cs2370 Notes: 25 Web Scraping

··1 min 
import requests

resp = requests.get("https://homework.quest/")
resp.status_code
resp.raise_for_status()
resp.text

Scraping Wikipedia:

  • Trying to use regex
  • main
import requests
import bs4

resp = requests.get("https://homework.quest/")
resp.status_code
resp.raise_for_status()
resp.text
tree = bs4.BeautifulSoup(resp.text, 'html.parser')
xs = tree.select('a')
for x in xs: print("[", x, "]")