I have a problem on scraping an e-commerce site using BeautifulSoup. I did some Googling but I still can’t solve the problem.
Please refer on the pictures:
1 Chrome F12 :
2 Result :
Here is the site that I tried to scrape: “https://shopee.com.my/search?keyword=h370m“
When I tried to open up Inspect Element on Google Chrome (F12), I can see the for the product’s name, price, etc. But when I run my python program, I could not get the same code and tag in the python result. After some googling, I found out that this website used AJAX query to get the data.
Anyone can help me on the best methods to get these product’s data by scraping an AJAX site? I would like to display the data in a table form.
import requests from bs4 import BeautifulSoup source = requests.get('https://shopee.com.my/search?keyword=h370m') soup = BeautifulSoup(source.text, 'html.parser') print(soup)