Launching an Ubuntu 20.04 EC2 in AWS is a server edition with no GUI. These days scraper faced with higher security where CDN blocks puppeteer, wget or curl. I’ve tried impersonating all the header on a real browser in wget to get the same content but with no success.
The workaround I found was to install a real browser and run keyboard macro to scrape the content I needed with PyAutoGUI. It is not possible to run PyAutoGUI headless. First you need to setup TightVNC on Ubuntu Server.
sudo apt update
sudo apt install ubuntu-desktop
sudo apt install…