Scrapy - Web crawler

Links

Headless Browser Scraping

Scraping Onepage Apps, JavaScript Frameworks (React etc.)

Scrapy does not interpret JavaScript statements and therefore will probably not show what your browser renders if the website you are scraping relies on JavaScript a lot (for example single-page apps). There's no immediate plan to have Scrapy interpret JavaScript or render pages like a browser does. Splash is one solution to render JavaScript. There are other solutions, for example Selenium.

Possible Solutions

https://github.com/scrapy/scrapy/issues/4484
https://github.com/joelgriffith/navalia
Splash
Google Puppeteer - The best option, fully customizable
Selenium - I would rate it as 2nd number, not bad but not the best!