François Pelletier
46ee4cc650
fix: update Goodreads scrapers for scrapy 2.12 and new site markup
- bump scrapy API: from scrapy import Item/Field, .get() instead of .extract_first()
- replace pandas with stdlib json for reading the list
- extract book metadata from schema.org JSON-LD (old #metacol selectors obsolete)
- add browser UA + headers (book pages return 403 otherwise)
- FEEDS overwrite=True to avoid corrupt append
- pin deps in requirements.txt (scrapy 2.12, Twisted 24.10)
|
2026-09-02 18:44:51 -04:00 |
|
clean_data
|
ajustements faits en écrivant l'article
|
2022-02-06 18:26:27 -05:00 |
|
mask_img
|
premiere version pour git
|
2021-12-29 00:52:54 -05:00 |
|
out_img
|
ajustements faits en écrivant l'article
|
2022-02-06 18:26:27 -05:00 |
|
raw_data
|
premiere version pour git
|
2021-12-29 00:52:54 -05:00 |
|
scraped_data
|
ajustements faits en écrivant l'article
|
2022-02-06 18:26:27 -05:00 |
|
.gitignore
|
fix: update Goodreads scrapers for scrapy 2.12 and new site markup
|
2026-09-02 18:44:51 -04:00 |
|
convert_krausebooklist.py
|
premiere version pour git
|
2021-12-29 00:52:54 -05:00 |
|
Enrichissement.ipynb
|
ajustements faits en écrivant l'article
|
2022-02-06 18:26:27 -05:00 |
|
installation.ipynb
|
premiere version pour git
|
2021-12-29 00:52:54 -05:00 |
|
README.md
|
premiere version pour git
|
2021-12-29 00:52:54 -05:00 |
|
requirements.txt
|
fix: update Goodreads scrapers for scrapy 2.12 and new site markup
|
2026-09-02 18:44:51 -04:00 |
|
scraping_goodreads_books.py
|
fix: update Goodreads scrapers for scrapy 2.12 and new site markup
|
2026-09-02 18:44:51 -04:00 |
|
scraping_goodreads_list.py
|
fix: update Goodreads scrapers for scrapy 2.12 and new site markup
|
2026-09-02 18:44:51 -04:00 |
|
scraping_ongig_diversity_list.py
|
premiere version pour git
|
2021-12-29 00:52:54 -05:00 |
|
sources.md
|
premiere version pour git
|
2021-12-29 00:52:54 -05:00 |