Project Description
I have several websites that need to be crawled and their data captured, cleaned, and delivered in a single, well-structured Excel workbook. The exact fields vary by site, but in most cases I’ll want core page information such as titles, visible text blocks, image URLs, and any clearly marked product or contact snippets that appear.
You are free to use the stack you’re most comfortable with—Python + BeautifulSoup, Scrapy, or Selenium are all fine—as long as the final result arrives as an Excel file with tidy column headers and no duplicate rows.
Deliverables
• An executable script (or notebook) with inline comments so I can rerun the scrape later.
• The completed Excel file containing all records pulled from the target sites.
Acceptance criteria
• Every URL I supply is scraped without missing data.
• Workbook opens without errors and all columns are consistently populated.
• Script respects robots.txt and rate limits; no site is blocked during the process.
If you can turn this around quickly while keeping the data clean, let’s get started right away.