forked from HaydenKaufmann/McMaster_Scraper
-
Notifications
You must be signed in to change notification settings - Fork 0
cmarhoover/McMaster_Scraper
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
========================================== McMaster-Carr Part Scraping and CAD Downloader ========================================== This project contains two Python scripts designed to automate the process of extracting part numbers from a text file, fetching product details from McMaster-Carr, and downloading CAD files for each part. ----------------------------------------- Requirements: ----------------------------------------- - Python 3.x - Required Python libraries: - `selenium` - `pandas` - `webdriver_manager` ----------------------------------------- Setup Instructions: ----------------------------------------- 1. Paste McMasterCarr information into a .txt file in the project directory (similar to 'ExampleNutList.txt'). 2. Change file names in 'scrapePartNumbers.py' 3. Run `scrapePartNumbers.py` to generate `PartNumbers.csv`. 4. Run `fetchCad.py` to scrape product details and download CAD files. 5. Scraped data will be saved in the folder `part_files`, under subfolders for each part number. ----------------------------------------- Output: ----------------------------------------- 1. **PartNumbers.csv**: A CSV file containing the extracted part numbers. 2. **Part-specific folders**: Contain the product details and CAD files for each part number. ----------------------------------------- 1. Part Number Extraction Script (scrapePartNumbers.py) ----------------------------------------- Purpose: - Extracts part numbers from a text file (ExampleNutList.txt) and saves them in a CSV file (PartNumbers.csv). How it works: 1. Reads part numbers from `ExampleNutList.txt`. 2. Uses a regular expression to extract part numbers (e.g., "95462A029"). 3. Saves the extracted part numbers into `PartNumbers.csv`. Usage: - Place the part numbers in the text file (`ExampleNutList.txt`). - Run the script to extract part numbers and save them into `PartNumbers.csv`. ----------------------------------------- 2. Product and CAD Scraping Script (fetchCad.py) ----------------------------------------- Purpose: - Automates web scraping of product details and downloading CAD files from McMaster-Carr for each part number listed in `PartNumbers.csv`. How it works: 1. Reads part numbers from `PartNumbers.csv`. 2. Visits the McMaster-Carr website to retrieve product details (name and specifications) for each part. 3. Saves the product information in a text file and downloads the CAD file for each part (if available). 4. Stores the scraped data and CAD file in a folder corresponding to each part number. Usage: - Ensure you have a valid `PartNumbers.csv` file generated by the first script. - Run the script to scrape product details and download CAD files.
About
Scrapes McMaster-Carr CAD files and product details
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published
Languages
- Python 100.0%