Changelog
v3 - 8/11/24
Added
- Tor support for .onion websites
- TorManager class for handling Tor connections
- TorScraper class for scraping .onion sites
- Tor configuration and setup in Dockerfile
- Health check mechanism for container monitoring
Changed
- Modified WebExtractor to handle both regular and .onion URLs
- Updated Dockerfile with proper Tor configuration
- Improved error handling for different URL types
- Updated proxy handling to avoid affecting non-onion traffic
- Modified startup scripts for better service management
Fixed
- Global socket modification affecting all traffic
- Proxy configuration issues
- Docker networking configuration
- Error handling for failed Tor connections
Known Issues
- Docker container: Regular scraping works but Tor functionality needs fixing
Todo
- Fix Tor connectivity in Docker container
- Improve Docker service startup sequence
- Add better error handling for Tor service failures
- Implement proper circuit isolation for Tor connections
- Add documentation for Tor feature usage
- Improve Docker networking configuration
Developer Notes
The Tor integration is currently working in the native installation but needs fixes in the Docker environment. When using Docker, currently:
- Regular website scraping works normally
- Tor connections need additional configuration
- Service startup sequence needs optimization
Installation Notes
Until the Docker issues are resolved, users are advised to:
- Use native installation for full functionality
- If using Docker, be aware that Tor features may not work properly
- Follow updated installation instructions in README
Please report any issues or bugs on the GitHub issue tracker.
Full Changelog: CyberScraper-2077-v3...CyberScraper-2077-HotFix