• : USA : +16-468109281, INDIA : +91-9811118919 | +91-9891020000

Web content crawler

This application visits various websites and downloads content of various types including text, images, and videos. The application can parse HTML pages of varying degree of complexity.

The key features are:

  1. Ability to visit and crawl password protected websites
  2. Ability to detect and follow links created dynamically such as by Javascript
  3. Ability to find images and videos which are protected by a session and locked on a device
  4. Ability to configure rules to filter URLs to follow
  5. Ability to extract information contained in a page, for example ability to extract from an ecommerce page
    1. Vendor name
    2. Product name
    3. Product price
    4. MOQ
    5. Product features
    6. Delivery features