Spidering https://hateball.com/

Disallowed files and directories in robots.txt:
https://hateball.com/admin
https://hateball.com/cart
https://hateball.com/orders
https://hateball.com/checkouts/
https://hateball.com/checkout
https://hateball.com/25031802957/checkouts
https://hateball.com/25031802957/orders
https://hateball.com/carts
https://hateball.com/account
https://hateball.com/collections/*sort_by*
https://hateball.com/*/collections/*sort_by*
https://hateball.com/collections/*+*
https://hateball.com/collections/*%2B*
https://hateball.com/collections/*%2b*
https://hateball.com/*/collections/*+*
https://hateball.com/*/collections/*%2B*
https://hateball.com/*/collections/*%2b*
https://hateball.com/blogs/*+*
https://hateball.com/blogs/*%2B*
https://hateball.com/blogs/*%2b*
https://hateball.com/*/blogs/*+*
https://hateball.com/*/blogs/*%2B*
https://hateball.com/*/blogs/*%2b*
https://hateball.com/*?*oseid=*
https://hateball.com/*preview_theme_id*
https://hateball.com/*preview_script_id*
https://hateball.com/policies/
https://hateball.com/*/*?*ls=*&ls=*
https://hateball.com/*/*?*ls%3D*%3Fls%3D*
https://hateball.com/*/*?*ls%3d*%3fls%3d*
https://hateball.com/search
https://hateball.com/apple-app-site-association

1. Link https://hateball.com/: file checking forbidden in robots.txt file

Completed at 18:54:57.