Spidering https://www.mit.edu/

1. Retrieving: https://www.mit.edu/ at 23:23:37.
Size of page: 48.89kb. Starting indexing at 23:23:38. MD5 sum checked. Page content not changed
Links found: 0. New links: 0
2. Retrieving: http://calendar.mit.edu/ at 23:23:39.
Size of page: 120.29kb. Starting indexing at 23:23:40. Re-indexed
Links found: 167. New links: 167
3. Retrieving: http://news.mit.edu/ at 23:24:28.
Size of page: 184.61kb. Starting indexing at 23:24:28. Re-indexed
Links found: 117. New links: 114
4. Retrieving: http://news.mit.edu/2022/bringing-hope-transformation-congo-milain-fayulu-0524 at 23:25:10.
Size of page: 123.02kb. Starting indexing at 23:25:11. Re-indexed
Links found: 129. New links: 45
5. Retrieving: http://news.mit.edu/2022/gudni-thorlacius-johannesson-iceland-0524 at 23:25:39.
Size of page: 127.92kb. Starting indexing at 23:25:44. Re-indexed
Links found: 117. New links: 31
6. Retrieving: http://news.mit.edu/2022/mit-simmons-hall-interactive-canvas-0525 at 23:26:00.
Size of page: 115.65kb. Starting indexing at 23:26:00. Re-indexed
Links found: 111. New links: 21
7. Retrieving: http://news.mit.edu/2022/ngozi-okonjo-iweala-commencement-2022 at 23:26:14.
Size of page: 149.50kb. Starting indexing at 23:26:21. Re-indexed
Links found: 121. New links: 32
8. Retrieving: http://news.mit.edu/2022/ngozi-okonjo-iweala-commencement-2022-0217 at 23:26:35.
Size of page: 125.61kb. Starting indexing at 23:26:41. MD5 sum checked. Page content not changed
Links found: 0. New links: 0
9. Retrieving: http://news.mit.edu/2022/voice-for-change-jessica-chomik-morales-0524 at 23:26:41.
Size of page: 119.32kb. Starting indexing at 23:26:47. Re-indexed
Links found: 113. New links: 25
10. Retrieving: http://socialmediahub.mit.edu/ at 23:27:00.
Size of page: 16.23kb. Starting indexing at 23:27:06. Re-indexed
Links found: 30. New links: 26
11. Retrieving: http://whereis.mit.edu/ at 23:27:20.
Size of page: 7.66kb. Starting indexing at 23:27:20. MD5 sum checked. Page content not changed
Links found: 0. New links: 0
12. Retrieving: https://commencement.mit.edu/webcasts at 23:27:21.
Size of page: 59.88kb. Starting indexing at 23:27:22. Re-indexed
Links found: 59. New links: 58
13. Retrieving: https://twitter.com/mit at 23:27:44.
Size of page: 98.61kb. Starting indexing at 23:27:47. Re-indexed
Links found: 68. New links: 67
14. Retrieving: https://twitter.com/share?url=http://web.mit.edu/spotlight/congratulations-class-2022-0 at 23:28:07.
Size of page: 98.71kb. Starting indexing at 23:28:10. Re-indexed
Links found: 68. New links: 48
15. Retrieving: http://architecture.mit.edu at 23:29:01.
Unreachable: http 400
Links found: 0. New links: 0
16. Retrieving: http://arts.mit.edu/camit at 23:29:02.
Size of page: 94.49kb. Starting indexing at 23:29:04.
Indexed
Links found: 88. New links: 85
17. Retrieving: http://bcs.mit.edu at 23:29:33.
Unreachable: http 400
Links found: 0. New links: 0
18. Retrieving: http://calendar.mit.edu/ at 23:29:35.
Size of page: 120.29kb. Starting indexing at 23:29:41. MD5 sum checked. Page content not changed
Links found: 0. New links: 0
19. Retrieving: http://calendar.mit.edu/auth/facebook_login?previous_url=https%3A%2F%2Fcalendar.mit.edu%2F at 23:29:42.
Size of page: 109.74kb. Starting indexing at 23:29:55.