A smaller site, like this one is at present, is much easier to walk through and make sure things are working the way you expect them to. Larger sites, especially ones involving dynamic pages - usually with query string parameters - are far more labor intensive, which makes the work dull, meanial, more error prone. Html was intended to be a communications language between a web server and a browser; both are software, making software an ideal solution.
Several log analyzer programs will do this, using as their data requests made by clients. Microsoft's Visual Studio Team Foundation Server, Tester Edition, will test your web application for scalibility, security, bug regression, and more simple things, like which pages return 404 ( not found ) errors. Web Status Tester is the latest entry to the automated web site testing applications.
Web Status Tester
A public beta has become available: PageStatus.exe, and continue to evolve afterwards. Until then, the core functionality is available to developers, as PageStatusTester.cs. Documentation is available in an article on this site called Requesting Web URL Status From .net Code.