Free Site Validator

FreeSiteValidator.com will be moving to a new server this evening, March 24th and the service will be unavailable for about 30 minutes. All new reports are being queued up until the move is complete. Our apologies for any inconvenience.

Update: 25 March 2009
This is taking quite a bit longer than expected. Once the move is complete we'll ramp back up the validator processing engine. Bear with us...

Update II: 25 March 2009
The move is complete. The site is running on a brand new shiny machine and your reports should be processing again.

I just want to post a reminder that reports older than 30 days will start getting purged as of this coming Sunday, February 1st. If you have an old report that you would like saved, please download an exportable copy of your report. Report summary data will not be purged, just the error details.

You can now export your site validation report to either JSON or PDF formats. This can be accessed at the top of the report page by clicking on the respective links. CSV format will follow shortly.

Please note, starting February 1st, any site report older than 30 days will have its details deleted. You will still be able to see the summary statistics for all reports you run, you just won't have access to the specific validation or request errors that were found. This is being done to try and free up space in the database which is currently sitting at nearly 40GB.

The PDF version of the report contains a list of all of the invalid pages, their errors, and a clickable link to the W3C's validator so that you can see the current status the page's validation. It also includes a list of all of the dead links along with where they were found in a clickable link format.

If you have any questions, problems, or suggestions, feel free to hop on over to the forum and let them be known.

Validating large sites can take quite a while. I've seen one report that took 6 days to process. In these instances I recommend breaking your site into smaller logical sub-sites when you can. For instance, you could break digg.com into many smaller sites based on category such as http://digg.com/apple, http://digg.com/design, http://digg.com/gadgets, etc. The validator bot will stay within the base URL that you provide for your sites so it will only visit URLs that exist within it. This will allow the entire site to be validated much quicker and should also make reading the reports a lot easier. This will result in increased validator bot traffic on your site, but most large sites can handle this additional overhead with no problem.

As usage of the Free Site Validator continues to grow, the stress and strain on my available server resources grows too. To accommodate for this, I am in the process of making two major changes to the service which should allow me to keep the service going without having to make additional investments in infrastructure for quite a while.

Exporting Reports

As requested in the forums, you will soon have the ability to export your report data in XML, JSON, CSV, and PDF formats. This will allow you to take the data generated by the validator and manipulate or use it any way you wish to help with fixing errors.

Report Expiration

In order to free up space on the server for additional reports, you will have 30 days from the date that your report finished processing to view and/or export your report data. After this time period, the report data will be purged from the system to make room for additional reports. If you want to keep your report data longer than that, you should make sure to export the report before it expires. Your report summary and statistics WILL NOT be deleted so you can see how your site has improved (or not) over time by looking at the total number of errors or dead links.

Stay tuned for future updates and if you have ideas for changes you'd like to see in future updates, let them be known in the Feature Requests forum.

The recent traffic spikes have wreaked a bit of havoc on my tiny VPS. While I'm working to sort the issues out, I know a few reports have failed due to the server crashing a few times. If this is the case for your site, just restart a new report. I've had to scale back the number of crawler processes, so processing might take a while now until things settle down.

Thanks for everyone's patience and understanding!

Roger Johanson wrote an article about the Free Site Validator today which has caused a massive surge in activity on the site. Although the site is holding up, as you can tell its a bit slow and I've had to throttle site report processing until I can upgrade my server's resources. So, if your site report doesn't begin processing right away, its in the queue and will be processed ASAP - I promise. In the meantime, I would highly recommend using the email notification setting on your sites so that you get an email when your report is complete.

If you're curious as to what's different between the service provided at FreeSiteValidator.com and say, the W3C's validation service, the answer is simple. The W3C's service allows you to enter in a single URL, validate it, and see the results. This is a very handy, and accurate service. However, if you manage an entire site that is changing constantly by multiple people, staying on top of validation throughout the full site can become challenging. Authors don't always remember to validate their work after they submit a new story or create a new page within your website, and if you're committed to a completely valid (X)HTML site, it can be a tedious task to stay on top of.

The FreeSiteValidator allows you to enter the base URL of your site which it then crawls and validates. In addition to validating that single URL it also extracts other URLs from the page and validates those as well. It continues this "spidering" process until every publicly available page on your site has been validated. During and after the process of crawling your site, you can see reports on which pages are not valid and the validation errors contained within t hem. In addition, the crawler checks all links, images, scripts, and javascript files to ensure they are still active. This helps to ensure that your pages are not serving what's called "dead content". So, in addition to finding out what pages on your site are invalid, you'll also be able to see a list of links that were referenced on your site which are now "dead".

It is important to note that the FreeSiteValidator uses the W3C's validation service on each page of your site. So the validation results you see for an individual page here on the FreeSiteValidator are identical to what you would see if you submitted the page manually to the W3C's validation service. The FreeSiteValidator just gives you the added convenience of bulk processing your entire website instead of having to do it one page at a time.

All you need to do to get started using the FreeSiteValidator is signup for a new account or login with your OpenID account if you have one. Once you've verified your email address you will be able to manage your sites, and initiate new reports. These reports can take a while depending on the size of your website, so you will receive an email once the entire site has been completely validated.

If you have any questions or suggestions about the service, feel free to stop by the forums and let us know.

I am now looking for beta testers to give this thing a shot. If you're interested, please sign up and you will receive an email when your account has been approved.

Although I've used this tool for some time, there were a few "user-friendliness" type features that I wanted to add before releasing this into the wild. The tweaks are almost done and soon enough you too will be able to go validation crazy with your websites.

If you're interested in being one of the first to use the service, be sure to sign up. I will be letting people onto the service slowly as I'm not 100% sure how the system will perform under load and the last thing I want is people coming here and nothing working at all.

Stay tuned...