In order to pass this test you must create and properly install a robots.txt file.

For this, you can use any program that produces a text file or you can use an online tool (Google Webmaster Tools has this feature).

Remember to use all lower case for the filename: robots.txt, not ROBOTS.TXT.

A simple robots.txt file looks like this:

User-agent: *
Disallow: /cgi-bin/
Disallow: /images/
Disallow: /pages/thankyou.html

This would block all search engine robots from visiting "cgi-bin" and "images" directories and the page "http://www.yoursite.com/pages/thankyou.html"

TIPS:

• You need a separate Disallow line for every URL prefix you want to exclude
• You may not have blank lines in a record because they are used to delimit multiple records
• Notice that before the Disallow command, you have the command: User-agent: *. The User-agent: part specifies which robot you want to block. Major known crawlers are: Googlebot (Google), Googlebot-Image (Google Image Search), Baiduspider (Baidu), Bingbot (Bing)
• One important thing to know if you are creating your own robots.txt file is that although the wildcard (*) is used in the User-agent line (meaning "any robot"), it is not allowed in the Disallow line.
• Regular expressions are not supported in either the User-agent or Disallow lines

Once you have your robots.txt file, you can upload it in the top-level directory of your web server. After that, make sure you set the permissions on the file so that visitors (like search engines) can read it.

In order to pass this test you must create a sitemap.xml file for your website. Some of the best practices are listed below:

• It is strongly recommended that you place your sitemap at the root directory of your website: http://yourwebsite.com/sitemap.xml But in some situations, you may want to produce different sitemaps for different paths on your site (e.g., security permission issues)
• Sitemaps should be no larger than 10MB (10,485,760 bytes) and can contain a maximum of 50,000 URLs. This means that if your site contains more than 50,000 URLs or your sitemap is bigger than 10MB, you must create multiple sitemap files and use a Sitemap index file
• All URLs listed in the sitemap must reside on the same host as the sitemap. For instance, if the sitemap is located at http://www.yourwebsite.com/sitemap.xml, it can't include URLs from http://subdomain.yourwebsite.com
• Once you have created your sitemap, let search engines know about it by submitting directly to them, pinging them, or adding the sitemap location to your robots.txt file
• Sitemaps can be compressed using gzip, reducing bandwidth consumption

sitemap.xml example:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
 <loc>http://www.yourwebsite.com</loc>
 <lastmod>2013-01-01</lastmod>
 <changefreq>weekly</changefreq>
 <priority>0.9</priority>
</url>
<url>
 <loc>http://www.yourwebsite.com/articles/100</loc>
 <changefreq>weekly</changefreq>
</url>
<url>
 <loc>http://www.yourwebsite.com/articles/101</loc>
 <lastmod>2013-01-02</lastmod>
 <changefreq>weekly</changefreq>
</url>
<url>
 <loc>http://www.yourwebsite.com/articles/102</loc>
 <lastmod>2013-01-02T13:00:12+00:00</lastmod>
 <priority>0.5</priority>
</url>
</urlset>

In order to pass this test you must add an alt attribute to every <img> tag used into your webpage.

An image with an alternate text specified is inserted using the following HTML line:

<img src="image.png" alt="text_to_describe_your_image">

Remember that the point of alt text is to provide the same functional information that a visual user would see. Search engines, users who disabled images in their browsers and other agents who are unable to see the images on your webpage can read the alt attributes assigned to the image since they cannot view it.

Learn more about optimizing images for SEO.

In order to resolve this problem you are advised to:

• use gzip compression
• move all CSS style rules into a single, external and minified CSS file
• minify all JS files and, if possible, try combining them into a single external JS file
• use CSS layouts

Your two options for file compression are Deflate and GZIP.

• Deflate is an option which comes automatically with the Apache server and which is simple to set up.
• GZIP on the other hand needs to be installed and requires a bit more work to install. However, GZIP does achieve a higher compression rate and therefore might be a better choice if your website uses pages which have a lot of images or large file sizes.

Setting up file compression for your website will depend on which type of server you're using for your website. Most likely, you'll be using Apache, which means you can enable compression by adding a few deflate codes to your .htaccess file.

# compress text, html, javascript, css, xml:
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/x-javascript

For more advanced information regarding deflate you can check this Apache documentation.

<IfModule mod_expires.c>
    ExpiresActive on
 
    ExpiresByType image/jpg "access plus 1 month"
    ExpiresByType image/jpeg "access plus 1 month"
    ExpiresByType image/gif "access plus 1 month"
    ExpiresByType image/png "access plus 1 month"
</IfModule>

<IfModule mod_expires.c>
    ExpiresActive on
 
    ExpiresByType text/javascript "access plus 1 month"
    ExpiresByType application/javascript "access plus 1 month"
   </IfModule>

<IfModule mod_expires.c>
    ExpiresActive on
 
    ExpiresByType text/css "access plus 1 month"
 </IfModule>

Creating a custom 404 error page can be relatively simple or more complicated, depending on what technologies were used to build your website (your web server, framework used, etc). Learn more about the basics of creating a custom 404 error page.

On your custom error page, you should provide relevant information to keep the user's attention so they remain on your website. Some steps to achieve this goal include:

• Adding links to your most important pages (your best articles articles/content, latest posts, etc.)
• If you have an e-commerce business, you can display special offers, product recommendations, discounts, etc.
• Include a search box or a sitemap link to help users find the information they wanted.
• Use simple language (non-technical) to explain what went wrong.
• Add a contact form or provide an email address to allow users to get in touch with you for further help.

You can also use your custom 404 error page to track the source of broken links in your site.