 |
|
 |
|
Staying on Top |
|
Part of staying on top of the search engines is demonstrating that the site is being actively maintained. Changes to the home page every now and then show that. It can be a minor change in one paragraph.
Adding new links and pages are not a problem. This also shows the site is active and doing things. It is when you change the name of a web page or folder that effects finding you on the search engines.
The following assumes you do not have access to the server settings for your site and must do things via standard FTP. Also waiting for the hosting company to set up 301 redirects may not work out well. |
|
Best File Names |
|
Try to use web page file names that include the most important keywords for its contents. That will give you one more count for keyword density. Some search engines will give the site and file names some extra points for matches.
Don't get carried away. One or two words works the best. Run the words together or use dashes as separators. Do not use spaces or punctuation used elsewhere in URL's in folder or file names. Spaces in URL's can cause problems along with using colons (:), commas (,), slants (\/), or periods (.). The semicolon (;) can create some really messy problems. Computers do not understand being cute or inventive. Stick to the basics.
I avoid using underlines in a URL because when underlined as clickable, you can't tell if it has spaces or underlines. People will usually try to type it with spaces.
If you are using a system that generates dynamic URL's, use one that gets the keywords into them instead of cryptic numbers. Note the URL in the Address line above. This page is about "Changes". |
|
Renamed Web Pages |
|
Replace your original web page text with code including a Meta tag Refresh redirect line in the heading. It can look something like the following:
 !DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"
HTML
HEAD
TITLEBIZyCart - Redirect/TITLE
META HTTP-EQUIV="Content-Style-Type" CONTENT="text/css"
LINK REL="stylesheet" TYPE="text/css" HREF="BIZyCart.css"
META NAME="ROBOTS" CONTENT="NOINDEX,NOARCHIVE"
META HTTP-EQUIV="REFRESH" CONTENT="0;URL=Index.htm"
/HEAD
BODY
TABLE BORDER=0 CELLPADDING=0 CELLSPACING=0 WIDTH="100%" HEIGHT="100%"
TR>TH
A HREF="Index.htm"
Please click here if the page does not appear./A/TH/TR
/TABLE
/BODY>/HTML
|
Note: You'll need to restore the arrow brackets if you copy and paste the above
The above example displays the link text centered and in the middle of the web page. Make the following corrections before uploading the page:
- Replace the TITLE with what you feel is appropriate.
- Change the style sheet to your preference or leave those lines out and set the font inside the web page body.
- Change where "Index.htm" appears to the name of new web page serving the same purpose.
- You can change the appearance of the table to suit.
- In some cases, you may want the page to remain visible for a while. If so, change the "0" in the CONTENT= to the number of seconds you want. If the page has images to display, the browser will start the delay after all of the images are loaded.
The Robots Meta tag in this example tells the browsers to stop indexing the old web page name and start indexing the new one. It serves the same purpose as a 301 redirect. In the process, any search ranking for the original web page will be passed to the new one.
The example above also ensures that any links to the old page on other web sites won't stop working. After a month or two and you are sure all of the old links have been replaced, you can remove the redirect page. If your site visitors may have bookmarked the old page, you may want to wait longer. |
|
Deleting Web Pages |
|
Just deleting a web page does not tell the robots what you want to happen. They will leave it indexed and archived until they decide to take action on the "Not Found" condition. That can take months.
Replace the web page text with code including a Meta tag Robots line in the heading. It can look something like the following:
!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"
HTML
HEAD
TITLEBIZyCart - Deleted Page/TITLE
META NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW,NOARCHIVE"
/HEAD
BODY
/BODY>/HTML
|
Note: You'll need to restore the arrow brackets if you copy and paste the above
Replace the TITLE with what you feel is appropriate. The web page Body can include text if you want to tell visitors to the page why it's gone now. See Robot Control for more information on using the Robots Meta tag.
The Robots Meta tag in this example tells the browsers to stop indexing the old web page and remove any archived copies. |
|
Renamed ASP Pages |
|
There are several ways to put an *.ASP page to work. Renaming or shifting to dynamic content for a site or folder are the main ones.
To relocate a MyOldPage.asp page to MyNewPage.asp you would replace the contents of the MyOldPage.asp with the following:
%
Response.Redirect "MyNewPage.asp"
%
|
Using Response.Redirect automatically returns the heading information for a 302 redirect and the search engines will shift to indexing the new web page. This works the same as the Meta tag Refresh line above. Additional code can be added to force a 301 redirect. For example:
%
Response.Status="301 Moved Permanently"
Response.AddHeader "Location", "MyNewPage.asp"
%
|
The examples use relative addressing. You can use a fully qualified URL (like http://bizycart.com/MyNewPage.asp) if going to a different domain.
I use the following as the default ASP page for a web site to launch BIZyCart with the parameters for the home page.
%
Response.Redirect "BIZyCart.asp?ACTION=Home&CLIENT=SiteName"
%
|
Note: You'll need to restore the arrow brackets if you copy and paste the above
You can redirect to another web site by including the full URL with the domain name. Similar redirects can be done with PHP, but requires more coding. I prefer ASP or VB scripting because it dovetails with the Windows operating system instead of the C++ style slower overlays with PHP. It's a toss up on Linux servers. |
|
Renaming Folders |
|
I do not recommend renaming folders or directories that contain web pages. Instead, create a new folder and leave the old folder in place.
Use the redirect examples above to replace all of the web pages in the old folder. This avoids breaking any existing links on the search engines and other web sites. |
|
Moving to a New Server |
|
Sometimes it is necessary to move a web site from one hosting server to another. If done properly, only minimal disruptions will occur. The search engine may have a few problems during the transfer but otherwise the search rankings should be retained.
An exception would be changing countries on Google and other search engines that try to use geolocation. Unlike Zip codes and telephone numbers, IP numbers are scattered all over the world without any geographical organization. There is probably a main data base with who is using what IP blocks but I don't think it's available for on-line query or useful for country lookup. there are 255 x 255 x 255 x 255 IP numbers so it would be a rather massive data base.
To do it right, both old and new web sites should remain active during the first 72 hours. Until the Name Servers for the domain record are changed, the old site will continue operating.
- When the new site is fully functional using the IP number or the Preview Link, the domain record should be updated with the name servers for the new host.
- The DNS records for the old site location should be updated to point to the new IP number.
- Use an entry like: "NSLOOKUP example.com" from the DOS prompt to verify the IP numbers have shifted to the new location. You can expect it to change back and forth for 24 to 72 hours depending on how well the two versions of sites are hosted.
- When the DNS lookup for the domain name seems to be stable, or at the 72 hour mark, the old location of the web site should be completely uninstalled. Just stopping it leaves the DNS Bind files active.
If you do not have access to the DNS settings for the old site and the old hosting company will not cooperate on changing the DNS, you will have to make do with insisting the old site be completely uninstalled on schedule. This means some people will not be able to reach your new site and the search engines may not be able to reach the web pages while the DNS Root Servers get updated and stabilized.
Which web site location visitors and the search engines see will bounce back and forth while both locations are trying to keep the DNS Root Servers up to date. If the site changes shopping carts, it can be a bit snarly making sure orders don't get lost. |
|
Moving to New Domain Name |
|
Moving to a new domain name should be as easy as setting up hosting for the new name and closing down the old one. However, that will not retain any of the search engine rankings the old site may have earned and it breaks any links or bookmarks.
Program wise, redirecting a web page functions similar to links between web sites. Ideally, the ranking points for the source page should be passed to the destination page. However, that can only happen properly the first time a search engine finds the redirect and the destination page does not have any points yet. Thereafter, only the usual link points would be added.
Since the search engines are trying to protect themselves against buying better rankings, purchasing domain names with good ranking and redirecting them to an existing web site won't gain much. At best, there would be a one time boost for having a link from another web site.
If at all possible, the old web site needs to stay active for several months. After the new site is active and ready for visitors, use the following stages to finish the move:
- Use page by page Meta Refresh (302) redirects to transfer visitors from your old web site to matching pages on the new site. If you have a really massive number of pages, you can use one of the methods to 302 redirect the entire site to the home page or a site map for the new web pages. If the web page names stay the same, it makes it easier.
The 302 Found Elsewhere keeps both pages active in the search engine data bases so they retain their points. A 301 Moved Permanently redirect will remove the old URL's along with their ranking values.
- When the new site is doing as well or better than the old site for searches, then you are ready to start phasing out the old web pages. At this point, many of the old web pages may already have stopped displaying for searches. However, you may still have many clients who have added pages to their favorites or bookmarked them.
Start your phase out by shifting to 301 redirects to tell the search engines to remove the old pages completely. This can take some time and you'll need to do some checking to make sure they go away.
- When all of the old pages are gone and you are comfortable your existing clients have found the new pages and links from other sites have been fixed, you can disable or turn off the old site. One of the better ways is to have the old domain name set up as an alias of the new domain or add a CNAME record for DNS. By keeping the old domain name active, old links from other sites will keep working.
Every web site is different. The above are only some of the basics and you should think through how you want to proceed carefully. It must be done right the first time if you want to retain as much of the search engine rankings as you can. |
|
Redirect Codes |
|
There are three Redirect codes you can put to work if your hosting company will help with them or inside your scripting.
301 Moved Permanently
302 Found Elsewhere (Which is the default)
307 Temporarily Redirected
The search engines will use the above codes to make decisions about how to index the pages involved.
503 Service Temporarily Unavailable
Is another handy code if you need to have the search engines come back again later. |
|
HTTP Informational Response Codes (1xx) |
|
This class of status code indicates a provisional response, consisting only of the Status-Line and optional headers, and is terminated by an empty line. Since HTTP/1.0 did not define any 1xx status codes, servers must not send a 1xx response to an HTTP/1.0 client except under experimental conditions.
100 Continue
This means that the server has received the request headers, and that the client should proceed to send the request body (in the case of a request for which a body needs to be sent; for example, a POST request). If the request body is large, sending it to a server when a request has already been rejected based upon inappropriate headers is inefficient. To have a server check if the request could be accepted based on the request's headers alone, a client must send Expect: 100-continue as a header in its initial request (see RFC 2616 §14.20 – Expect header) and check if a 100 Continue status code is received in response before continuing (or receive 417 Expectation Failed and not continue).
101 Switching Protocols
This means the requester has asked the server to switch protocols and the server is acknowledging that it will do so.
102 Processing (WebDAV) (RFC 2518 )
As a WebDAV request may contain many sub-requests involving file operations, it may take a long time to complete the request. This code indicates that the server has received and is processing the request, but no response is available yet. This prevents the client from timing out and assuming the request was lost. |
|
HTTP Success Response Codes (2xx) |
|
This class of status code indicates that the client's request was successfully received, understood, and accepted.
200 OK
Standard response for successful HTTP requests. The actual response will depend on the request method used. In a GET request, the response will contain an entity corresponding to the requested resource. In a POST request the response will contain an entity describing or containing the result of the action.
201 Created
The request has been fulfilled and resulted in a new resource being created.
202 Accepted
The request has been accepted for processing, but the processing has not been completed. The request might or might not eventually be acted upon, as it might be disallowed when processing actually takes place.
203 Non-Authoritative Information (since HTTP/1.1)
The server successfully processed the request, but is returning information that may be from another source.
204 No Content
The server successfully processed the request, but is not returning any content.
205 Reset Content
The server successfully processed the request, but is not returning any content. Unlike a 204 response, this response requires that the requester reset the document view.
206 Partial Content
The server is delivering only part of the resource due to a range header sent by the client. This is used by tools like wget to enable resuming of interrupted downloads, or split a download into multiple simultaneous streams.
207 Multi-Status (WebDAV) (RFC 2518 )
The message body that follows is an XML message and can contain a number of separate response codes, depending on how many sub-requests were made. |
|
HTTP Redirection Response Codes (3xx) |
|
This class of status code indicates that further action needs to be taken by the user agent in order to fulfil the request. The action required may be carried out by the user agent without interaction with the user if and only if the method used in the second request is GET or HEAD. A user agent should not automatically redirect a request more than five times, since such redirections usually indicate an infinite loop.
300 Multiple Choices
Indicates multiple options for the resource that the client may follow. It, for instance, could be used to present different format options for video, list files with different extensions, or word sense disambiguation.
301 Moved Permanently
This and all future requests should be directed to the given URI.
302 Found
This is the most popular redirect code[citation needed], but also an example of industrial practice contradicting the standard. HTTP/1.0 specification (RFC 1945 ) required the client to perform a temporary redirect (the original describing phrase was "Moved Temporarily"), but popular browsers implemented it as a 303 See Other. Therefore, HTTP/1.1 added status codes 303 and 307 to distinguish between the two behaviours. However, the majority of Web applications and frameworks still use the 302 status code as if it were the 303.
303 See Other (since HTTP/1.1)
The response to the request can be found under another URI using a GET method. When received in response to a PUT, it should be assumed that the server has received the data and the redirect should be issued with a separate GET message.
304 Not Modified
Indicates the resource has not been modified since last requested. Typically, the HTTP client provides a header like the If-Modified-Since header to provide a time against which to compare. Utilizing this saves bandwidth and reprocessing on both the server and client, as only the header data must be sent and received in comparison to the entirety of the page being re-processed by the server, then resent using more bandwidth of the server and client.
305 Use Proxy (since HTTP/1.1)
Many HTTP clients (such as Mozilla[4] and Internet Explorer) do not correctly handle responses with this status code, primarily for security reasons.
306 Switch Proxy
No longer used.
307 Temporary Redirect (since HTTP/1.1)
In this occasion, the request should be repeated with another URI, but future requests can still use the original URI. In contrast to 303, the request method should not be changed when reissuing the original request. For instance, a POST request must be repeated using another POST request. |
|
HTTP Client Error Response Codes (4xx) |
|
The 4xx class of status code is intended for cases in which the client seems to have erred. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and whether it is a temporary or permanent condition. These status codes are applicable to any request method. User agents should display any included entity to the user. These are typically the most common error codes encountered while online.
400 Bad Request
The request contains bad syntax or cannot be fulfilled.
401 Unauthorized
Similar to 403 Forbidden, but specifically for use when authentication is possible but has failed or not yet been provided. The response must include a WWW-Authenticate header field containing a challenge applicable to the requested resource. See Basic access authentication and Digest access authentication.
402 Payment Required
The original intention was that this code might be used as part of some form of digital cash or micropayment scheme, but that has not happened, and this code has never been used.
403 Forbidden
The request was a legal request, but the server is refusing to respond to it. Unlike a 401 Unauthorized response, authenticating will make no difference.
404 Not Found
The requested resource could not be found but may be available again in the future. Subsequent requests by the client are permissible.
405 Method Not Allowed
A request was made of a resource using a request method not supported by that resource; for example, using GET on a form which requires data to be presented via POST, or using PUT on a read-only resource.
406 Not Acceptable
The requested resource is only capable of generating content not acceptable according to the Accept headers sent in the request.
407 Proxy Authentication Required
408 Request Timeout
The server timed out waiting for the request.
409 Conflict
Indicates that the request could not be processed because of conflict in the request, such as an edit conflict.
410 Gone
Indicates that the resource requested is no longer available and will not be available again. This should be used when a resource has been intentionally removed; however, it is not necessary to return this code and a 404 Not Found can be issued instead. Upon receiving a 410 status code, the client should not request the resource again in the future. Clients such as search engines should remove the resource from their indexes.
411 Length Required
The request did not specify the length of its content, which is required by the requested resource.
412 Precondition Failed
The server does not meet one of the preconditions that the requester put on the request.
413 Request Entity Too Large
The request is larger than the server is willing or able to process.
414 Request-URI Too Long
The URI provided was too long for the server to process.
415 Unsupported Media Type
The request did not specify any media types that the server or resource supports. For example the client specified that an image resource should be served as image/svg+xml, but the server cannot find a matching version of the image.
416 Requested Range Not Satisfiable
The client has asked for a portion of the file, but the server cannot supply that portion (for example, if the client asked for a part of the file that lies beyond the end of the file).
417 Expectation Failed
The server cannot meet the requirements of the Expect request-header field.
418 I'm a teapot
The HTCPCP server is a teapot. The responding entity MAY be short and stout. Defined by the April Fools' specification RFC 2324. See Hyper Text Coffee Pot Control Protocol for more information.
422 Unprocessable Entity (WebDAV) (RFC 4918 )
The request was well-formed but was unable to be followed due to semantic errors.
423 Locked (WebDAV) (RFC 4918 )
The resource that is being accessed is locked
424 Failed Dependency (WebDAV) (RFC 4918 )
The request failed due to failure of a previous request (e.g. a PROPPATCH).
425 Unordered Collection (RFC 3648 )
Defined in drafts of WebDav Advanced Collections, but not present in "Web Distributed Authoring and Versioning (WebDAV) Ordered Collections Protocol".
426 Upgrade Required (RFC 2817 )
The client should switch to TLS/1.0.
449 Retry With
A Microsoft extension. The request should be retried after doing the appropriate action. |
|
HTTP Server Error Response Codes (5xx) |
|
Response status codes beginning with the digit "5" indicate cases in which the server is aware that it has encountered an error or is otherwise incapable of performing the request. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and indicate whether it is a temporary or permanent condition. Likewise, user agents should display any included entity to the user. These response codes are applicable to any request method.
500 Internal Server Error
A generic error message, given when no more specific message is suitable.
501 Not Implemented
The server either does not recognise the request method, or it lacks the ability to fulfil the request.
502 Bad Gateway
The server was acting as a gateway or proxy and received an invalid response from the downstream server.
503 Service Unavailable
The server is currently unavailable (because it is overloaded or down for maintenance). Generally, this is a temporary state.
504 Gateway Timeout
The server was acting as a gateway or proxy and did not receive a timely request from the downstream server.
505 HTTP Version Not Supported
The server does not support the HTTP protocol version used in the request.
506 Variant Also Negotiates (RFC 2295 )
Transparent content negotiation for the request, results in a circular reference.
507 Insufficient Storage (WebDAV) (RFC 4918 )
509 Bandwidth Limit Exceeded (Apache bw/limited extension)
This status code, while used by many servers, is not specified in any RFCs.
510 Not Extended (RFC 2774 )
Further extensions to the request are required for the server to fulfil it. |
|
