Yandex Webmaster allows you to:
- add Sitemap;
- update;
- delete;
- analyze.
How to add a Sitemap to Yandex Webmaster
To add a Sitemap file to the queue for indexing, you need, as always, to go to the Webmaster service - webmaster.yandex.ru. Further:
- In the "Indexing" section, you will find the "Sitemaps" item.
- Add the full address of the sitemap files, for example, https://site/sitemap.xml.
- Click Add.
- Wait while Yandex Webmaster checks the file.
After adding, the file is queued for processing. The robot will add it within two weeks. Each added file, including those attached to the Sitemap index, is processed by the robot separately.
After loading, opposite each sitemap, you will see one of the statuses in Yandex Webmaster:
| Status | Description | Note |
|---|---|---|
| OK | The file is formed correctly and uploaded to the robot database | The date of the last upload will be displayed next to the file. Indexed pages will appear in search results within two weeks |
| « » | The specified URL redirects to another address | Delete the redirect and tell the robot about the update |
| "Error" | The file is not well formed | Click the Error link for more details. After making changes to the file, inform the robot about the update |
| "Not Indexed" | When accessing the Sitemap, the server returns an HTTP code other than 200 | Check if the file is available to the robot using the Check Server Response tool by specifying the full path to the file. If the file is not available, contact the administrator of the site or server where it is located. |
| Access to the file is denied in robots.txt using the Disallow directive | Allow access to the Sitemap and tell the robot to update |
Update Sitemap
When changes occur in the Sitemap, it does not need to be removed from Yandex Webmaster and re-added. The search engine checks it for changes and errors from time to time.
If your Sitemap is not indexed, then this can be accelerated. You need to go to the "Indexing" section, then to the "Sitemaps" item. Click the refresh icon next to the desired sitemap. Yandex will upload the data within three days. You can use the function up to 10 times for one host.
When you use up all attempts, the next one will be available 30 days after the first one. The exact date is displayed in Webmaster's interface.
Remove Sitemap from Webmaster
In the Yandex.Webmaster interface, you can delete those files that were added on the "Sitemap files" page:
If a directive was added to the Sitemap in robots file.txt, delete it. After making changes, information about the Sitemap will disappear from the database of the robot and Yandex.Webmaster within a few weeks.
Supported formats by Yandex Webmaster
Yandex accepts Sitemap in the following formats:
XML sitemap format
It is preferable to send in XML format, because it can carry additional data.
Yandex supports the Sitemap protocol. Use this format if you want to report about:
- date latest update pages (lastmod element);
- page change frequency (changefreq element);
- page significance (priority element) - the robot loads pages one by one, taking into account the presence and value of the coefficient from 0.0 to 1.0. Specify a coefficient for those URLs that are most important for the site.
TXT format Sitemap
V this format only page addresses can be passed. And that's it. Yes, sadness and sadness, better use XML 🙂
Use this format to report page addresses only.
https://website/ssylki-v-yandex-webmaster/ https://website/zerkala-sajta-v-yandex-webmaster/
To send information about a video, specify links to XML feeds in the Sitemap (for details, see the Help of the Ya.Video service).
Search does not support the transmission of information about images, RSS, and Atom feeds using a Sitemap file.
When do you need a sitemap file?
Yandex develops special algorithms by which the indexing robot learns about the site. For example, using internal and external links- moving from one page to another. Sometimes the robot may skip pages. Use a Sitemap if your site:
- a large number of pages;
- separate pages without navigation links;
- deep nesting.
How to create a file
Add Sitemap.xml to Yandex Webmaster - video
With our sitemap generator create XML files that can be submitted to Google, Yandex, Bing, Yahoo and more search engines ah, to help them index your site.
Do it in three easy steps:
Sitemap is a site map in XML format, which in 2005 the Google search engine began to use to index website pages. A sitemap is a way of organizing a website, identifying the address and data for each section. Previously, sitemaps were mainly directed to site users. The XML format was developed for search engines, allowing them to find data faster and more efficiently.
The new Sitemap protocol was developed in response to the increase in the size and complexity of websites. Business websites often contain thousands of products in their catalogs, and the popularity of blogs, forums, and message boards compels webmasters to update their content at least once a day. It is becoming increasingly difficult for search engines to keep track of all material. Through the XML protocol, search engines can track addresses more efficiently, optimizing their search by putting all the information on one page. The XML also shows how often a particular website is updated and records the latest changes. XML maps are not a tool for search engine optimization. This does not affect rankings, but it allows search engines to make more accurate rankings and search terms. It does this by providing data that is easy to read by search engines.
The general acceptance of the XML protocol means that web developers no longer need to create different types sitemaps for various search engines. They can create one file to submit and then update it when they make changes to the site. This simplifies the entire process of fine-tuning and expanding the website. Webmasters themselves have begun to see the benefits of using this format. Search engines rank pages according to the relevance of specific content. keywords, but before the XML format, often the content of the pages was not correctly represented. This is often frustrating for webmasters who realize that their efforts to build a website have gone unnoticed. Blogs, additional pages, adding multimedia files take several hours. Through the XML file, these hours will not be wasted, they will be seen by all known search engines.
To create your sitemap in XML format and keep search engines up to date with any changes to your site, try our free sitemap generator.
In this article, you will learn how to create a sitemap file and share it with Googlebots.
Creating and Submitting Sitemaps
Sitemap file formats
Google supports several sitemap file formats, described below. All formats should use the standard protocol. Google does not currently respect the attribute
All formats are subject to the following restrictions: A Sitemap file can contain no more than 50,000 URLs, and its uncompressed size can not exceed 50 MB. If the file size or the number of addresses listed in it exceeds these limits, split it into several parts. You can create a Sitemap index file listing all Sitemaps in it and submit them to Google all at once.
An example URL that contains a non-ASCII character (ü) and a character that needs to be escaped (&):
http://www.example.com/umlat.html&q=name
The same URL in ISO-8859-1 encoding (to be hosted on a server using this encoding) and with escaping applied:
http://www.example.com/%FCmlat.html&q=name
The same UTF-8 encoded URL (to be hosted on a server using that encoding) and with escaping applied:
The same URL with an escaped sequence instead of an ampersand character:
http://www.example.com/%C3%BCmlat.html&q=name
How to Make a Sitemap Available to Google
A Sitemap is only analyzed the first time it is found, not every time the site is crawled. In order for the file to be processed again, you need to inform us that the data in it has changed. This is done with a ping request. Don't add multiple times identical files Sitemap and don't ping unless the file has been modified and is not new.
There are several ways to grant Googlebots access to your Sitemap:
- Submit it to Google using a dedicated tool in Search Console
- Insert a line anywhere in the robots.txt file indicating the location of the Sitemap file:
Sitemap: http://example.com/sitemap_location.xml - Use a ping request. Send an HTTP GET request:
http://www.google.com/ping?sitemap=<polniy_url_faila_sitemap>
Example:
http://www.google.com/ping?sitemap=https://example.com/sitemap.xml
You're just a nerd if you didn't pay the right attention to the sitemap at the time. It is enough to understand the issue once and avoid it in the future. a large number mistakes, which let's do now.
Your obedient servant in his youth was also such a cretin when he just started promoting sites in one office. At that time, I came across one site for promotion, which, it should be said, was just shit. And this shit had problems with indexing. Naturally, if the site had been of sufficient quality, both search engines would have indexed it no matter what the problems, but the owners clung to a normal designer, layout designer and programmer, and in this case, the SEO can only, so to speak, open the bottle with scissors. I tried everything on it - both the last-modified setting, and the acceleration of indexing by a fastbot that was fashionable at that time, and the purchase of links. And only then it turned out that the point was that the sitemap was not automatically updated there! When I updated it, all the pages flew into the index.
What is a sitemap and why is it needed
What is a sitemap? This is a file with information about the pages of the site that need to be indexed. Usually a sitemap is created for Yandex and Google in order to notify search robots about the pages that need to be indexed. The sitemap also checks how often updates occur, and which web documents are indexed most importantly. In general, they spoke very well about it at Yandex Webmaster:
Does the presence of a sitemap affect promotion
If you do not have a sitemap, this does not mean that search engines will not index the resource. Search robots often do a good job of crawling sites and including them in the search. But sometimes failures can occur, due to which sometimes it is not possible to find all web documents. The main reasons are:
- Sections of the site that can only be reached by making a long chain of transitions;
- Dynamic URLs.
So, creating a sitemap.xml helps a lot to solve this problem. This file affects SEO only insofar as it facilitates / speeds up the indexing of pages. It also increases the chance that web pages will be indexed before your competitors can copy the content by publishing it on their site.
What other format is a sitemap in and why is it made in XML format
Why do we need a sitemap, we figured it out. Now let's look at what formats it can be done in:
- In html format. It is created in the form of an ordinary page with addresses leading to the main sections of the resource. This type of map helps you quickly navigate, and is designed more for people than for search robots. You can put a limited number of links in the HTML sitemap (no more than 100), because if there are more of them, then not all of them will be included in the index. Or search robots may completely exclude such a page from the search for an excessive number of URLs, even internal ones.
- Creating an xml file sitemap. There are no too critical restrictions on the number of links here, and search engines index it better, because the xml-format sitemap file contains complete information in a form understandable to the robot. It is especially important for projects where there are hundreds and thousands of documents of the same importance, and the placement of all links to them is necessary. In this type of sitemap, it is possible to place up to 50 thousand URLs and, in addition, you can set the update frequency and approximate priority (priority), which cannot be said about the map in HTML format. It is for these reasons that sitemaps are almost always created in xml.
Here is more info about this file:
How to make the right sitemap
Consider how to make the correct xml-map. The following requirements must be met here:
- The file size should be no more than 10 MB;
- The map must contain no more than 50,000 links. In cases where there are more links, you can create several maps and include them in the main xml map;
- The sitemap address should be written in robots.txt;
- Also upload the sitemap to Yandex and Google (how to add a file is described below);
- Search engines must have access to the map. It is necessary to use special tags that let search engines know that this is a map, and not something else;
- The sitemap must be UTF-8 encoded.
Here is a simple map example:
< url > < loc >http: //site.ru/ < lastmod >2016-11-20T19:45:08+03:00< / lastmod > < changefreq >always< / changefreq > < priority > 0 , 9 < / priority > < / url > < url > < loc >http: //site.ru/category/ < lastmod >2016 - 11 - 20T19 : 46 : 38 + 03 : 00< / lastmod > < changefreq >monthly< / changefreq > < priority > 0 , 6 < / priority > < / url > < url > < loc >http: //site.ru/page/ < lastmod >2016 - 11 - 20T19 : 48 : 41 + 03 : 00< / lastmod > < changefreq >yearly< / changefreq > < priority > 0.4 < / priority > < / url > |
The url and loc tags are required. The first prescribes all the information about a particular URL. The second contains the address itself.
The lastmod, changefreq, priority tags are optional, but still recommended.
Lastmod in the sitemap is responsible for the date of the last update.
Changefreq indicates how often the page changes. Values can be:
- Hourly - update hourly;
- Always - always updated;
- Weekly - updated once a week;
- Daily - updates occur daily;
- Monthly - updates occur once a month;
- Yearly - once a year;
- Never - does not update (it is better not to use such a value).
Priority tells search engines how important a page is compared to others. The priority can be set from 0.1(low) to 1(high).
This was just an example of a map, you don't need to specify those exact values. In general, priority is recommended to be set as follows: maximum for home page(1), for headings the average (0.6), and for entries - the minimum (0.4).
Now consider an example when there are more than 50 thousand links. In this case, the file includes other maps:
< sitemap > < loc >http : //site.ru/sitemaps/sitemap01.xml < lastmod >2016 - 11 - 20T21 : 37 : 28 + 03 : 00< / lastmod > < / sitemap > < sitemap > < loc >http : //site.ru/sitemaps/sitemap02.xml < lastmod >2016 - 11 - 20T21 : 37 : 29 + 03 : 00< / lastmod > < / sitemap > |
How to create a sitemap
There are several ways to create an xml map, consider them:
- Download a map using an online generator from another resource;
- Generate with special program. But it is worth considering that programs of this kind are mostly paid. An example of such a generator is the Wonder WebWare SiteMap Generator. Screaming Frog also has this feature;
- Create sitemap manually;
- Automatically create a map using CMS (for example, such a function is available on WordPress).
Here is an option on how to make a sitemap without the help of plugins:
WordPress sitemap plugins
You can create a sitemap in WordPress using a special plugin called Google XML Sitemaps. Everything is simple here: download the plugin, install it, then proceed to create the file. To do this, open the Console-Settings, and select XML-sitemap. Next, set the settings. Leave the priority at the default.
A site map can be compared to a table of contents in a book containing a list of all the most important sections. On the site, this is a directory where links to the main sections and pages of the web resource are indicated. In general, the sitemap contains the following information:
- list of URLs of site pages in hierarchical order;
- the format of the content that appeared on the site: text, video, pictures;
- information about videos and images necessary for robots to enter them into the database and generate search results;
- page priority information;
- data on the latest changes to documents and the regularity of their updates;
- information about other versions of the web page.
Sitemap helps visitors and especially search engine robots to quickly determine the location of all the pages most necessary for indexing, as well as their importance in relation to other documents, frequency and time. last change, while doing as few "extra" transitions as possible. Sitemap helps the system to index the resource more efficiently.
Yes, on small sites you can go to any document in just a couple of clicks, but large ones contain several thousand pages and their structure is much more complicated. Without a Sitemap on such resources it will be difficult for both visitors and search bots.
Based on the above, there are three main situations in which there are problems with resource indexing and a sitemap is needed:
- a huge number of pages;
- complex resource structure;
- lack of external or internal backlinks to important pages for indexing.
Types of cards
Any search engine introduces site pages into the index gradually, starting from the main page and ending with documents with deep nesting, it depends on what the site has.
TXT
Another method of designing a sitemap is a Sitemap file in txt (text format). It contains a list of all links important for indexing. But in general this option relevant for webmasters who do not want to bother too much.
Yes, the TXT map can also contain 50,000 links, but there is no way to indicate the date of the last change and the relative priority of documents.
Example
1. http://site.com/; 2. http://site.com/page1/; 3. http://site.com/page2.Yandex and Google requirements for maps
I have identified 6 general requirements that two of the most popular search engines put forward for a Sitemap file:
- You must use UTF-8 encoding.
- The limit on the number of URLs is up to 50,000 links.
- If the file is larger than 10 MB, split it into multiple maps and list them all in the Sitemap index file.
- In the process of accessing the sitemap, the server should send a 200 OK response.
- Enter only canonical document URLs: there should be no session IDs or GET parameters.
- Links in the file must point to the same domain as the sitemap itself. It would seem that this is an elementary requirement. But in fact, many webmasters who have several web resources, when transferring a template with a Sitemap file, forgot to change the existing links to the actual ones.
Search engines have their own unique requirements.
Yandex:
- the maximum allowable size is 10 MB;
- it is allowed to use URLs with Cyrillic characters.
- the maximum allowable file size is 50 MB;
- works only with Latin characters and numbers (Cyrillic domains can be converted to Latin using Punycode).
Important! To reduce the requirements for bandwidth channel, the gzip archiver is used. 10 and 50 MB are the sizes of Sitemap files before compression by archivers, and not after.
Does a sitemap affect promotion?
I note that without a sitemap, search engines will not lower your resource in the search results. Search engines aren't stupid, and even if you don't have a Sitemap on your site, they still have one.
Now search engines, even without a sitemap, crawl pages and include them in the search results quite well. And in theory, they should do it on their own, without any help from webmasters.
But problems often arise when the robots fail and cannot crawl all the pages. This often happens due to:
- deep nesting of some sections, to go to which you need to go to a lot of links;
- dynamic URLs.
Therefore, it is in the interests of webmasters to help search engines.
Having a sitemap helps you avoid these situations. Sitemap to a certain extent positively affects SEO promotion web resource, because it makes the process of bypassing and including pages in the search faster and easier for robots.
Even with this file, your resource reduces the risk of turning from the original source into copy-paste due to the fact that the pages get into the index before competitors manage to steal content from it and host it.
Important! Add to the Sitemap the main pages that play a key role in website optimization so that they get into the search results in the first place. Do not write in the file pages with service information, dynamic URLs, tags and prohibited for indexing.
Is it possible without Sitemap?
The answer is obvious - yes. Search engines perceive sitemap.xml only as a recommendation for robots. In practice, for web resources where there is no sitemap, Yandex marks the presence of possible, rather than critical or fatal problems.
As it was written above, search robots are able to independently and efficiently index resources, but for large sites and those with dynamic URLs, it is better to make a Sitemap, moreover, it will not take much time, and in the future it can decently save your nerves.
How to create a sitemap?
TO this process you need to approach responsibly, because here you need to indicate the main pages that you want to see in the search results.
HTML
Sitemap in HTML format is, first of all, navigation through sections of the site for users. If a web resource has only a few hundred pages, it has a simple structure and a clear menu, it is not necessary to include an HTML map.
But when the site is large and the structure is quite complex, it makes sense to create a page with a list of links to the main pages.
Consider the niche to which your resource belongs and, accordingly, its target audience. If your visitors are predominantly elderly, a sitemap will help them navigate the site, since the main menu is more difficult to figure out.
There are many ways to create an HTML Sitemap on the . The best way to do this is through the Hierarchical HTML Sitemap plugin and the like. Download the plugin for yours, install it, and then activate the add-on to install the file. After - place the shortcode issued by the plugin on the page where you want to connect the sitemap.
The sitemap will be available at:
http://site.com/sitemap.html
For other engines there are such solutions:
- Drupal - Sitemap plugin.
- Joomla! - OSMap plugin.
Those using OpenCart don't have to worry about creating an HTML sitemap, because it appears there automatically.
XML
If you decide to create an XML sitemap, decide how you will do it:
- Generate sitemap using resource management engine, if it has a corresponding built-in function, using various plugins and modules.
- Download generator software. Most of these programs are paid, but thanks to them you can easily create XML file both for one and for several resources. The most popular are WonderWebWare SiteMap Generator and SiteMap XML Dynamic SiteMap Generator .
- Download sitemap file online generator. The choice of special services today is very large, they all have different limitations and capabilities. For example, on the xml-sitemaps.com site, an abundance of map parameters are configured, but a maximum of 500 links can be placed in it.
- Manual connection sitemap.
Using the generator online
Advantage this method is the lack of binding to any CMS. Online generators allow you to create a sitemap for web resources on any engines and even on static HTML sites.
The disadvantage of using generators is that free services are very limited in the number of pages and are more suitable, for example, for business card sites, where there are several pages and new ones will no longer appear.
If the resource is regularly updated with new articles, then free generators will not work. They don't update the Sitemap, so new URLs won't appear there, causing the file to quickly become out of date.
Most Popular Generator:
I will give an example of how to create an XML map with XML-Sitemaps.
Go to the service website and indicate the address of your resource, click on the START button.
In a few minutes, an XML sitemap will be generated. Next, upload the file to HDD computer.
Then you need to upload the Sitemap to the site's root folder using an FTP client, such as WinSCP. If you do not know where to look for the root directory, enter index.html in the search. Upload the file here.
Through software
Creating an XML sitemap in this way is not that difficult. Now I will show you how to do this using the example of the Majento PC program:
- resource.
- In the top tab, go to the "Sites" section, then click "Generate Sitemap".
- Save the file in .xml format.
- Everything is a map on your site.
Through plugins for CMS
This option is much more convenient than free online generators. Plugins for site management engines are more functional, they allow you to change the parameters of the sitemap. And one of the main advantages is the availability of add-ons for automatic update file immediately after the appearance of new documents, as well as notifications about them to search engines.
For WordPress sites, one of the best options is the Google XML Sitemaps module. Install and then activate the plugin. Next go to settings.
In the main settings, enable alerts for Google, specify the sitemap address in the robots file, and activate the Sitemap.xml compression function.
Go to "Article Priority" and turn off the automatic page prioritization feature. Then you will need to specify the priorities of publications yourself. Take a look at the "Sitemap Content" section. Include in the sitemap only the types of documents that you want to be indexed.
Don't forget to check the box next to Include the last modification time. In this way, the module will show the date of the last page update to the robots in the sitemap, which will help them calculate the date when the site was crawled again.
The Change Frequencies option allows you to set the frequency of reindexing certain types of documents. Often, search engines do not follow the changefreq tag specified in the Sitemap file, because they independently determine the crawling budget of the resource and start from it.
However, the plugin provides the ability to further configure the frequency of scanning, but this will also be a recommendation, not a command.
Excluded Items allows you to remove certain documents or categories from sitemap.xml.
The Priorities tab speaks for itself. The priority of the pages is indicated here, the values for them are set from 0.1 to 1. Initially, each page has a value of 0.5. Expose highest priority the most important pages for ranking.
At the end, be sure to save the changes in the settings. The plugin will continue to function in the background and there will no longer be a need to log into it - the sitemap will continue to be up to date, as everything is updated automatically. Next, you just need to add the file to the Yandex and Google webmaster panels.
For images
The search giant also works with image sitemaps. Thanks to it, Googlebots can get more data about the images on the site. The syntax also helps the search engine to quickly find and index pictures that it could not find during the resource scanning process.
You can make the syntax for images separately or add it to an already created Sitemap. The sitemap for images, like the standard sitemap, contains optional and required XML tags.
The mandatory ones include:
, containing the address and other information regarding the image.
The optional tags are as follows:
For video
Google also supports XML sitemaps specifically for video. The advantages of this syntax are:
- allows you to show search engines detailed description video file;
- shows bots the type of content on the resource;
- in the results of the issue, it can display a splash screen for the video, which potentially increases clicks and transitions to the site;
- content can be found in Google Video.
The XML map for the video includes the required tags:
The description and title of the video file must either be escaped or placed in a CDATA block. You can generate a video for the sitemap.xml using the WP Video SEO plugin.
TXT
Need to create text file in UTF-8 encoding. In it, write only the URLs of the pages that should be indexed.
Where to place the sitemap?
The Sitemap file is best placed in the root folder of the web resource. And the link to the sitemap is placed in . It should look something like this:
You can clearly see in our robots.txt example at the link
How to add to Yandex Webmaster
The sitemap provides an opportunity to inform the search engine about the current structure of the resource, so it can be added to the Yandex webmaster panel.
Here you can:
- Download sitemap.
- Notify me of an update.
- Exclude file.
- Select a web resource from the list.
- Specify the URL address of the file location in the corresponding field.
- Click Add.
This will send the sitemap for processing. The search bot will upload the file no later than 14 days later. The search engine processes all added files separately. When the files are uploaded, a status will be displayed next to each of them:
- OK. The file was created correctly and loaded into the database. Next to it, the date of the last update will be shown. Documents that the robots have already indexed will be displayed in the search results no later than 2 weeks later.
- Redirect. The file address redirects to another URL. V this case you need to remove the redirect and notify the system about the update.
- Error. The file was made incorrectly. Follow this link Error where its causes will be described. Correct everything that the search engine requires and then report the changes.
- Not indexed. The server sends an HTTP code that is different from 200 OK. Go to the Server Response Check tool and paste the full sitemap location URL to see if the search bot has access to it. If the file is not available, contact the administrator of the server or resource hosting it.
Also make sure you don't apply a Disallow directive to the file in robots.txt to prevent it from being indexed.
Update
When making any changes to the sitemap, there is no need to delete it and then reload it, as bots often check the sitemap for updates and errors. It is recommended to just click on the icon manual update, which will allow bots to traverse the file faster.
On a note. The number of attempts to manually update is limited, so use it wisely.
- Go to Google Webmaster Dashboard.
- Find the "Crawling" section, in it go to "Sitemaps".
- Click Add/Check File.
- Enter the address where the sitemap is available.
- Click Submit.
If you want to check the Sitemap for errors, next to the Submit button is Test - click to check. After verification, a window will be displayed with a message about the presence or absence of errors.
In Google Help, you can find a list of all possible errors and their solutions.
Removal
If you have a new interface Search Console, write to search engine support.
But most best option– switch to the old version at the bottom of the page. And then everything is the same as in Yandex.Webmaster: go to selecting a site from the list and then check the box next to Sitemap to delete the file.
Conclusion
Think carefully about which pages of your web resource have the highest priority in relation to others, and indicate them in the Sitemap file. This effective tool, the competent use of which will help search engine robots to quickly bypass the site and receive information about its structure.
