--- title: MkDocs sitemap.xml file generation url: https://blog.octoperf.com/mkdocs-sitemapxml-file-generation/ date: '2015-06-17' authors: - jloisel categories: - Development tags: - Open Source description: Sitemaps lets crawlers easily index your web pages. Learn how to get a sitemap.xml file for MkDocs. --- # MkDocs sitemap.xml file generation Here at OctoPerf we love **static website generators**. Even if our load testing tool is made completely dynamic by [AngularJS](https://blog.octoperf.com/building-better-software/#our-frontend), our [website and blog are generated using Jekyll](https://blog.octoperf.com/hosting-jekyll-on-amazon-web-services/). The [OctoPerf documentation](https://api.octoperf.com/doc/) is created with [MkDocs](https://www.mkdocs.org/). This wonderful tool allowed us to concentrate on the documentation content as the markdown syntax is really easy to comprehend. Moreover, the responsive themes let us integrate the documentation directly into our [performance testing tool](https://octoperf.com): ![OctoPerf integrated documentation](https://blog.octoperf.com/img/blog/mkdocs-sitemap/integrated-documentation.png) When we first wrote it, MkDocs version was 0.12.0. It did not support *sitemap.xml* file generation. This guide helps migrating to the latest version (0.14.0). ## Sitemaps The format and [Sitemap protocol](https://www.sitemaps.org/) are quite old as they were initiated by Google in 2005. This is a solution to provide the search engine crawlers (Google, Bing, etc.) a plan of your site. A Sitemap is a comprehensive list of your URLs in XML format. These robots can then identify and fetch all available pages, according to information provided in the file. ### sitemap.xml The concept of this format is extremely simple: you create an XML file that lists the pages on your site, as well as some information about them (update frequency, crawl priority, etc.): ```xml https://api.octoperf.com/doc/ 2015-06-15 daily https://api.octoperf.com/doc/account/ 2015-06-15 daily ``` You point out its presence to the search engine through an administration interface. Then the search engine robots read then it and use the proposed URLs to exhaustively index your site in depth. * Using a **Sitemap** does not guarantee that the search engines will index all listed pages. They remain master of the way they index websites (It still eases this process). * A **Sitemap** file is not a guarantee that your site will be better positioned. This tool is only an index tool and not a ranking tool. * Finally, a **Sitemap** does not replace the crawling of your website by search engine robots, done by following the web pages links. Both methods are quite complementary indeed. ### Webmaster tools Your *sitemap.xml* file can be submitted to Google using [Google Webmaster Tools](https://www.google.com/webmasters/tools/home). It can be submitted to Bing via their [Bing Webmaster Tools](https://www.bing.com/webmaster/home/mysites). ## MkDocs MkDocs is a **static site generator**. Its goal is to build projects documentation. Documentation source files are written in Markdown, and configured with a single YAML configuration file. It is quite [easy to install](https://www.mkdocs.org/#installation). **MkDocs** is currently still in development, and evolves quickly. When we started to write OctoPerf's documentation, MkDoc was in version 0.12.0 and did not support *sitemap.xml* file generation. The version 0.13.0 [added this feature](https://www.mkdocs.org/about/release-notes/#version-0130-2015-05-26). But it also came with changes in the configuration file (mkdocs.yml) format. ### What is your version? First you may **display your current MkDocs version**: ```sh mkdocs --version ``` If it is above 0.13.0, you may already have a *sitemap.xml* generated and available at /sitemap.xml. Otherwise, you should upgrade MkDocs version. ### Version upgrade To **upgrade MkDocs**, simply run the following command in a shell: ```sh pip install mkdocs --upgrade ``` > > *pip* stands for Python Index Package and is a package manager for Python. It is mandatory to install MkDocs so you probably already have it installed on your machine. > > If you get an error like *OSError: [Errno 13] Permission denied: '/usr/local/bin/mkdocs'* and you are running on Ubuntu, you may prefix the command with *sudo*. > Once upgraded, you can type *mkdocs --version* to ensure that you use the latest version. ### Configuration migration The *mkdocs.yml* **[multilevel pages format](https://www.mkdocs.org/user-guide/writing-your-docs/#configure-pages-and-navigation)** changed for the better. It was redundant and it is now more succinct. E.g. our [load testing tool documentation](https://api.octoperf.com/doc/) configuration file included: ```yaml pages: - ['index.md', 'Getting Started','Introduction'] - ['account/index.md', 'Getting Started','Account'] - ['project/index.md', 'Getting Started','Project'] - ['design/record.md', 'Design','Recording'] - ['design/index.md', 'Design', 'Virtual Users'] - ['design/logicActions.md', 'Design', 'Logical Actions'] - ['design/httpActions.md', 'Design', 'HTTP Actions'] - ['design/postProcessors.md', 'Design', 'Post Processors'] - ['variable/index.md', 'Design','Variables'] - ['server/index.md', 'Design','Servers'] - ['correlation/index.md', 'Design','Correlation Rules'] - ['scenario/index.md', 'Execution', 'Scenarios'] - ['analysis/index.md', 'Analysis','Bench Reports'] - ['glossary.md', 'Glossary'] - ['privacy.md', 'Privacy Policy'] - ['tos.md', 'Terms Of Service'] ``` We had to repeat each section ('Getting Started', 'Design', etc.) multiple times. Now you define such hierarchy in a clearer way: ```yaml pages: - Getting Started: - 'Introduction': 'index.md' - 'Account': 'account/index.md' - 'Project': 'project/index.md' - Design: - 'Recording': 'design/record.md' - 'Virtual Users': 'design/index.md' - 'Logical Actions': 'design/logicActions.md' - 'HTTP Actions': 'design/httpActions.md' - 'Post Processors': 'design/postProcessors.md' - 'Variables': 'variable/index.md' - 'Servers': 'server/index.md' - 'Correlation Rules': 'correlation/index.md' - Execution: - 'Scenarios': 'scenario/index.md' - Analysis: - 'Bench Reports': 'analysis/index.md' - Glossary: 'glossary.md' - Privacy Policy: 'privacy.md' - Terms Of Service: 'tos.md' ``` ### Tips One last thing with withMkDocs, to avoid **broken links** you can add the line *strict: true* in your *mkdocs.yml* file. MkDocs will check for internal links (links between your documentation pages) and won't run or build if there is a broken one. > > Unfortunately it does not check for images URLs. > Another great tool to check for broken links (both internal and external) is the [Chrome extension Check My Links](https://chrome.google.com/webstore/detail/check-my-links/ojkcdipcgfaekbeaelaapakgnjflfglf).