Search engine crawlers (like Google) look for a list of pages/documents available in a website. There is a standard XML file format for the presentation of the list of files in a website. A software called Sitemap Generator can scan a whole website and generate an XML file according to the standard format.
XML format is similar to HTML format of tags, but not identical. The list of documents will be included between the <urlset> and </urlset> tags.
Below I have included the first three lines of the XML file for this website. The first two lines are standard. The third line represents the opening page of https://www.science.co.il/. The last line marks the end of the XML list. The complete list of files identified in the website will appear in place of the three dots below.
<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url><loc>https://www.science.co.il/</loc><lastmod>2026-03-12</lastmod></url>
...
</urlset>
As can be seen in this example, each line for a document includes the URL, followed by the date of last modification.
Installation of Sitemap Generator
On the internet there are many versions of Sitemap Generator including online services. This page provides information about Sitemap Generator that is installed from the NPM software registry and package manager used to install JavaScript packages.
To install NPM enter the following Bash command:
sudo apt install npm
After installing npm, enter the following command to install Sitemap Generator
sudo npm install -g sitemap-generator-cli
The "cli" at the end of sitemap-generator-cli means Command Line Interface.
To see the options available enter: sitemap-generator -h
Recommended options:
-c: max-concurrency - maximum number of requests the crawler will run simultaneously. Recommended: -c 100.-l: Include "last modified" field.-q: Include URLs with a query. URL example: https://www.science.co.il/elements/?s=Symbol-f: Path to save XML file. Example: /var/www/sitemap.xml (default filename: "sitemap.xml")
Command example:
sitemap-generator -q -l -c 100 -f /var/www/science.co.il/html/Sitemap.xml http://localhost/