Constructing a Website

Jump to Examples

This site started as a few HTML files on my NAS, and it is still basically that – just more files. Along the way I learned that the hard part of a website is not making it look nice. The hard part is everything around it: the domain, DNS, certificates, and making sure that somebody (or Google) can actually find what you wrote. This note is about that part.

A Bit of Web History

In 1989 Tim Berners-Lee at CERN wrote a proposal for linking documents between computers. By the end of 1990 he had the three pieces that still run the web today: HTML to write a page, HTTP to send it, and URLs to say where it is. The first website, info.cern.ch, explained what the web is – and you can still open it.

Names on the Internet are older. Before 1983 every computer kept one file, HOSTS.TXT, with the names and addresses of all the others, and it was copied around by hand. Paul Mockapetris designed the Domain Name System in 1983 to replace it. A funny leftover: Windows and Linux still have a hosts file, and it still wins over DNS.

For a long time most of the web was not encrypted, because certificates cost money and were a pain to set up. That changed when Let's Encrypt started giving them away for free and automatically at the end of 2015. Today a page without HTTPS gets a warning in every browser.

The newest chapter is AI, and it changed websites from two sides at once. Since 2023 AI assistants can write HTML, CSS and JavaScript from a plain description, so making a page is no longer the hard part for anybody. And AI is now also a big reader of the web: chatbots and AI search answer questions from what their crawlers collected, often without the person ever opening the page. More about both in Websites and AI below.

What Happens When You Type an Address

Say you type jpnas.net and press Enter. In less than a second:

1. DNS: the browser asks "what is the IP address of jpnas.net?" The question goes to your router or your DNS server, and if they don't know, down the chain from the root servers to the servers responsible for .net and then for jpnas.net. The answer is cached so the next time is instant.
2. Connection: the browser opens a TCP connection to that address on port 443 (more about ports in Computer Networks).
3. TLS: the server shows its certificate, the browser checks that it is really for jpnas.net and signed by somebody it trusts, and both sides agree on a key. From here on everything is encrypted.
4. HTTP: the browser asks GET /, and the server answers with 200 OK and the HTML.
5. The page: the browser reads the HTML, finds the CSS, JavaScript and images, and asks for each of them the same way.

You can watch all of this: press F12 in the browser and open the Network tab before loading a page.

Static or Dynamic?

A dynamic site (WordPress, an e-shop) builds every page when somebody asks for it, from a database and PHP or similar. That is great for comments, users and shopping carts – and it also means a server program that has to be updated, a database to back up, and a login page that bots try to break into all day long.

A static site is just files. The server only sends them. Nothing to hack, nothing to update, it is very fast and can run on almost anything – a NAS, a Raspberry Pi or a free hosting service. For a personal site with notes it is usually all you need. Tools like Hugo make static sites from Markdown; I did it my own way (Example 2).

Making a Site Findable

This is the part I learned only recently. My Digital Garden draws the notes with JavaScript from .js files – nice for search and filters, but Google sees an almost empty page. What helps:

• Real HTML pages that contain the text without JavaScript.
• A good <title> and <meta name="description"> on every page – that's what you see in the search results.
• <link rel="canonical"> – tells Google which address is the "official" one when the same page can be opened in several ways.
• Open Graph tags (og:title, og:description) – they make the preview when somebody shares a link on WhatsApp, Viber or Facebook.
• robots.txt – rules for crawlers, invented by Martijn Koster in 1994. Mine just allows everything and points to the sitemap.
• sitemap.xml – a list of all pages with the date of the last change. Google introduced it in 2005, the other search engines joined soon after.
• Google Search Console – add the site, submit the sitemap, and see what Google found and what it didn't like.

Be patient with Search Console. The first days it shows almost nothing, and then it takes weeks before most pages are in. The Pages report is the useful one: "Crawled – currently not indexed" usually just means "seen, maybe later"; "Duplicate without user-selected canonical" means the same text is reachable at two addresses and a canonical tag is missing; and "Page with redirect" is normal for old http:// addresses that now go to https://. The Performance report then shows which searches people actually used to find you – often not the ones you expected.

Websites and AI

AI already plays a big role in building websites, and an even bigger one in how they are found. Both matter for a small personal site.

Building with AI. A lot of this site was built together with an AI assistant: the interactive tools, the Python script from Example 2, and checking all the pages and links for mistakes. I describe what I want, it writes the code, I read it, test it in the browser and change what I don't like. It is much faster than searching for every CSS property myself – but the same rules apply as in Constructing a Program: AI code can look perfect and still be wrong, and it happily adds libraries and trackers you don't need. A static page with a few files is also the easiest kind of project to check, because you can open every file and see everything there is.

Who reads your site now. Next to Googlebot there are now the crawlers of AI companies: GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot and others. In a measurement by Vercel from December 2024, the biggest AI crawlers together already made more than a quarter as many requests as Googlebot – and none of them ran JavaScript. They see only the HTML the server sends. For my JavaScript garden that means exactly the same as for Google: without the static pages from Example 2, an AI would see an empty page.

Answers instead of clicks. Google's AI Overviews and chatbots with web search answer the question directly and show the source only as a small link. Fewer people click through. What still helps: clear headings, the answer close to the top, real examples and your own experience – the things a summary can't replace and the reason somebody still opens the page.

You decide. robots.txt works for AI crawlers too. You can allow everything (like I do – I write these notes so that they are read), or block only the AI training crawlers:

# robots.txt - normal search engines yes, AI training no
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: *
Allow: /

Sitemap: https://example.com/sitemap.xml

Google-Extended is not a separate crawler, only a token: it tells Google not to use your pages for its AI models, while normal Google search still works. And robots.txt is a polite request, not a lock – the big companies respect it, but not every bot does. There is also a newer proposal, llms.txt (2024): a short Markdown file at the root of the site that tells AI tools what the site is about and where the important pages are. It is not an official standard, and it is not clear how much the big AI systems use it yet.

Examples

Example 1: The Smallest Proper Page

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>My first page</title>
  <meta name="description" content="One sentence about what is on this page.">
  <link rel="canonical" href="https://example.com/">
</head>
<body>
  <h1>Hello</h1>
  <p>This is a real web page.</p>
</body>
</html>

Save it as index.html and open it in a browser – that's a website. The viewport line is the one people forget, and without it the page looks tiny on a phone.

Example 2: My Notes as Static Pages

I write every note as one .js file (this one too). A small Python script, build_static.py, reads all of them and writes:

• one HTML page per note in digital-garden/articles/, with title, description, canonical, Open Graph and structured data for Google,
• one page per resource section in digital-resources/topics/,
• a fresh sitemap.xml.

The script needs only Python, no extra packages, and it writes a file only when something really changed, so the NAS doesn't sync everything every time. I run it with a double-click on a .bat file after I change a note, then copy the changed files to the server. The interactive garden stays as it is; the static pages are for Google and for people with JavaScript turned off. Each of them has a button back to the interactive version.

Example 3: Domain, DNS and HTTPS on Your Own Server

Files on a disk are not a website yet. Three things are missing: a name, a way for that name to find your server, and a certificate. Here is how it goes for a server at home.

1. The domain. Buy it at any registrar – depending on the ending (.net, .com, .si …) and the registrar it costs roughly 10–25 € a year. In the registrar's DNS settings you then add the records:

2. Check that you can be reached. Compare the WAN address shown in your router with the one a page like ifconfig.me shows. If they are the same, good. If the router shows something between 100.64.0.0 and 100.127.255.255, your provider uses CGNAT – you share one public address with other customers and nobody from outside can reach you. Then ask the provider for a public IP (often free on request), use IPv6, or use a tunnel or a small rented server in front.

3. A home IP that changes. Most home connections get a new address now and then. A dynamic DNS service fixes that: a small program (or the router, or the NAS) checks the address every few minutes and changes the A record when it changes. Synology has this built in (Control Panel → External Access → DDNS), and many DNS providers have an API for it. Set the TTL of the record low, for example 300 seconds, so the change spreads quickly.

4. Open only what you need. In the router, forward ports 80 and 443 to the server – nothing else. Port 80 is needed only so that the certificate can be issued and so that http:// visitors get redirected to https://.

5. HTTPS, automatically. The easiest way on Linux is Caddy. This is the whole configuration file (Caddyfile) for a static site:

example.com, www.example.com {
    root * /srv/www
    file_server
    encode gzip
}

Caddy sees the domain names, gets free certificates from Let's Encrypt on its own, renews them on time and redirects http to https. On a Synology NAS the same is done with clicks: Web Station serves the files, Control Panel → Security → Certificate → Add gets a Let's Encrypt certificate and renews it, and Control Panel → Login Portal → Advanced → Reverse Proxy sends the domain to the right service if more of them run on the NAS.

Automatic renewal is not just comfortable, it is becoming necessary. Let's Encrypt certificates are valid for 90 days, and Let's Encrypt has announced that this goes down to 64 days in February 2027 and to 45 days in February 2028. Renewing by hand is no longer an option.

6. Test from outside. Use the phone on mobile data (not your Wi-Fi), or these commands:

# does the name point to the right address?
nslookup example.com

# does the server answer over HTTPS? (the first line should say 200)
curl -I https://example.com

# when does the certificate expire? (Linux or macOS)
openssl s_client -connect example.com:443 -servername example.com </dev/null 2>/dev/null | openssl x509 -noout -dates

If something doesn't work, go in the same order as the picture above: first DNS, then the port, then the certificate. Most of the time it is DNS – a typo in the record, or the old value still in a cache. Troubleshooting Networks has more tools for that.

Keeping It Safe

• HTTPS everywhere, also for a "simple" site. It protects visitors from changed content on public Wi-Fi.
• Don't publish more than you mean to. Folders with backups, .git or old test files on a web server are found by bots within days.
• If you host at home, open only ports 80 and 443 to one machine, keep it updated, and keep it separate from the rest of the network if you can (see Security Considerations).
• Back up the source, not only the published site – in my case the .js notes and the scripts.

A Few More Thoughts

What I like about the web is that the basics haven't changed since 1991. A text file with some tags, put on a server, is still a web page that every browser in the world can open. You don't need a framework, a subscription or a platform to have your own place on the Internet – a few files and a domain are enough, and they will still work in twenty years.

AI made the first part – writing the page – easy for everybody. What it didn't change is the rest: owning your name, knowing where your files are, and writing something worth reading. A chatbot can summarise a page, but it can only summarise what somebody wrote first.