Wait, where is it? Refresh. It was here yesterday. Refresh. I spent all this time moving everything over, and now I can’t Google it? Refresh. That’s not point. Refresh. How am I meant to find anything again? Refresh. If I could remember it, I wouldn’t have to write it down and then search for it. Refresh.
And that’s how I found out that I hadn’t configured a robots.txt file for my updated Hugo site, and Google was politely ignoring it.
It’s been a very long time since I had anything to do with HTML, CSS and JavaScript (which I have been studiously avoiding as it annoys me), and things have changed a lot. That’s probably to be expected, as the last time I poking around in HTML no one thought you’d ever be looking at it on your phone.
As a sort-of follow up to moving my website from Blogger to Hugo , below are some of the steps I took to try and make the site more useful. Most of these are embodied in the Grey Book theme I created for it. This is all based on the current, 0.85, version of Hugo.
What settings and features are useful to you depends entirely on what you’re trying to achieve with a website. In my case it’s: easy to find, quick to load and pleasant to read; so that’s what to expect below.
Searchable
First problem to fix is that the site doesn’t appear properly in Google’s search results, and while I’m not so concerned about other people finding the website, I do want to be able to use the world’s biggest search engine to search it myself.
The first stop is then Google Search Console , to see in more detail what Google thinks it knows about my site. Before this works you have to validate your domain ownership, which can either be done semi-automatically, which I found a bit disconcerting, or by adding a specific TXT entry to your DNS record.
Once in, I can see the error “Indexed, though blocked by robots.txt ”, and realise that Google’s default is to ignore a site’s content if there’s no robots.txt file to say it can (not all bots are so polite), and that Hugo doesn’t create one without being told to. By adding the below option to the config.toml file it will create a default robots.txt file that allows all search engines to index any part of the site.
enableRobotsTXT = true
The whole Search Console is huge, and a whole world to itself. I’ve never had to deal with search engine optimisation, or trying to increase website traffic, but I guess if you do, this is your starting point.
It does allow you to analyse your site in many different ways, such as load speed and mobile usability. Things I’d never really considered in detail other than ‘make it small’ and ‘keep the layout simple’. Talking of which…
Appearance
My theme started as the mini theme , which was closest to what I had in mind, but as the changelog shows I’ve made a few adjustments.
The first changes were to make the font into a serif, which is easier to read, and make minor adjustments in the sizes of various headings. Also changing the background from pure white to a grey reduces the contrast and makes it slightly easier on the eyes.
The other thing missing, from a reading perspective, were any navigation links from one article to the next. You would either have to go back to the home page, or the archive, to move to the next post in the history. This was solved by adding next/previous links at the bottom.
Featured Image
The last major change was adding a ‘featured image’ to each posts summary on the home page. The site is faster and smaller without this, but also very boring looking, and a bit of colour and variety on the front page is nice.
This resulted in my biggest dive into Hugo templating
, which seems OK, but awkward. Also Emacs HTML mode doesn’t like indenting the double curly brackets correctly, so large amounts of nested statements look confusing. actually it does1. Perhaps other template systems are easier, but I’ve not tried them, and at first glance other templating systems like Liquid seem very similar.
Code Block Resizing
One part that is not very satisfying was the code blocks combined with the relatively narrow column of the page text. Scrolling the box seems a bit cumbersome, so I tried to copy the behaviour seen on the Hugo site, where if there is a long line of code, the box expands when hovering over it by changing the width to ‘max-content’ on hover, as below. The one catch is that if your code is narrower than the normal column width, on hover it will shrink, looking odd. This was sort-of fixed with the min-width to 100% (I believe inherited from the style above), but this actually would make it slightly wider than the column. Choosing 98% seemed to work, but at a few screen widths it still causes a slight ‘flicker’ in the width on mouse-over if the code is narrower than the column width. Answers on a postcard if you know how to fix this.
.highlight pre:hover {
width: max-content;
min-width: 98%;
}
Org-Mode Differences
The last few tweaks were to adjust some of the CSS styles so that if the page to rendered from Org-Mode markup, then it looks closer to what a native Markdown page would look like.
The most noticeable difference was lists, where the lists from Org-Mode contain an extra <p>
tag, the effect of which has to removed with display: inline;
in the CSS.
<ol>
<li><p>First item</p></li>
<li><p>Second item</p></li>
<li><p>Third item</p></li>
</ol>
Footnotes are another difference. Once they’ve been through the rendering process starting with Org-mode, there a number of differences in HTML and classes it produces, and they don’t include the little return arrows at the end2. Instead you have to click the list number, which is less easy, especially on smaller screens.
Another difference is that if you add a caption to something, like a picture or a quote block, this is rendered as a <figure>
with a <figcaption>
. This then appears smaller and in the middle of the main column. It’s not terrible, but if you have pictures with and without captions in the same post, it can look disjointed.
The last issue is that it’s not as easy to create internal cross-reference links to headlines in the same document. The rendered HTML output doesn’t give the heading a useful id, just “headline-15”, which you would then have to link to, and hope the sequence doesn’t change. Getting the headlines to work as links both in native Org-Mode and in the rendered HTML document isn’t something I’ve managed to get to work, and I’m not sure there is an answer.
Mobile Friendly
As mentioned at the top, the last time I seriously spent any time with HTML, the idea of reading a real website on a phone wasn’t yet born. There are not many changes made to the site to make it more mobile compatible, thanks to its simple one column design. There were really only two major adjustments. To see them check the site on a screen less than 680 pixels (or the equivalent) wide.
First, the top navigation would bunch up in a ugly way. Creating a whole fold out/pop over menu as is the fashion also seemed overkill what are only a few sections. Instead on small screens the navigation now becomes scrollable and you can push it left and right to select First, the top navigation would bunch up in a ugly way. Creating a whole fold out/pop over menu as is the fashion also seemed overkill what are only a few sections. Instead on small screens the navigation now becomes scrollable and you can push it left and right to select what to click. Hopefully the little lower border that appears is hint enough.
The second item was the next/previous links at the bottom of articles. With longer titles these would also bunch up in a ugly way, so I cheated an hid them on smaller screens and increased the size of the arrows slightly to make it easier to click.
Statistics
One thing that the previous Blogger site had baked in were simple page analytics, even without using the more advanced Google Analytics. This was powered by the unavoidable use of cookies on Blogger, and naturally fed the vast tracking data stores of Google.
While I’m not interested in the great details of how people come to the site, click through rates, time on site etc. (I don’t have any targets to hit), I am curious to have a few statistics, like number of visitors, and where they come from. Since I’m not managing the hosting, I can’t use server logs for this.
I could find a number of options for ‘privacy focused’ analytics, which seem like a good compromise. You don’t have to store cookies on someone else’s computer, register as a data processor with all the other GDPR or related legalities, and you leave people alone. All you get are some basic, anonymous, statistics.
To my surprise almost none of these privacy focused providers had a free, and monthly charges were much higher that say, a Spotify subscription. For a hobby site that’s hosted for free, that seems too much. In the end Insights seemed to be the best solution, beating the bigger contenders like Fathom by having a small free plan, that I suspect I will never exceed with my small exclusive readership.
Future Ideas
Tweaking a website’s function and appearance can be fun, but it’s hard to know when to stop, as you could carry on forever making tiny changes. I can already think of a few, and if I looked at the results of any speed test sites I could probably find plenty more. The main one that remains would be to get Hugo to automatically turn all the images into WebP. This should make the site a bit smaller and faster to load, more than most other tweaks.
Currently (at Hugo version 0.85) it is possible to use image processing to turn an image into WebP, but there doesn’t seem to be a way to do it across the board. It would have to be added into every partial that uses and image, and preferably controlled by a site parameter in case you don’t want to use it. Additionally it would increase the website’s render time considerably, unless you can save the processed images from previous renders to be re-used, something that doesn’t seem so easy on the current hosting solution.
Web-Mode does support correctly indenting and treating templates. The only tricky part is associating the correct engine, which you can do manually, or by adding
-*- engine:ENGINE_NAME -*-
in a comment at the start of the file whenweb-mode-enable-engine-detection
is set tot
. ↩︎I’m talking about this > ↩︎