How to Integrate a Local Asset Server with Hugo

Hugo's remote data fetch APIs such as =getJSON= See Get Remote Data allow grabbing data over a network for use with pipe functions. This is a very cool feature and I use it, for example, to create programming language badges from the information I pull from gitlab APIs.

Now even though these APIs are great for fetching remote data, processing and rendering data using Hugo templates can quickly become a mess. This is especially true if the data has complex JSON structure, or has to be assembled out of multiple API calls.

In this article we discusses how to use the getJSON API to write custom Hugo pipe like functions that can be used to perform complex asset processing and simplify Hugo template code.

We start with an example, the Open Library shortcode open-book.html. The shortcode fetches and renders details of a book using openlibrary.org APIs. For example, to add Fantastic Mr.Fox to a page, ISBN 9780140328721 we can use the short-code as shown below:

{{<<open-book>}}9780140328721{{</open-book>>}}

• Fantastic Mr. Fox
• Roald Dahl
• Tony Ross (Illustrator)
• October 1, 1988
• Puffin
• https://openlibrary.org/books/OL7353617M

The main character of Fantastic Mr. Fox is an extremely clever anthropomorphized fox named Mr. Fox. He lives with his wife and four little foxes. In order to feed his family, he steals food from the cruel, brutish farmers named Boggis, Bunce, and Bean every night.

The implementation of the short-code is as follows,

{{- $isbn     := (trim .Inner " ") -}}
{{- $bookURL  :=  printf "http://localhost:8383/openlib/books/%s" $isbn -}}
{{ with getJSON $bookURL  }}
{{with .Err }}
    {{ warnf "BUILD: failed to fetch book information: %s" $bookURL }}
{{ else }}
<article>
 <!-- begin: cover image -->
 {{ with resources.GetRemote .cover_image }}
    {{ with .Err }}
    {{ warnf "BUILD: failed to fetch image: %s"  . }}
 {{ else }}
    {{ $data := resources.Fingerprint .}}
    <img src="{{ $data.RelPermalink }}"
         width="{{ $data.Width }}"
         height="{{ $data.Height }}"
         alt="$data.title"
         integrity="{{$data.Data.integrity}}">
 {{ end }}
 {{ end }}
 <!-- end: cover image -->
 <!-- begin: book details -->
   <ul>
    <li><strong>{{ .title | plainify }}</strong></li>
    <li> {{range .authors}}<a href="{{.link}}">{{.name}}</a>{{end}} </li>
    <li> {{range .contributors}}{{.}} {{end}}</li>
    <li>{{.publish_date | plainify }}</li>
    <li>{{range .publishers}} <span>{{- . -}}</span> {{end}}</li>
    <li> <a href="{{.url}}" alt="{{.title}}y">{{.url}}</a> </li>
   </ul>
   <div>{{.description | plainify }}</div>
 <!-- end: book details -->
</article>
{{ end }}
{{ end }}

The important bit to notice is in the first few lines. In particular, the line that invokes getJSON on a local URL, which fetches data from a local server, http://localhost:8383/openlib/books/9780140328721 instead of directly fetching it from openlibrary.org.

{{- $isbn     := (trim .Inner " ") -}}
{{- $bookURL  := (printf "http://localhost:8383/openlib/books/%s" $isbn) -}}
{{ with getJSON $bookURL  }}
{{with .Err }}
    {{ warnf "BUILD: failed to fetch resource: %s" $bookURL }}
{{else}}

  ...

{{end}}
{{end}}

The local URL points to a Python code open-book.py, written using bottle.py, that actually generates simplified JSON data Warning: In general, a local server can execute arbitrary code. This can lead to security issues. In fact, security concern is the main reason why Hugo lacks support for running custom scripts, /see/ issue# 796. .

import requests
from bottle import get

def openlib_authors(authors):
    '''Returns simplifed list of authors--name and link only'''
    result  = [ ]
    for author in authors:
        author_key  = author["key"]
        author_url  = 'https://openlibrary.org{0}.json'.format(author_key)
        response    = requests.get(author_url)
        author_json = response.json()
        result.append({
            "name":  author_json["name"],
            "link":  'https://openlibrary.org{0}'.format(author_key)
        })
    return result

def openlib_description( book ):
    'Returns summary of a book'
    book_id     = [ w["key"] for w in book["works"] ][0]
    detail_url  = 'https://openlibrary.org{0}.json'.format(book_id)
    response    = requests.get( detail_url)
    description = response.json()["description"]
    description = re.split(r'(?<=[.:;])\s', description)
    return ' '.join( description[:5] )


@get('/openlib/books/<isbn>')
def openlib_book(isbn):
    'Returns details of a book given its ISBN'
    api_url  = 'https://openlibrary.org/isbn/{0}.json'.format(isbn)
    response = requests.get(api_url)
    book     = response.json()
    book_key = book["key"]
    links    = [
        'https://openlibrary.org{0}'.format( w["key"] )
        for w in book["works"]
    ]
    return {
        "title":           book["title"],
        "authors":         openlib_authors(book["authors"]),
        "description":     openlib_description(book)
        "publish_date":    book["publish_date"],
        "number_of_pages": book["number_of_pages"],
        "contributors":    book["contributions"],
        "publishers":      book["publishers"],
        "url":
        'https://openlibrary.org{0}'.format(book["key"]),
        "cover_image":
        'https://covers.openlibrary.org/b/isbn/{0}-M.jpg'.format(isbn),
    }

The code makes use of several APIs from the Open Library project. Specifically the Books, Cover, and Authors APIs. It assembles pieces of data it gathers from these APIs into a single JSON object, that is much easier to handle with Hugo templates.

To make the generated resource available to Hugo during a build process, the asset server must be running already. This involves running the command below,

 python3 -m bottle --debug --reload --bind localhost:8383 open-book.py

Once the server starts, the book resource implemented in the open-book.py file, will be available on http://localhost:8383/openlib/books/<isbn>. And Hugo getJSON function can be used to access it.

Template Code Readability Hugo's templating language is not a general programming language. For example, it lacks basic code organization constructs such as functions and modules. It makes sense, therefore, to offload tasks not suited for its original intended design elsewhere to improve code readability.

Fine Grained API Control API calls have quota limits, cost money to use, and can significantly slow down site build process. Setting up an environment where utter control over the life cycle of an API call can be exercised is useful.

Complex Asset Processing the getRemote data fetch API supports multiple MIME types and HTTP post method. This means it is possible to upload data in a Hugo project to a local server for processing in all sorts of ways. For an example, creating time-series plot from CSV data, see time-series.

Continuous Integration & Deployment (CI/CD) Considerations adding an asset server on a Hugo build process will involve additional configuration. For instance, integrating a local asset server with an example(src,demo) built for this article involved finding a suitable docker image with versions of Hugo and Python capable of properly building it.

That's it!

This article covered, through an example, how to use a local asset server along with Hugo's getJSON pipe function to simplify template code. For a working demo, check out the accompanying code repository on gitlab.