Porting a Python web app to Go

The other day I was describing in this post the structure I use for my Django backend, strongly inspired by HackSoft's styleguide.

However, as a lot of design patterns the beauty of this one is that it's not specific to Django, and not even to Python; it can actually be used for pretty much any programming language and framework! 🙂

Let's see how the "Gin Rummy leaderboard" I was describing in my 'From scratch to online in production' in a single day series can be ported to another language I love, Go! :fontawesome-brands-golang:

"Go" or "Golang" ?

Go is also often referred as "Golang": this term is mostly used when it comes to searching stuff on the Web related to Go - the name of this programming language was certainly not optimised for optimal results using only its 2 letters name 😅

File structure

We can apply pretty much the same file structures than with the Django app, with only a slight adaptation to embrace the Go idioms - such as the package name internal to namespace the "private" code of the app / package.

There is no "standard" files layout for Go, but one of the general principles recommended by its authors is to keep the directories structure as flat as possible - some Go apps/pakages even have their whole code in a single folder!

Here is the whole content of my Go Web app:

gin-scoring/
├── cmd/
│     ├── http_server/
│     │     └── main.go
│     └── quicktests/
│         ├── get_hall_of_fame_global.go
│         ├── get_hall_of_fame_monthly.go
│         ├── get_last_games.go
│         └── save_game_result.go
├── internal/
│     ├── domain/
│     │     ├── mutations/
│     │     │     └── save_game_result.go
│     │     ├── queries/
│     │     │     ├── calculate_hall_of_fame_global.go
│     │     │     ├── calculate_hall_of_fame_monthly.go
│     │     │     └── get_last_games.go
│     │     ├── gin_rummy.go
│     │     └── types.go
│     ├── http/
│     │     ├── templates/
│     │     │     ├── layouts/
│     │     │     │     └── main.gohtml
│     │     │     └── homepage.gohtml
│     │     └── handlers.go
│     ├── models/ # (1)
│     │     ├── boil_queries.go
│     │     ├── boil_table_names.go
│     │     ├── boil_types.go
│     │     ├── boil_view_names.go
│     │     ├── game_result.go
│     │     └── psql_upsert.go
│     ├── config.go
│     └── db.go
├── db_schema.sql
├── docker-compose.yml
├── go.mod
├── go.sum
├── Makefile
└── sqlboiler.toml

The content of this folder is not versioned, as it's code generated by SQLBoiler

(file tree generated as usual with tree --dirsfirst -F . - see tree's MAN page)

The main components of that Go app, in a nutshell

Database schema management

For this Go port of my Django app I opted for the tool sqldef to manage my database migrations. In a nutshell, I just have to describe my schema in a plain SQL file, and then run psqldef (because I'm using Postgres) to apply the schema updates to my database. Very handy - especially for such a small project! 🙂

-- file: /db_schema.sql
create table if not exists game_result (
    id integer primary key generated by default as identity,
    player_north_name varchar not null,
    player_south_name varchar not null,
    outcome varchar not null,
    winner_name varchar,
    deadwood_value smallint not null,
    winner_score smallint,
    created_at timestamp not null
);

create index on game_result(created_at);

ORM

On the ORM-ish side of things, I chose SQLBoiler. It will introspect the schema of my database, and generate strongly typed Go code that allows me to create, read, update and delete data 👌 As the generated code is quite verbose, for such a small project I chose to not version the package - running go generate can re-generate it when I need to. I also have a Make target that simply runs sqlboiler psql to do the same :-)

# file: .gitignore
/internal/models

HTTP routing

The HTTP layer is in the internal/http package, powered by the classic gorilla/mux HTTP router - although I could also simply have used Go's built-in URL routing engine, as it does the job pretty well too. 🙂 Here is the HTTP entry point of the app, that initialises that stuff:

 // file: cmd/http_server/main.go
 package main

 import (
     "fmt"
     apphttp "github.com/olivierphi/gin-scoring/internal/http"
     appinternal "github.com/olivierphi/gin-scoring/internal"
     "github.com/gorilla/mux"
     "log"
     "net/http"
     "time"
 )

 func main() {
     err := apphttp.LoadTemplates()
     if err != nil {
         panic(err)
     }

     srv := &http.Server{
         Handler:      createRouter(),
         Addr:         appinternal.Config().Addr, // (1)
         WriteTimeout: 15 * time.Second,
         ReadTimeout:  15 * time.Second,
     }

     fmt.Printf("Server starting on '%s'\n", srv.Addr)
     log.Fatal(srv.ListenAndServe())
 }

 func createRouter() http.Handler {
    r := mux.NewRouter()
    r.HandleFunc("/", apphttp.HomepageHandler).Methods("GET")
    r.HandleFunc("/game/result", apphttp.PostGameResultHandler).Methods("POST")
    r.HandleFunc("/ping", apphttp.PingHandler)

    return r
}

"Controllers"

The "HTTP Handlers" of this app are pretty standard - that's the Go equivalent of what we would call "Controllers" in frameworks such as Ruby On Rails, Symfony or Laravel, or "Views" in Django: they receive an HTTP request, and are in charge or sending back an HTTP response. For example, here is the one that displays the main HTML page, with the current stats for the leaderboard:

// in file: internal/http/handlers.go
func HomepageHandler(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()

    t, ok := templates["homepage.gohtml"] // (1)
    if !ok {
        http.Error(w, "Could not load template", 500)
        return
    }

    db := internal.DB()

    lastGames, err := queries.GetLastGames(ctx, db)
    if err != nil {
        http.Error(w, err.Error(), 500)
        return
    }

    hallOfFameGlobal, err := queries.CalculateHallOfFameGlobal(ctx, db)
    if err != nil {
        http.Error(w, err.Error(), 500)
        return
    }

    hallOfFameMonthly, err := queries.CalculateHallOfFameMonthly(ctx, db)
    if err != nil {
        http.Error(w, err.Error(), 500)
        return
    }

    templateData := make(map[string]interface{})
    templateData["LastGames"] = lastGames
    templateData["HallOfFameGlobal"] = hallOfFameGlobal
    templateData["HallOfFameMonthly"] = hallOfFameMonthly

    if err := t.Execute(w, templateData); err != nil {
        http.Error(w, fmt.Sprintf("Error while rendering template: %v", err), 500)
        return
    }
}

Yes, using short names (even reduced to one single character) for variables is quite idiomatic in Go 😄 The typical signature for a HTTP handler for example is to receive the Request as r and the ResponseWriter as w :-)

HTML templates

My HTML templates, powered by Go's built-in template engine, live in the internal/http/templates folder. These HTML templates are embedded in the generated executable, using Go's built-in embed machinery.

Here are 2 articles that I found very useful to build this: 👓

"HTML templating and inheritance", from the excellent book Let's Go by Alex Edwards: https://lets-go.alexedwards.net/sample/02.07-html-templating-and-inheritance.html
"Learn how to use the embed package in Go by building a web page easily": https://charly3pins.dev/blog/learn-how-to-use-the-embed-package-in-go-by-building-a-web-page-easily/

Quick tests, in lieu of a REPL

In my cmd/quicktests folder I have Go files that I use for some CLI quick tests of my domain layer - the same way I would have used the REPL (powered by IPython) of the Django shell on my Django app :-) Here is an example of such a file:

// file: cmd/quicktests/get_hall_of_fame_monthly.go
package main

import (
    "context"
    "fmt"

    "github.com/olivierphi/gin-scoring/internal"
    "github.com/olivierphi/gin-scoring/internal/domain/queries"
)

func main() {
    ctx := context.Background()
    db := internal.DB()
    hallOfFameRows, err := queries.CalculateHallOfFameMonthly(ctx, db)
    if err != nil {
        panic(err)
    }
    fmt.Printf("Found %d monthly hall of fame rows\n", len(hallOfFameRows))
    for _, row := range hallOfFameRows {
        fmt.Printf("Row: %#v\n", row)
    }
}

To compile and execute such a file I just run the following command in the terminal:

go run cmd/quicktests/get_hall_of_fame_monthly.go

Implementing the business logic

Last but not least, the business logic of the app lives in the internal/domain package - which really is a port of the Python/Django domain package, almost translated as-is in Go, using the same principles! 👇

The `internal/domain` package

The logic is the same as for the Django app, that I described there:

Is it code that creates, updates or deletes data in a database? 👉 Let's create a new Go file in the internal/domain/mutations package. Each of these files exposes one single function - its name will start with a verb -, and the input of this function is a struct that describes the data we need to execute that business action.
Is it code that reads data from a database? 👉 Let's create a new Go file in the internal/domain/queries package.
Is it code that expresses the business logic but neither alters nor reads data from a database? 👉 Let's put that in a new Go file of the internal/domain/ package.

As they are interacting with the database, all the "domain" functions we create in the "mutations" and "queries" packages will always use the following 2 first arguments: ctx context.Context, db boil.ContextExecutor

The first one is a classic Go context - detailed in this article for example.
The second one is a SQLBoiler ContextExecutor - basically, an entry point to the API generated by SQLBoiler

Ok, let's see some code snippets that illustrate this!

Disclaimer: this is not 'production level' code

The following Go code is pretty naive, just because this is a side project only used by my partner and me to track the results of our Gin Rummy :material-cards-playing: games 🙂

If that was a professional project some parts would be more "industrial" - the errors management or the data validation for example would be managed with more care. 👀

Disclaimer II: I'm not a seasoned Go developer

Although I used Go to build some microservices in my professional coding activity, I still have way less experience with Go than I have with my "core" stacks (these days: Python, Django, TypeScript, React); consequently, the following code is likely not the best and most idiomatic Go code ever written 😅

These 2 disclaimers in mind, let's proceed with some examples of the code that powers this Go version of my "Gin Rummy leaderboard"...

A "mutation"

// file: internal/domain/mutations/save_game_result.go
package mutations

import (
    "context"
    "fmt"
    "strings"

    "github.com/volatiletech/null/v8"
    "github.com/volatiletech/sqlboiler/v4/boil"

    "github.com/olivierphi/gin-scoring/internal/domain"
    "github.com/olivierphi/gin-scoring/internal/models"
)

type SaveGameResultCommand struct {
    PlayerNorthName string
    PlayerSouthName string
    Outcome         domain.GameOutcome
    WinnerName      *string
    DeadwoodValue   uint
}

func SaveGameResult(ctx context.Context, db boil.ContextExecutor, cmd SaveGameResultCommand) (*models.GameResult, error) {
    err := checkSaveGameResultInput(cmd)
    if err != nil {
        return nil, err
    }

    winnerScore, err := domain.CalculateRoundScore(cmd.Outcome, cmd.DeadwoodValue)
    if err != nil {
        return nil, err
    }

    playerNorthName := strings.ToLower(cmd.PlayerNorthName)
    playerSouthName := strings.ToLower(cmd.PlayerSouthName)
    var winnerNamePtr *string
    if cmd.WinnerName != nil {
        winnerName := strings.ToLower(*cmd.WinnerName)
        winnerNamePtr = &winnerName
    }

    resultModel := models.GameResult{ // (1)
        PlayerNorthName: playerNorthName,
        PlayerSouthName: playerSouthName,
        Outcome:         string(cmd.Outcome),
        DeadwoodValue:   int16(cmd.DeadwoodValue),
        WinnerName:      null.StringFromPtr(winnerNamePtr),
        WinnerScore:     null.Int16From(int16(winnerScore)),
    }
    err = resultModel.Insert(ctx, db, boil.Infer())
    if err != nil {
        return nil, err
    }
    return &resultModel, nil
}

func checkSaveGameResultInput(cmd SaveGameResultCommand) error {
    //TODO: use proper validation, powered by the "validator" package :-)

    // Check the outcome
    outcomeOk := false
    for _, outcome := range domain.ValidGameOutcomes {
        if outcome == cmd.Outcome {
            outcomeOk = true
            break
        }
    }
    if !outcomeOk {
        return fmt.Errorf("invalid game outcome '%s'", cmd.Outcome)
    }

    // Winner name must be one of the 2 players's name
    winnerName := *cmd.WinnerName
    if cmd.Outcome != domain.GameOutcomeDraw && winnerName != cmd.PlayerNorthName && winnerName != cmd.PlayerSouthName {
        return fmt.Errorf("winner name '%s' is neither '%s' or '%s'", winnerName, cmd.PlayerNorthName, cmd.PlayerSouthName)
    }
    return nil
}

models.GameResult is a strongly-typed struct generated from our database schema by SQLBoiler. Using wrong types for it will prevent the compilation of the Go program.

Info

On a "real" project I would have used the validator package to handle the data validation: 🙂 https://pkg.go.dev/github.com/go-playground/validator/v10

A "query"

// file: internal/domain/queries/calculate_hall_of_fame_monthly.go
package queries

import (
    "context"
    "sort"
    "time"

    "github.com/volatiletech/sqlboiler/v4/boil"
    "github.com/volatiletech/sqlboiler/v4/queries"
)

type hallOfFameMonthlyRowRaw struct {
    Month      time.Time `boil:"month"`
    WinnerName string    `boil:"winner_name"`
    WinCounts  int       `boil:"win_counts"`
    GrandTotal int       `boil:"grand_total"`
}

type HallOfFameMonthlyRow struct {
    Month      time.Time `boil:"month"`
    GameCounts int       `boil:"-"`
    WinnerName string    `boil:"winner_name"`
    WinCounts  int       `boil:"win_counts"`
    GrandTotal int       `boil:"grand_total"`
    Delta      int       `boil:"-"`
}

const getHallOFameMonthlySQL = `
with first_pass as (
    select
        date_trunc('month', created_at) as month,
        winner_name,
        count(*) as win_counts,
        sum(winner_score) as total_score
    from
        game_result
    where
        winner_score is not null
    group by
        winner_name,
        month
    order by
        month desc,
        win_counts desc
)
select
    month,
    winner_name,
    win_counts,
    total_score,
   (total_score + (win_counts * 25)) as grand_total
from
    first_pass
order by
    grand_total desc
`

func CalculateHallOfFameMonthly(ctx context.Context, db boil.ContextExecutor) ([]*HallOfFameMonthlyRow, error) {
    var rawRes []*hallOfFameMonthlyRowRaw
    err := queries.Raw(getHallOFameMonthlySQL).Bind(ctx, db, &rawRes)
    if err != nil {
        return nil, err
    }

    // Let's group our rows by month:
    resByMonth := make(map[time.Time][]*hallOfFameMonthlyRowRaw)
    for _, rawRow := range rawRes {
        monthRows, ok := resByMonth[rawRow.Month]
        if !ok {
            monthRows := []*hallOfFameMonthlyRowRaw{}
            resByMonth[rawRow.Month] = monthRows
        }
        resByMonth[rawRow.Month] = append(monthRows, &hallOfFameMonthlyRowRaw{
            Month:      rawRow.Month,
            WinnerName: rawRow.WinnerName,
            WinCounts:  rawRow.WinCounts,
            GrandTotal: rawRow.GrandTotal,
        })
    }

    res := make([]*HallOfFameMonthlyRow, 0, len(resByMonth))
    for month, rawRows := range resByMonth {
        gameCounts := 0
        for _, rawRow := range rawRows {
            gameCounts += rawRow.WinCounts
        }

        winner := rawRows[0]
        winnerGrandTotal := winner.GrandTotal
        var delta int
        if len(rawRows) > 1 {
            secondBest := rawRows[1]
            delta = winnerGrandTotal - secondBest.GrandTotal
        } else {
            delta = winnerGrandTotal
        }

        res = append(res, &HallOfFameMonthlyRow{
            Month:      month,
            GameCounts: gameCounts,
            WinnerName: winner.WinnerName,
            WinCounts:  winner.WinCounts,
            GrandTotal: winnerGrandTotal,
            Delta:      delta,
        })
    }

    sort.Slice(res, func(i, j int) bool {
        return res[i].Month.After(res[j].Month)
    })

    return res, nil
}

A "pure domain" function

// file: internal/domain/gin_rummy.go
package domain

import "fmt"

func CalculateRoundScore(outcome GameOutcome, deadwood uint) (score uint, err error) {
    switch outcome {
    case GameOutcomeDraw:
        return
    case GameOutcomeKnock:
        score = deadwood
        return
    case GameOutcomeGin:
        score = deadwood + 25
        return
    case GameOutcomeBigGin:
        score = deadwood + 31
        return
    case GameOutcomeUndercut:
        score = deadwood + 15
        return
    }
    return 0, fmt.Errorf("invalid game outcome '%#v'", outcome)
}

And that's it! 🙂

Even though my day-to-day job mainly involves Python and Django - as well as some React and TypeScript here and there -, I like keeping some other skills "warm" ♨️ - and Go certainly is one of them! 💙

Porting this "Gin Rummy leaderboard" Django app to Go took me around 8 hours, but it was totally worth it, as it allows me to keep a bit of my "coding in Go" muscle memory 🙂