Friday, May 24, 2013

Writing a Go ("golang") Web App with nginx, FastCGI, MySQL, JSON

Want to write a web app in Go ("golang") like you write a PHP app? Go is cool since it's kind-of multi-threaded and has some other neat advantages over PHP. I've had fun setting up a small web app in Go. In my case, it's a simple API which accepts JSON input, decodes it into a struct, does a MySQL database request, and returns a JSON response.

NOTE: Go's built-in net/http web server is production-ready, and I recommend just using it. You shouldn't need to use nginx in front of your Go application in most cases. But this guide will show you how to use JSON and MySQL with your Go app, which is good to know regardless of your server configuration!

Install Go, nginx, and MySQL if not already installed

I will assume you already have these installed and added to your $PATH. If you don't need a database, then don't install MySQL. On my Mac, homebrew was the easiest way to install these -- trust me. (brew install go, brew install nginx, etc.)

If you're using MySQL, then you'll also want to install the Go-MySQL-Driver for Go. Install it simply by doing:

$ go get github.com/go-sql-driver/mysql

Configure nginx/FastCGI

This is actually pretty easy. I assume you already have some experience configuring nginx.conf. (Each install seemingly has different defaults as to the conf file's location, and contents, so I won't go over it here. Mine is in /usr/local/etc/nginx.)

I assume too that you've configured PHP with FastCGI before. If not, you may still understand what's happening.

All you have to do is tell nginx to pass certain requests, or maybe all of them if you wish, to FastCGI on a certain port. Our Go program will have a FastCGI handler listening on that same port. If you need a reference, my entire server { ... } block looks like this:

server {
        listen 80;
        server_name go.dev;
        root /Users/matt/Sites/go;
        index index.html;
        #gzip off;
        #proxy_buffering off;

        location / {
                 try_files $uri $uri/;
        }

        location ~ /app.* {
                include         fastcgi.conf;
                fastcgi_pass    127.0.0.1:9001;
        }

        try_files $uri $uri.html =404;
}

Notice that the server_name is go.dev. I added this to my hosts file with the loopback IP address, 127.0.0.1. So when I type http://go.dev in my browser, it resolves to my own box and nginx receives the request.

Also notice that the fastcgi_pass port is not the default 9000, but is 9001. I did this because I already have php-fpm listening on port 9000. My Go app will listen on 9001.

Further notice that the fastcgi_pass is in a location ~ { ... } block such that any page under /app of go.dev will be redirected to my Go program. You can change this path to whatever you'd like or even replace it with the location / { ... } block above if you want your Go program to be executed out of the root of the site.

Let's see it working

Make a .go file (I'm calling mine app.go) with these contents (though I encourage you to study why/how it's working):

package main

import (
"net"
"net/http"
"net/http/fcgi"
)

type FastCGIServer struct{}

func (s FastCGIServer) ServeHTTP(resp http.ResponseWriter, req *http.Request) {
resp.Write([]byte("<h1>Hello, 世界</h1>\n<p>Behold my Go web app.</p>"))
}

func main() {
listener, _ := net.Listen("tcp", "127.0.0.1:9001")
srv := new(FastCGIServer)
fcgi.Serve(listener, srv)
}

Here's what's happening: The main() function creates a network listener at localhost on port 9001, which is our FastCGI pass-thru port from the nginx config. Then we make a new FastCGIServer and serve requests that pop into that port. The fcgi package has one function: Serve, which blocks and waits for incoming requests. The interface is defined in the net/http package, which is where the ServeHTTP function comes from.

Notice that it receives a ResponseWriter and a Request. From just these, you can write out to your response, and get everything about the incoming request. In this case, all we're doing is writing a simple HTML string.

So, type this in your terminal:

$ go run app.go

Your Go program is now waiting for requests. If you set up nginx.conf like mine, go to http://go.dev/app and you should see, in all its glory:


That was pretty easy!

Now let's receive some simple input from GET/POST

This depends how you want to receive input. If the request was form-URL-encoded (Content-Type: application/x-www-form-urlencoded), like when a form is submitted on a web page, then you can get the form values like this, inside your ServeHTTP function:

fieldValue := req.FormValue("field_name")

This, and more, is documented in the net/http package. Note that fieldValue will be a string unless you convert it to another type yourself. This will nab the field from either POST or GET, but POST takes precedence over GET.

Getting the raw POST request body

But what if you want to get the raw contents of a POST request, like a JSON string? That's also pretty easy. First, make sure a request body exists, then read it. You'll need to import "io/ioutil" since the request Body is a stream. 

if req.Body == nil {
return
}
body, err := ioutil.ReadAll(req.Body)
req.Body.Close()

Decoding JSON

Decoding the JSON is also easy, but it depends on how you want to do it. Don't forget to import "encoding/json". You'll want to make a type to hold your deserialized JSON data. You can use JSON-to-Go to quickly convert JSON into a Go type definition, or make your own manually:

type UserInput struct {
SomeField string
AnotherField int
LastOne string
}

Then you can decode the JSON and write out a value, like so:

var inp UserInput
json.Unmarshal(body, &inp)
resp.Write([]byte(m.SomeField))

Simple!

The nice thing is that you don't need to include all the JSON fields that the JSON input has. Any fields you don't include in your struct, or that don't match the JSON input's structure will be ignored.

Encoding (serializing) JSON

Now let's JSON-encode a struct to output.

bytesOfJSON, _ := json.Marshal(myStructOrSlice)

Here I'm throwing away the error, which isn't smart, but for this example, whatever. The encoded string is now contained as a []byte (byte array, or more technically, a byte slice) which conveniently can be passed right into the ResponseWriter's Write function:

resp.Write(bytesOfJSON)

Tada!

A simple MySQL database request

Now let's do something with the database. Assuming you ran the "go get" line at the very beginning of this article and have the driver's package installed, import these two packages:

"database/sql"
_ "github.com/go-sql-driver/mysql"

I chose the Go SQL Driver over MyMySQL because the Go SQL Driver uses the native database/sql package and is a little newer. But both packages are excellent and work without any real problems.

Making the connection (under the hood, it doesn't actually make the connection until needed, from what I understand) is simple:

conn, err := sql.Open("mysql", "/test")
defer conn.Close()

if err != nil {
fmt.Println("Oh noez, could not connect to database")
return
}

Let me explain the first line. The first argument is the type of database, in our case, "mysql" will do. The second argument is the connection string, which, if you're familiar with PEAR DB, works almost the same way. In my case, I simply specified "/test" because the default user is "root" and I have no password on my dev machine's MySQL server, so those all go away because of defaults, and the name of my database is "test". See the docs for a brief overview of the format for the connection string (it's really quite easy).

The defer line ensures that the connection is closed when we're through with it.

Then to query a row and get a single result, I do this:

var zipcode string
sqlerr := conn.QueryRow("SELECT ZipCode FROM Cities WHERE CityName=? LIMIT 1", "Las Vegas").Scan(&zipcode)
switch {
case sqlerr == sql.ErrNoRows:
    fmt.Printf("No rows")
case sqlerr != nil:
    fmt.Println(sqlerr)
default:
    fmt.Printf("ZIP code is %s\n", zipcode)
}

Now, this only gets one ZIP code for Las Vegas, but there are actually many. How do we get the rest of them?

Easy:

var zipcodes []string

rows, err := db.Query("SELECT ZipCode FROM Cities WHERE CityName=?", "Las Vegas")

if err != nil {
fmt.Println(err)
return
}

for rows.Next() {
var zip string
if err := rows.Scan(&zip); err != nil {
fmt.Println(err)
return
}
zipcodes = append(zipcodes, zip)
}

if err := rows.Err(); err != nil {
fmt.Println(err)
return
}

Ahhhh... much better. Writing that out to the response (as a JSON array, of course) gives me a list of ZIP codes in Las Vegas:


["87701","87745","89044","89054","89101","89102","89103","89104","89105","89106",
"89107","89108","89109","89110","89111","89112","89113","89114","89115","89116",
"89117","89118","89119","89120","89121","89122","89123","89124","89125","89126",
"89127","89128","89129","89130","89131","89132","89133","89134","89135","89136",
"89137","89138","89139","89140","89141","89142","89143","89144","89145","89146",
"89147","89148","89149","89150","89151","89152","89153","89154","89155","89156",
"89157","89158","89159","89160","89161","89162","89163","89164","89165","89166",
"89169","89170","89173","89177","89178","89179","89180","89183","89185","89191",
"89193","89195","89199"]

So... now what?

Well, now that we can read requests, send responses, use MySQL, encode/decode JSON, and all through nginx and FastCGI which is available even on shared hosting, I'd say the possibilities are nearly endless. Ready? Set... Go!