In this post we will try to serialize some struct data into redis (via redigo library), by using gob package for serializing and deserialing to and from redis.

Overview

In software development, serializing means transforming data, from less knowledgable format to others (like golang struct, ruby object or java object) into something more common (like bytes). Bytes are usually easier to store and transfer (via network).

Deserializing is the reverse of serializing.

Check the wikipedia entry for more reference.

Transferring data via network may require serialization so we won’t mix the data with the protocol specification.

Storing data to RDBMS or NoSQL may also require serialization, because those database don’t know anything about golang struct.

In this article below, we are going to save a struct data into redis. The flow will be like this:

• Create sample data
• Encode/Serialize the data
• Send encoded data to redis
• Receive encoded data from redis
• Decode/Deserialize the data, get the struct data back

With this mechanism in place, we can create caching or just storing temporary data to redis, or transfer reliably it via network. But do note that if we transfer it via network, the recipient of data knows how to handle deserialized data.

Sample Data

Let’s create our new Golang app for demonstration purpose.

There are several struct types here, here we are going to demonstrate serializing and deserializing struct with embedded and array structs.

One struct type is Address, it has other structs embedded in it.

package address

type Address struct {
	/*
		2 examples:
		4317 Lewis Street
		Jl. Sukamaju No. 23, RT 2 RW 014 Kelurahan Sukabakti
	*/
	DetailAddress string

	City        City
	Province    Province
	Country     Country
	PostalCode  string
	PhoneNumber string
}

type City struct {
	Name string
}

type Province struct {
	Name string
}

type Country struct {
	Name string
}

package customer

import "github.com/dwahyudi/golang-serialize-to-redis/address"

type Customer struct {
	Name      string
	Addresses []address.Address
}

With this struct types, we can then proceed creating a data sample.

// Specify some sample data
address1 := address.Address{
    DetailAddress: "Jalan Kecapi no. 23 Desa Sukasayur",
    City:          address.City{Name: "Sukabumi"},
    Province:      address.Province{Name: "Jawa Barat"},
    Country:       address.Country{Name: "Indonesia"},
    PostalCode:    "99999",
    PhoneNumber:   "+62333333333",
}

address2 := address.Address{
    DetailAddress: "274 Brookside Drive",
    City:          address.City{Name: "New York City"},
    Province:      address.Province{Name: "New York"},
    Country:       address.Country{Name: "United States of America"},
    PostalCode:    "77777",
    PhoneNumber:   "+1222222222",
}

var addresses = []address.Address{address1, address2}
customer1 := customer.Customer{
    Name:      "Jono",
    Addresses: addresses,
}

Encode the Data

Before we move further, we might want to create a simple function for handling error:

func panicErr(err error) {
	if err != nil {
		panic(err)
	}
}

Now, let’s move to serializing…

There are plenty ways to serialize data, ex: msgpack, protobuf and json. But Golang has built-in package for serializing data, gob, it will transform our data into binary values (which are bytes).

gob is simple and easy to use, and it is built-in, and it’s real fast, it gets the job done, but.. if we need more extreme performance, we may want to go with protobuf instead.

To serialize data with gob, it is quite easy:

// Encode data with gob
encodedToBeSent := new(bytes.Buffer)
err := gob.NewEncoder(encodedToBeSent).Encode(customer1)
panicErr(err)

customer1 data has been transformed into bytes by gob into encodedToBeSent.

Sending and Receiving To And From Redis

Before we write the code, make sure a redis server is up.

We will be using redigo, so make sure to install the library first in our go app:

go get github.com/gomodule/redigo@v2.0.0

package util

import (
	"github.com/gomodule/redigo/redis"
	"log"
	"os"
)

func RedisConn() (redis.Conn, error) {
	conn, err := redis.Dial("tcp", ":6379")

	if err != nil {
		log.Printf("ERROR: fail initializing the redis pool: %s", err.Error())
		os.Exit(1)
	}
	return conn, err
}

Opening the connection will be simple:

// Open redis connection
redisConn, err := util.RedisConn()
panicErr(err)

Now, let’s send the encoded data to redis, customer1 is the redis key, in real-world application, we might want to design a specific redis key naming.

// Send gob-encoded data with redis specific key
_, err = redisConn.Do("SET", "customer1", encodedToBeSent.Bytes())
panicErr(err)

_, err = redisConn.Do("SET", "customer1-with-expire", encodedToBeSent.Bytes(), "EX", 60)

Let’s pretend to retrieve it in later use (for caching (so we don’t need to query the database again) or something else):

// Get gob-encoded data from redis, with the same key
reply, err := redisConn.Do("GET", "customer1")
panicErr(err)

Decode the Data

We can then decode the reply (encoded customer data above):

Be very careful when deserializing, make sure that the struct assigned for deserializing via Decode function, is correct, otherwise gob will return error.

// Decode data
encodedReceived := reply.([]byte)
var decodedCustomer *customer.Customer
encodedReceivedIOReader := bytes.NewBuffer(encodedReceived)
err = gob.NewDecoder(encodedReceivedIOReader).Decode(&decodedCustomer)
panicErr(err)

We have captured the data from redis into decodedCustomer.

When we try to inspect decodedCustomer, the result should be good.

fmt.Println(decodedCustomer)
fmt.Println(decodedCustomer.Name)
fmt.Println(decodedCustomer.Addresses[0].City.Name)

And here’s how redis monitor looks like: