Sunday, December 6, 2009

Reflection Package in Go

While Go does not have the level of introspection that allows a language like Python to define new types during runtime, it at least has the ability to examine and manipulate arbitrary types. One can, for example, pass a random object into a function that will identify the object's type and, in the case of a struct, its fields.

Indeed, if one looks into the 'fmt' package at the Printf family of functions, we see that Go's implementation of arbitrary parameters (the ... parameter) requires the use of 'reflect' to extract those parameters. Most interesting, however, is the 'xml' package, which contains a function call 'Unmarshal'. Unmarshal parses an xml file and attempts to implant the data into a given struct, matching xml elements with struct fields.

Unfortunately, reflection has a major issue: a lack of documentation. All we have to work with is the package specification and the aforementioned examples of the 'fmt' and 'xml' packages to learn from. Furthermore, these examples are rather complex. For this reason, I will make an attempt at producing a series of simple examples.

First off, let us have Go identify a parameter of arbitrary type.

Go allows one to take a parameter of any type and store it or pass it into a function via the empty interface: 'interface{}'. Since the empty interface defines no methods, all types implement it -- even basic types like integers or booleans. Thus we have the skeleton of our 'Identify' function:

func Identify (val interface{}) {

}

The first thing that 'reflect' must do for us is turn 'val' into something that the package can work with. The function that does this is NewValue, which takes as a parameter any value and returns a reflection value. Note that the type 'Value' is an interface. This will be more important later.

func Identify (val interface{}) {
    v := reflect.NewValue (val)
}


From here, the most basic way to identify the type is to simply ask for it via reflect.Value's 'Type' method. This returns a type object, which in turn implements a method call 'Name' that returns a string.

func Identify (val interface{}) {
    v := reflect.NewValue (val);

    fmt.Println (v.Type ().Name ())
}


Thus, when we call 'Identify (5)' the output is 'int'. Call 'Identify (Stuff{3})' and the output is 'Stuff'.

Not a very useful function except, perhaps, for debugging. Let's try something more complex, like diving into a struct instance to extract all field names and values and print them. Basically, it will be a generic Print function.

type Stuff struct {
    Num int;
    Str string;
    Boo bool
}

func PrintAny (val interface{}) {
}


We start the same way as with 'Identify', but creating a reflection value. However, to keep things generic and to avoid massive indentation, we'll use PrintAny simply as a feeder for the function that will do the actual work.

func printAny (val reflect.Value) {

}

func PrintAny (val interface{}) {
    v := reflect.NewValue (val);
    printAny (v)
}


To start we'll just have printAny deal with basic types. For the sake of simplicity, I'll just deal with the three basic types we'll need for our struct: int, string, and bool. First we'll use an if-statement to determine the type. Since the second value returned by a type assertion tells us whether the cast was successful, we can use that to determine whether we've found the correct type. The cast variable is of a type that implements a function called 'Get', which returns the value of the variable. The basic interface Value does not implement Get because it does not know what type to return.

func printAny (val reflect.Value) {
    if v, ok := val.(*reflect.IntValue); ok {
        fmt.Printf ("%d\n", v.Get ())
    } else if v, ok := val.(*reflect.StringValue); ok {
        fmt.Println (v.Get ())
    } else if v, ok := val.(*reflect.BoolValue); ok {
        fmt.Printf ("%t\n", v.Get ())
    } else fmt.Println ("Unknown Type")
}

This is the cumbersome way of doing this. I show it simply to demonstrate type assertions with reflect. Now let's simplify it using the ability of the switch-statement to deal with types. This way, we do not have to worry about 'ok'. The switch-statement automatically looks over the cases to decide which is valid, with 'default' handling the final 'else' case.


func printAny (val reflect.Value) {
    switch v := val.(type) {
        case *reflect.IntValue:
            fmt.Printf ("%d\n", v.Get ())
        case *reflect.StringValue:
            fmt.Println (v.Get ())
        case *reflect.BoolValue:
            fmt.Printf ("%t\n", v.Get ())
        default:
            fmt.Println ("Unknown Type")
    }

}

Thus:

func main () {
    var num int = 4;
    PrintAny (num)
}

. . . will output '4'. However, what if we call 'PrintAny (&num)'? It'll run into the default case. Let's get around that.

The reflect package handles pointers via the 'PtrValue' type. The value held by the memory pointed to is accessed by PtrValue's 'Elem' method, which returns a generic reflect 'Value'. To get the actual value, we must run it through the switch-statement again, but instead of nesting one, we'll simply recurse by calling printAny again with the result of 'Elem' as the parameter.

func printAny (val reflect.Value) {
    switch v := val.(type) {
        case *reflect.PtrValue:
            if v.IsNil () {
                fmt.Println ("nil")
            }
            printAny (v.Elem ())
        case *reflect.IntValue:
            fmt.Printf ("%d\n", v.Get ())
        case *reflect.StringValue:
            fmt.Println (v.Get ())
        case *reflect.BoolValue:
            fmt.Printf ("%t\n", v.Get ())
        default:
            fmt.Println ("Unknown Type")
    }
}


This is why I separated the bulk of the code from the original PrintAny function.

Note that I check to see if the pointer is nil. If I don't, when the function recurses it'll land in the default case.

Next is dealing with structs. All structs, regardless of the actual type, are defined by 'reflect.StructValue'. It implements all methods necessary to iterate through its fields and extract their data. The fields are accessed individually by calling the 'Field' method with an index as the parameter. 'Field' returns a basic 'Value', so we can recurse again.

func printAny (val reflect.Value) {
    switch v := val.(type) {
        case *reflect.PtrValue:
            if v.IsNil () {
                fmt.Println ("nil")
            }
            printAny (v.Elem ())
        case *reflect.StructValue:
            for i := 0; i < v.NumField (); i++ {
                printAny (v.Field (i))
            }
        case *reflect.IntValue:
            fmt.Printf ("%d\n", v.Get ())
        case *reflect.StringValue:
            fmt.Println (v.Get ())
        case *reflect.BoolValue:
            fmt.Printf ("%t\n", v.Get ())
        default:
            fmt.Println ("Unknown Type")
    }
}


Wait, though: in the introduction to this example I said that I wanted the field names, too. Fortunately 'reflect' allows us to find those via the 'StructType' type. 'StructType' and other '*Type' types in 'reflect' provide information on the type itself. In the case of structs, this includes information on the fields. This is accessed by casting the value returned by the 'Type' method.

The methods for accessing the fields of a 'StructType' do not return a 'Value', but a 'StructField', a struct that contains data about the field, including what we are looking for: the field's name. As such, we iterate through it much as we would the 'StructValue''s fields.

func printAny (val reflect.Value) {
    switch v := val.(type) {
        case *reflect.PtrValue:
            if v.IsNil () {
                fmt.Println ("nil")
            }
            printAny (v.Elem ())
        case *reflect.StructValue:
            typ := v.Type ().(*reflect.StructType);

            for i := 0; i < typ.NumField (); i++ {
                name := typ.Field (i).Name;
                field := v.FieldByName (name);

                fmt.Printf ("%s: ", name);
                printAny (field)
            }
        case *reflect.IntValue:
            fmt.Printf ("%d\n", v.Get ())
        case *reflect.StringValue:
            fmt.Println (v.Get ())
        case *reflect.BoolValue:
            fmt.Printf ("%t\n", v.Get ())
        default:
            fmt.Println ("Unknown Type")
    }

}

And that's it! Try it out with our 'Stuff' struct:


func main () {
    a := Stuff{4, "Hi!", true};
    PrintAny (&a)
}


The obvious application for a similar function would be serialization, though I would not use it for complex structures. It would indeed be best only for small structures tailor made to save only that which needs to be saved. Everything else can be derived when reading from the saved data.

Another application is the ... parameter, as I mentioned at the beginning of this article. The ... parameter is treated as a struct containing the values passed into it. For example:

fmt.Printf (format, 1, 2.3, true, "me");

The types that Printf deals with internally are actually a string and a struct with an int field, a float field, a bool field, and a string field. Of course, it also handles more types that my little example does, but it is not difficult to implement them.

4 comments:

  1. Why do you write on a black background. It suffers from the problem that the text bleeds into the background and becomes harder to read. A dull background or a white background would be preferable for people who no longer have 20/20 vision.

    ReplyDelete
  2. this is good but no longer reflects the function name in reflect. would be nice if it were updated.

    ReplyDelete

Followers