---
title: Reflection in Go
date: 2020-08-17
description: A practical walk-through of Go's reflect package using a real CLI flags library as the guiding example.
tags: [go]
---

I'm going to walk you through how to understand reflection in Go using the [`reflect`](https://golang.org/pkg/reflect/) package. We'll do this by looking at an open-source package I created called [`go-flags`](https://github.com/integralist/go-flags) that utilizes reflection _heavily_.

The use of reflection is often frowned upon because it side steps the 'compile time' type safety we get in Go. So I'll also explain how we can still ensure a level of type safety even at runtime.

## What is reflection?

> Reflection in computing is the ability of a program to examine its own structure, particularly through types; it's a form of metaprogramming. It's also a great source of confusion.\
> -- [The Laws of Reflection](https://blog.golang.org/laws-of-reflection)

The above quote comes from the opening of the Go Blog post [The Laws of Reflection](https://blog.golang.org/laws-of-reflection) and I _strongly_ recommend you read it, as it breaks down how types and interfaces work in Go (which is a prerequisite to understanding reflection) before leading into how reflection itself works.

There will be a summarized version of that information here, but ultimately what differentiates their post from mine is that I'll be discussing examples that are more practical and based on a _real_ project (e.g. [`go-flags`](https://github.com/integralist/go-flags)).

## Understanding Interfaces in Go

In Go an 'interface type' is a collection of methods which can be defined like so:

```go
type Foo interface {
    Bar(s string)
    Baz(i int)
}
```

If you define a variable to be of type `Foo`, then you're saying the value that is assigned to the variable can be _any_ type that implements the methods as defined by the `Foo` interface.

Conceptually the variable containing the assigned value can be thought of as a tuple that contains the underlying concrete value and the concrete type.

This is best demonstrated in an example. Below we create an interface type `Foo`, and then we define a struct type that will implement that interface. We'll assign an instance of the struct to a variable defined as being an interface type of `Foo` and we'll inspect the code at runtime using the `reflect` package to see what it can tell us about the variable and what it contains:

```go
package main

import (
    "fmt"
    "reflect"
)

type Foo interface {
    Bar(s string)
    Baz(i int)
}

type T struct{
    s string
    i int
}

func (t T) Bar(s string) {
    //
}

func (t T) Baz(i int) {
    //
}

func main() {
    var f Foo
    f = T{"testing", 123}

    fmt.Printf("(%+v, %+v)\n", reflect.ValueOf(f), reflect.TypeOf(f))
}
```

The output of the above program is:

```go
({s:testing i:123}, main.T)
```

OK, so we can see that the 'value' assigned to the `f` variable (of interface type `Foo`) is a struct with the relevant fields assigned with the appropriate value types. We can also see that the 'type' of the value is `main.T`, which is correct and expected.

So this clearly demonstrates that although we've stated the variable `f` should hold an interface type of `Foo`, it's actually holding a different type entirely.

This probably sounds like common sense (and as such not that interesting) but if you really think about it, that's incredible. We've not explicitly declared the `T` struct as implementing the `Foo` interface.

Instead the fact it does implement the interface, and doing so is being _dynamically_ understood by the compiler is a super powerful feature.

## Why do we need Reflection?

You need reflection whenever dealing with information that wasn't known at the time of the program being written. One such example might be that you need to populate a struct with configuration data that comes from a external file.

There are many other examples, and the one I'll be using today is similar in that I wanted to populate a struct with values provided by a user at runtime, specifically CLI flag values.

## The building blocks of Reflection

So let's consider what are probably the two _primary_ reflection methods people learn about first when dealing with reflection (these were used in our example code earlier):

- `reflect.ValueOf()`: returns a `reflect.Value`.
- `reflect.TypeOf()`: returns a `reflect.Type`.

Both returned types have a bunch of methods associated with them that are unique to the purpose of those types. So for example, you want a `reflect.Value` if you're interested in querying the concrete value, where as `reflect.Type` is more useful when you need to know information about the specific type.

So the use case for getting back a `reflect.Type` vs `reflect.Value` depends on what you're trying to ascertain at runtime.

Once you have either a `reflect.Value` or a `reflect.Type`, then you can start digging further down into the data that is exposed via these types.

This is where exploring a real-world use case can be handy, as it helps to clarify _why_ certain methods on these objects are called (compared to using a more _contrived_ example).

## Package Walk-through: Go-Flags

If you've ever written a CLI program in Go you'll inevitably use its [`flag`](https://golang.org/pkg/flag/) package. Most people find this package unintuitive and in some cases just downright difficult to work with or to build nice patterns of use from.

This leads people to build their own abstraction patterns on top of the standard library, but from my experience the majority (if not _all_) of third-party flag packages are convoluted and confusing to work with. The user experience feels very poor IMHO.

> [!INFO]
> I'm not knocking these packages. They all do very complex things, and solve real problems, and are written by much smarter people than myself. But it doesn't change the fact that rarely do they come across as 'simple' and 'easy to use'.

I wanted to solve the problem of handling flags and commands for a CLI based program in a simple way, that didn't require me to construct a whole bunch of boilerplate code and have to negotiate lots of types.

In essence I wanted to define a 'schema' of what flags and commands (and flags for those commands) that I was expecting, and to have a package magically populate that struct with the values provided by a user of my program.

This is where utilizing reflection was the only real solution to my problem (if I indeed wanted the user experience to be as simple as possible).

So thus [`go-flags`](https://github.com/integralist/go-flags) was born.

Here's an example of how you would use it:

```go
package main

import (
    "fmt"
    "os"

    "github.com/integralist/go-flags/flags"
)

type Schema struct {
    Debug   bool   `short:"d" usage:"enable debug level logs"`
    Number  int    `short:"n" usage:"a number field"`
    Message string `short:"m" usage:"a message field"`
    Foo     struct {
        AAA string `short:"a" usage:"does A"`
        BBB string `short:"b" usage:"does B"`
    }
    Bar struct {
        CCC bool `short:"c" usage:"does C"`
    }
}

func main() {
    var s Schema

    err := flags.Parse(&s)
    if err != nil {
        fmt.Printf("error parsing schema: %v\n", err)
        os.Exit(1)
    }
}
```

This is very simple and concise from a user's perspective.

We can see that there are a bunch of top-level, non command specific, flags defined:

- `-d/-debug`
- `-n/-number`
- `-m/-message`

You'll notice that we utilize [struct tags](https://golang.org/ref/spec#Struct_types) (the backticks that follow the struct field's type) for defining the 'short' flag (while the field name itself is defining the 'long' flag variation) as well as defining the 'usage' description for each flag.

We can also see that we've defined two separate commands (`Foo` and `Bar`), and each command has its own set of flags:

- `foo`: `-a/-aaa` (`string`), `-b/-bbb` (`string`).
- `bar`: `-c/-ccc` (`bool`).

An example of how a user might then run this program would be:

```bash
my_cli_app -debug -n 123 -m "something here" foo -a beepboop -b 666
```

## Explaining how go-flags uses Reflection

So how exactly does [`go-flags`](https://github.com/integralist/go-flags) achieve this magic using reflection?

Let's step through the code and find out...

> [!INFO]
> I'm not an expert on the `reflect` package, nor was a lot of time spent on this package outside getting it functional, so there's likely many improvements that can be made to the code.

We'll start with the [`Parse()`](https://github.com/Integralist/go-flags/blob/4704c0e/flags/flags.go#L22) function. Here we can see that the function accepts an argument of type `interface{}` which is the empty interface.

The reason for this is because we want the schema for the flags to be defined by the consumer of this package, and so at runtime we have no idea what that value will look like (hence the use of `interface{}` to mean: we'll take any type).

Once we've got this unknown value we'll execute the following code to get at the actual value:

```go
v := reflect.Indirect(reflect.ValueOf(s))
```

We can see the familiar `reflect.ValueOf()` but we have also wrapped it in a call to `reflect.Indirect()`, so why do that? Well, the `Indirect()` method will dereference the struct pointer to the struct concrete value!

It's also safe to use `Indirect()` because if you give it a non-pointer value it'll just return the value from `reflect.ValueOf()`. Don't worry, in just a moment we'll do some validation of the given argument to ensure the user has followed our package instructions and indeed given us a pointer to a struct containing their flag schema.

Next we acquire a `reflect.Type` type from the given argument:

```go
st := v.Type()
```

We get the type structure from the `reflect.Value` type returned from calling `reflect.Indirect()`, and we do this because later on we'll pass it into a function called [`IterFields`](https://github.com/Integralist/go-flags/blob/4704c0e/flags/flags.go#L160) which iterates over the user's schema struct. We pass it into that function because within there we'll try to get the struct's individual fields as a `reflect.StructField` (I'll explain why we want that later).

Now it's at this point we do some runtime type validation:

```go
if v.Kind() != reflect.Struct {
  return ErrWrongType
}
```

We're expecting a struct and there's no point trying to go any further with something that's _not_ a struct, so we short-circuit the program if we're given anything unexpected.

OK, at this point of our package we have a few stages that involve, among many things, iterating the given struct (we use a custom iterator function called `IterFields` and we pass it a callback function to execute for each 'field' in the given struct).

**Here are the stages at a high-level:**

1. iterate over the top level fields of the user provided struct, and dynamically create flags.
1. parse the flag values the user of the CLI program have provided (using the flags our package has now dynamically generated).
1. iterate over the top level fields of the user provided struct, and populate the fields with the parsed flag values.
1. identify the command the user provided when running the CLI program (e.g. `foo` or `bar`).
1. parse the flag values the user provided after specifying the command.
1. dynamically create a `flag.FlagSet` for the command.
1. parse the flagset values the user provided (using the flagset our package has now dynamically generated).
1. iterate over the command fields of the user provided struct, and populate the fields with the parsed flagset values.

Now we understand the general steps taken, we can dig into each of those and understand what reflection work needs to be carried out.

I won't be discussing the general code logic, just picking out the bits related to reflection. So if you want to understand the full go-flags implementation, then I recommend you read through the code base in its entirety after finishing this post.

> [!INFO]
> if I was working with Go 2.0 then I'd have access to generics and parts of this code could be reduced/simplified. Hello future readers who are lucky enough to have Go 2.0.

As the majority of these steps are related to iterating over the user provided struct, let's start with the `IterFields` function that enables that.

First thing we do in `IterFields()` is we call `.NumField()` on the `reflect.Value`, which gives us back the number of fields the struct contains.

This will panic if the `reflect.Kind()` is not a `reflect.Struct` type, hence why at the beginning of our `Parse()` function we do that validation there.

We could have moved the check for a struct down into the `IterFields()` function but then we'd end up doing extra unnecessary logic processing only to have to just stop the program any way (so best to short circuit the code as soon as possible).

Once we have the number of struct fields, we'll create a loop for that number and we'll use the incrementing `i` value to access each individual struct field by calling `.Field()` on the `v` variable's `reflect.Value` type.

> [!INFO]
> you'll find that when you call methods on a `reflect.Value` type you'll likely end up with ...another `reflect.Value` type!

```go
for i := 0; i < v.NumField(); i++ {
  field := v.Field(i)

  ...
}
```

Next we call the `.Field()` method again but this time on the `reflect.Type` type. The reason we do this is because (as I mentioned earlier) we want a `reflect.StructField` type (which is what the `.Field()` call will give us) so we can inspect it later on in the callback function and parse out the 'tags' assigned to the current struct field.

We have at this point a if/else condition which I'm going to skip over the `if` block briefly and move onto discussing the `else` block which states:

```go
if !recurse && field.CanSet() {
    callback(field, sf)
}
```

You'll see we call `.CanSet()` on the `field` variable which (as noted earlier) is actually a `reflect.Value` type. The reason we do this is because one of the given callback functions will attempt to set a value onto the struct field and that is only valid if the struct field is public (i.e. _exported_) otherwise it would panic.

Let's jump back up to the `if` statement, in there we're checking the condition:

```go
if field.Kind() == reflect.Struct {
  ...
}
```

This is a similar check to what we did at the start of the `Parse()` function, so why are we doing it again? Well, this time (as we iterate over the user's schema struct and look at each field) we might discover one of the fields is itself a struct! So we want to also iterate over the nested struct's fields looking for flag information.

All the code within this `if` block is effectively the same as what came before it with regards to getting access to the underlying struct 'value' that is assigned to the current struct's field, and then looping over that nested structs own fields.

We could have done this with recursion but then it makes the `IterFields()` function more complex and as I was designing go-flags to work with only one level of nesting supported it wasn't worth the effort to implementing a strict recursive function.

That said, there is one aspect of the `if` block we should look at which is the _nested_ `if` block. The nested `if` block is what actually does the looping over the nested struct's fields, and it does this by first getting at the nested struct concrete value (as I mentioned a moment ago).

To get at that struct value we do:

```go
reflect.TypeOf(field.Interface())
```

OK, so this needs some explaining! Firstly, we call `.Interface()` on the `field` variable (which is a `reflect.Value` type) and then we pass that 'result' to `reflect.TypeOf()`.

The `.Interface()` method will get the underlying concrete value as an empty interface (i.e. `interface{}`) type, and so that's what we pass to `reflect.TypeOf()`.

The reason we use `.Interface()` first is because if we had just passed the `field` variable into `reflect.TypeOf()`, then we would have gotten `reflect.Value` as the type (and that's no use to us! we want the struct type).

So why did we need the struct type? Well, after we start iterating over the nested struct's fields, we again need to get a `reflect.StructField` so that when we execute the given callback function it'll be able to parse any 'tags' found on the nested struct fields.

At this point in time we've explained some key bits as far as looping _safely_ over the user's struct, so let's go back to what we do in some of the 'callback' functions that get executed for each struct field.

One thing we do is access the struct field's tags, and we do this by calling `.Tag.Get()` on the `reflect.StructField` type. So in go-flags we ask users to add struct tags like so:

```go
short:"..." usage:"..."
```

That's two tags 'short' and 'usage'. So if I want the 'short' tag I'd call `.Tag.Get("short")` and if I want the 'usage' tag I'd call `.Tag.Get("usage")`. Nice and simple.

When it comes to setting the field values we have to check the kind of the field (remember this is a `reflect.Value` type). The following switch statement demonstrates this:

```go
switch field.Kind() {
case reflect.Bool:
    if b, ok := getter.Get().(bool); ok {
        field.Set(reflect.ValueOf(b))
    }
case reflect.Int:
    if i, ok := getter.Get().(int); ok {
        field.Set(reflect.ValueOf(i))
    }
case reflect.String:
    if s, ok := getter.Get().(string); ok {
        field.Set(reflect.ValueOf(s))
    }
}
```

The call to `.Kind()` returns a `reflect.Kind` type, which is an integer iota that maps constants to a numerical value for easier comparison. Hence each case is a check against some of those constants like `reflect.Bool`, `reflect.Int` and `reflect.String`.

Once we know what type the field is we type assert the flag value to the relevant type and then call `.Set()` on the field. You'll notice that we can't just pass the value into the `.Set()` method as we need to provide an argument of type `reflect.Value`.

To fix that we call `reflect.ValueOf()` and pass it the value we're trying to set, and then pass that resulting `reflect.Value` type into `.Set()`.

## Conclusion

Well, that's it! There is _sooooo_ much more to reflection than what I have had time to describe in this post, but hopefully this has been enough to help you in the future if you ever stumble across a need for reflection.

One thing I would say before I go is: avoid reflection wherever possible. It's a nightmare to work with and although we can code defensively and get back some runtime safety, that's still no comparison to compile time safety. There's also a performance cost to runtime reflection that can't be ignored.

So do please think twice before reaching for reflection.