go-testdeep

Maxime Soulé

2020-07-20

2

3

Go testing

Why a test framework?

To avoid boilerplate code, especially error reports, like:

import "testing"

func TestGetPerson(t *testing.T) {
    person, err := GetPerson("Bob")
    if err != nil {
        t.Errorf("GetPerson returned error %s", err)
    } else {
        if person.Name != "Bob" {
            t.Errorf("Name: got=%s expected=Bob", person.Name)
        }
        if person.Age != 42 {
            t.Errorf("Age: got=%s expected=42", person.Age)
        }
    }
}
4

Using a test framework

For example using go-testdeep:

import (
    "testing"

    "github.com/maxatome/go-testdeep/td"
)

func TestGetPerson(t *testing.T) {
    person, err := GetPerson("Bob")
    if td.CmpNoError(t, err, "GetPerson does not return an error") {
        td.Cmp(t, person, Person{Name: "Bob", Age: 42}, "GetPerson returns Bob")
    }
}

In most cases, there is not even test names:

func TestGetPerson(t *testing.T) {
    person, err := GetPerson("Bob")
    if td.CmpNoError(t, err) {
        td.Cmp(t, person, Person{Name: "Bob", Age: 42})
    }
}
5

Why go-testdeep instead of an existing framework?

Custom comparison engine allowing the use of operators

Accurate (and colored) error reports

60 operators to match in all circumstances

Fully documented with plenty examples

Consistent API: got parameter is always before expected one

Very few basic functions, all others are operators shortcuts (args... allow to name tests):

Cmp(t TestingT, got, expected interface{}, args ...interface{}) bool
CmpError(t TestingT, got error, args ...interface{}) bool
CmpFalse(t TestingT, got interface{}, args ...interface{}) bool
CmpLax(t TestingT, got interface{}, expected interface{}, args ...interface{}) bool
CmpNoError(t TestingT, got error, args ...interface{}) bool
CmpNot(t TestingT, got interface{}, notExpected interface{}, args ...interface{}) bool
CmpNotPanic(t TestingT, fn func(), args ...interface{}) bool
CmpPanic(t TestingT, fn func(), expectedPanic interface{}, args ...interface{}) bool
CmpTrue(t TestingT, got interface{}, args ...interface{}) bool
6

Test names

As many testing frameworks, tests can be named:

td.Cmp(t, got, "Bob", `Hey! got has to be "Bob" here!`)

Each Cmp* function cleverly accepts fmt.Fprintf or fmt.Fprint parameters in args

func Cmp(t TestingT, got, expected interface{}, args ...interface{}) bool

The doc says:

// "args..." are optional and allow to name the test. This name is
// used in case of failure to qualify the test. If len(args) > 1 and
// the first item of "args" is a string and contains a '%' rune then
// fmt.Fprintf is used to compose the name, else "args" are passed to
// fmt.Fprint. Do not forget it is the name of the test, not the
// reason of a potential failure.

So, no risk of mistake between Cmp and a (nonexistent) Cmpf: only use Cmp!

td.Cmp(td, got, 12, "Check got is", 12)    → fmt.Fprint
td.Cmp(td, got, 12, "Check got is %d", 12) → fmt.Fprintf
td.Cmp(td, got, 12, lastErr)               → fmt.Fprint
7

Custom comparison engine

Derived from reflect.DeepEqual and heavily modified to integrate operators handling

It allows go-testdeep to know exactly where a test fails in a big structure, and even to continue testing in this structure to report several mismatches at the same time (up to 10 by default)

Reports are also very accurate and, cherry on the cake, colorized:

instead of awful diffs you see elsewhere…

8

60 operators to match in all circumstances

Some examples, see the expected (3rd) parameter:

              here ↴
td.Cmp(t, age,     td.Between(40, 45))
td.Cmp(t, headers, td.ContainsKey("X-Ovh-Account"))
td.Cmp(t, err,     td.Contains("Internal server error"))
td.Cmp(t, grants,  td.Empty())
td.Cmp(t, price,   td.N(float64(12.03), float64(0.01)))
td.Cmp(t, name,    td.Re(`^[A-Z][A-Za-z-]+\z`))
td.Cmp(t, ids,     td.Set(789, 456, 123))
td.Cmp(t, tags,    td.SuperMapOf(map[string]bool{"enabled": true, "shared": true}, nil))

All operators:

All           Contains      Isa           N             NotZero       SStruct       SuperMapOf
Any           ContainsKey   JSON          NaN           PPtr          String        SuperSetOf
Array         Delay         Keys          Nil           Ptr           Struct        Tag
ArrayEach     Empty         Lax           None          Re            SubBagOf      TruncTime
Bag           Gt            Len           Not           ReAll         SubJSONOf     Values
Between       Gte           Lt            NotAny        Set           SubMapOf      Zero
Cap           HasPrefix     Lte           NotEmpty      Shallow       SubSetOf
Catch         HasSuffix     Map           NotNaN        Slice         SuperBagOf
Code          Ignore        MapEach       NotNil        Smuggle       SuperJSONOf
9

Almost all operators have shortcuts

Always the same pattern:

td.Cmp(t, got, td.HasPrefix(expectedPrefix), …) → td.CmpHasPrefix(t, got, expectedPrefix, …)
td.Cmp(t, got, td.HasSuffix(expectedSuffix), …) → td.CmpHasSuffix(t, got, expectedSuffix, …)
                  ¯¯¯¯¯¯¯¯¯                             ¯¯¯¯¯¯¯¯¯
td.Cmp(t, got, td.NotEmpty(), …) → td.CmpNotEmpty(t, got, …)
                  ¯¯¯¯¯¯¯¯               ¯¯¯¯¯¯¯¯

You just understood CmpNot and CmpLax were in fact shortcuts :)

td.Cmp(t, got, td.Not(notExpected)) → td.CmpNot(t, got, notExpected)
td.Cmp(t, got, td.Lax(expected))    → td.CmpLax(t, got, expected)
                  ¯¯¯                       ¯¯¯

Using a shortcut is not mandatory, it could just be more readable in some cases (or not)

4 operators without shortcut: Catch, Delay, Ignore, Tag, because having a shortcut in these cases is a nonsense

10

Matching nested structs/slices/maps

Take this structure:

type Person struct {
    ID       int64     `json:"id"`
    Name     string    `json:"name"`
    Age      int       `json:"age"`
    Children []*Person `json:"children"`
}

We want to check:

11

Operators in nested structs/slices/maps — 1/4 classic way, like others

Like other frameworks, we can do:

    got := GetPerson("Bob")

    td.Cmp(t, got.ID, td.NotZero())
    td.Cmp(t, got.Name, "Bob")
    td.Cmp(t, got.Age, td.Between(40, 45))
    if td.Cmp(t, got.Children, td.Len(2)) {
        // Alice
        td.Cmp(t, got.Children[0].ID, td.NotZero())
        td.Cmp(t, got.Children[0].Name, "Alice")
        td.Cmp(t, got.Children[0].Age, 20)
        td.Cmp(t, got.Children[0].Children, td.Len(0))
        // Brian
        td.Cmp(t, got.Children[1].ID, td.NotZero())
        td.Cmp(t, got.Children[1].Name, "Brian")
        td.Cmp(t, got.Children[1].Age, 18)
        td.Cmp(t, got.Children[1].Children, td.Len(0))
    }

Exercise: replace the following shortcuts in the code above ↑

CmpNotZero(t, …) — CmpBetween(t, …) — CmpLen(t, …)
12

Operators in nested structs/slices/maps — 2/4 using SStruct operator

SStruct is the strict-Struct operator:

SStruct(model interface{}, expectedFields StructFields)

Strict because omitted fields are checked against their zero value:

    got := GetPerson("Bob")
    td.Cmp(t, got,
        td.SStruct(
            Person{Name: "Bob"},
            td.StructFields{
                "ID":  td.NotZero(),
                "Age": td.Between(40, 45),
                "Children": td.Bag(
                    td.SStruct(&Person{Name: "Alice", Age: 20},
                        td.StructFields{"ID": td.NotZero()}),
                    td.SStruct(&Person{Name: "Brian", Age: 18},
                        td.StructFields{"ID": td.NotZero()}),
                ),
            },
        ))
13

Operators in nested structs/slices/maps — 3/4 using JSON operator

JSON allows to compare the JSON representation (comments are allowed!):

    td.Cmp(t, got, td.JSON(`
    {
        "id":       $^NotZero, // ← simple operator (8 others are eligible)
        "name":     "Bob"
        "age":      $1,        // ← placeholder (could be "$1" or $BobAge, see JSON operator doc)
        "children": [
            {
                "id":       $^NotZero,
                "name":     "Alice",
                "age":      20,
                "children": null,
            },
            {
                "id":       $^NotZero,
                "name":     "Brian",
                "age":      18,
                "children": null,
            }
        ]
    }`,
        td.Between(40, 45),
    ))
14

Operators in nested structs/slices/maps — 4/4 using anchoring

Anchoring feature allows to put operators directly in litterals

To keep track of anchors, a td.T instance is needed:

    tdt := td.NewT(t)
    tdt.Cmp(got,
        Person{
            ID:   tdt.A(td.NotZero(), int64(0)).(int64),
            Name: "Bob",
            Age:  tdt.A(td.Between(40, 45)).(int),
            Children: []*Person{
                {
                    ID:   tdt.A(td.NotZero(), int64(0)).(int64),
                    Name: "Alice",
                    Age:  20,
                },
                {
                    ID:   tdt.A(td.NotZero(), int64(0)).(int64),
                    Name: "Brian",
                    Age:  18,
                },
            },
        })
15

Anatomy of an anchor

Anchors are created using A (or its alias Anchor) method of *td.T

It generates a specific value that can be retrieved during the comparison process

func A(operator TestDeep, model ...interface{}) interface{}
          │                └ if the type can not be guessed from the operator
          └ the operator to use

// model is not needed when operator knows the type behind the operator
tdt.A(td.Between(40, 45)).(int)

// model is mandatory if the type behind cannot be guessed
tdt.A(td.NotZero(), int64(666)).(int64)

// for reflect lovers, they can use the longer version
tdt.A(td.NotZero(), reflect.TypeOf(int64(666))).(int64)

Conflicts are possible, so be careful with 8 and 16 bits types

Work for pointers, slices, maps, but not available for bool types

Specific handling is needed for structs, see AddAnchorableStructType function

16

Encapsulating testing.T

Instead of doing:

func TestVals(t *testing.T) {
    got := GetPerson("Bob")
    td.Cmp(t, got.Age, td.Between(40, 45))
    td.Cmp(t, got.Children, td.Len(2))
}

one can build a td.T instance encapsulating the testing.T one:

func TestVals(t *testing.T) {
    tt := td.NewT(t)

    got := GetPerson("Bob")
    tt.Cmp(got.Age, td.Between(40, 45))
    tt.Cmp(got.Children, td.Len(2))
}

Building a td.T instance provides some advantages over using td.Cmp* functions directly

17

td.T — 1/5

type T struct {
    testing.TB               // implemented by *testing.T
    Config     ContextConfig // defaults to DefaultContextConfig
}

See testing.TB interface, ContextConfig struct and DefaultContextConfig variable

Construction:

func NewT(t testing.TB, config ...ContextConfig) *T                 // inherit properties from t
func Assert(t testing.TB, config ...ContextConfig) *T               // test failures are not fatal
func Require(t testing.TB, config ...ContextConfig) *T              // t.Fatal if a test fails
func AssertRequire(t testing.TB, config ...ContextConfig) (*T, *T)  // Assert() + Require()

Configuring *td.T instance (return a new instance):

func (t *T) BeLax(enable ...bool) *T           // enable/disable strict type comparison
func (t *T) FailureIsFatal(enable ...bool) *T  // enable/disable failure "fatality"
func (t *T) RootName(rootName string) *T       // change data root name, "DATA" by default
func (t *T) UseEqual(enable ...bool) *T        // delegate cmp to UseEqual() method if available
18

td.T — 2/5

Main methods:

func (t *T) Cmp(got, expected interface{}, args ...interface{}) bool
func (t *T) CmpError(got error, args ...interface{}) bool
func (t *T) CmpLax(got interface{}, expected interface{}, args ...interface{}) bool
func (t *T) CmpNoError(got error, args ...interface{}) bool
func (t *T) CmpNotPanic(fn func(), args ...interface{}) bool
func (t *T) CmpPanic(fn func(), expected interface{}, args ...interface{}) bool
func (t *T) False(got interface{}, args ...interface{}) bool
func (t *T) Not(got interface{}, notExpected interface{}, args ...interface{}) bool
func (t *T) Run(name string, f func(t *T)) bool
func (t *T) RunAssertRequire(name string, f func(assert *T, require *T)) bool
func (t *T) True(got interface{}, args ...interface{}) bool

In fact mostly the same as main functions, but:

See documentation for details

19

td.T — 3/5

Shortcuts, as for td functions, follow always the same pattern:

t.Cmp(got, td.HasPrefix(expected), …) → t.HasPrefix(got, expected, …)
t.Cmp(got, td.HasSuffix(expected), …) → t.HasSuffix(got, expected, …)
              ¯¯¯¯¯¯¯¯¯                   ¯¯¯¯¯¯¯¯¯
t.Cmp(got, td.NotEmpty(), …) → t.NotEmpty(t, got, …)
              ¯¯¯¯¯¯¯¯           ¯¯¯¯¯¯¯¯

So yes, T.Not is in fact a shortcut:

t.Cmp(got, td.Not(notExpected)) → t.Not(got, notExpected)
              ¯¯¯                   ¯¯¯

The only exception is T.CmpLax method, shortcut of td.Lax operator, it is more relevant than a T.Lax method

Same 4 operators without shortcut: Catch, Delay, Ignore, Tag, because having a shortcut in these cases is a nonsense

20

td.T — 4/5

Anchoring related methods:

func (t *T) A(operator TestDeep, model ...interface{}) interface{}
func (t *T) Anchor(operator TestDeep, model ...interface{}) interface{}
func (t *T) AnchorsPersistTemporarily() func()
func (t *T) DoAnchorsPersist() bool
func (t *T) ResetAnchors()
func (t *T) SetAnchorsPersist(persist bool)

A and Anchor allow both to anchor an operator, the first is just shorter to write

By default, anchoring is effective only for the next Cmp* call, but this can be overridden thanks to SetAnchorsPersist and AnchorsPersistTemporarily

Useful for helpers writers

21

td.T — 5/5

Simple example:

import (
    "testing"

    "github.com/maxatome/go-testdeep/td"
)

func TestExample(t *testing.T) {
    assert, require := td.AssertRequire(t)

    person, err := GetPerson("Bob")

    require.CmpNoError(err) // exits test if it fails

    assert.RootName("PERSON").
        Cmp(person, &Person{
            ID:       assert.A(td.NotZero(), int64(0)).(int64),
            Name:     "Bob",
            Age:      assert.A(td.Between(40, 45)).(int),
            Children: assert.A(td.Len(2), ([]*Person)(nil)).([]*Person),
        })
}
22

Table driven tests — the heaven of go-testdeep operators

var personTests = []struct {
    name           string
    expectedErr    td.TestDeep
    expectedPerson td.TestDeep
}{
    {"Bob", nil,
        td.SStruct(
            &Person{Name: "Bob", Age: 41},
            td.StructFields{"ID": td.NotZero(), "Children": td.Len(2)})},
    {"Marcel", td.String("User not found"), td.Nil()},
    {"Alice", nil, td.SStruct(&Person{Name: "Alice", Age: 20}, td.StructFields{"ID": td.NotZero()})},
    {"Brian", nil, td.SStruct(&Person{Name: "Brian", Age: 18}, td.StructFields{"ID": td.NotZero()})},
}
                                                        === RUN   TestGetPerson
func TestGetPerson(tt *testing.T) {                     === RUN   TestGetPerson/Bob
    t := td.Assert(tt)                                  === RUN   TestGetPerson/Marcel
    for _, pt := range personTests {                    === RUN   TestGetPerson/Alice
        t.Run(pt.name, func(t *td.T) {                  === RUN   TestGetPerson/Brian
            person, err := GetPerson(pt.name)           --- PASS: TestGetPerson (0.00s)
            t.Cmp(err, pt.expectedErr)                      --- PASS: TestGetPerson/Bob (0.00s)
            t.Cmp(person, pt.expectedPerson)                --- PASS: TestGetPerson/Marcel (0.00s)
        })                                                  --- PASS: TestGetPerson/Alice (0.00s)
    }                                                       --- PASS: TestGetPerson/Brian (0.00s)
}                                                       PASS
23

Operators types

There is two kinds of operators: classic ones and smuggler ones

A smuggler operator is an operator able to transform the value (by changing its value or even its type) before comparing it

Smuggler operators follows:

Cap           Contains      Keys          Len           Ptr           Tag
Catch         ContainsKey   Lax           PPtr          Smuggle       Values

Some examples:

td.Cmp(t, list,    td.Len(td.Between(3, 4)))
td.Cmp(t, headers, td.ContainsKey(td.HasPrefix("X-Ovh")))
td.Cmp(t, &age,    td.Ptr(td.Gt(18))))
td.Cmp(t, ageStr,  td.Smuggle(strconv.Atoi, td.Catch(&age, td.Gt(18))))
td.Cmp(t, headers, td.Keys(td.SuperSetOf("X-Ovh-Account", "X-Remote-IP")))
24

Custom operators — for beginners

Two operators Code and Smuggle allow to achieve what others cannot for your very special use cases

Below, only the year of time.Time is important

With Code, you do the test:

    td.Cmp(t, gotTime,
        td.Code(func(date time.Time) bool {
            return date.Year() == 2018
        }))

With Smuggle, you transform the value and delegate the test:

    td.Cmp(t, gotTime,
        td.Smuggle(func(date time.Time) int { return date.Year() },
            td.Between(2010, 2020)),
    )

Discover more features in each operator description

25

Custom operators — master class 1/2

Sometimes you need to test something over and over, let's do your own operator!

func CheckDateGte(t time.Time, catch *time.Time) td.TestDeep {
    op := td.Gte(t.Truncate(time.Millisecond))
    if catch != nil {
        op = td.Catch(catch, op)
    }
    return td.All(
        td.HasSuffix("Z"),
        td.Smuggle(func(s string) (time.Time, error) {
            t, err := time.Parse(time.RFC3339Nano, s)
            if err == nil && t.IsZero() {
                err = fmt.Errorf("zero time")
            }
            return t, err
        }, op))
}

Ensures that a RFC3339-stringified date has "Z" suffix and is well RFC3339-formatted. Then check it is greater or equal than t truncated to milliseconds

Additionally, if catch is non-nil, stores the resulting time.Time in *catch

26

Custom operators — master class 2/2

This new operator is useful when used with JSON operator

func TestCreateArticle(t *testing.T) {
    type Article struct {
        ID        int64     `json:"id"`
        Code      string    `json:"code"`
        CreatedAt time.Time `json:"created_at"`
    }

    beforeCreation := time.Now()
    var createdAt time.Time
    td.Cmp(t, CreateArticle("Car"),
        td.JSON(`{"id": $^NotZero, "code": "Car", "created_at": $1}`,
            CheckDateGte(beforeCreation, &createdAt)))

    // If the test succeeds, then "created_at" value is well a RFC3339
    // datetime in UTC timezone and its value is directly exploitable as
    // time.Time thanks to createdAt variable
}
27

tdhttp or how to easily test a http.Handler

And now you want to test your API, aka a http.Handler

Thanks to the tdhttp helper and all these *#@!# operators, nothing is easier!

import (
    "testing"

    "github.com/maxatome/go-testdeep/helpers/tdhttp"
    "github.com/maxatome/go-testdeep/td"
)

func TestMyApi(t *testing.T) {
    ta := tdhttp.NewTestAPI(t, myAPI)

    ta.Get("/person/Bob", "Accept", "application/json").
        CmpStatus(htp.StatusOK).
        CmpHeader(td.ContainsKey("X-Custom-Header")).
        CmpJSONBody(td.JSON(`{"id": $1, "name": "Bob", "age":  26}`, td.NotZero()))

    if !ta.Failed() {
        t.Log("Good job pal!")
    }
}
28

tdhttp for any framework

myAPI is here a http.Handler

    ta := tdhttp.NewTestAPI(t, myAPI)

That's pretty cool as:

implement all http.Handler!

You can now change your web framework and keep your test framework :)

29

tdhttp for any content

Ready to use GET, POST, PATCH, PUT, DELETE, HEAD requests, but can be fed by any already created http.Request

Support out of the box application/x-www-form-urlencoded, application/json and application/xml encoding

Support string and []byte bodies so you can handle the encoding by yourself

Operator anchoring works out of the box too

func TestMyApiAnchor(t *testing.T) {
    ta := tdhttp.NewTestAPI(t, myAPI)

    ta.Get("/person/Bob", "Accept", "application/json").
        CmpStatus(htp.StatusOK).
        CmpHeader(td.ContainsKey("X-Custom-Header")).
        CmpJSONBody(Person{
            ID:   ta.A(td.NotZero(), int64(0)).(int64),
            Name: "Bob",
            Age:  ta.A(td.Between(40, 45)).(int),
        })
}
30

End

Some stats:

Links:

31

Thank you

Maxime Soulé

2020-07-20

Use the left and right arrow keys or click the left and right edges of the page to navigate between slides.
(Press 'H' or navigate to hide this message.)