go-testdeep
Maxime SoulΓ©
2022-01-28
Maxime SoulΓ©
2022-01-28
Why a test framework?
To avoid boilerplate code, especially error reports π
import "testing" func TestGetPerson(t *testing.T) { person, err := GetPerson("Bob") if err != nil { t.Errorf("GetPerson returned error %s", err) } else { if person.Name != "Bob" { t.Errorf(`Name: got=%q expected="Bob"`, person.Name) } if person.Age != 42 { t.Errorf("Age: got=%s expected=42", person.Age) } } }
For example using go-testdeep π
import ( "testing" "github.com/maxatome/go-testdeep/td" ) func TestGetPerson(t *testing.T) { person, err := GetPerson("Bob") if td.CmpNoError(t, err, "GetPerson does not return an error") { td.Cmp(t, person, Person{Name: "Bob", Age: 42}, "GetPerson returns Bob") } }
In most cases, there is not even test names π
func TestGetPerson(t *testing.T) { person, err := GetPerson("Bob") if td.CmpNoError(t, err) { td.Cmp(t, person, Person{Name: "Bob", Age: 42}) } }
Custom comparison engine allowing the use of powerful operators
Accurate and colored error reports
62 operators to match in all circumstances
Fully documented with plenty examples
Consistent API: got
parameter is always before expected
one
Very few basic functions, all others are operators shortcuts (args... allow to name tests):
Cmp(t TestingT, got, expected interface{}, args ...interface{}) bool CmpError(t TestingT, got error, args ...interface{}) bool CmpFalse(t TestingT, got interface{}, args ...interface{}) bool CmpLax(t TestingT, got, expected interface{}, args ...interface{}) bool CmpNoError(t TestingT, got error, args ...interface{}) bool CmpNot(t TestingT, got, notExpected interface{}, args ...interface{}) bool CmpNotPanic(t TestingT, fn func(), args ...interface{}) bool CmpPanic(t TestingT, fn func(), expectedPanic interface{}, args ...interface{}) bool CmpTrue(t TestingT, got interface{}, args ...interface{}) bool
Unique anchoring feature, to easily test literals
JSON content testable like never before
Table driven tests are simple to write and maintain
Comparison engine can be configured to be lax, to ignore struct unexported fields, to treat specifically some types, to display N errors before giving up
Efficient flattening
of []X
slices into ...interface{}
of variadic functions
A function returning several values can be tested in one call, thanks to tuples
tdhttp helper allows to easily test HTTP APIs regardless of the web framework used
tdsuite helper handles consistent and flexible suites of tests
Probably some others reasons you will discover by yourself :)
7As many testing frameworks, tests can optionally be named π
td.Cmp(t, got, "Bob", `Hey! got has to be "Bob" here!`)
Each Cmp*
function cleverly accepts
fmt.Fprintf or
fmt.Fprint parameters in args
func Cmp(t TestingT, got, expected interface{}, args ...interface{}) bool
The doc says:
// "args..." are optional and allow to name the test. This name is // used in case of failure to qualify the test. If len(args) > 1 and // the first item of "args" is a string and contains a '%' rune then // fmt.Fprintf is used to compose the name, else "args" are passed to // fmt.Fprint. Do not forget it is the name of the test, not the // reason of a potential failure.
So, no risk of mistake between Cmp and a (nonexistent) Cmpf: only use Cmp! π
td.Cmp(t, got, 12, "Check got is ", 12) β fmt.Fprint td.Cmp(t, got, 12, "Check got is %d", 12) β fmt.Fprintf td.Cmp(t, got, 12, lastErr) β fmt.Fprint
Derived from reflect.DeepEqual and heavily modified to integrate operators handling
It allows go-testdeep to know exactly where a test fails in a big structure, and even to continue testing in this structure to report several mismatches at the same time (up to 10 by default)
Reports are also very accurate and colorized, instead of awful diffs you see elsewhereβ¦
Some examples, see the expected (3rd) parameter π
here β΄ td.Cmp(t, age, td.Between(40, 45)) td.Cmp(t, headers, td.ContainsKey("X-Ovh")) td.Cmp(t, err, td.Contains("Internal server error")) td.Cmp(t, grants, td.Len(td.Gt(2))) td.Cmp(t, price, td.N(float64(12.03), float64(0.01))) td.Cmp(t, name, td.Re(`^[A-Z][A-Za-z-]+\z`)) td.Cmp(t, ids, td.Set(789, 456, 123)) td.Cmp(t, tags, td.SuperMapOf(map[string]bool{"enabled": true, "shared": true}, nil))
All operators follow
All Contains Isa MapEach NotNil Smuggle SuperJSONOf Any ContainsKey JSON N NotZero SStruct SuperMapOf Array Delay JSONPointer NaN PPtr String SuperSetOf ArrayEach Empty Keys Nil Ptr Struct SuperSliceOf Bag Gt Lax None Re SubBagOf Tag Between Gte Len Not ReAll SubJSONOf TruncTime Cap HasPrefix Lt NotAny Set SubMapOf Values Catch HasSuffix Lte NotEmpty Shallow SubSetOf Zero Code Ignore Map NotNaN Slice SuperBagOf
Always the same pattern
td.Cmp(t, got, td.HasPrefix(expectedPrefix), β¦) β td.CmpHasPrefix(t, got, expectedPrefix, β¦) td.Cmp(t, got, td.HasSuffix(expectedSuffix), β¦) β td.CmpHasSuffix(t, got, expectedSuffix, β¦) Β―Β―Β―Β―Β―Β―Β―Β―Β― Β―Β―Β―Β―Β―Β―Β―Β―Β― td.Cmp(t, got, td.NotEmpty(), β¦) β td.CmpNotEmpty(t, got, β¦) Β―Β―Β―Β―Β―Β―Β―Β― Β―Β―Β―Β―Β―Β―Β―Β―
You just understood CmpNot and CmpLax were in fact shortcuts :)
td.Cmp(t, got, td.Not(notExpected)) β td.CmpNot(t, got, notExpected) td.Cmp(t, got, td.Lax(expected)) β td.CmpLax(t, got, expected) Β―Β―Β― Β―Β―Β―
Using a shortcut is not mandatory, it could just be more readable in some cases (or not)
4 operators without shortcut: Catch, Delay, Ignore, Tag, because having a shortcut in these cases is a nonsense
11Take this structure returned by a GetPerson
function:
type Person struct { ID int64 `json:"id"` Name string `json:"name"` Age int `json:"age"` Children []*Person `json:"children"` }
We want to check:
Like other frameworks, we can do π
got := GetPerson("Bob") td.Cmp(t, got.ID, td.NotZero()) td.Cmp(t, got.Name, "Bob") td.Cmp(t, got.Age, td.Between(40, 45)) if td.Cmp(t, got.Children, td.Len(2)) { // Alice td.Cmp(t, got.Children[0].ID, td.NotZero()) td.Cmp(t, got.Children[0].Name, "Alice") td.Cmp(t, got.Children[0].Age, 20) td.Cmp(t, got.Children[0].Children, td.Len(0)) // Brian td.Cmp(t, got.Children[1].ID, td.NotZero()) td.Cmp(t, got.Children[1].Name, "Brian") td.Cmp(t, got.Children[1].Age, 18) td.Cmp(t, got.Children[1].Children, td.Len(0)) }
Exercise: replace the following shortcuts in the code above β π
CmpNotZero(t, β¦) β CmpBetween(t, β¦) β CmpLen(t, β¦)
SStruct is the strict-Struct operator
SStruct(model interface{}, expectedFields StructFields)
Strict because omitted fields are checked against their zero value, instead of ignoring them π
td.Cmp(t, GetPerson("Bob"), td.SStruct(Person{Name: "Bob"}, td.StructFields{ "ID": td.NotZero(), "Age": td.Between(40, 45), "Children": td.Bag( td.SStruct(&Person{Name: "Alice", Age: 20}, td.StructFields{"ID": td.NotZero()}), td.SStruct(&Person{Name: "Brian", Age: 18}, td.StructFields{"ID": td.NotZero()}), ), }, ))
JSON allows to compare the JSON representation (comments are allowed!) π
td.Cmp(t, GetPerson("Bob"), td.JSON(` { "id": $1, // β placeholder (could be "$1" or $BobAge, see JSON operator doc) "name": "Bob", "age": Between(40, 45), // yes, most operators are embedable "children": [ { "id": NotZero(), "name": "Alice", "age": 20, "children": Empty(), /* null is "empty" */ }, { "id": NotZero(), "name": "Brian", "age": 18, "children": Nil(), } ] }`, td.Catch(&bobID, td.NotZero()), // $1 catches the ID of Bob on the fly and sets bobID var ))
JSON allows to compare the JSON representation (comments are allowed!) π
td.Cmp(t, GetPerson("Bob"), td.JSON(` { "id": $1, // β placeholder (could be "$1" or $BobAge, see JSON operator doc) "name": "Bob", "age": Between(40, 45), // yes, most operators are embedable "children": Bag( // β Bag HERE { "id": NotZero(), "name": "Brian", "age": 18, "children": Nil(), }, { "id": NotZero(), "name": "Alice", "age": 20, "children": Empty(), /* null is "empty" */ }, ) }`, td.Catch(&bobID, td.NotZero()), // $1 catches the ID of Bob on the fly and sets bobID var ))
Anchoring feature allows to put operators directly in literals
To keep track of anchors, a td.T instance is needed π
assert := td.Assert(t) assert.Cmp(GetPerson("Bob"), Person{ ID: assert.A(td.Catch(&bobID, td.NotZero())).(int64), Name: "Bob", Age: assert.A(td.Between(40, 45)).(int), Children: []*Person{ { ID: assert.A(td.NotZero(), int64(0)).(int64), Name: "Alice", Age: 20, }, { ID: assert.A(td.NotZero(), int64(0)).(int64), Name: "Brian", Age: 18, }, }, })
Anchors are created using A (or its alias Anchor) method of *td.T
It generates a specific value that can be retrieved during the comparison process
func (t *T) A(operator TestDeep, model ...interface{}) interface{} β β mandatory if the type can not be guessed from the operator β the operator to use // model is not needed when operator knows the type behind the operator assert.A(td.Between(40, 45)).(int) // model is mandatory if the type behind the operator cannot be guessed assert.A(td.NotZero(), int64(666)).(int64) // for reflect lovers, they can use the longer version assert.A(td.NotZero(), reflect.TypeOf(int64(666))).(int64)
Conflicts are possible, so be careful with 8 and 16 bits types
Work for pointers, slices, maps, but not available for bool
types
Specific handling is needed for structs, see AddAnchorableStructType function
18Instead of doing π
func TestVals(t *testing.T) { got := GetPerson("Bob") td.Cmp(t, got.Age, td.Between(40, 45)) td.Cmp(t, got.Children, td.Len(2)) }
one can build a td.T instance encapsulating the testing.T one π
func TestVals(t *testing.T) { assert := td.Assert(t) got := GetPerson("Bob") assert.Cmp(got.Age, td.Between(40, 45)) assert.Cmp(got.Children, td.Len(2)) }
Building a td.T instance provides some advantages over using td.Cmp* functions directly
19type T struct { testing.TB // implemented by *testing.T Config ContextConfig // defaults to td.DefaultContextConfig }
See testing.TB interface, ContextConfig struct and DefaultContextConfig variable
Construction:
func NewT(t testing.TB, config ...ContextConfig) *T // inherit properties from t func Assert(t testing.TB, config ...ContextConfig) *T // test failures are not fatal func Require(t testing.TB, config ...ContextConfig) *T // t.Fatal if a test fails func AssertRequire(t testing.TB, config ...ContextConfig) (*T, *T) // Assert() + Require()
Configuring *td.T instance (return a new instance):
func (t *T) BeLax(enable ...bool) *T // enable/disable strict type comparison func (t *T) FailureIsFatal(enable ...bool) *T // enable/disable failure "fatality" func (t *T) IgnoreUnexported(types ...interface{}) *T // ignore unexported fields of some structs func (t *T) RootName(rootName string) *T // change data root name, "DATA" by default func (t *T) UseEqual(types ...interface{}) *T // delegate cmp to Equal() method if available
func (t *T) Cmp(got, expected interface{}, args ...interface{}) bool func (t *T) CmpError(got error, args ...interface{}) bool func (t *T) CmpLax(got, expected interface{}, args ...interface{}) bool func (t *T) CmpNoError(got error, args ...interface{}) bool func (t *T) CmpNotPanic(fn func(), args ...interface{}) bool func (t *T) CmpPanic(fn func(), expected interface{}, args ...interface{}) bool func (t *T) False(got interface{}, args ...interface{}) bool func (t *T) Not(got, notExpected interface{}, args ...interface{}) bool func (t *T) True(got interface{}, args ...interface{}) bool func (t *T) Run(name string, f func(t *T)) bool func (t *T) RunAssertRequire(name string, f func(assert, require *T)) bool
In fact mostly the same as main functions, but:
See documentation for details
21Shortcuts, as for td functions, follow always the same pattern:
t.Cmp(got, td.HasPrefix(expected), β¦) β t.HasPrefix(got, expected, β¦) t.Cmp(got, td.HasSuffix(expected), β¦) β t.HasSuffix(got, expected, β¦) Β―Β―Β―Β―Β―Β―Β―Β―Β― Β―Β―Β―Β―Β―Β―Β―Β―Β― t.Cmp(got, td.NotEmpty(), β¦) β t.NotEmpty(t, got, β¦) Β―Β―Β―Β―Β―Β―Β―Β― Β―Β―Β―Β―Β―Β―Β―Β―
So yes, T.Not is in fact a shortcut:
t.Cmp(got, td.Not(notExpected)) β t.Not(got, notExpected) Β―Β―Β― Β―Β―Β―
The only exception is T.CmpLax method, shortcut of td.Lax operator, it is more relevant than a T.Lax method
Same 4 operators without shortcut: Catch, Delay, Ignore, Tag, because having a shortcut in these cases is a nonsense
22Anchoring related methods:
func (t *T) A(operator TestDeep, model ...interface{}) interface{} func (t *T) Anchor(operator TestDeep, model ...interface{}) interface{} func (t *T) AnchorsPersistTemporarily() func() func (t *T) DoAnchorsPersist() bool func (t *T) ResetAnchors() func (t *T) SetAnchorsPersist(persist bool)
A and Anchor allow both to anchor an operator, the first is just shorter to write
By default, anchoring is effective only for the next Cmp* call, but this can be overridden thanks to SetAnchorsPersist and AnchorsPersistTemporarily
Useful for helpers writers or table driven tests afficionados :)
23import ( "testing" "github.com/maxatome/go-testdeep/td" ) func TestExample(t *testing.T) { assert, require := td.AssertRequire(t) person, err := GetPerson("Bob") require.CmpNoError(err) // exits test if it fails assert.RootName("PERSON"). Cmp(person, &Person{ ID: assert.A(td.NotZero(), int64(0)).(int64), Name: "Bob", Age: assert.A(td.Between(40, 45)).(int), Children: assert.A(td.Len(2), ([]*Person)(nil)).([]*Person), }) }
WithCmpHooks allows to register comparison functions for some types π
assert = assert.WithCmpHooks( func (got, expected reflect.Value) bool { return td.EqDeeply(got.Interface(), expected.Interface()) }, (time.Time).Equal, // bypasses the UseEqual flag ) x := 123 assert.Cmp(reflect.ValueOf(x), reflect.ValueOf(123)) // succeeds
WithSmuggleHooks allows to register functions to alter the data before comparing it π
assert = assert.WithSmuggleHooks( func (got int) bool { return got != 0 }, // each int is changed to a bool strconv.Atoi, // each string is converted to an int ) assert.Cmp("123", 123) // succeeds
Smuggle hooks are run just before Cmp hooks and are not run again for their returned value
25var personTests = []struct { name string expectedErr td.TestDeep expectedPerson td.TestDeep }{ {"Bob", nil, td.JSON(`{"name":"Bob","age":41,"id":NotZero(),"children":Len(2)}`)}, {"Marcel", td.String("User not found"), td.Nil()}, {"Alice", nil, td.SStruct(&Person{Name: "Alice", Age: 20}, td.StructFields{"ID": td.NotZero()})}, {"Brian", nil, td.SStruct(&Person{Name: "Brian", Age: 18}, td.StructFields{"ID": td.NotZero()})}, } === RUN TestGetPerson func TestGetPerson(t *testing.T) { === RUN TestGetPerson/Bob assert := td.Assert(t) === RUN TestGetPerson/Marcel for _, pt := range personTests { === RUN TestGetPerson/Alice assert.Run(pt.name, func(assert *td.T) { === RUN TestGetPerson/Brian person, err := GetPerson(pt.name) --- PASS: TestGetPerson (0.00s) assert.Cmp(err, pt.expectedErr) --- PASS: TestGetPerson/Bob (0.00s) assert.Cmp(person, pt.expectedPerson) --- PASS: TestGetPerson/Marcel (0.00s) }) --- PASS: TestGetPerson/Alice (0.00s) } --- PASS: TestGetPerson/Brian (0.00s) } PASS
There is two kinds of operators: classic ones and smuggler ones
A smuggler operator is an operator able to transform the value (by changing its value or even its type) before comparing it
Smuggler operators follows:
Cap Contains JSONPointer Lax PPtr Smuggle Values Catch ContainsKey Keys Len Ptr Tag
Some examples π
td.Cmp(t, slice, td.Len(td.Between(3, 4))) td.Cmp(t, headers, td.ContainsKey(td.HasPrefix("X-Ovh"))) td.Cmp(t, &age, td.Ptr(td.Gt(18))) td.Cmp(t, ageStr, td.Smuggle(strconv.Atoi, td.Catch(&age, td.Gt(18)))) td.Cmp(t, body1, td.Smuggle(json.RawMessage{}, td.JSON(`{"name": $br}`, td.Tag("br", "Brian")))) td.Cmp(t, body2, td.Smuggle("Service.Owner.Children[0].Name", "Alice")) td.Cmp(t, body2, td.JSONPointer("/service/owner/children/0/name", "Alice")) td.Cmp(t, headers, td.Keys(td.SuperSetOf("X-Ovh", "X-Remote-IP"))) td.Cmp(t, err, td.Contains("integrity constraint")) td.Cmp(t, bytes, td.Lax("pipo bingo!"))
Two operators Code and Smuggle allow to achieve what others cannot for your very special use cases
Below, only the year of time.Time is important
With Code, you do the test π
td.Cmp(t, gotTime, td.Code(func(date time.Time) bool { return date.Year() == 2018 }))
With Smuggle, you transform the value and delegate the test π
td.Cmp(t, gotTime, td.Smuggle(func(date time.Time) int { return date.Year() }, td.Between(2010, 2020)), )
Discover more features in each operator description
28Sometimes you need to test something over and over, let's do your own operator!
func CheckDateGte(t time.Time, catch *time.Time) td.TestDeep { op := td.Gte(t.Truncate(time.Millisecond)) if catch != nil { op = td.Catch(catch, op) } return td.All( td.HasSuffix("Z"), td.Smuggle(func(s string) (time.Time, error) { t, err := time.Parse(time.RFC3339Nano, s) if err == nil && t.IsZero() { err = errors.New("zero time") } return t, err }, op)) }
Ensures that a RFC3339-stringified date has "Z" suffix and is well
RFC3339-formatted. Then check it is greater or equal than t
truncated to milliseconds
Additionally, if catch
is non-nil, stores the resulting
time.Time
in *catch
This new operator is useful when used with JSON operator π
func TestCreateArticle(t *testing.T) { type Article struct { ID int64 `json:"id"` Code string `json:"code"` CreatedAt time.Time `json:"created_at"` } var createdAt time.Time beforeCreation := time.Now() td.Cmp(t, CreateArticle("Car"), td.JSON(`{"id": NotZero(), "code": "Car", "created_at": $1}`, CheckDateGte(beforeCreation, &createdAt))) // If the test succeeds, then "created_at" value is well a RFC3339 // datetime in UTC timezone and its value is directly exploitable as // time.Time thanks to createdAt variable t.Logf("Article created at %s", createdAt) }
And now you want to test your API, aka a http.Handler
Thanks to the tdhttp helper and all these *#@!# operators, nothing is easier! π
import ( "testing" "github.com/maxatome/go-testdeep/helpers/tdhttp" "github.com/maxatome/go-testdeep/td" ) func TestMyApi(t *testing.T) { ta := tdhttp.NewTestAPI(t, myAPI) var id int64 ta.Name("Retrieve a person"). Get("/person/Bob", "Accept", "application/json"). CmpStatus(http.StatusOK). CmpHeader(td.ContainsKey("X-Custom-Header")). CmpJSONBody(td.JSON(`{"id": $1, "name": "Bob", "age": 26}`, td.Catch(&id, td.NotZero()))) t.Logf("Did the test succeeded? %t, ID of Bob is %d", !ta.Failed(), id) }
myAPI is here a http.Handler β΄
ta := tdhttp.NewTestAPI(t, myAPI)
That's pretty cool as:
implement all http.Handler! Check the HTTP frameworks section of the FAQ
You can now change your web framework and keep your test framework :)
32Ready to use GET, POST, PATCH, PUT, DELETE, HEAD requests, but can be fed by any already created http.Request
Supports out of the box application/x-www-form-urlencoded
,
application/json
, application/xml
and multipart/form-data
encoding &
cookies
Supports string
and []byte
bodies so you can handle the encoding by yourself
Operator anchoring works out of the box too π
func TestMyApiAnchor(t *testing.T) { ta := tdhttp.NewTestAPI(t, myAPI) var id int64 ta.Get("/person/Bob", "Accept", "application/json"). CmpStatus(http.StatusOK). CmpJSONBody(Person{ ID: ta.A(td.Catch(&id, td.NotZero())).(int64), Name: "Bob", Age: ta.A(td.Between(25, 30)).(int), }) }
Thanks to AutoDumpResponse, Or & OrDumpResponse methods, you can inpect the HTTP response to see what happened in case of failure π
func TestMyApiDumpIfFailure(t *testing.T) { ta := tdhttp.NewTestAPI(t, myAPI) var id int64 ta.Name("Person creation"). PostJSON("/person", PersonNew{Name: "Bob"}). CmpStatus(http.StatusCreated). CmpJSONBody(td.JSON(` { "id": $1, "name": "Bob", "created_at": $2 }`, td.Catch(&id, td.NotZero()), // catch just created ID td.Gte(ta.SentAt()), // check that created_at is β₯ request sent date )). OrDumpResponse() // if some test fails, the response is dumped }
Or tests suites with one hand tied behind your back π
import ( "testing" "github.com/maxatome/go-testdeep/helpers/tdsuite" "github.com/maxatome/go-testdeep/td" ) func TestPerson(t *testing.T) { tdsuite.Run(t, &PersonSuite{ // entrypoint of the suite db: InitDB(), // a DB handler probably used in each tests }) } type PersonSuite struct{ db MyDBHandler } func (ps *PersonSuite) TestGet(assert *td.T) { // β¦ } func (ps *PersonSuite) TestPost(assert, require *td.T) { // β¦ }
Each method Test* of the suite is run in a sub-test
assert
or assert+require
, you choose
Several hooks can be implemented:
Setup(t *td.T) error Destroy(t *td.T) error PreTest(t *td.T, testName string) error PostTest(t *td.T, testName string) error BetweenTests(t *td.T, previousTestName, nextTestName string) error
Setup and Destroy are respectively called before any test is run and after all tests ran
PreTest and PostTest are respectively called before and after each test
BetweenTests is called between 2 consecutive tests (after the PostTest call of previous test but before the PreTest call of next test)
If a hook returns a non-nil error, the suite fails immediately
36func TestAnother(t *testing.T) { // π https://goplay.tools/snippet/5cbM9eHbx33 tdsuite.Run(t, &AnotherSuite{}) // entrypoint of the suite } // BaseSuite is the base test suite used by all tests suite using the DB. type BaseSuite struct{ db MyDBHandler } func (bs *BaseSuite) Setup(t *td.T) (err error) { bs.db, err = InitDB() return } func (bs *BaseSuite) Destroy(t *td.T) error { return bs.db.Exec(`TRUNCATE x, y, z CASCADE`) } // AnotherSuite is the final test suite blah blah blahβ¦ type AnotherSuite struct{ BaseSuite } func (as *AnotherSuite) TestGet(assert, require *td.T) { res, err := as.db.Query(`SELECT 42`) require.CmpNoError(err) assert.Cmp(res, 42) }
Some stats:
Links: