Userfacing API

@check [key=val]... function...

The main way to declare & run a property based test. Called like so:

julia> using Supposition, Supposition.Data

julia> Supposition.@check [options...] function foo(a = Data.Text(Data.Characters(); max_len=10))
          length(a) > 8
       end

Supported options, passed as key=value:

• rng::Random.AbstractRNG: Pass an RNG to use. Defaults to Random.Xoshiro(rand(Random.RandomDevice(), UInt)).
• max_examples::Int: The maximum number of generated examples that are passed to the property.
• broken::Bool: Mark a property that should pass but doesn't as broken, so that failures are not counted.
• record::Bool: Whether the result of the invocation should be recorded with any parent testsets.
• db: Either a Boolean (true uses a fallback database, false stops recording examples) or an ExampleDB.
• config: A CheckConfig object that will be used as a default for all previous options. Options that are passed explicitly to @check will override whatever is provided through config.

The arguments to the given function are expected to be generator strategies. The names they are bound to are the names the generated object will have in the test. These arguments will be shown should the property fail!

Extended help

Reusing existing properties

If you already have a predicate defined, you can also use the calling syntax in @check. Here, the generator is passed purely positionally to the given function; no argument name is necessary.

julia> using Supposition, Supposition.Data

julia> isuint8(x) = x isa UInt8

julia> intgen = Data.Integers{UInt8}()

julia> Supposition.@check isuint8(intgen)

Passing a custom RNG

It is possible to optionally give a custom RNG object that will be used for random data generation. If none is given, Xoshiro(rand(Random.RandomDevice(), UInt)) is used instead.

julia> using Supposition, Supposition.Data, Random

# use a custom Xoshiro instance
julia> Supposition.@check rng=Xoshiro(1234) function foo(a = Data.Text(Data.Characters(); max_len=10))
          length(a) > 8
       end

Additional Syntax

In addition to passing a whole function like above, the following syntax are also supported:

text = Data.Text(Data.AsciiCharacters(); max_len=10)

# If no name is needed, use an anonymous function
@check (a = text) -> a*a
@check (a = text,) -> "foo: "*a
@check (a = text, num = Data.Integers(0,10)) -> a^num

# ..or give the anonymous function a name too - works with all three of the above
@check build_sentence(a = text, num = Data.Floats{Float16}()) -> "The $a is $num!"
build_sentence("foo", 0.5) # returns "The foo is 0.5!"

@composed

A way to compose multiple Possibility into one, by applying a function.

The return type is inferred as a best-effort!

Used like so:

julia> using Supposition, Supposition.Data

julia> text = Data.Text(Data.AsciiCharacters(); max_len=10)

julia> gen = Supposition.@composed function foo(a = text, num=Data.Integers(0, 10))
              lpad(num, 2) * ": " * a
       end

julia> example(gen)
" 8:  giR2YL\rl"

In addition to passing a whole function like above, the following syntax are also supported:

# If no name is needed, use an anonymous function
double_up =  @composed (a = text) -> a*a
prepend_foo = @composed (a = text,) -> "foo: "*a
expo_str = @composed (a = text, num = Data.Integers(0,10)) -> a^num

# ..or give the anonymous function a name too - works with all three of the above
sentence = @composed build_sentence(a = text, num = Data.Floats{Float16}()) -> "The $a is $num!"
build_sentence("foo", 0.5) # returns "The foo is 0.5!"

# or compose a new generator out of an existing function
my_func(str, number) = number * "? " * str
ask_number = @composed my_func(text, num)

target!(score)

Update the currently running testcase to track the given score as its target.

score must be convertible to a Float64.

assume!(precondition::Bool)

If this precondition is not met, abort the test and mark the currently running testcase as invalid.

produce!(p::Possibility{T}) -> T

Produces a value from the given Possibility, recording the required choices in the currently active TestCase.

reject!()

Reject the current testcase as invalid, meaning the generated example should not be considered as producing a valid counterexample.

event!(obj)
event!(label::AbstractString, obj)

Record obj as an event in the current testcase that occured while running your property. If no label is given, a default one will be chosen.

err_less(e1::E, e2::E) where E

A comparison function for exceptions, used when encountering an error in a property. Returns true if e1 is considered to be "easier" or "simpler" than e2. Only definable when both e1 and e2 have the same type.

This is optional to implement, but may be beneficial for shrinking counterexamples leading to an error with rich metadata, in which case err_less will be used to compare errors of the same type from different counterexamples. In particular, this function will likely be helpful for errors with metadata that is far removed from the input that caused the error itself, but would nevertheless be helpful when investigating the failure.

DEFAULT_CONFIG

A ScopedValue holding the CheckConfig that will be used by default & as a fallback.

Currently uses these values:

• rng: Random.Xoshiro(rand(Random.RandomDevice(), UInt))
• max_examples: 10_000
• record: true
• verbose: false
• broken: false
• db: UnsetDB()
• buffer_size: 100_000

@check will use a new instance of Random.Xoshiro by itself.

|(::Possibility{T}, ::Possibility{S}) where {T,S} -> OneOf{Union{T,S}}

Combine two Possibility into one, sampling uniformly from either.

If either of the two arguments is a OneOf, the resulting object acts as if all original non-OneOf had be given to OneOf instead. That is, e.g. OneOf(a, b) | c will act like OneOf(a,b,c).

See also OneOf.

filter(f, pos::Possibility)

Filter the output of produce! on pos by applying the predicate f.

map(f, pos::Possibility)
map(f, pos::Vararg{Possibility})

Apply f to the result of calling produce! on pos (lazy mapping).

Equivalent to calling Map(pos, f), or MultiMap(pos, f) for the Vararg case (where pos is the tuple of Possibility). In the Vararg case, f is expected to take as many arguments as Possibility are passed to map.

See also Map.

bind(f, pos::Possibility)

Maps the output of produce! on pos through f, and calls produce! on the result again. f is expected to take a value and return a Possibility.

Equivalent to calling Bind(pos, f).

See also Bind.

postype(::P) where P <: Possibility

Gives the type of objects this Possibility object will generate.

postype(::Type{P<:Possibility})

Gives the type of objects this Possibility type will generate.

recursive(f, pos::Possibility; max_layers=5)

Recursively expand pos into deeper nested Possibility by repeatedly passing pos itself to f. f returns a new Possibility, which is then passed into f again until the maximum depth is achieved.

Equivalent to calling Recursive(pos, f).

See also Recursive.

AsciiCharacters() <: Possibility{Char}

A Possibility of producing arbitrary Char instances that are isascii. More efficient than filtering Characters.

julia> using Supposition

julia> ascii = Data.AsciiCharacters()

julia> example(ascii, 5)
5-element Vector{Char}:
 '8': ASCII/Unicode U+0038 (category Nd: Number, decimal digit)
 'i': ASCII/Unicode U+0069 (category Ll: Letter, lowercase)
 'R': ASCII/Unicode U+0052 (category Lu: Letter, uppercase)
 '\f': ASCII/Unicode U+000C (category Cc: Other, control)
 '>': ASCII/Unicode U+003E (category Sm: Symbol, math)

Bind(source::Possibility, f)

Binds f to source, i.e., on produce!(::Bind, ::TestCase) this calls produce! on source, the result of which is passed to f, the output of which will be used as input to produce! again.

In other words, f takes a value produce!d by source and gives back a Possibility that is then immediately produce!d from.

Equivalent to bind(f, source).

BitIntegers() <: Possibility{Union{Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8}}

A Possibility for generating all possible bitintegers with fixed size.

Booleans() <: Possibility{Bool}

A Possibility for sampling boolean values.

julia> using Supposition

julia> bools = Data.Booleans()

julia> example(bools, 4)
4-element Vector{Bool}:
 0
 1
 0
 1

Characters(;valid::Bool = false) <: Possibility{Char}

A Possibility of producing arbitrary well-formed Char instances.

Keyword arguments:

• valid: Whether the produced Char must be valid, i.e. not malformed and not have Unicode category Invalid.

julia> using Supposition

julia> chars = Data.Characters()

julia> example(chars, 5)
5-element Vector{Char}:
 '⠺': Unicode U+283A (category So: Symbol, other)
 '𰳍': Unicode U+30CCD (category Lo: Letter, other)
 '\U6ec9c': Unicode U+6EC9C (category Cn: Other, not assigned)
 '\U1a05c5': Unicode U+1A05C5 (category In: Invalid, too high)
 '𓂫': Unicode U+130AB (category Lo: Letter, other)

Dicts(keys::Possibility, values::Possibility; min_size=0, max_size=10_000)

A Possibility for generating Dict objects. The keys are drawn from keys, while the values are drawn from values. min_size/max_size control the number of objects placed into the resulting Dict, respectively.

julia> dicts = Data.Dicts(Data.Integers{UInt8}(), Data.Integers{Int8}(); max_size=3);

julia> example(dicts)
Dict{UInt8, Int8} with 2 entries:
  0x54 => -29
  0x1f => -28

Floats{T <: Union{Float16,Float32,Float64}}(;infs=true, nans=true, minimum=-T(Inf), maximum=+T(Inf)) <: Possibility{T}

A Possibility for sampling floating point values.

The keyword infs controls whether infinities can be generated. nans controls whether any NaN (signaling & quiet) will be generated.

julia> using Supposition

julia> floats = Data.Floats{Float16}()

julia> example(floats, 5)
5-element Vector{Float16}:
  -8.3e-6
   1.459e4
   3.277
 NaN
  -0.0001688

Floats(;nans=true, infs=true) <: Possibility{Union{Float64,Float32,Float16}}

A catch-all for generating instances of all three IEEE floating point types.

Integers(minimum::T, maximum::T) <: Possibility{T <: Integer}
Integers{T}() <: Possibility{T <: Integer}

A Possibility representing drawing integers from [minimum, maximum]. The second constructors draws from the entirety of T.

Produced values are of type T.

julia> using Supposition

julia> is = Data.Integers{Int}()

julia> example(is, 5)
5-element Vector{Int64}:
 -5854403925308846160
  4430062772779972974
    -9995351034504801
  2894734754389242339
 -6640496903289665416

Just(value::T) <: Possibility{T}

A Possibility that always produces value.

julia> using Supposition

julia> three = Data.Just(3)

julia> example(three, 3)
3-element Vector{Int64}:
 3
 3
 3

Map(source::Possibility, f) <: Possibility

A Possibility representing mapping values from source through f.

Equivalent to calling map(f, source).

The pre-calculated return type of Map is a best effort and may be wider than necessary.

julia> using Supposition

julia> makeeven(x) = (x ÷ 2) * 2

julia> pos = map(makeeven, Data.Integers{Int8}())

julia> all(iseven, example(pos, 10_000))
true

OneOf(pos::Possibility...) <: Possibility

A Possibility able to generate any of the examples one of the given Possibility can produce. The given Possibility are sampled from uniformly.

At least one Possibility needs to be given to OneOf.

postype(::OneOf) is inferred as a best effort, and may be wider than necessary.

OneOf can also be constructed through use of a | b on Possibility. Constructed in this way, if either a or b is already a OneOf, the resulting OneOf acts as if it had been given the original Possibility objects in the first place. That is, OneOf(a, b) | c acts like OneOf(a, b, c).

See also WeightedNumbers and WeightedSample.

julia> of = Data.OneOf(Data.Integers{Int8}(), Data.Integers{UInt8}());

julia> Data.postype(of)
Union{Int8, UInt8}

julia> ex = map(of) do i
           (i, typeof(i))
       end;

julia> example(ex)
(-83, Int8)

julia> example(ex)
(0x9f, UInt8)

Pairs(first::Possibility{T}, second::Possibility{S}) where {T,S} <: Possibility{Pair{T,S}}

A Possibility for producing a => b pairs. a is produced by first, b is produced by second.

julia> p = Data.Pairs(Data.Integers{UInt8}(), Data.Floats{Float64}());

julia> example(p, 4)
4-element Vector{Pair{UInt8, Float64}}:
 0x41 => 4.1183566661848205e-230
 0x48 => -2.2653631095108555e-119
 0x2a => -6.564396855333643e224
 0xec => 1.9330751262581671e-53

Recursive(base::Possibility, extend; max_layers::Int=5) <: Possibility{T}

A Possibility for generating recursive data structures. base is the basecase of the recursion. extend is a function returning a new Possibility when given a Possibility, called to recursively expand a tree starting from base. The returned Possibility is fed back into extend again, expanding the recursion by one layer.

max_layers designates the maximum layers Recursive should keep track of. This must be at least 1, so that at least the base case can always be generated. Note that this implies extend will be used at most max_layers-1 times, since the base case of the recursion will not be wrapped.

Equivalent to calling recursive(extend, base).

Examples

julia> base = Data.Integers{UInt8}()

julia> wrap(pos) = Data.Vectors(pos; min_size=2, max_size=3)

julia> rec = Data.recursive(wrap, base; max_layers=3);

julia> Data.postype(rec) # the result is formatted here for legibility
Union{UInt8,
      Vector{UInt8},
      Vector{Union{UInt8, Vector{UInt8}}}
}

julia> example(rec)
0x31

julia> example(rec)
2-element Vector{Union{UInt8, Vector{UInt8}}}:
     UInt8[0xa9, 0xb4]
 0x9b

julia> example(rec)
2-element Vector{UInt8}:
 0xbd
 0x25

SampledFrom(collection) <: Possibility{eltype(collection)}

A Possibility for sampling uniformly from collection.

collection, as well as its eachindex, is assumed to be indexable.

julia> using Supposition

julia> sampler = Data.SampledFrom([1, 1, 1, 2])

julia> example(sampler, 4)
4-element Vector{Int64}:
 1
 1
 2
 1

Satisfying(source::Possibility, pred) <: Possibility

A Possibility representing values from source fulfilling pred.

Equivalent to calling filter(f, source).

julia> using Supposition

julia> pos = filter(iseven, Data.Integers{Int8}())

julia> all(iseven, example(pos, 10_000))
true

Text(alphabet::Possibility{Char}; min_len=0, max_len=10_000) <: Possibility{String}

A Possibility for producing Strings containing Chars of a given alphabet.

julia> using Supposition

julia> text = Data.Text(Data.AsciiCharacters(); max_len=15)

julia> example(text, 5)
5-element Vector{String}:
 "U\x127lxf"
 "hm\x172SJ-("
 "h`\x03\0\x01[[il"
 "\x0ep4"
 "9+Hk3 ii\x1eT"

UnicodeCharacters(;valid::Bool = false, malformed = true) <: Possibility{Char}

A Possibility of producing arbitrary Char instances.

Keyword arguments:

• malformed: Whether produced Char are allowed to be malformed. This only has an effect when valid=false.
• valid: Whether the produced Char must be valid, i.e. not malformed and not have Unicode category Invalid.

julia> using Supposition

julia> chars = Data.UnicodeCharacters()

julia> example(chars, 5)
5-element Vector{Char}:
 '\xd2\x5a\x96\x07': Malformed UTF-8 (category Ma: Malformed, bad data)
 '\x44\x45\xc5\x64': Malformed UTF-8 (category Ma: Malformed, bad data)
 '\x04': Unicode U+0004 (category Cc: Other, control)
 '\x2b\xe0\x6a\x89': Malformed UTF-8 (category Ma: Malformed, bad data)
 '\xf5\x9b\x63\x05': Malformed UTF-8 (category Ma: Malformed, bad data)

Vectors(elements::Possibility{T}; min_size=0, max_size=10_000) <: Possibility{Vector{T}}

A Possibility representing drawing vectors with length l in min_size <= l <= max_size, holding elements of type T.

min_size and max_size must be positive numbers, with min_size <= max_size.

julia> using Supposition

julia> vs = Data.Vectors(Data.Floats{Float16}(); max_size=5)

julia> example(vs, 3)
3-element Vector{Vector{Float16}}:
 [9.64e-5, 9.03e3, 0.04172, -0.0003352]
 [9.793e-5, -2.893, 62.62, 0.0001961]
 [-0.007023, NaN, 3.805, 0.1943]

WeightedNumbers(weights::Vector{Float64}) <: Possibility{Int}

Sample the numbers from 1:length(weights), each with a weight of weights[i].

The weights may be any number > 0.0.

See also OneOf.

julia> using Supposition

julia> bn = Data.WeightedNumbers([1.0, 1.0, 3.0]);

julia> example(Data.Vectors(bn; min_size=3, max_size=15), 5)
5-element Vector{Vector{Int64}}:
 [3, 2, 3, 3, 2, 3, 3]
 [1, 1, 1, 2, 1, 3, 1, 3]
 [2, 3, 3, 3, 3, 3, 3, 1, 1, 3, 3, 3]
 [3, 3, 2, 3, 3]
 [1, 3, 3, 3, 2, 2]

WeightedSample{T}(colllection, weights::Vector{Float64}) <: Possibility{T}

Draw an element from the indexable collection, biasing the drawing process by assigning each index i of col the weight at weights[i].

length of col and weights must be equal and eachindex(col) must be indexable.

See also OneOf.

julia> bs = Data.WeightedSample(["foo", "bar", "baz"], [3.0, 1.0, 1.0]);

julia> example(bs, 10)
10-element Vector{String}:
 "foo"
 "foo"
 "bar"
 "baz"
 "baz"
 "foo"
 "bar"
 "foo"
 "foo"
 "bar"

Possibility{T}

Abstract supertype for all generators. The T type parameter describes the kinds of objects generated by this integrated shrinker.

Required methods:

• produce!(::TestCase, ::P) where P <: Possibility

Fallback definitions:

• postype(::Possibility{T}) -> Type{T}
• example(::Possibility{T}) -> T
• ==(::Possibility, ::Possibility) = false

ExampleDB

An abstract type representing a database of previous counterexamples.

Required methods:

• records(::ExampleDB): Returns an iterable of all currently recorded counterexamples.
• record!(::ExampleDB, key, value): Record the counterexample value under the key key.
• retrieve(::ExampleDB, key)::Option: Retrieve the previously recorded counterexample stored under key. Return nothing if no counterexample was stored under that key.