Documentation Reference

This section contains a complete reference of everything Supposition.jl contains, on one page. This is not a devdocs section, but a reference, for quick lookups of what something does, without having to hunt for the exact definition in the source code. A proper devdocs section with a high level introduction will be added at a later date.

Stability

The entries written on this page are automatically generated and DO NOT represent the currently supported API surface. Feel free to use anything you can find here, but be aware that just because it's listed here, does not mean it's covered under semver (though it may be - check Userfacing API if you're unsure).

Data reference

The Data module contains most everyday objects you're going to use when writing property based tests with Supposition.jl. For example, the basic generators for integers, strings, floating point values etc. are defined here. Everything listed in this section is considered supported under semver.

Functions

Base.:|Method
|(::Possibility{T}, ::Possibility{S}) where {T,S} -> OneOf{Union{T,S}}

Combine two Possibility into one, sampling uniformly from either.

If either of the two arguments is a OneOf, the resulting object acts as if all original non-OneOf had be given to OneOf instead. That is, e.g. OneOf(a, b) | c will act like OneOf(a,b,c).

See also OneOf.

source
Base.filterMethod
filter(f, pos::Possibility)

Filter the output of produce! on pos by applying the predicate f.

No stalling

In order not to stall generation of values, this will not try to produce a value from pos forever, but reject the testcase after some attempts.

source
Base.mapMethod
map(f, pos::Possibility)
map(f, pos::Vararg{Possibility})

Apply f to the result of calling produce! on pos (lazy mapping).

Equivalent to calling Map(pos, f), or MultiMap(pos, f) for the Vararg case (where pos is the tuple of Possibility). In the Vararg case, f is expected to take as many arguments as Possibility are passed to map.

See also Map.

source
Supposition.Data.bindMethod
bind(f, pos::Possibility)

Maps the output of produce! on pos through f, and calls produce! on the result again. f is expected to take a value and return a Possibility.

Equivalent to calling Bind(pos, f).

See also Bind.

source
Supposition.Data.produce!Function
produce!(tc::TestCase, pos::Possibility{T}) -> T

Produces a value from the given Possibility, recording the required choices in the TestCase tc.

This needs to be implemented for custom Possibility objects, passing the given tc to any inner requirements directly.

See also Supposition.produce!

Examples

You should not call this function when you have a Possibility and want to inspect what an object produced by that Possibility looks like - use example for that instead.

source
Supposition.Data.recursiveMethod
recursive(f, pos::Possibility; max_layers=5)

Recursively expand pos into deeper nested Possibility by repeatedly passing pos itself to f. f returns a new Possibility, which is then passed into f again until the maximum depth is achieved.

Equivalent to calling Recursive(pos, f).

See also Recursive.

source
Supposition.Data.AsciiCharactersType
AsciiCharacters() <: Possibility{Char}

A Possibility of producing arbitrary Char instances that are isascii. More efficient than filtering Characters.

julia> using Supposition

julia> ascii = Data.AsciiCharacters()

julia> example(ascii, 5)
5-element Vector{Char}:
 '8': ASCII/Unicode U+0038 (category Nd: Number, decimal digit)
 'i': ASCII/Unicode U+0069 (category Ll: Letter, lowercase)
 'R': ASCII/Unicode U+0052 (category Lu: Letter, uppercase)
 '\f': ASCII/Unicode U+000C (category Cc: Other, control)
 '>': ASCII/Unicode U+003E (category Sm: Symbol, math)
source
Supposition.Data.BindType
Bind(source::Possibility, f)

Binds f to source, i.e., on produce!(::Bind, ::TestCase) this calls produce! on source, the result of which is passed to f, the output of which will be used as input to produce! again.

In other words, f takes a value produce!d by source and gives back a Possibility that is then immediately produce!d from.

Equivalent to bind(f, source).

source
Supposition.Data.BitIntegersType
BitIntegers() <: Possibility{Union{Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8}}

A Possibility for generating all possible bitintegers with fixed size.

source
Supposition.Data.BooleansType
Booleans() <: Possibility{Bool}

A Possibility for sampling boolean values.

julia> using Supposition

julia> bools = Data.Booleans()

julia> example(bools, 4)
4-element Vector{Bool}:
 0
 1
 0
 1
source
Supposition.Data.CharactersType
Characters(;valid::Bool = false) <: Possibility{Char}

A Possibility of producing arbitrary well-formed Char instances.

Unicode

This will produce! ANY well-formed Char by default, not just valid unicode codepoints! Notably, this includes overlong Char. To only produce valid unicode codepoints, pass valid=true as a keyword argument.

Keyword arguments:

  • valid: Whether the produced Char must be valid, i.e. not malformed and not have Unicode category Invalid.
julia> using Supposition

julia> chars = Data.Characters()

julia> example(chars, 5)
5-element Vector{Char}:
 '⠺': Unicode U+283A (category So: Symbol, other)
 '𰳍': Unicode U+30CCD (category Lo: Letter, other)
 '\U6ec9c': Unicode U+6EC9C (category Cn: Other, not assigned)
 '\U1a05c5': Unicode U+1A05C5 (category In: Invalid, too high)
 '𓂫': Unicode U+130AB (category Lo: Letter, other)
source
Supposition.Data.DictsType
Dicts(keys::Possibility, values::Possibility; min_size=0, max_size=10_000)

A Possibility for generating Dict objects. The keys are drawn from keys, while the values are drawn from values. min_size/max_size control the number of objects placed into the resulting Dict, respectively.

julia> dicts = Data.Dicts(Data.Integers{UInt8}(), Data.Integers{Int8}(); max_size=3);

julia> example(dicts)
Dict{UInt8, Int8} with 2 entries:
  0x54 => -29
  0x1f => -28
source
Supposition.Data.FloatsMethod
Floats(;nans=true, infs=true) <: Possibility{Union{Float64,Float32,Float16}}

A catch-all for generating instances of all three IEEE floating point types.

source
Supposition.Data.FloatsType
Floats{T <: Union{Float16,Float32,Float64}}(;infs=true, nans=true, minimum=-T(Inf), maximum=+T(Inf)) <: Possibility{T}

A Possibility for sampling floating point values.

The keyword infs controls whether infinities can be generated. nans controls whether any NaN (signaling & quiet) will be generated.

Inf, Nan

This possibility will generate any valid instance, including positive and negative infinities, signaling and quiet NaNs and every possible float.

julia> using Supposition

julia> floats = Data.Floats{Float16}()

julia> example(floats, 5)
5-element Vector{Float16}:
  -8.3e-6
   1.459e4
   3.277
 NaN
  -0.0001688
source
Supposition.Data.IntegersType
Integers(minimum::T, maximum::T) <: Possibility{T <: Integer}
Integers{T}() <: Possibility{T <: Integer}

A Possibility representing drawing integers from [minimum, maximum]. The second constructors draws from the entirety of T.

Produced values are of type T.

julia> using Supposition

julia> is = Data.Integers{Int}()

julia> example(is, 5)
5-element Vector{Int64}:
 -5854403925308846160
  4430062772779972974
    -9995351034504801
  2894734754389242339
 -6640496903289665416
source
Supposition.Data.JustType
Just(value::T) <: Possibility{T}

A Possibility that always produces value.

Mutable Data

The source object given to this Just is not copied when produce! is called. Be careful with mutable data!

julia> using Supposition

julia> three = Data.Just(3)

julia> example(three, 3)
3-element Vector{Int64}:
 3
 3
 3
source
Supposition.Data.MapType
Map(source::Possibility, f) <: Possibility

A Possibility representing mapping values from source through f.

Equivalent to calling map(f, source).

The pre-calculated return type of Map is a best effort and may be wider than necessary.

julia> using Supposition

julia> makeeven(x) = (x ÷ 2) * 2

julia> pos = map(makeeven, Data.Integers{Int8}())

julia> all(iseven, example(pos, 10_000))
true
source
Supposition.Data.OneOfType
OneOf(pos::Possibility...) <: Possibility

A Possibility able to generate any of the examples one of the given Possibility can produce. The given Possibility are sampled from uniformly.

At least one Possibility needs to be given to OneOf.

postype(::OneOf) is inferred as a best effort, and may be wider than necessary.

OneOf can also be constructed through use of a | b on Possibility. Constructed in this way, if either a or b is already a OneOf, the resulting OneOf acts as if it had been given the original Possibility objects in the first place. That is, OneOf(a, b) | c acts like OneOf(a, b, c).

See also WeightedNumbers and WeightedSample.

julia> of = Data.OneOf(Data.Integers{Int8}(), Data.Integers{UInt8}());

julia> Data.postype(of)
Union{Int8, UInt8}

julia> ex = map(of) do i
           (i, typeof(i))
       end;

julia> example(ex)
(-83, Int8)

julia> example(ex)
(0x9f, UInt8)
source
Supposition.Data.PairsType
Pairs(first::Possibility{T}, second::Possibility{S}) where {T,S} <: Possibility{Pair{T,S}}

A Possibility for producing a => b pairs. a is produced by first, b is produced by second.

julia> p = Data.Pairs(Data.Integers{UInt8}(), Data.Floats{Float64}());

julia> example(p, 4)
4-element Vector{Pair{UInt8, Float64}}:
 0x41 => 4.1183566661848205e-230
 0x48 => -2.2653631095108555e-119
 0x2a => -6.564396855333643e224
 0xec => 1.9330751262581671e-53
source
Supposition.Data.RecursiveType
Recursive(base::Possibility, extend; max_layers::Int=5) <: Possibility{T}

A Possibility for generating recursive data structures. base is the basecase of the recursion. extend is a function returning a new Possibility when given a Possibility, called to recursively expand a tree starting from base. The returned Possibility is fed back into extend again, expanding the recursion by one layer.

max_layers designates the maximum layers Recursive should keep track of. This must be at least 1, so that at least the base case can always be generated. Note that this implies extend will be used at most max_layers-1 times, since the base case of the recursion will not be wrapped.

Equivalent to calling recursive(extend, base).

Examples

julia> base = Data.Integers{UInt8}()

julia> wrap(pos) = Data.Vectors(pos; min_size=2, max_size=3)

julia> rec = Data.recursive(wrap, base; max_layers=3);

julia> Data.postype(rec) # the result is formatted here for legibility
Union{UInt8,
      Vector{UInt8},
      Vector{Union{UInt8, Vector{UInt8}}}
}

julia> example(rec)
0x31

julia> example(rec)
2-element Vector{Union{UInt8, Vector{UInt8}}}:
     UInt8[0xa9, 0xb4]
 0x9b

julia> example(rec)
2-element Vector{UInt8}:
 0xbd
 0x25
source
Supposition.Data.SampledFromType
SampledFrom(collection) <: Possibility{eltype(collection)}

A Possibility for sampling uniformly from collection.

collection, as well as its eachindex, is assumed to be indexable.

Mutable Data

The source objects from the collection given to this SampledFrom is not copied when produce! is called. Be careful with mutable data!

Sampling from `String`

To sample from a String, you can collect the string first to get a Vector{Char}. This is necessary because Strings use the variable-length UTF-8 encoding, which isn't arbitrarily indexable in constant time.

julia> using Supposition

julia> sampler = Data.SampledFrom([1, 1, 1, 2])

julia> example(sampler, 4)
4-element Vector{Int64}:
 1
 1
 2
 1
source
Supposition.Data.SatisfyingType
Satisfying(source::Possibility, pred) <: Possibility

A Possibility representing values from source fulfilling pred.

Equivalent to calling filter(f, source).

julia> using Supposition

julia> pos = filter(iseven, Data.Integers{Int8}())

julia> all(iseven, example(pos, 10_000))
true
source
Supposition.Data.TextType
Text(alphabet::Possibility{Char}; min_len=0, max_len=10_000) <: Possibility{String}

A Possibility for producing Strings containing Chars of a given alphabet.

julia> using Supposition

julia> text = Data.Text(Data.AsciiCharacters(); max_len=15)

julia> example(text, 5)
5-element Vector{String}:
 "U\x127lxf"
 "hm\x172SJ-("
 "h`\x03\0\x01[[il"
 "\x0ep4"
 "9+Hk3 ii\x1eT"
source
Supposition.Data.UnicodeCharactersType
UnicodeCharacters(;valid::Bool = false, malformed = true) <: Possibility{Char}

A Possibility of producing arbitrary Char instances.

Unicode

This will produce! ANY Char by default, not just valid or well-formed unicode codepoints! To only produce valid unicode codepoints, pass valid=true as a keyword argument. To produce well-formed unicode codepoints, pass malformed=false as a keyword argument.

Keyword arguments:

  • malformed: Whether produced Char are allowed to be malformed. This only has an effect when valid=false.
  • valid: Whether the produced Char must be valid, i.e. not malformed and not have Unicode category Invalid.
julia> using Supposition

julia> chars = Data.UnicodeCharacters()

julia> example(chars, 5)
5-element Vector{Char}:
 '\xd2\x5a\x96\x07': Malformed UTF-8 (category Ma: Malformed, bad data)
 '\x44\x45\xc5\x64': Malformed UTF-8 (category Ma: Malformed, bad data)
 '\x04': Unicode U+0004 (category Cc: Other, control)
 '\x2b\xe0\x6a\x89': Malformed UTF-8 (category Ma: Malformed, bad data)
 '\xf5\x9b\x63\x05': Malformed UTF-8 (category Ma: Malformed, bad data)
source
Supposition.Data.VectorsType
Vectors(elements::Possibility{T}; min_size=0, max_size=10_000) <: Possibility{Vector{T}}

A Possibility representing drawing vectors with length l in min_size <= l <= max_size, holding elements of type T.

min_size and max_size must be positive numbers, with min_size <= max_size.

julia> using Supposition

julia> vs = Data.Vectors(Data.Floats{Float16}(); max_size=5)

julia> example(vs, 3)
3-element Vector{Vector{Float16}}:
 [9.64e-5, 9.03e3, 0.04172, -0.0003352]
 [9.793e-5, -2.893, 62.62, 0.0001961]
 [-0.007023, NaN, 3.805, 0.1943]
source
Supposition.Data.WeightedNumbersType
WeightedNumbers(weights::Vector{Float64}) <: Possibility{Int}

Sample the numbers from 1:length(weights), each with a weight of weights[i].

The weights may be any number > 0.0.

See also OneOf.

julia> using Supposition

julia> bn = Data.WeightedNumbers([1.0, 1.0, 3.0]);

julia> example(Data.Vectors(bn; min_size=3, max_size=15), 5)
5-element Vector{Vector{Int64}}:
 [3, 2, 3, 3, 2, 3, 3]
 [1, 1, 1, 2, 1, 3, 1, 3]
 [2, 3, 3, 3, 3, 3, 3, 1, 1, 3, 3, 3]
 [3, 3, 2, 3, 3]
 [1, 3, 3, 3, 2, 2]
source
Supposition.Data.WeightedSampleType
WeightedSample{T}(colllection, weights::Vector{Float64}) <: Possibility{T}

Draw an element from the indexable collection, biasing the drawing process by assigning each index i of col the weight at weights[i].

length of col and weights must be equal and eachindex(col) must be indexable.

See also OneOf.

Sampling from `String`

To sample from a String, you can collect the string first to get a Vector{Char}. This is necessary because Strings use the variable-length UTF-8 encoding, which isn't arbitrarily indexable in constant time.

julia> bs = Data.WeightedSample(["foo", "bar", "baz"], [3.0, 1.0, 1.0]);

julia> example(bs, 10)
10-element Vector{String}:
 "foo"
 "foo"
 "bar"
 "baz"
 "baz"
 "foo"
 "bar"
 "foo"
 "foo"
 "bar"
source

Supposition reference

Supposition.@checkMacro
@check [key=val]... function...

The main way to declare & run a property based test. Called like so:

julia> using Supposition, Supposition.Data

julia> Supposition.@check [options...] function foo(a = Data.Text(Data.Characters(); max_len=10))
          length(a) > 8
       end

Supported options, passed as key=value:

  • rng::Random.AbstractRNG: Pass an RNG to use. Defaults to Random.Xoshiro(rand(Random.RandomDevice(), UInt)).
  • max_examples::Int: The maximum number of generated examples that are passed to the property.
  • broken::Bool: Mark a property that should pass but doesn't as broken, so that failures are not counted.
  • record::Bool: Whether the result of the invocation should be recorded with any parent testsets.
  • db: Either a Boolean (true uses a fallback database, false stops recording examples) or an ExampleDB.
  • config: A CheckConfig object that will be used as a default for all previous options. Options that are passed explicitly to @check will override whatever is provided through config.

The arguments to the given function are expected to be generator strategies. The names they are bound to are the names the generated object will have in the test. These arguments will be shown should the property fail!

Extended help

Reusing existing properties

If you already have a predicate defined, you can also use the calling syntax in @check. Here, the generator is passed purely positionally to the given function; no argument name is necessary.

julia> using Supposition, Supposition.Data

julia> isuint8(x) = x isa UInt8

julia> intgen = Data.Integers{UInt8}()

julia> Supposition.@check isuint8(intgen)

Passing a custom RNG

It is possible to optionally give a custom RNG object that will be used for random data generation. If none is given, Xoshiro(rand(Random.RandomDevice(), UInt)) is used instead.

julia> using Supposition, Supposition.Data, Random

# use a custom Xoshiro instance
julia> Supposition.@check rng=Xoshiro(1234) function foo(a = Data.Text(Data.Characters(); max_len=10))
          length(a) > 8
       end
Hardware RNG

Be aware that you cannot pass a hardware RNG to @check directly. If you want to randomize based on hardware entropy, seed a copyable RNG like Xoshiro from your hardware RNG and pass that to @check instead. The RNG needs to be copyable for reproducibility.

Additional Syntax

In addition to passing a whole function like above, the following syntax are also supported:

text = Data.Text(Data.AsciiCharacters(); max_len=10)

# If no name is needed, use an anonymous function
@check (a = text) -> a*a
@check (a = text,) -> "foo: "*a
@check (a = text, num = Data.Integers(0,10)) -> a^num

# ..or give the anonymous function a name too - works with all three of the above
@check build_sentence(a = text, num = Data.Floats{Float16}()) -> "The $a is $num!"
build_sentence("foo", 0.5) # returns "The foo is 0.5!"
Replayability

While you can pass an anonymous function to @check, be aware that doing so may hinder replayability of found test cases when surrounding invocations of @check are moved. Only named functions are resistant to this.

source
Supposition.@composedMacro
@composed

A way to compose multiple Possibility into one, by applying a function.

The return type is inferred as a best-effort!

Used like so:

julia> using Supposition, Supposition.Data

julia> text = Data.Text(Data.AsciiCharacters(); max_len=10)

julia> gen = Supposition.@composed function foo(a = text, num=Data.Integers(0, 10))
              lpad(num, 2) * ": " * a
       end

julia> example(gen)
" 8:  giR2YL\rl"

In addition to passing a whole function like above, the following syntax are also supported:

# If no name is needed, use an anonymous function
double_up =  @composed (a = text) -> a*a
prepend_foo = @composed (a = text,) -> "foo: "*a
expo_str = @composed (a = text, num = Data.Integers(0,10)) -> a^num

# ..or give the anonymous function a name too - works with all three of the above
sentence = @composed build_sentence(a = text, num = Data.Floats{Float16}()) -> "The $a is $num!"
build_sentence("foo", 0.5) # returns "The foo is 0.5!"

# or compose a new generator out of an existing function
my_func(str, number) = number * "? " * str
ask_number = @composed my_func(text, num)
source
Supposition.Data.produce!Method
produce!(p::Possibility{T}) -> T

Produces a value from the given Possibility, recording the required choices in the currently active TestCase.

Callability

This can only be called while a testcase is currently being examined or an example for a Possibility is being actively generated. It is ok to call this inside of @composed or @check, as well as any functions only intended to be called from one of those places.

source
Supposition.adjustMethod
adjust(ts::TestState, attempt)

Adjust ts by testing for the choices given by attempt.

Returns whether attempt was by some measure better than the previously best attempt.

source
Supposition.assembleMethod
assemble(::T, sign::I, expo::I, frac::I) where {I, T <: Union{Float16, Float32, Float64}} -> T

Assembles sign, expo and frac arguments into the floating point number of type T it represents. sizeof(T) must match sizeof(I).

source
Supposition.assume!Method
assume!(precondition::Bool)

If this precondition is not met, abort the test and mark the currently running testcase as invalid.

Callability

This can only be called while a testcase is currently being examined or an example for a Possibility is being actively generated. It is ok to call this inside of @composed or @check, as well as any functions only intended to be called from one of those places.

source
Supposition.choice!Method
choice!(tc::TestCase, n)

Force a number of choices to occur, taking from the existing prefix first. If the prefix is exhausted, draw from [zero(n), n] instead.

source
Supposition.considerMethod
consider(ts::TestState, attempt::Attempt) -> Bool

Returns whether the given choices are a conceivable example for the testcase given by ts.

source
Supposition.err_lessFunction
err_less(e1::E, e2::E) where E

A comparison function for exceptions, used when encountering an error in a property. Returns true if e1 is considered to be "easier" or "simpler" than e2. Only definable when both e1 and e2 have the same type.

This is optional to implement, but may be beneficial for shrinking counterexamples leading to an error with rich metadata, in which case err_less will be used to compare errors of the same type from different counterexamples. In particular, this function will likely be helpful for errors with metadata that is far removed from the input that caused the error itself, but would nevertheless be helpful when investigating the failure.

Coincidental Errors

There may also be situations where defining err_less won't help to find a smaller counterexample if the cause of the error is unrelated to the choices taken during generation. For instance, this is the case when there is no network connection and a Sockets.DNSError is thrown during the test, or there is a network connection but the host your program is trying to connect to does not have an entry in DNS.

source
Supposition.event!Function
event!(obj)
event!(label::AbstractString, obj)

Record obj as an event in the current testcase that occured while running your property. If no label is given, a default one will be chosen.

Callability

This can only be called while a testcase is currently being examined or an example for a Possibility is being actively generated. It is ok to call this inside of @composed or @check, as well as any functions only intended to be called from one of those places.

source
Supposition.exampleMethod
example(pos::Possibility; tries=100_000, generation::Int)

Generate an example for the given Possibility.

example tries to have pos produce an example tries times and throws an error if pos doesn't produce one in that timeframe. generation indicates how "late" in a usual run of @check the example might have been generated.

Usage:

julia> using Supposition, Supposition.Data

julia> example(Data.Integers(0, 10))
7
source
Supposition.exampleMethod
example(gen::Possibility, n::Integer; tries=100_000)

Generate n examples for the given Possibility. Each example is given tries attempts to generate. If any fail, the entire process is aborted.

julia> using Supposition, Supposition.Data

julia> is = Data.Integers(0, 10);

julia> example(is, 10)
10-element Vector{Int64}:
  9
  1
  4
  4
  7
  4
  6
 10
  1
  8
source
Supposition.find_user_error_frameFunction
find_user_error_frame(err, trace)

Try to heuristically guess where an error was actually coming from.

For example, ErrorException is (generally) thrown from the error function, which would always report the same location if we'd naively take the first frame of the trace. This tries to be a bit smarter (but still fairly conservative) and return something other than the first frame for a small number of known error-throwing functions.

source
Supposition.find_user_stack_depthMethod
find_user_stack_depth

Return a heuristic guess for how many functions deep in user code an error was thrown. Falls back to the full length of the stacktrace.

source
Supposition.forced_choice!Method
forced_choice(tc::TestCase, n::UInt64)

Insert a definite choice in the choice sequence.

Note that all integrity checks happen here!

source
Supposition.reject!Method
reject!()

Reject the current testcase as invalid, meaning the generated example should not be considered as producing a valid counterexample.

Callability

This can only be called while a testcase is currently being examined or an example for a Possibility is being actively generated. It is ok to call this inside of @composed or @check, as well as any functions only intended to be called from one of those places.

source
Supposition.runMethod
run(ts::TestState)

Run the checking algorithm on ts, generating values until we should stop, targeting the score we want to target on and finally shrinking the result.

source
Supposition.should_keep_generatingMethod
should_keep_generating(ts::TestState)

Whether ts should keep generating new test cases, or whether ts is finished.

true returned here means that the given property is not trivial, there is no result yet and we have room for more examples.

source
Supposition.shrink_redistributeMethod
shrink_redistribute(ts::TestState, attempt::Attempt, k::UInt)

Try to shrink attempt by redistributing value between two elements length k apart.

source
Supposition.shrink_sortMethod
shrink_sort(::TestState, attempt::Attempt, k::UInt)

Try to shrink attempt by sorting k contiguous elements at a time.

source
Supposition.shrink_swapMethod
shrink_swap(::TestState, attempt::Attempt, k::UInt)

Try to shrink attempt by swapping two elements length k apart.

source
Supposition.shrink_zerosMethod
shrink_zeros(::TestSTate, attempt::Attempt, k::UInt)

Try to shrink attempt by setting k elements at a time to zero.

source
Supposition.target!Method
target!(score)

Update the currently running testcase to track the given score as its target.

score must be convertible to a Float64.

Multiple Updates

This score can only be set once! Repeated calls will be ignored.

Callability

This can only be called while a testcase is currently being examined or an example for a Possibility is being actively generated. It is ok to call this inside of @composed or @check, as well as any functions only intended to be called from one of those places.

source
Supposition.target!Method
target!(tc::TestCase, score::Float64)

Update tc to use score as the score this TestCase achieves during optimization.

Multiple Updates

This score can only be set once! Repeated calls will be ignored.

source
Supposition.target!Method
target!(ts::TestState)

If ts has a target to go towards set, this will try to climb towards that target by adjusting the choice sequence until ts shouldn't generate anymore.

If ts is currently tracking an error it encountered, it will try to minimize the stacktrace there instead.

source
Supposition.tearMethod
tear(x::T) where T <: Union{Float16, Float32, Float64} -> Tuple{I, I, I}

Returns the sign, exponent and fractional parts of a floating point number. The returned tuple consists of three unsigned integer types I of the same bitwidth as T.

source
Supposition.test_functionMethod
test_function(ts::TestState, tc::TestCase)

Test the function given to ts on the test case tc.

Returns a NTuple{Bool, 2} indicating whether tc is interesting and whether it is "better" than the previously best recorded example in ts.

source
Supposition.windowsMethod
windows(array, a, b)

Split array into three windows, with split points at a and b. The split points belong to the middle window.

source
Supposition.CheckConfigType
CheckConfig(;options...)

A struct holding the initial configuration for an invocation of @check.

Options:

  • rng: The initial RNG object given to @check. Defaults to a copyable Random.AbstractRNG.
  • max_examples: The maximum number of examples allowed to be drawn with this config. -1 means infinite drawing (careful!). Defaults to 10_000.
  • record: Whether the result should be recorded in the parent testset, if there is one. Defaults to true.
  • verbose: Whether the printing should be verbose, i.e. print even if it's a Pass. Defaults to false.
  • broken: Whether the invocation is expected to fail. Defaults to false.
  • db: An ExampleDB for recording failure cases for later replaying. Defaults to default_directory_db().
  • buffer_size: The default maximum buffer size to use for a test case. Defaults to 100_000.
Buffer Size

At any one point, there may be more than one active buffer being worked on. You can try to increase this value when you encounter a lot of Overrun. Do not set this too large, or you're very likely to run out of memory; the default results in ~800kB worth of choices being possible, which should be plenty for most fuzzing tasks. It's generally unlikely that failures only occur with very large values here, and not with smaller ones.

source
Supposition.ComposedType
Composed{S,T} <: Possibility{T}

A Possibility composed from multiple different Possibility through @composed. A tiny bit more fancy/convenient compared to map if multiple Possibility are required to be mapped over at the same time.

Should not be instantiated manually; keep the object returned by @composed around instead.

source
Supposition.ErrorType
Error

A result indicating that an error was encountered while generating or shrinking.

source
Supposition.TestCaseType
TestCase

A struct representing a single (ongoing) test case.

  • prefix: A fixed set of choices that must be made first.
  • rng: The RNG this testcase ultimately uses to draw from. This is used to seed the task-local RNG object before generating begins.
  • generation: The "generation" this TestCase was made in. Can be used for determining how far along in the generation process we are (higher is further).
  • max_generation: The maximum "generation" this TestCase could have been made in. Does not necessarily exist.
  • max_size: The maximum number of choices this TestCase is allowed to make.
  • choices: The binary choices made so far.
  • targeting_score: The score this TestCase attempts to maximize.
source
Supposition.TestStateType
TestState
  • config: The configuration this TestState is running with
  • is_interesting: The user given property to investigate
  • rng: The currently used RNG
  • valid_test_cases: The count of (so far) valid encountered testcases
  • calls: The number of times is_interesting was called in total
  • result: The choice sequence leading to a non-throwing counterexample
  • best_scoring: The best scoring result that was encountered during targeting
  • target_err: The error this test has previously encountered and the smallest choice sequence leading to it
  • error_cache: A cache of errors encountered during shrinking that were not of the same type as the first found one, or are from a different location
  • test_is_trivial: Whether is_interesting is trivial, i.e. led to no choices being required
  • previous_example: The previously recorded attempt (if any).
source
Supposition.UnsetDBType
UnsetDB

An ExampleDB that is only used by the default CheckConfig, to mark as "no config has been set". If this is the database given in a config to @check and no other explicit database has been given, @check will choose the default_directory_db() instead.

Cannot be used during testing.

source
Supposition.CURRENT_TESTCASEConstant
CURRENT_TESTCASE

A ScopedValue containing the currently active test case. Intended for use in user-facing functions like target! or assume! that need access to the current testcase, but shouldn't require it as an argument to make the API more user friendly.

Not intended for user-side access, thus considered internal and not supported under semver.

source
Supposition.DEFAULT_CONFIGConstant
DEFAULT_CONFIG

A ScopedValue holding the CheckConfig that will be used by default & as a fallback.

Currently uses these values:

  • rng: Random.Xoshiro(rand(Random.RandomDevice(), UInt))
  • max_examples: 10_000
  • record: true
  • verbose: false
  • broken: false
  • db: UnsetDB()
  • buffer_size: 100_000

@check will use a new instance of Random.Xoshiro by itself.

source
Supposition.MESSAGE_BASED_ERRORType
MESSAGE_BASED_ERROR

A Union of some some in Base that are known to contain only the field :msg.

If you're using one of these errors and require specialized shrinking on them, define a custom exception type and throw that instead of overriding err_less. The definition of err_less for these types is written for the most generality, not perfect accuracy.

Unstable

This heavily relies on internals of Base, and may break & change in future versions. THIS IS NOT SUPPORTED API.

source