gridtest

Decorators

If you haven’t installed gridtest, you should do this first.

GridTest Decorators

GridTest provides a suite of it’s own decorators to measure metrics. For a simple example of using the gridtest timeit decorator, see this tutorial. For a custom decorator example, you can see the tutorial here or keep reading.

Custom Decorator

If you want to run a gridtest that uses a custom metric, you can easily do this by defining your own decorator. For example, let’s say we have a function to do some kind of text processing. It takes some number of inputs, and returns raw text. We would then want to count the unique words in the raw text. Let’s go!

Create your Functions

Let’s first write our functions. We will write a simple function to take some text input and parse it (multiply_sentences) and a decorator to run any function that returns a string of text, and count the unique words (countwords)

# These are functions in my script

def multiply_sentence(sentence, count):
    return sentence * count

def countwords(func):
    """this is a simple example of a custom decorator - the idea would be that
       the function we are decorating returns some texty value, and we split
       this value by a blank space and then count the number of tokens (words).
    """
    def counter(*args, **kwargs):
        result = func(*args, **kwargs)
        words = len(set(result.split(' ')))
        print(f"@script.countwords {words} words")
        return result

    return counter

An important note about the decorator - it needs to be importable, meaning either the module is already on your Python path, or it’s included somewhere in the library that you are testing. Also note that in order for gridtest to parse the result, you need to print something to stdout that is prefixed exactly with the name of the decorator defined under metrics. E.g., if we changed script.countwords to just countwords the result wouldn’t be properly parsed, because gridtest is looking for the the first.

Generate your Template

Let’s generate a simple template that we can fill in to include a grid. We can first preview it:

$ gridtest generate script.py

script:
  filename: /home/vanessa/Desktop/Code/gridtest/examples/custom-decorator/script.py
  tests:
    script.countwords:
    - args:
        func: null
    script.multiply_sentence:
    - args:
        count: null
        sentence: null

We don’t want a test for the decorator, so we will write this to file, and remove it.

$ gridtest generate script.py gridtest.yml

script:
  filename: /home/vanessa/Desktop/Code/gridtest/examples/custom-decorator/script.py
  tests:
    script.multiply_sentence:
    - args:
        count: null
        sentence: null

Next, let’s better refine our arguments.

script:
  filename: /home/vanessa/Desktop/Code/gridtest/examples/custom-decorator/script.py
  tests:
    script.multiply_sentence:
    - metrics:
      - '@script.countwords'
      args:
        count: [1, 5, 10]
        sentence:
          - "He ran for the hills."
          - "Skiddery-a rinky dinky dinky, skittery rinky doo."
          - "You are my sunshine, my only sunshine."

Just for your FYI - if you had wanted to have some set of arguments shared between tests, you could have defined them as a named grid:

grids:
  script_inputs:
    args:
      count: [1, 5, 10]
      sentence:
        - "He ran for the hills."
        - "Skiddery-a rinky dinky dinky, skittery rinky doo."
        - "You are my sunshine, my only sunshine."

and then instead pointed to it for your test:

script:
  filename: /home/vanessa/Desktop/Code/gridtest/examples/custom-decorator/script.py
  tests:
    script.multiply_sentence:
    - metrics:
      - '@script.countwords'
      grid: script_inputs

By default, a grid is generated at the test creation time. However, if you have a grid shared by many functions that you want to calculate once and cache, just set the cache variable to true:

grids:
  script_inputs:
    cache: true
    args:
      count: [1, 5, 10]
      sentence:
        - "He ran for the hills."
        - "Skiddery-a rinky dinky dinky, skittery rinky doo."
        - "You are my sunshine, my only sunshine."

Regardless of how we specify our grid (globally or inline) the grid says that for each sentence under the list of sentences, we will run the function multiply_sentence with counts of 1,5, and 10. This would come down to 3x3 or 9 total tests.

Running Tests

Let’s run the tests! We should see a count of words for each function.

$ gridtest test
[9/9] |===================================| 100.0% 
Name                           Status                         Summary                       
________________________________________________________________________________________________________________________
script.multiply_sentence.0     success                                                      
script.multiply_sentence.1     success                                                      
script.multiply_sentence.2     success                                                      
script.multiply_sentence.3     success                                                      
script.multiply_sentence.4     success                                                      
script.multiply_sentence.5     success                                                      
script.multiply_sentence.6     success                                                      
script.multiply_sentence.7     success                                                      
script.multiply_sentence.8     success                                                      

________________________________________________________________________________________________________________________
script.multiply_sentence.0     @script.countwords             5 words                       
script.multiply_sentence.1     @script.countwords             7 words                       
script.multiply_sentence.2     @script.countwords             7 words                       
script.multiply_sentence.3     @script.countwords             21 words                      
script.multiply_sentence.4     @script.countwords             31 words                      
script.multiply_sentence.5     @script.countwords             31 words                      
script.multiply_sentence.6     @script.countwords             41 words                      
script.multiply_sentence.7     @script.countwords             61 words                      
script.multiply_sentence.8     @script.countwords             61 words                      

9/9 tests passed

If you don’t see the result of the decorator under the summary, likely you didn’t write the stdout print within the function to match the name that you supplied under metrics. For this example, this should be @script.countwords in both cases. And that’s it! We’ve run a grid of tests over a grid of inputs, and measured some metric with our custom decorators.

You might next want to browse other tutorials available.