*Negative Tests
Date & Time:  November  7 2013  2:20pm -  3:10pm
Session Proposer: David Kranz

Tempest has a lot of negative tests but there is no shared view of how many there should be, or whether they should really
be part of unit tests. The same issue regarding coverage applies to positive
tests as well. We have no policy about what adequate tempest coverage for an api actually is. The coming plan for
using tempest as part of refstack means we need to make some proposals in this area.

There have also been discussions about fuzz testing. Here are a few things that have been discussed and from which a new consensus might emerge:

 - We could have a decorator or other kind of syntax that allows a declarative way to define negative tests, but which run in the
    same way as existing tests. These would be easier to write and review. Also, they could be run as either unit tests or in tempest.
    Projects are moving towards json schema
    dkranz and mkoderer to come up with proposal (three weeks)

- We should come up with a policy for negative test coverage and possibly move most to unit tests.  
Moratorium on new negative tests unless of high value. 
sdague to communicate about this. Will accept patches in flight if reasonable code
sdague to investigate project capability for neutron-style functional testing
mnewby ambassador for functional tests in projects

- We could have a fuzz testing framework, possibly supported by some kind of type signature for apis to allow checking for
    fencepost errors rather than just slinging random arguments 

- Services must enforce valid utf-8 input encoding and rejecting the incorrect request  with 400 
 https://bugs.launchpad.net/tempest/+bug/1191508