Testing Concluded; Git

CS 300 Lecture 6-1
Bart Massey



Testing Concluded

• Reminder: Last week we talked about test cases and where they come from, and principles of testing

• Today we will pick that up where we left off

  

Black Box vs White Box

• Black Box: Tests given without knowledge of implementation

• White Box: Use implementation knowledge to construct tests

• Both are valuable

  

Test Domains

• Idea: Divide input or output space up such that only one representative in each domain need be tested

  • How to draw domain boundaries?
  • There are still a lot

  • Better idea: formal methods plus testing

    

Testing Kinds

• What should be tested?

  • System (acceptance)
  • Integration
  • Unit
  • Regression

• Integration testing can be

  • bottom-up

  • top-down

    

Stubs and Drivers

• Stub: piece of code that "simulates" a function or module so that others can call it during testing

• Driver: piece of code that calls a function or module to test it 

Testing Methods

• Domain-driven

• Random (e.g. "fuzz testing")

• "User Tests"

• Coverage-driven

  

Regression Tests

• Bugs can come back, tests are expensive

  • Run tests after every change

• Debugging: Write tests for every fix

  • Add tests to regression test suite

    

Coverage Testing

• How much of the program has been tested?

  • All statements?
  • All branches (each way)?
  • All code paths?
  • All data patterns?

• Automated tools (e.g. gcov) exist

• 100% coverage is impossible

• Untested code is broken code

  

Fault Seeding

• Attempt to find out how good the tests are

• Use SCMS to reliably remove seeded faults (!)

  

Test Frameworks

• "Test-driven development" says write tests first, then code to pass tests

  • Typically unit tests and/or system tests

• Idea is to always run tests ("regression test")

  • So build environment + code is always full of tests
  • Without automatic support from tools, this gets ugly fast

• Integrated with build environment for "continuous integration"

• Lots and lots of this out there

  • Flavors of the week: "JUnit" and friends, "Travis"

    

Testing Infrastructure

• Tests need to be maintained with code

• Tests need to be runnable automatically

• Test failures need to be logged as tickets until fixed

  

Git

• DVCS concepts

• Repo setup

• Commits

• Remotes

  • Push, Pull

• Branching

• Merging

• Rebasing

  

Github

• Accounts

• Services

  • Repo storage
  • Issue tracker
  • Wiki
  • Email list
  • etc

• SSH keys