Lecture 6-1

Testing Concluded; Git

CS 300 Lecture 6-1
Bart Massey

Testing Concluded

  • Reminder: Last week we talked about test cases and where they come from, and principles of testing

  • Today we will pick that up where we left off

Black Box vs White Box

  • Black Box: Tests given without knowledge of implementation

  • White Box: Use implementation knowledge to construct tests

  • Both are valuable

Test Domains

  • Idea: Divide input or output space up such that only one representative in each domain need be tested

    • How to draw domain boundaries?
    • There are still a lot
    • Better idea: formal methods plus testing

Testing Kinds

  • What should be tested?

    • System (acceptance)
    • Integration
    • Unit
    • Regression
  • Integration testing can be

    • bottom-up
    • top-down

Stubs and Drivers

  • Stub: piece of code that "simulates" a function or module so that others can call it during testing

  • Driver: piece of code that calls a function or module to test it

Testing Methods

  • Domain-driven

  • Random (e.g. "fuzz testing")

  • "User Tests"

  • Coverage-driven

Regression Tests

  • Bugs can come back, tests are expensive

    • Run tests after every change
  • Debugging: Write tests for every fix

    • Add tests to regression test suite

Coverage Testing

  • How much of the program has been tested?

    • All statements?
    • All branches (each way)?
    • All code paths?
    • All data patterns?
  • Automated tools (e.g. gcov) exist

  • 100% coverage is impossible

  • Untested code is broken code

Fault Seeding

  • Attempt to find out how good the tests are

  • Use SCMS to reliably remove seeded faults (!)

Test Frameworks

  • "Test-driven development" says write tests first, then code to pass tests

    • Typically unit tests and/or system tests
  • Idea is to always run tests ("regression test")

    • So build environment + code is always full of tests
    • Without automatic support from tools, this gets ugly fast
  • Integrated with build environment for "continuous integration"

  • Lots and lots of this out there

    • Flavors of the week: "JUnit" and friends, "Travis"

Testing Infrastructure

  • Tests need to be maintained with code

  • Tests need to be runnable automatically

  • Test failures need to be logged as tickets until fixed


  • DVCS concepts

  • Repo setup

  • Commits

  • Remotes

    • Push, Pull
  • Branching

  • Merging

  • Rebasing


  • Accounts

  • Services

    • Repo storage
    • Issue tracker
    • Wiki
    • Email list
    • etc
  • SSH keys

Last modified: Wednesday, 4 November 2015, 12:58 AM