Software Testing Ideas and Tools

Collection of resources, tools, and libraries related to testing software. Blogs, YouTube videos can get dated quicker than libraries.

Learn

• The Oracle Problem
  • https://en.wikipedia.org/wiki/Test_oracle
  • https://discovery.ucl.ac.uk/id/eprint/1471263/1/06963470.pdf
• https://en.wikipedia.org/wiki/Model-based_testing
• https://en.wikipedia.org/wiki/Metamorphic_testing

Autonomous Testing

• https://www.youtube.com/watch?v=fFSPwJFXVlw
• https://apple.github.io/foundationdb/testing.html

Property Based Testing

• John Hughes - Testing the Hard Stuff and Staying Sane: https://youtu.be/zi0rHwfiX1Q
• Paper: http://publications.lib.chalmers.se/records/fulltext/232550/local_232550.pdf
• Haskell QuickCheck https://begriffs.com/posts/2017-01-14-design-use-quickcheck.html
• Test Fast, Fix More Property based testing with Hypothesis by Alexander Hultnér: https://www.youtube.com/watch?v=MKf6KfdTems

Blogs

• https://martinfowler.com/testing/

Distributed Systems

• https://jepsen.io/

Kubernetes

• https://github.com/kubernetes/test-infra
• Prow CI/CD https://github.com/kubernetes/test-infra/tree/master/prow
• https://jenkins-x.io/

Research

• https://github.com/utsaslab/crashmonkey

GitHub

• https://github.com/topics/testing

Test Automation

Frameworks

• Language Agnostic: https://gauge.org
• Python: https://docs.pytest.org
• UI: [Selenim]https://www.selenium.dev),[Playwright](https://playwright.dev/)

Build CLIs

• https://click.palletsprojects.com/
• https://typer.tiangolo.com/
• Advanced TUI: https://www.textualize.io/

Test Rest APIs

• API mocking https://stoplight.io/open-source/prism
• Data generation: https://www.getsynth.com/
• API Clients https://insomnia.rest, https://www.postman.com

Parse CLI output

• https://pypi.org/project/pyparsing/

Execute Commands on Remote hosts

• https://www.fabfile.org/
• Mitogen

Testing Libraries

• Property based testing: https://hypothesis.readthedocs.io/en/latest/
• Load Testing: https://locust.io/
• Code Coverage: https://github.com/nedbat/coveragepy

Fuzzing

• http://lcamtuf.coredump.cx/afl/
• https://github.com/google/clusterfuzz

Linux

• https://github.com/linux-test-project
• https://linux-test-project.github.io/

File Systems

• From MacOS: https://macosforge.github.io/fstools/tools/
  • fsTorture
  • fsx
• https://github.com/pjd/pjdfstest
• Block and some file IO: https://github.com/axboe/fio
• https://wiki.lustre.org/VDBench
• https://github.com/filebench/filebench (No longer active)

Windows

• Disk, I/O: https://github.com/Microsoft/diskspd
• Protocol: https://github.com/microsoft/WindowsProtocolTestSuites

Online only tools

• SSL: https://www.ssllabs.com/ssltest/analyze.html
• Regex: https://regex101.com/
• Jellyfish Video Bitrate Test Files: http://jell.yfish.us/

Code Quality

• https://www.sonarqube.org/
• https://github.com/astral-sh/ruff

Reporting

• Tests :http://allure.qatools.ru/
• Tests: https://extentreports.com/
• Error Reporting: https://github.com/getsentry/sentry