about:me

• TYPO3 since 2003
• Community since 2007
• Education Committee
• Task Force Lead TCCD
• based in Cologne
• Freelancer, focusing on large projects and hard problems

A word of warning

• disappointment ahead
• getting to a testing strategy is hard
• we need to talk about testing in TYPO3

In this talk

• The state of testing in TYPO3
• Overcoming challenges in testing
• What makes a robust testing strategy
• How to make testing work day-to-day

Why do we need tests?

The general answer

• tests safeguard changes
• tests improve software design
• finding bugs early is cheaper

Why do we need tests?

The general answer

• high quality software needs QA
• tests are one crucial part
• … but not an end in itself

⇒ tests are one QA tool

Why do we need tests?

The answer for TYPO3 projects

Recently, TYPO3 projects

• … tend to live longer
• … become more complex
• … become more critical for clients

Audience question

Who would say…

• …their critical projects have a good test coverage?
• …writing tests for TYPO3 is fun?

TYPO3 testing ecosystem

• TYPO3 Testing Framework
• + PHPUnit (unit/functional tests)
• acceptance tests: no universal solution
• Core uses Codeception

Testing in TYPO3

The missing parts

• off-the-shelf solution for acceptance tests
• base test cases for standard situations
• … like Extbase controllers, form finishers, DataHandler hooks, …
• good practices on how to test what

Unit tests

UnitTestCase in TYPO3 Testing Framework

(TYPO3) unit tests are constrained:

• no database
• no filesystem
• very limited bootstrap

Unit tests are good

• they run fast
• easy to implement
• … for decoupled, well-structured code

Unit tests suck

• for glue code
• (highly coupled code, e.g. controllers)
• for code with external dependencies
• (database, filesystem, APIs)

Testing beyond unit tests

To test our whole system, we need more:

• does our database work?
• do we treat files correctly?
• do our external API calls work?
• do we use the framework correctly?

Testing beyond unit tests

Functional tests

FunctionalTestCase in TYPO3 Testing Framework

• bootstrap a separate TYPO3 instance
• extensions defined in test case
• provide database + filesystem
• essentially a "child TYPO3 system"

Acceptance tests

• fetch pages via HTTP
• run in a browser
• run against a full TYPO3 system
• simulate a real user's interactions

Acceptance tests for projects

• no standardized setup
• must provide your own data
• much manual work

State of the ecosystem

• good approaches in the core/TF
• not much support for projects
• steep learning curve
• ⇒ challenges for writing tests

Drafting a testing strategy

1. find your pain points
2. define most critical parts
3. start covering those
4. define when to run which tests

Climbing “Mt. Test”

• start small
• Rome wasn't built in a day as well…
• pairing/reviews
• standardize testing approaches

• “this code is untestable”
• “my test fails randomly”
• “this test needs 40 lines of setup code”

Why is code “untestable”?

• it does too many things
• it has too many dependencies
• it relies on hard-to-setup code
• ⇒ code must be decoupled

Tightly coupled code

What is tight coupling?

• mixing business logic with I/O, API calls, …
• logic in controllers, form finishers, …
• running your code needs TYPO3 knowledge + setup
• ⇒ your code is tightly coupled to TYPO3

Decouple code (from TYPO3)

• separate I/O, API calls, business logic
• move code to services
• define clear interface TYPO3↔service
• use interface in controllers/finishers etc.

Testing decoupled code

• unit/functional tests for services
• functional/acceptance test for glue code

Write modular code

separation of concerns: use separate classes for…

• holding data (value objects!)
• data conversion
• API calls
• …

Writing decoupled code: a practical example

What do we want?

• submit data to an external API (job application)
• form + uploaded files

Writing decoupled code: a practical example

What do we need to do?

• analyze necessary steps
• split steps into components
• develop components w/ tests
• test = first user of your code

Writing decoupled code: a practical example

Testing tightly coupled code

• move higher in the testing pyramid
• only write unit tests for refactored code
• … or code w/o dependencies

Misconceptions about testing

• “We cannot write tests for everything”
• “We must cover 100% of lines”
• “Tests are not useful if only I write them”
• “We're slower when we write tests”

What to focus on when testing

• “code coverage” is meaningless
• focus on critical areas
• execute tests often
• tests are a safety net e.g. for upgrades
• test happy part + fault cases

Tests must run often

“Tests that don't run break over time”

• run tests with a single command
• tests must be part of CI pipeline
• unit/functional for every change
• acceptance at least daily

Avoid mocks where possible

• Mocks increase coupling
  … to internals of tested code
• after upgrade: test may be green, but code broken
• "Do not mock what you don't own" – Sebastian Bergmann

On code coverage…

• do not use coverage as quality gate
• 100% coverage is no sign for anything
• indirectly testing code is better than nothing

Write less tests

• do not test framework/PHP functionality
• use strong type hints
• use PhpStan to catch errors
• write own PhpStan rules

Q&A

Conclusion

Testing in TYPO3 projects

• lack of standardization/foundations
• lack of best practices
• common misconceptions about how to test
• rework on the way

Misconceptions on testing

• unit tests for everything
• covering everything with tests
• isolating a class with mocks

Testing tightly coupled code

• focus on large-scale tests (acceptance)
• use the safety net for refactorings
• move out code w/o dependencies for unit tests

Thanks for listening!

Final Questions?

mail@a-w.io – @andreaswo