Ideas for maintaining large projects
Andreas Wolf
2024-08-02 – TYPO3 Developer Days 2024, Karlsruhe
about:me
- TYPO3 since 2003
- Community since 2007
- Education Committee
- Task Force Lead TCCD
- Freelancer, focusing on large projects and hard problems
Marathons and… maintaining software?
- we focus on short iterations…
- …sprints
- but we maintain long-term…
- … like a marathon?
Projects are a marathon!?
- systems are more crucial today
- total rewrites (almost) never happen
- long-term maintenance required
- = evolution instead of revolution
Projects are a marathon!?
- sprint = 100-400m
- in IT: usually two to four weeks
- "~ 25-200m/week"
- marathon = 42.195km
- translates to ~4-32 years
Projects are a marathon!?
Lifecycles:
- new PHP version every year
- new TYPO3 LTS every 1.5 years
- new Symfony major version every 2 years
⇒ major changes at least every three years
Projects are a marathon!?
- so, looking at running…
- … can we learn for long-term maintenance?
Running a marathon
It boils down to:
- proper equipment
- good training
- the right techniques
Running a marathon
The equipment
- good shoes
- comfortable clothing
- water and snacks
Running a marathon
The training
- extended prep time
- mixture of distances + speeds
- increasing distance over time
- monitor training goals
Running a marathon
The techniques
- rest and sleep well before
- find a proper running form
- keep your personal pace
- prevent injuries
So… how does that translate?
Maintaining a project
The equipment
- version control
- CI/CD
- QA tools
- documentation tools
Maintaining a project
The “training”
- properly onboard new people
- continuous maintenance
- adopt new technology
Maintaining a project
The techniques
- code reviews
- automation
- proper documentation
- good modularization
The equipment
Version control
- one project = one monorepo
- choose a branching strategy
- protect important branches (CI)
The equiment
Version control: trunk-based development
- aka "small PRs merged to main"
- simple model
- everything goes to main
- short-lived branches
The equiment
Version control: Git Flow
- pretty complex
- variations in merge strategies
- fits for products, not projects
The equipment
CI/CD
- regularly run all checks
- no merge for broken code!
- ensure pipeline is fast
- collect code metrics
- act when code starts to rot
The equipment
QA tools
- code formatter
- static analyzer
- automated refactoring (Rector)
- tests
The equipment
QA tools: code formatter
- apply defined code standard
- reduces unnecessary discussions
- consistent style improves readability
The equipment
QA tools: static analyzer
- checks code w/ type information
- types rule out whole classes of bugs
- types are also documentation
The equipment
QA tools: automated refactoring
- e.g. Rector for PHP
- refactoring + version upgrades
- prevents adding outdated code
- powerful technology 🚀
The equipment
QA tools: automated refactoring
- for non-PHP code: Fractor (github.com/andreaswolf/fractor)
- supports TypoScript, XML, YAML, composer.json
- more to come
- sponsored by TYPO3 Association ❤️
The equipment
QA tools: tests
- use all levels of tests
- do not strive for 100% coverage
- run tests in CI
- more: talk @ Sat 13:30
The equipment
Documentation tools
- keep documentation with sources
- build + make available for everyone
- diagrams: MermaidJS, PlantUML
The “training”
Onboarding
- document local dev setup
- … or automate it
- document project rules/standards
- … or enforce them via software
The “training”
Continuous maintenance
- frequent dependency updates
- e.g. with Renovate or dependabot
- adopt new PHP versions (Rector!)
- refactor when necessary
The “training”
Adopting new technology
- start with small experiments
- adopt on success
- rewrite existing code w/ Rector
The techniques
Automation
- automation is also documentation
- … on steroids: it's executable 🚀
- run in CI: notice broken things
- reduces mental load for devs
The techniques
Code reviews
- foster understandable code
- foster knowledge sharing
- spot errors early on
- keep code consistent
The techniques
Proper documentation
- document why, less how/what
- as much as necessary…
- …as little as required
- keep technical docs with code
The techniques
Good modularization
- break down problems into smaller pieces
- keep extensions small
- no large sitepackages
The techniques
Good modularization
- separate business logic from frontend
- keep controllers, form finishers, … small
- improves testability
Project maintenance
Challenges
- longer project lifecycles
- + more complexity
- requires different strategies
Long-term project maintenance
- evolution instead of revolution
- continuous changes (“steady pace”)
- automation is your friend
Maintenance: rules no. 1 + 2
Automate it
Standardize it
Maintenance 101
How to get started
- version control + CI pipeline
- code formatter + static analyzer
- code reviews
Maintenance 201
Advanced steps
- add tests
- automate project setup
- automate version updates
Thanks for listening!
Final Questions?
mail@a-w.io – @andreaswo