Ideas for maintaining large projects

Andreas Wolf
2024-08-02 – TYPO3 Developer Days 2024, Karlsruhe

Introduction

about:me

  • TYPO3 since 2003
  • Community since 2007
  • Education Committee
  • Task Force Lead TCCD
  • Freelancer, focusing on large projects and hard problems

Marathons and… maintaining software?

  • we focus on short iterations…
  • …sprints
  • but we maintain long-term…
  • … like a marathon?

Projects are a marathon!?

  • systems are more crucial today
  • total rewrites (almost) never happen
  • long-term maintenance required
  • = evolution instead of revolution

Projects are a marathon!?

  • sprint = 100-400m
  • in IT: usually two to four weeks
  • "~ 25-200m/week"
  • marathon = 42.195km
  • translates to ~4-32 years

Projects are a marathon!?

Lifecycles:

  • new PHP version every year
  • new TYPO3 LTS every 1.5 years
  • new Symfony major version every 2 years

⇒ major changes at least every three years

Projects are a marathon!?

  • so, looking at running…
  • … can we learn for long-term maintenance?

Running a marathon

It boils down to:

  • proper equipment
  • good training
  • the right techniques

Running a marathon

The equipment

  • good shoes
  • comfortable clothing
  • water and snacks

Running a marathon

The training

  • extended prep time
  • mixture of distances + speeds
  • increasing distance over time
  • monitor training goals

Running a marathon

The techniques

  • rest and sleep well before
  • find a proper running form
  • keep your personal pace
  • prevent injuries

So… how does that translate?

Maintaining a project

The equipment

  • version control
  • CI/CD
  • QA tools
  • documentation tools

Maintaining a project

The “training”

  • properly onboard new people
  • continuous maintenance
  • adopt new technology

Maintaining a project

The techniques

  • code reviews
  • automation
  • proper documentation
  • good modularization

Maintaining a project

The equipment

Version control

  • one project = one monorepo
  • choose a branching strategy
  • protect important branches (CI)

The equiment

Version control: trunk-based development

  • aka "small PRs merged to main"
  • simple model
  • everything goes to main
  • short-lived branches

The equiment

Version control: Git Flow

  • pretty complex
  • variations in merge strategies
  • fits for products, not projects

The equipment

CI/CD

  • regularly run all checks
  • no merge for broken code!
  • ensure pipeline is fast
  • collect code metrics
  • act when code starts to rot

The equipment

QA tools

  • code formatter
  • static analyzer
  • automated refactoring (Rector)
  • tests

The equipment

QA tools: code formatter

  • apply defined code standard
  • reduces unnecessary discussions
  • consistent style improves readability

The equipment

QA tools: static analyzer

  • checks code w/ type information
  • types rule out whole classes of bugs
  • types are also documentation

The equipment

QA tools: automated refactoring

  • e.g. Rector for PHP
  • refactoring + version upgrades
  • prevents adding outdated code
  • powerful technology 🚀

The equipment

QA tools: automated refactoring

  • for non-PHP code: Fractor (github.com/andreaswolf/fractor)
  • supports TypoScript, XML, YAML, composer.json
  • more to come
  • sponsored by TYPO3 Association ❤️

The equipment

QA tools: tests

  • use all levels of tests
  • do not strive for 100% coverage
  • run tests in CI
  • more: talk @ Sat 13:30

The equipment

Documentation tools

  • keep documentation with sources
  • build + make available for everyone
  • diagrams: MermaidJS, PlantUML

The “training”

Onboarding

  • document local dev setup
  • … or automate it
  • document project rules/standards
  • … or enforce them via software

The “training”

Continuous maintenance

  • frequent dependency updates
  • e.g. with Renovate or dependabot
  • adopt new PHP versions (Rector!)
  • refactor when necessary

The “training”

Adopting new technology

  • start with small experiments
  • adopt on success
  • rewrite existing code w/ Rector

The techniques

Automation

  • automation is also documentation
  • … on steroids: it's executable 🚀
  • run in CI: notice broken things
  • reduces mental load for devs

The techniques

Code reviews

  • foster understandable code
  • foster knowledge sharing
  • spot errors early on
  • keep code consistent

The techniques

Proper documentation

  • document why, less how/what
  • as much as necessary…
  • …as little as required
  • keep technical docs with code

The techniques

Good modularization

  • break down problems into smaller pieces
  • keep extensions small
  • no large sitepackages

The techniques

Good modularization

  • separate business logic from frontend
  • keep controllers, form finishers, … small
  • improves testability

Conclusion

Q&A

Project maintenance

Challenges

  • longer project lifecycles
  • + more complexity
  • requires different strategies

Long-term project maintenance

  • evolution instead of revolution
  • continuous changes (“steady pace”)
  • automation is your friend

Bonus slides

Maintenance: rules no. 1 + 2

Automate it

Standardize it

Maintenance 101

How to get started

  • version control + CI pipeline
  • code formatter + static analyzer
  • code reviews

Maintenance 201

Advanced steps

  • add tests
  • automate project setup
  • automate version updates

Thanks for listening!

Final Questions?

mail@a-w.io – @andreaswo