03. Audits

Independent party verifying that the processes are being followed and the end product meets the requirements.

Formal software audits:

• Uses formal methods and mathematically functions to prove correctness
• Requires code freeze
• Takes a lot of time and resources
• Useful for critical systems - where there are legal standards (e.g. health, financial)

Less-formal software audits:

• Does the software:
  • Have good quality
  • Do what the customer wants
• Attempts to do this with a much lower cost and time
• Lower confidence and degree of correctness, but may be enough
• Development can continue

In SENG401, a less formal software audit will be done on SENG302 teams.

Software outcomes can be divided into two strands:

1. Does it meet the acceptance criteria?

• Explicit, implicit, and non-functional requirements

2. Does it adhere to a process that increases the changes of success:

• Development/quality process
• Examine work artifacts: backlog, scrum board, walkthroughs, logs, test plans/protocols
• Critique team processes
• Resourcing: is there enough staff?
• Examine training/on-boarding processes

SENG302 Audit

Part 1: Report

Observe:

• Potentially shippable products; check the ACs
• The codebase:
  • Architecture
  • Metrics
  • Code smells
  • Extensibility
  • Amount of over-engineering, gold-plating
  • Amount of re-engineering
  • Testing and metrics
  • And how it changes over time
  • Bugs and issues
  • Commit history
• The process:
  • Scrum board
  • Team processes (e.g. estimation)
  • Code review, merge request process
  • Wiki
• Teamwork:
  • Observe how the teams work
  • Tuckman model
  • Informal chats
  • Pair-programming, co-location
  • Workload distribution
• Culture
  • Bike shedding: spending time on trivial matters
  • Group think
  • Hero culture: one person doing all the work
  • Death march: doing everything at the last minute all while knowing that you will not meet the deadline
    • Not communicating with the PO due to the power-level difference and PO not recognizing this
  • Personalities
• Communication with:
  • Teaching staff
  • Other teams
  • PO, SM

Can ask Moffat for summarized peer feedback/self-reflection, but not the full submissions.

Then, the audit report:

• Diagnosis: what is the current state of the team
• Prognosis: what is the future state of the team
• Recommendations: what do you recommend to improve the team, and what will their future state look like if these recommendations are followed

There must be evidence, ideally multiple factors that corroborate the conclusions drawn.

Part 2: Live Review

Diagnosis, prognosis, recommendation.

Talk to the team - the patient, professionally:

• With respect
• With empathy
• Sitting down - standing up emphasizes power-difference
• Arrange the room so that you are talking to the team, not the audience
• Don’t present; talk and discuss with them

Prognosis:

• Long-term future with the current state of affairs
• Long-term future if recommendations are followed

Misc:

• Balance out both positives and negatives to not overwhelm the ‘patient’
• Tell them that we, not them, are the ones being assessed
• Place feedback in the context of industry, not the course
  • e.g. ‘log or else Moffat will get angry’; that only helps them pass the course
• No identifying information - no naming and shaming. anonymize graphs