04. When Good Design Goes Bad

UML requires big design up-front, synchronization of diagrams and code. However, it is useful for communication.

What we’ve learned:

• 201: Design is important and achievable
• 301: Quality is important and achievable
• 302:
  • Many things are important:
    • Business decisions
    • Time/cost/resources
    • Processes
    • The team is important
    • The individual is important
    • The customer is important
    • Priorities
    • Everything
  • All are achievable, but not by us
• 401:
  • What is design?
  • What is quality?
  • What is process?
  • Are any of these achievable?

Design Erosion

AKA: architectural drift, software aging, architecture erosion, software decay, software rot, software entropy.

When the initial design becomes more and more obsolete:

• The world changes, requiring the design to change:
  • New requirements, incremental
  • Technical debt
    • Hacks during rush jobs
  • Bug fixes/patches, local corrections
  • Changes in the business environment/strategy
    • Solving different problems that the architecture is not capable of meeting
  • Adoption of new solutions/technologies
  • People change
  • Vaporized design decisions
    • Original designers left
    • Undocumented design decisions
    • Documents not followed
  • Quick fixes
  • Inexperienced teams (new programmers and/or new teams)
  • Iterative methods/practices
  • Time/cost pressures
  • Not enough language support

Consequences:

• Accumulation of sub-optimal design solutions, leading to further design erosion
• Increased time/cost to add/modify/fix bugs or features
• Workarounds, ad-hoc approaches, trying to fix the symptom instead of the problem
  • Fixing one bug makes two more, fixing those causes a hundred more
• Negative impact on development
• Deployment problems
• A large and growing number of bugs/issues
• Spiraling descent towards worsening design

Eventually, a replacement, rewrite, re-engineering or refactor becomes required.

So what to do when changes occur?

1. Optimal design strategy

• No-compromise in design
• High local/immediate costs

2. Minimal effort strategy

• ‘Stretching’ design rules
• Could help with cost/time constraints

‘Natural’ Rot

… the design of a software project is documented primarily by its source code

Robert C. Martin

To destroy an abandoned building, cut a hole in the roof and wait for it to rot from the inside out.

Software works the same way; without proper maintenance, a small hole can lead it to decaying from the inside.

Broken window theory: hacks in software normalize other hacks, leading to a spiraling descent in quality.

Symptoms of rot:

• Rigidity:
  • Design resists change, even for simple changes
  • Changes cause cascading changes; high coupling
  • Code built on top of rigid code becomes more rigid
  • Leads to:
    • Fear of letting developers fix non-critical problems
    • Unknown time necessary to make fixes or changes
    • “Official rigidity”: management is not flexible, leading to inflexible teams and inflexible to design deficiencies
• Fragility:
  • A single change can break things in multiple places with (seemingly) no relationship to the changed area
  • Leads to:
    • Fear of making fixes and breaking more things, or of having no idea where the underlying issue is
    • Increase in fragility; fixes are minimal and likely add more fragility
• Immobility:
  • Difficult to get re-usable components - a tangled and highly-coupled mess
  • Component dependent on too much baggage
  • Leads to:
    • Rewrites instead of reuse
    • Copy/paste programming
    • Multiple points of failure
• Viscosity:
  • Software viscosity:
    • Easier to add a hack than to conform to the design
    • Leads to increased rot speed
  • Environmental viscosity:
    • Slow development environment
    • Long compile, check-in, and deployment times
    • Leads to:
      • Skipping processes to speed things up
      • Hacks, merge conflicts, bad designs
• Needless repetition::
  • YAGNI
  • Business rules not implemented in a clear and distinct manner
  • Copy pasting
  • Leads to:
    • Reduced trust in the software, team, management, company
    • Reduced comprehension of code
    • Rigidity: changes need to be implemented in many places
• Opacity:
  • How understandable is the code?
  • Code that evolves over time becomes more difficult to understand
  • Leads to:
    • Reduced comprehension of code and business rules
    • Fragility
    • Rigidity
• Needless complexity:
  • Due to the developer looking out for future extensions
    • May lead to a lot of additional classes that must be maintained and dead code (results not used)
  • Leads to:
    • Over-engineering
    • Other rot symptoms

Preventing Rot

Address problems immediately:

• Broken window theory: sloppy code invites more sloppy code
• Patch the holes; may take a lot of effort and require you to convince management (the hole that you made)
• Determine the root cause; don’t just fix the symptoms:
  • Developers should discuss and have a meeting
  • Code reviews
  • Do not allow even one hole or broken window
    • Very difficult to achieve
    • Fix the issues the moment you find them - requires open communication with PO
• Add processes to identify and measure problems, then make strategies to fix them
• SOLID principles

Class Discussion: How Do Classical and Modern Processes Influence Design Erosion

Waterfall:

• Big design up-front: designs can’t be changed, so no rot but bad designs can’t be fixed
• Iterative waterfall: changes can consider how the design needs to change and how to best change it to reduce erosion
• If requirements do change within the waterfall cycle, rigid structure means hacks may be required to fit them in

Agile:

• Prioritizes working software over code quality

Waterfall in business:

• Very well-defined process
• Predictable: budget, time-lines written down
• Legal can take a look at the project before it starts
• Used in other engineering disciples

Design/Code Smells

NB: code smells can also refer to good smells.

An indication/symptom that something may be wrong: but does this mean it should be fixed? Two approaches:

• Purist: Where there’s smoke, there’s fire: fix it
• Pragmatic: check it out and fix it if it is major

Smells: Within Classes

• Long methods:
  • Too much functionality: SRP, separation of concerns
  • Lines of code
  • Complexity
• Long parameter lists:
  • High coupling
  • May be doing too much
  • How many arguments are too many? 3? 4?
• Large classes:
  • Too much functionality: SRP
  • God class
• Comments:
  • A sign that the code is too hard to understand - unreadable code
  • Code should read like prose
  • May get out of date, leading to reduced trust in documentation, and may not be read
• Duplicated code:
  • Pull it out into its own utility method
  • Rule of three:
    • Duplicate once
      • Could be called over-engineering
    • Maybe duplicate twice
    • By the time you get to three, pull the code out into a method
• Combinatorial explosion
• Dead code, unreachable code
• Speculative generality:
  • Making things more general for perceived future needs; over-engineering
• Oddball solution:
  • Many different ways of solving the same problem

Smells: Between Classes

• Primitive obsession:
  • Using primitives for everything rather than classes/objects
  • Hard to read
  • Can’t be extended; can’t add constraints etc.
  • Keep data and methods together: can’t add methods to primitives
  • No type-safety
• Data class: Classes with properties but no methods/functionality Keep data and behavior together
• Refused bequest:
  • Children classes cannot implement parent contracts
• Inappropriate intimacy:
  • Severe coupling between classes
• Indecent exposure:
  • Fields that should be private but are public
• Lazy class:
  • Classes that don’t have much functionality
  • Increased complexity
• Message chains:
  • Law of demeter
• Shotgun surgery:
  • Change that requires multiple things to be changed

Metrics

If you can’t measure it, you can’t improve it

Peter Drucker

You can’t control what you can’t measure

Tom DeMarco

Some context-dependent measure of a project, usually measured over time to track how the project is improving or getting worse. However, the context can change over time, making interpreting the metrics and making comparisons over time more difficult.

Benefits:

• Quality assurance
• Software/project/development management
• Performance
  • Tracking issues, bugs
• Estimations
  • How to estimate?
• Management
  • Identifying parts/modules to improve
  • Prioritize work
  • Reduce costs
  • Understand where resources should be put
  • Return on investment; measurable improvement to the code base
  • Workload: can show management impacts of over-working the team

Dangers:

• Gaming the system - improving metrics but not the code

  • Goodhart’s Law:

    Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes

• Not having an understanding of the metric

• Metrics being used by management in performance reviews

  • e.g. LoC added, deleted, modified, survivability:
  • File being indented/formatted causes many changes
  • Moving methods to different files
  • Getters/setters low-effort code?

• Concentrating on a single metric

• Reducing a complex situation to a single metric

• Comparing betweeen two very different projects For a metric to be useful:

• They should be easily and quickly calculated

• They should be run often

• Several metrics should be collected

• Trends are more important than values

• Outliers should be checked

• TODO

Difficulties:

• Measuring code does not measure the design
  • TODo
• Limited tool support (but getting better)
  • Metrics require interpretation
  • Ignorance is prevalent

Alternativ ways to identify code smells:

• Ownership and expertise
  • TODO
• Social structures
  • Experts
  • Do people follow the most knowledgable or the loudest
  • TODO

Measurements

McCabe’s Cyclomatic Complexity:

• Complexity of a function; depends on control flows
• $M = E - N + 2P$
• Where:
  • $E$ is the number of edges - TODO
  • $N$ is the number of nodes - number of statements
  • $P$ is the number of conected components (exit paths)
  • TODO

Chidmaber and Kemerer OO Metrics:

1. Weighted methods per class (WMC)

• Sum of method complexities in a class

2. Depth of inheritance tree (DIT)

• Max inheritance depth
• Deeper trees: if something near the top of the tree breaks, everything below also breaks

3. Number of children (NOC)

• Specialization polymorphism in terms of contracts?
• Number of immediate children
• Closer to parent contract, so higher is better

4. Coupling between objects (CBO)

• Number of classes to which a class is coupled

5. Response for class (RFC)

• Number of class methods, plus number of remote methods called directly by the methods (or through the entire call tree)
• TODO
• Coupling

6. Lack of cohesion in methods (LCOM):

• Increase cohesion: keep related data and behavior together
• Decrease cohesion: single responsibility principle, separation of concerns, open-closed principle

TODO Lorenz and Kidd:

'Ar

Smells:

• Long methods:
  • LOC: too long
  • CYCLO: too many conditional branches
  • MAXNESTING: nesting too deep
  • NV: too many variables
• A lot more

Other:

• Process metrics:
  • Team velocity
    • Story points completed per unit time
  • Burn down/up chart
  • Code churn
    • How often methods/classes get modified
    • High churn:
      • Dependents require updating as well
      • Complex method that is not quite worked out
        • KISS - simple is hard
      • May lead to design erosion
  • App crash rate
  • Lead time
    • Time taken from PO request to finished feature
  • Active days
    • Time taken from picking up feature to finishing it
  • Mean time between failures (MTBF)
  • Mean time to recover/repair from failures (MTTR)

Refactoring

Refactoring will TODO

When to refactor?

• During development
  • During: start clean
  • After: to clean up
  • When fixing a bug
• Code reviews
• The rule of three; duplication and speculative generality

TODO

• Check metrics
• TODO
• Check metrics - have they imrpoved

Corerectness:

• Formal: prove semantics and correctness of program transformation
• Implementation: unit/regression tests - ensure the implementations meet specifications

Rewrites:

• TODO

Reengineering:

TODO