For sharing my view on stuff and ideas which at least I believe are worthwhile

Documentation aversion

Working software over documentation is fine but ‘only’ working software and no documentation is not a good idea, especially in case of a business process driven applications. It may be acceptable to go easy on technical documentation but there is no justification for not creating and maintaining business requirement documents. Absence of documented business rules and processes creates an enormous amount of confusion, application is hard to maintain, it’s difficult to communicate changes and reverse engineering code to drive business rules is highly unproductive and wasteful process.

Simulated uncertainty

I believe the two primary notions that give agile edge over waterfall methodology are

  • Built in process that encourages frequent and continuous review by end users
  • Ability to keep everybody on toes with daily stand-ups and rapid deliveries

Waterfall assumed that it’s possible to plan everything upfront, agile goes the opposite way and assumes that things are always changing so it’s better not to plan. While frequent changes may be possible in some projects but it’s not always the case, specifically for the projects that are business process centric.

While agile acknowledges the importance of bigger picture during initial project estimation, preliminary release plan, backlog prioritization, technology selection and architecture, it looks the other way when it comes to application design and identification of reusable components. I agree certain projects could have uncertain requirements and putting too much effort in designing would be a waste of time and energy but there are projects in which requirements tend toward a level of certainty.

I feel with agile methodologies like scrum considerable amount of effort goes in refactoring the design/code each sprint because we tried to do just enough or simplest thing possible in one of the previous sprint refusing to acknowledge the bigger picture. Lot of this effort could be saved, if depending on the context of project more time and effort is devoted towards design when project starts. Dedicated design sprint in the beginning may not be a bad idea!

Fixed amount + Fixed deadline + Fixed scope + Change friendly = Team burnout

In most of the scenarios before committing to a project, customer is primarily interested in getting estimated cost with reasonable accuracy for given requirements. The calculation of cost for executing a project is derived with some calculation based on team size and timelines. The customer is confident that cost more or less covers the defined requirements and since project will be executed with agile so change would not be an issue! It’s quite possible that sales team of an organization may further enforce this perception if not deny it.

So now we have a project with fixed cost which was calculated on basis of a fixed team size and timelines. Customer’s perception is that most of the requirements are in scope because cost was estimated on basis of these requirements, so generally customer would be averse to any change in scope. Customer may be open to possibility of varying team size and timelines but any changes to these factors would have a direct impact on the margins of the project so not a good option for company executing the project.

To top it all customer holds the perception that since it’s an agile project changes would not be an issue, process will accommodate them magically when requested. Now imagine being a member of the team who is caught in this situation with frugal initial estimates! While I acknowledge that with waterfall a project runs bigger risk of failure but team is still protected from frequent changes, also customer understands change averse nature of waterfall, so it may be a bit easier for the team to handle the situation.

I personally do not think fixed bid projects fit well with the philosophy of agile, but since they are an uneasy reality we should go easy on selling the ‘change embracing’ attitude of agile and make sure that customer understands that even with Agile change has a cost, you put something in something needs to be pushed out!

primarily

This post describes a simple framework that can be used for implementing breadcrumbs in a Grails application. lets get started…

The three main components of this framework are:

  • Breadcrumb configuration XML which is used to defined breadcrumb hierarchy and attributes.
  • Breadcrumb manager which is used for parsing/loading XML configuration file, finding correct breadcrumb for the page and in general managing breadcrumb implementation
  • Custom tag <g:breadCrumb> which is used to embedded breadcrumbs on pages.

Lets look at breadcrumbs.xml first, I placed this XML in WEB-INF folder of the application right next to applicationContext.xml

breadcrumbs.xml

<!-- Bread crumb configuration XML -->
<breadcrumbs>
	<!-- Mapping is used to define the navigation structure for a bread crumb, it is based on the site map.
		 matchController and matchAction are used to figure location of requested page on the site by matching
		 the controller/action of the request to matchController/matchAction combination. Once a match is found
		 bread crumb hierarchy is resolved and information crumbs tag is used to get details related to each
		 bread crumb. All attributes in <nav> tag are mandatory
	-->
	<map>
		<nav id="homeCrumb" matchController="samplePages" matchAction="homeBreadCrumbPage">
			<!-- levels navigation -->
			<nav id="itemsLevel1Crumb" matchController="samplePages" matchAction="level1BreadCrumbPage">
				<nav id="itemsLevel2Crumb" matchController="samplePages" matchAction="level2BreadCrumbPage">
					<nav id="itemsLevel3Crumb" matchController="samplePages" matchAction="level3BreadCrumbPage">
						<nav id="showItemCrumb" matchController="samplePages" matchAction="itemDetailsBreadCrumbPage"/>
					</nav>
				</nav>
			</nav>
			<nav id="simple1Crumb" matchController="samplePages" matchAction="simpleBreadCrumb"/>
			<nav id="simple2Crumb" matchController="samplePages" matchAction="simpleBreadCrumbWithAttr"/>
			<!-- levels navigation -->
		</nav>
	</map>

		<!-- Information in <crumb> tag is used to figure out detail of individual bread crumb, the navigation bread crumb
			 and details are matched using id attribute. If for a 'id' in navigation map no details bread crumb id is found
			 then the 'id' is ignored.

			  - Optional viewController and viewAction attributes in XML should be supplied together, if any one of them
			  is missing other will be ignored.

			  - viewController/viewAction combination has a higher precedence than viewTemplate, viewTemplate attribute
			  will be ignored if both are present

			  - linkToController/linkToAction combination has a higher precedent than link, link attribute  will be ignored
			  if both are present

			  - Optional linkToController and linkToAction attributes in XML should be supplied together, if any one of them
			  is missing other will be ignored

			  - If rendering of a bread crumb is delegated to action/template then bread crumb title needs to be generated by
				renderer

			  - Optional parameters can be supplied for each id in JSON format i.e. params = { foo: 'bar' , key: 'value'}

			  - Requested page is given an opportunity to define the title/link if the bread crumb by supplying title/link attribute
			    If these are supplied then configured title/link in XML are overridden
		-->
	<crumbs>
		<crumb id="homeCrumb" title="Home" linkToController="samplePages" linkToAction="homeBreadCrumbPage" />
		<crumb id="itemsLevel1Crumb" viewController="breadCrumb" viewAction="renderLevel1Crumb" />
		<crumb id="itemsLevel2Crumb" viewController="breadCrumb" viewAction="renderLevel2Crumb" />
		<crumb id="itemsLevel3Crumb" viewController="breadCrumb" viewAction="renderLevel3Crumb" />
		<crumb id="showItemCrumb" />
		<crumb id="simple1Crumb" title="Simple1"/>
		<crumb id="simple2Crumb"/>
	</crumbs>
</breadcrumbs>

XML file is divided into two sections, first section is used to define hierarchical relationship between the views for which we need to display breadcrumbs. For each element we define matchController/matchAction attributes which are used by our custom tag to figure out its position in breadcrumb hierarchy based on current page. We also define unique ‘id’ attribute for each element, this is used in second section of XML to define properties used while rendering the breadcrumb. Please go though the comments in the XML for more details.

Now lets look at elements that provide breadcrumb manager implementation

The first code element is BreadCrumb.groovy which provides an object for holding properties loaded from XML for each breadcrumb

BreadCrumb.groovy

package org.breadcrumbs

import org.apache.commons.lang.builder.EqualsBuilder
import org.apache.commons.lang.builder.HashCodeBuilder

/**
 * class for handling BreadCrumbs
 * 
 */
class BreadCrumb {
	
	// mandatory bread crumb identifier, checked while parsing XML
	String id
	
	// Title to be used for the Bread Crumb (optional)
	String title
	
	// Controller to be used to render the view (optional)
	String viewController
	
	// Action to be used to render the view (optional)
	String viewAction
	
	// If controller/action is not required, only template can 
	// be supplied for rendering the view (optional)
	String viewTemplate
	
	// link that needs to attached to bread crumb title (optional)
	String link
	
	// Controller that should be used to create link for title (optional)
	String linkToController
	
	// Action that should be used to create link for title (optional)
	String linkToAction
	
	// crumb parameters map, loaded from XML configuration
	def params 
	
	/*
	 * Create a copy of bread crumb instance
	 */
	def getCopy () {
		def paramList = [:]
		for ( e in params ) 
			paramList.put (e.key, e.value)
                 
		return new BreadCrumb(title: this.title, link: this.link, 
				viewController: this.viewController, viewAction: this.viewAction, viewTemplate: this.viewTemplate,
				linkToController: this.linkToController, linkToAction: this.linkToAction, params: paramList)
	}
	
	@Override
	int hashCode () { 
		return new HashCodeBuilder().append(this.id).toHashCode() 
	}
	
	@Override
	boolean equals (Object other) { 
		if(other?.is(this)) 
			return true 
		if(!(other instanceof BreadCrumb)) 
			return false 
		
		return new EqualsBuilder().append(this.id, other.id).isEquals() 
	}
}

Second element is custom exception for breadcrumbs
BreadCrumbException.groovy

package org.breadcrumbs

/**
 * class to handle bread crumb Exceptions 
 */
class BreadCrumbException extends Exception  {
    
	BreadCrumbException() {
		super();
	}
	
	BreadCrumbException(Exception e) {
		super(e);
	}
	
	BreadCrumbException (String text) {
		super (text);
	}
}

Third and last element is BreadCrumbManager.groovy which provides lazy loaded singelton instance as breadcrumb manager
BreadCrumbManager.groovy

package org.breadcrumbs

import org.apache.commons.logging.LogFactory
import grails.converters.XML
import org.codehaus.groovy.grails.commons.ApplicationHolder
import grails.converters.*
import org.codehaus.groovy.grails.web.json.*
import org.codehaus.groovy.grails.web.converters.exceptions.ConverterException

import org.breadcrumbs.BreadCrumbException


/**
 * Singleton class for managing BreadCrumbs
 */
class BreadCrumbManager {
	
	private static def log = LogFactory.getLog("grails.app.BreadCrumbManager")
	
	private static def NAVIGATION_TAG = "nav"
	private static def CRUMB_DETAIL_TAG = "crumb"
	private static def CRUMB_DELIMITER = "."
	private static def ALL_ACTION_WILDCARD = "*"	
	private static def XML_FILE_PATH = "WEB-INF/breadcrumbs.xml" 
	
	// instance is not lazy loading because bread crumbs are required on all pages
	private static BreadCrumbManager manager = new BreadCrumbManager()
	
	// Map stores bread crumb navigation id's for matchers
	private def matcherToCrumbIdsMap 
	// Map stores bread crumb instance for each bread crumb id
	private def crumbIdDetailMap                           
		
	/**
	 * private constructor for creating BreadCrumbManager instance
	 */
	private BreadCrumbManager() {
		matcherToCrumbIdsMap = [:]
		crumbIdDetailMap = [:]
		// load navigation maps and bread crumb attributes from XML instances
		loadBreadCrumbsFromXML()
	}
	
	/**
	 * Return existing BreadCrumb manager instance 
	 */
	static BreadCrumbManager getInstance() {
		return manager
	}
	
	/**
	 * Match and retrieve bread crumbs navigation list 
	 * 
	 * @param  controller: controller to be matched
	 * @param  action: action to be matched
	 * 
	 * @return list of applicable bread crumb instances
	 */
	def getBreadCrumbs (def controller, def action) {
		
		def key = getMatcherKey(controller, action)
		def crumbs = matcherToCrumbIdsMap.get(key)
		
		if (!crumbs) {
			// if id not for for specific controller/action combination, try to get
			// an id applicable to all actions of a controller
			def alternatekey = getMatcherKey(controller, ALL_ACTION_WILDCARD)
			crumbs = matcherToCrumbIdsMap.get(alternatekey)
		}
		
		// log if still not found
		if (!crumbs) {
			log.warn "No matching crumbs for key ${key}" 
		} else {
			log.debug "Matching crumbs for key ${key} : ${crumbs}" 
			
			def crumbArray = crumbs.split('\\.')
			def crumbList = []
			crumbArray.each { crumb ->
				def crumbObj = crumbIdDetailMap.get(crumb)
				
				if (crumbObj) {
					// create a copy of original crumb object and add to list
					crumbList.add (crumbObj.getCopy())
				} else {
					log.info "Crumb details not found for '${crumb}' "
				}
			}
			return crumbList
		}
	}
	
	/**
	 * Construct a matcher key from controller and action  
	 * 
	 * @param  controller: controller to be matched
	 * @param  action: action to be matched
	 * 
	 * @return matcher string
	 */
	def getMatcherKey(def controller, def action) {
		if (controller && action)
			return "${controller.toLowerCase()}/${action.toLowerCase()}"
	}
	
	/**
	 * load navigation map and crumb details from XML  
	 * 
	 */
	private loadBreadCrumbsFromXML () {
		// get application context and resource
		def file = ApplicationHolder.application.getParentContext().getResource(XML_FILE_PATH).getFile()  
		log.debug "Loading breadcrumbs from file " + file.getAbsolutePath()  
		
		def xml = new XmlParser().parse(file)
		xml.depthFirst().each { node ->
			
			if (node.name() == NAVIGATION_TAG) {
					// throw exception if any of the required attributes are missing
					if (!node.@id || !node.@matchController || !node.@matchAction) {
						throw new BreadCrumbException("Required attribute for navigation missing, please validate XML ")
					}	
					
					def key = getMatcherKey(node.@matchController, node.@matchAction)
					
					if (matcherToCrumbIdsMap.get(key)) {
						throw new BreadCrumbException("Duplicate matcher key found: ${key}")
					} else {
						def p = node.parent()
						def crumbs = node.@id
						// move towards top of tree from current node to get crumb hierarchy 	
						while(p?.parent() != null) {
							if (p.name() == NAVIGATION_TAG) {
								crumbs = p.@id + CRUMB_DELIMITER + crumbs
							}
							p = p.parent()
						}
						
						log.debug "Putting matcher key: ${key} for crumb navigation ${crumbs} "
						matcherToCrumbIdsMap.put(key, crumbs.toLowerCase())
					}
			} else {
				if (node.name() == CRUMB_DETAIL_TAG) {
					def id = node.@id
					if (!id) {
						throw new BreadCrumbException("Required attribute 'id' for crumb  missing, please validate XML ")
					}
					
					def crumbParams
					
					if (node.@params) {
						// try parsing params
						try {
								crumbParams = JSON.parse(node.@params) // Parse a JSON param string
						} catch(ConverterException exp) {
							throw new BreadCrumbException("Params: ${node.@params} specified in incorrect format " +
									"for id: '${id}' , please validate XML ")
						}
					}
				
					log.debug "Putting crumb detail for id: ${id}"
					// create a bread crumb object and save it on details map
					crumbIdDetailMap.put(id.toLowerCase(), new BreadCrumb(title: node.@title, link: node.@link, 
							viewController : node.@viewController, viewAction: node.@viewAction, 
							viewTemplate: node.@viewTemplate, linkToController: node.@linkToController,
							linkToAction: node.@linkToAction, params: crumbParams))
				}
			}
		}
	}
}

Now lets have a look at the BreadCrumbTagLib which is defined in grails-app/taglib directory
BreadCrumbTagLib.groovy

package org.breadcrumbs

import org.breadcrumbs.BreadCrumbManager;

import org.apache.commons.lang.StringUtils;
import org.apache.commons.logging.LogFactory; 

class BreadCrumbTagLib {
	
	static def log = LogFactory.getLog("grails.app.breadCrumbTag")
	
	def breadCrumb = { attrs , body ->
	
			def manager = BreadCrumbManager.getInstance()
			def uri = request.getRequestURI() 
			def context = request.getContextPath() 
			
			def controller = params.controller
			def action = params.action
			
			def attrTitle =  attrs.title
			def attrLink = attrs.link
				
			// if controller and action are missing from params try to get them from request url
			if (!controller && !action && uri && context && uri.indexOf(context) != -1) {
				def uriParams = uri.substring(uri.indexOf(context) + (context.length() + 1), uri.length())
				def uriArray = uriParams.split('/')
				
				if (uriArray.size() >= 2 ) {
					controller = uriArray[0]
					action = uriArray[1]                       
				}
			}
			
			def crumbs = manager.getBreadCrumbs(controller, action)
			
			if (crumbs) {
				out << '<div class="breadcrumb"><ul>'
				def size = crumbs.size()
				crumbs.eachWithIndex { crumb, index ->
					out << '<li>'
					
					// override title and link of breadcrumb on current page (i.e. last bread crumb in hierarchy)
					// if name, link attributes are supplied
					if (index == size - 1) {
						if (attrTitle)
							crumb.title = attrTitle
						if (attrLink)
							crumb.link = attrLink
					}
					// set title to undefined if not found, associated 
					// renderer if present can overwrite it
					if (!crumb.title)
						crumb.title = "undefined"
					if (crumb.title && crumb.title.size() > 40)
						crumb.title = crumb.title.substring(0, 40) + "..."
						
					if (crumb.viewController && crumb.viewAction) {
						def content = g.include(controller:crumb.viewController, action:crumb.viewAction, breadcrumb:crumb, params:params)
						out << content
					} else if (crumb.viewTemplate) {
						def content = g.include(view:crumb.viewTemplate, breadcrumb:crumb, params: params)
						out << content
					} else if (crumb.linkToController && crumb.linkToAction && (size - 1 > index)){
						out << "<a href=\"${g.createLink (controller: crumb.linkToController, action: crumb.linkToAction)}\">${crumb.title}</a>"
					// if crumb has a link and its not the last vread crumb then show link else
					// just show the text
					} else if (crumb.link && (size - 1 > index)){
						out << "<a href=\"${crumb.link}\">${crumb.title}</a>"
					} else {
						out << "${crumb.title}"
					}
					out << "</li>"
					// do not print for last bread crumb
					if (size - 1 > index)
						out << "<li>&raquo;</li>"
				}
				out << "</ul></div>"
			}
	}
}

Lets also define basic css that is used for displaying breadcrumbs on pages

breadcrumbs.css


.breadcrumblist_selected {
	color:#a4db2f;
}

.breadcrumblist_selected a {
	text-decoration:none!important;
}

.breadcrumb {
	padding-left:15px; margin-top:5px; width:600px;
}
.breadcrumb ul {
	margin:0; padding:0;
}
.breadcrumb ul li {
	display:inline; font-size:14px; color:#FF9900
}
.breadcrumb ul li a {
	color:#2cbe0c
}

Now lets look at the demo code for breadcrumb implementation
Starting URL for the demo application will be something like http://localhost:8080/sample/samplePages/homeBreadCrumbPage if deployed on localhost on port 8080

Following is the controller code used for displaying sample pages in demo, each action defined in the controller refers to a demo page
samplePagesController.groovy

package com.pages

class samplePagesController {
	
	def homeBreadCrumbPage = { }

	def allItemsBreadCrumbPage = { }

	def level1BreadCrumbPage = { }
	
	def level2BreadCrumbPage = { }
	
	def level3BreadCrumbPage = { }
	
	def itemDetailsBreadCrumbPage = { }
	
	def simpleBreadCrumb = { }
	
	def simpleBreadCrumbWithAttr = { }
}

Lets look at the code for some of the views, code for remaining views can be found in zipped code attached with the post

Simple breadcrumb page includes the css that we defined for breadcrumbs and has a <g:breadCrumb/> tag at a place where breadcrumb needs to be displayed

simpleBreadCrumb.gsp

<html>
	<head>
		<link rel="stylesheet" href="${resource(dir:'css',file:'breadcrumbs.css')}" />
	</head>
  	<body>
	<g:breadCrumb/>
	<h3>Simple breadcrumbs</h3>
	</body>
</html>

The image below how breadcrumb will be shown in running application, The ‘Home’ link is clickable and will take user to page defined in configuration XML, please note that for the displayed page, controller is ‘samplePages’ and action is ‘simpleBreadCrumb’. If you look at first section of configuration XML you will figure out how breadcrumb hierarchy is being displayed

The next sample view that we are going to look at is itemDetailsBreadCrumbPage.gsp

itemDetailsBreadCrumbPage.gsp

<html>
	<head>
		<link rel="stylesheet" href="${resource(dir:'css',file:'breadcrumbs.css')}" />
	</head>
  	<body>
	<g:breadCrumb title='Selected Item'/>
	<h3>Item Details Page</h3>
	</body>
</html>

The image below shows how the breadcrumb will get rendered on runtime

Please note that the drop down displayed for breadcrumbs are clickable, we will shortly discuss how we displayed drop downs instead of text for these breadcrumbs but important point to note here is that you can have complex elements like sliders and popups here instead of drop downs as breadcrumb elements.

OK, lets see how these drop downs were displayed
We have following configuration in breadcrumbs.xml for this page

	<nav id="showItemCrumb" matchController="samplePages" matchAction="itemDetailsBreadCrumbPage"/>

The navigation hierarchy that we get for this page is homeCrumb->itemsLevel1Crumb->itemsLevel2Crumb->itemsLevel3Crumb->showItemCrumb

When we start rendering each breadcrumb in hierarchy and encounter ‘itemsLevel1Crumb’ we know that for this breadcrumb we need to execute controller ‘breadCrumb’ and action ‘renderLevel1Crumb’ to get the content so we invoke it

		<crumb id="itemsLevel1Crumb" viewController="breadCrumb" viewAction="renderLevel1Crumb" />
		<crumb id="itemsLevel2Crumb" viewController="breadCrumb" viewAction="renderLevel2Crumb" />
		<crumb id="itemsLevel3Crumb" viewController="breadCrumb" viewAction="renderLevel3Crumb" />

Following is the source code for controller breadCrumb
breadCrumbController.groovy

package org.breadcrumb

class breadCrumbController {

    def renderLevel1Crumb = { }
	
	def renderLevel2Crumb = { }
	
	def renderLevel3Crumb = { }
}

Following are the contents if renderLevel1Crumb.gsp
renderLevel1Crumb.gsp

<select>
<option>Level1 - A</option>
<option>Level1 - B</option>
<option>Level1 - C</option>
<option>Level1 - D</option>
</select>

Hence the breadcrumb with id ‘itemsLevel1Crumb’ gets rendered into a drop down, similar is the case with other drop down breadcrumbs.

You can download the code here. Note that application was created with grails 1.2.2

Use following URL for running the demo after deploying it on grails (please update host/port configuration)

http://localhost:8080/sample/samplePages/homeBreadCrumbPage

Designing for Dummies

I’ve been involved with software design and analysis for many years now and I still learn something new each day.  Over the years I’ve come across many confused and at times overwhelmed newbies struggling to understand and produce decent designs, this blog is an effort to share some learning with them

1)    Take a plunge
Designing is like swimming, you can’t just read stuff and be good at it! You have to jump in water and practice to be a good swimmer.  Similar is case with design, don’t just get stuck at reading books get working on it, practice and you will get better at it. Once you know your way around things you read will make more sense.

2)    Work full cycle
Try to take up different roles in software life cycle if possible work as a requirement analyst, tester, developer, production support analyst. Each of these roles will give you a different perspective and will make you a better designer.

3)    Understand the functional domain of problem
The functional domain provides all the essential clues required for building a good design, it is important to understand the domain before working on the design. Understandably all information may not be readily available but make a conscious effort to gather knowns.

4)    Work your way Top down
Work in layers, start with domain, build a layer of high level abstractions below it and keep moving in this direction building lower layers to the lowest level. Try to focus on current layer and its interactions with the one below it.

5)    Encapsulate what varies
The golden rule for a good design is ‘encapsulate what varies’ i.e. try to segregate things that change from those which do not change.  Initially try to follow this simple rule only and save yourself from being overwhelmed trying to follow all design rules and principals.

6)    Do not over engineer
At the outset it might seem that lot of things can vary and you need to segregate everything! Fight this urge, your domain would give you a good idea about things that can change, look at the probability of each kind of change, and segregate the ones that have high probability of change.

7)    Let patterns show the way
When you have identified a segregation point, look at the interactions between stable and varying components. Try to figure out and use a design pattern which would provide similar kind of segregation.

8)    Design to interfaces
The stable components should always be designed to interact with the interfaces of varying components rather that the implementations, remember they were encapsulated out because they can change, designing to interfaces provides flexibility required for change.

I’ve been involved with software design and analysis for many years now and I still learn something new each day.  Over the years I’ve come across many confused and at times overwhelmed newbies. This blog is an effort to share few simple tips that can be followed to put forth a decent design.

1) Take a plunge

Designing is like swimming, you can’t just read stuff and be good at it! You have to take a plunge and practice to be a good swimmer.  Similarly don’t just get stuck at reading books get working on it, practice and you will get better at it. Once you know your way around things you read will make more sense.

2) Work full cycle

Try to take up different roles in software life cycle, work as a requirement analyst, tester, developer, production support analyst. Each of these roles will give you a different perspective and will make you a better designer.

3) Understand the functional domain

The functional domain provides all the essential clues required for building a good design, it is important to understand the domain before working on the design. Understandably all information may not be readily available but make a conscious effort to gather known information.

4) Work your way Top down

Work in layers, start with domain, build a layer of high level abstractions below it and keep moving in this direction building layers to the lowest level. Try to focus on current layer and its interactions with the one below it.

5) Encapsulate what varies

The golden rule for a good design is ‘encapsulate what varies’ i.e. try to segregate things that change from those which do not change.  Initially try to follow this simple rule only and save yourself from being overwhelmed trying to follow all design rules and principals.

6) Do not over engineer

At the outset it might seem that lot of things can vary and you need to segregate everything! Fight this urge, your domain would give you a good idea about things that can change, look at the probability of each kind of change, and segregate the ones that have high probability of change.

7) Let patterns show the way

When you have identified a segregation point, look at the interactions between stable and varying components. Try to figure out and use a design pattern which would provide similar kind of segregation.

8) Design to interfaces

The stable components should always be designed to interact with the interfaces of varying components rather that the implementations, remember they were encapsulated out because they can change.

After working for considerable period of time with waterfall I recently joined Xebia. I believe it is one of the few organizations which live and breathe agile. From what I’ve seen and learnt so far I’ll say Agile Rocks!!! To me agile is simple no nonsense methodology which is all about common sense. I do not claim to be an expert on Agile or even waterfall, I just want to share what I’ve learned and understood so far working as a guy on the ground.

If somebody asked me to describe core idea of agile in fewest words I would say “frequent inspection”. Everything else in agile methodology is laid around this simple yet powerful notion. I’ve worked on lot of waterfall projects and fortunately none of them failed, all of them had a varied amount of success. For some it was smooth ride to production and others barely made it through. From last so many years I have heard and read about abysmal success record of IT projects and have often wondered why do so many of them fail?  I have always held this opinion that building software is just like building anything else… like maybe building a housing project or maybe some piece of furniture. I can already see heads nodding in disagreement! I understand these analogies may not be perfect or a right fit for all kind of software projects but I still feel on a certain level its valid for significant number of them (or at least for the ones that I worked on!). It sounds almost logical that in order to build something, analysis of what needs to be built is required (consider projects in which users have a fairly good idea of what they want). The abstract requirements discovered from analysis are taken to designing table where an effort is made to propose a domain specific solution. Once that’s done and accepted actual work starts to provide something tangible that can be delivered, deliveries go through cycles of construction/inspection/rework and then the final delivery is done.  Of course the events described in sequence above do not have such strict boundaries, but it can generally be acknowledged as high level framework of building and delivering goods.

This high level framework which I believe inspired waterfall methodology has for years worked successfully in various industries for building and delivering goods but has been a disappointment for software industry. Why this tested and tried logical process would not yield the same kind of results when it comes to software?  When I started working with Agile I sort of got answers to these questions and I’ll share those shortly, but before that I’ll like to point out two subtle differences that exist between process of construction in software and other industries for example lets say building construction.

  1. Prospects of inspection by end users
  2. Ability to accommodate change

Prospects of inspection by end users
For a building construction project end users may not fully comprehend or visualize the architectural plans but they can very well understand the progress by inspecting the construction site. They can see the limits of land by looking at outer walls, they may look at bricks laid for foundation to judge the room areas, they can see when walls are erected and plastered, location of stair case etc. With the ability to inspect they can catch anomalies early and request for change. Now compare this with a software project using similar methodology, at each stage of build nothing tangible that can be inspected by end user is available. End user mostly do not understand the design documents, they do not comprehend software design, code or component interactions. The artifacts produced at various stages of software construction do not make any sense to them, since they do not understand they cannot inspect or suggest change! The opportunity of inspection is provided towards the end of the project and often it’s too late by that time to change.

Ability to accommodate change
The notion of being able to change something significantly after it’s fully built is very powerful, software artifacts offer this rare ability to its builders (developers/designers) like no other. Can you imagine plight of construction project where teams unknowingly/knowingly did not lay down water supply pipes in washrooms? The ramification of such a blunder would be huge!  Builders of a construction project cannot afford such mistakes because a mistake like this is not easy to fix, this very fact forces them to be more rigorous.  Software developers on other hand often believe that they do not need to be too rigorous because they can change at any stage.  The ability to change later, time constraints and hassle of clarification with end user often drive developers/designers to take an easy way out. They code with what they know making assumptions, because they can fix it later if assumption was not up to the mark. This behavior can add quite substantially to project debt. Also, it’s certainly not true that consequences of changes to software would be less severe than that of a construction project, but perceived ability and ease with which it can be done can subconsciously drive procrastination in minds of some individuals.

I understand that ability to accommodate change is one of the biggest strengths of software and if done correctly can provide software construction an edge over others. Agile in fact embraces change and uses the ability of software to change as a tool to deliver.  It would be worth noting that point that I made above with regard to software ‘rigor’ should not be considered in violation of agile principal “Do the simplest thing that could possibly work”. By being rigorous I am no way suggesting that one should make baseless assumptions and try to accommodate everything, all I am saying that ability to change should not be misused to defer legitimate stuff.

Enter agile!
I think Agile was born with core idea of enabling frequent inspections of software artifact by end user, everything else in various agile methodologies was layered around this notion. Agile makes sure that artifacts are always in a form that can be inspected by end user, this enables early feedback and failure is detected early.  Waterfall is considered to be change averse because by the time project is ready for feedback from the end user it is so deep into the build that accommodating change is not easy and in certain cases not even possible without considerable time and effort.

Agile also provides a solution to second problem, though the lack of rigor may still exist but most of the issues introduced are detected early and fixed, unlike waterfall where they keep accumulating making a huge unmanageable lump of project debt.  Since Agile is able to recognize and accommodate for these two subtle differences the chances of success with Agile are much better than waterfall.

After a long stint of working with waterfall I recently joined my current organization Xebia Architects which I believe is one of the few organizations which live and breathe agile. From what I’ve seen and learnt so far I’ll say I love agile! To me agile is simple no nonsense methodology which is all about common sense. I do not claim to be an expert on Agile or even waterfall, I just want to blog what I’ve learned and understood so far working as a guy on the ground.

If somebody asked me to describe core idea of agile in fewest words I would say “frequent inspection”. Everything else in agile methodology is laid around this simple yet powerful notion. I’ve worked on lot of waterfall project and fortunately none of them failed, all of them had a varied amount of success. For some it was smooth ride to production and others barely made it through. From last so many years I have heard and read about abysmal success record of IT projects and have often wondered why do so many of them just fail? We in software industry must be doing something wrong.

I guess I have always held this opinion that building software is just like building anything else like maybe building a housing project or maybe some piece of furniture. I can already see heads nodding in disagreement! I understand these analogies may not be perfect or a right fit for all kind of software projects but I still feel on a certain level its valid for significant number of them (or at least for the ones that I worked on!). It sounds almost logical that in order to build something, analysis of what needs to be built is required, also it would be nice to assume that in kind of project I am referring to users have a fairly good idea of what they want. The abstract requirements discovered from analysis are taken to designing table where an effort is made to propose a domain specific solution. Once that’s done and accepted actual work starts to provide something tangible that can be delivered, deliveries go through cycles of construction/inspection/rework and then the final delivery is done.  Of course the events described in sequence above do not have such strict boundaries, but it can generally be acknowledged as high level framework of building and delivering goods.

This high level framework which I believe inspired waterfall methodology has for years worked successfully in various industries for building and delivering goods but has been a disappointment for software industry. Why this tested and tried logical process would not yield the same kind of results when it comes to software?  When I started working with Agile I sort of got answers to these questions and I’ll share those shortly but before that I’ll like to point out two subtle differences that exist between process of construction in software and other industries for example building construction.

1) Prospects of inspection by end users

2) Ability to accommodate change

Prospects of inspection by end users

For a building construction project end users may not fully comprehend or visualize the architectural plans but they can very well understand the progress by inspecting the construction site. They can see the limits of land by looking at outer walls, they look at bricks laid for foundation to judge the room areas, they can see when walls are erected and plastered, location of stair case etc. With the ability to inspect they can catch anomalies early and request for change. Now compare this with a software project using similar methodology, at each stage of build nothing tangible that can be inspected by end user is available. End user mostly do not understand the design documents, they do not comprehend software design, code or component interactions. The artifacts produced at various stages of software construction do not make any sense to them since they do not understand they cannot inspect or suggest change! The opportunity of inspection is provided towards the end of the project and often it’s too late by that time to change.

Ability to accommodate change

The notion of being able to change something significantly after it’s built is very powerful, software artifacts offer this rare ability to its builders (developers/designer) like no other. Can you imagine plight of construction project where teams unknowingly/knowingly did not lay down water supply pipes in master bedroom? The ramification of such a blunder would be huge!  Builders of a construction project cannot afford such mistakes because a mistake like this is not easy to fix, this very fact forces them to be more rigorous.  Software developers on other hand often believe that they do not need to be too rigorous because they can change at any stage.  The ability to change later, time constraints and hassle of clarification with end user often drive developers/designers to take an easy way out. They code with what they know on assumption, because they can fix it later if assumption was not up to the mark. This behavior can add quite substantially to “construction debt” of the project. Also, it’s certainly not true that consequences of changes to software would be less than that of a construction project, but ability and ease with which it can be done subconsciously can drive procrastination in minds of some individuals.

I understand that ability to accommodate change is one of the biggest strengths of software and if done correctly can provide software construction an edge over others. Agile in fact embraces change and uses the ability of software to change as a tool to deliver.  It would be worth noting that point that I made above with regard to software ‘rigor’ should not be considered in violation of agile principal “Do the simplest thing that could possibly work”. By being rigorous I am no way suggesting that one should make baseless assumptions and try to accommodate everything, all I am saying that ability to change should not be misused to defer legitimate stuff.

Enter agile!

I think idea of Agile was born with core idea of enabling frequent inspection of software artifact by end user and everything else in various agile methodologies was layered around this notion. Agile makes sure that artifacts are always in a form that can be inspected by end user, this enables early feedback and failure is detected early.  Waterfall is considered to be change averse because by the time we get feedback from the end user we are so deep into the build that accommodating change is not easy and in some cases not even possible without considerable time and effort.

Agile also provides a solution to second problem, though the lack of rigor may still exist but most of the issues introduced are detected early and fixed, unlike waterfall where they keep accumulating making a huge unmanageable lump of project debt.

The Only Thing That Is Constant Is Change”

Like everything else ecosystem around java professional is also changing and from what I gather future holds bigger changes then we are used to. Experts and veterans are already debating if java is dying, some feel that Java has reached its peak and nothing major can be done with its present language core. Only time will tell what future of java is going to be, but consensus is that whether or not java survives, JVM is here to stay i.e. dominant language of the future will have to support JVM. I personally feel that java is not going anywhere soon and may still be one of dominant language for years to come. In any case java professionals should be aware of uncertainty that java as a language faces in future and should prepare accordingly.

Learn Functional Language

As you may have already observed processors have stopped getting faster, and the number of cores available in a CPU are growing rapidly. In order to utilize a multi core processor, the operating system must be able to recognize multi-threading and the software must have simultaneous multi-threading technology (SMT) written into its code. SMT enables parallel multi-threading wherein the cores are served multi-threaded instructions in parallel. Without SMT the software will only recognize one core so  an average user with multi-core processor will notice the difference in performance if software is SMT aware. So bottom line is that we need to build concurrent software’s to make best use of hardware available to us.

Compilers can’t automatically parallelize imperative/object-oriented code, so programming styles have to change. Algorithmically simple concurrency can be achieved using libraries and extensions to existing languages. For example we can use the concurrent package in Java. But building complex, fine-grained concurrent systems quickly runs into a complexity wall in languages like Java. The answer is functional programming, functional languages like Erlang, Haskell and Scala offer better support for concurrency with features  like actor concurrency, software transactional memory and nested data parallelism. So learn a functional programming language. As a Java programmer, the easiest way to go is learning Scala. Parallelization is going to be huge and knowing how to write scalable and maintainable code for parallel systems will be a big plus.

Follow the Cloud

Another trend that has already gained a level of momentum is Cloud computing. Cloud computing that way I understand it is a virtualized, self monitoring, self healing and automatically configured network of computers managed with awareness of the user load and service level required for each of the applications deployed on it. As the load on an application increases more resources are allocated to it automatically from the network to maintain the service level. As the load decreases the resources are de-allocated automatically. Considering that applications deployed on the network will have different peak times, instead of allocating dedicated hardware to each of the application we can have a shared virtualized hardware infrastructure which translates into cost saving.  This network of computers is called a cloud because users and applications running on the network will not have the awareness of the physical hardware being used to service the request at a given point of time, so from this point of view the network is essentially a cloud of similar machines processing requests. The word “cloud” emphasizes the thinking which considers the whole network as a unit rather than a collection of units or machines.

Cloud development paradigms like distributed computing and scalability are not new to java, synergy between java and cloud computing is evident.  Java EE was designed to be deployed to a distributed environment. Cluster management and extensive monitoring and management capabilities are supported by major application servers. Most of the technological changes for supporting the cloud are focused at a system level and it’s largely unclear how exactly to prepare for this as a java professional, but be sure to watch trends carefully and study accordingly. One emergent impact of cloud is the implementation of persistence services. Some cloud providers, like Google or Amazon, don’t offer support for relational databases. So you either have to live with the map-like structure or try to install a relational database in the cloud. This necessitates some rethinking about how the persistence should be designed and can even impact the design of the user interface.

Chain of Creation

Allow me to propose a new creational design pattern “Chain of Creation”, as the name suggests the pattern is inspired by well known Behavioral pattern “Chain of Responsibility”. The Chain of Creation pattern delegates the task of creating the most suitable instance for processing a set of data to a group of related classes. Each of these classes is unaware of the capabilities of other classes in the chain and is only aware of its own capabilities in handling the data.  The only common link in these classes is the data that is passed between them, the data is passed along until an object which is capable of handling the data is created or the length of chain is exhausted in which case a null is returned.

Applicability

Chain of Creation can be used when

  • Without specifying explicitly you want to create an instance of class from several possible implementing classes based on available data.
  • A class can’t anticipate which kind of object it must create.
  • You want to encapsulate the capability of each class in the class itself so that it can be reused.
  • You want to be able to modify the set of related creational classes that are capable of handling data dynamically at run time.

Consequences of the Chain of Creation

  • The main purpose for this pattern is to reduce dependency between creator and created object.
  • The class made aware of its capabilities and hence can be reused.
  • Any changes to the capabilities of a class does not have impact other classes.

The example below provides a sample implementation for Chain of Creation pattern. The objective is to create a terrain from possible types available and then let the chain of classes provide most suitable vehicle for the terrain.

Terrain.java

package pattern.coc;

public interface Terrain {

	public boolean hasRoads();

	public boolean isRugged();

	public boolean isSandy();

}

City.java

package pattern.coc;

public class City implements Terrain {

	public String getName() {
		return "New Delhi";
	}

	public boolean hasRoads() {
		return true;
	}

	public boolean isRugged() {
		return false;
	}

	public boolean isSandy() {
		return false;
	}
}

Desert.java

package pattern.coc;

public class Desert implements Terrain {

	public String getName() {
		return "Thar Desert";
	}

	public boolean hasRoads() {
		return false;
	}

	public boolean isRugged() {
		return true;
	}

	public boolean isSandy() {
		return true;
	}
}

Forest.java

package pattern.coc;

public class Forest implements Terrain {

	public String getName() {
		return “Sariska Forest”;
	}

	public boolean hasRoads() {
		return false;
	}

	public boolean isRugged() {
		return true;
	}

	public boolean isSandy() {
		return false;
	}
}

Vehicle.java

package pattern.coc;

public interface Vehicle {

	public String getName();

	public void ignition();

	public void accelerate();

	public void brake();

	public void turnOff();
}

Creator.java

package pattern.coc;

public abstract class Creator {

	private Creator successor;

	public Vehicle create(Terrain terrain) {
		if (successor != null)
			return successor.create(terrain);
		else // no suitable creator found
			return null;
	}

	public void setSuccessor(Creator c) {
		this.successor = c;
	}
}

Car.java

package pattern.coc;

public class Car implements Vehicle {

	private static class CarCreator extends Creator {

		public Vehicle create(Terrain terrain) {

			if (!terrain.isRugged() && !terrain.isSandy() && terrain.hasRoads())
				return new Car();
			else
				return super.create(terrain);
		}
	}

	public static Creator getCreator() {
		return new CarCreator();
	}

	public String getName() {
		return "Car";
	}

	public void ignition() {
		System.out.println("Car ignition");
	}

	public void accelerate() {
		System.out.println("Car accelerated");
	}

	public void brake() {
		System.out.println("Car brake");
	}

	public void turnOff() {
		System.out.println("Car engine off");
	}
}

Jeep.java

package pattern.coc;

public class Jeep implements Vehicle {

	private static class JeepCreator extends Creator {

		public Vehicle create(Terrain terrain) {

			if (terrain.isRugged() && !terrain.isSandy() && !terrain.hasRoads())
				return new Jeep();
			else
				return super.create(terrain);
		}
	}

	public static Creator getCreator() {
		return new JeepCreator();
	}

	public String getName() {
		return "Jeep";
	}

	public void ignition() {
		System.out.println("Jeep ignition");
	}

	public void accelerate() {
		System.out.println("Jeep accelerated");
	}

	public void brake() {
		System.out.println("Jeep brake");
	}

	public void turnOff() {
		System.out.println("Jeep engine off");
	}
}

AVT.java

package pattern.coc;

public class AVT implements Vehicle {

	private static class AVTCreator extends Creator {

		public Vehicle create(Terrain terrain) {

		if (terrain.isRugged() && terrain.isSandy() && !terrain.hasRoads())
			return new AVT();
		else
			return super.create(terrain);
		}
	}

	public static Creator getCreator() {
		return new AVTCreator();
	}

	public String getName() {
		return “All-terrain Vehicle”;
	}

	public void ignition() {
		System.out.println(“AVT ignition”);
	}

	public void accelerate() {
		System.out.println(“AVT accelerated”);
	}

	public void brake() {
		System.out.println(“AVT brake”);
	}

	public void turnOff() {
		System.out.println(“AVT engine off”);
	}
}

Client.java

package pattern.coc;

public class Client {

	public static void main(String args[]) {

		Desert terrain = new Desert();
		Creator creator = Car.getCreator();
		Creator jeep = Jeep.getCreator();
		Creator avt = AVT.getCreator();
		creator.setSuccessor(jeep);
		jeep.setSuccessor(avt);

		Vehicle v = creator.create(terrain);
		System.out.println("Driving in " + terrain.getName() + " using " + v.getName());

		v.ignition();
		v.accelerate();
		v.brake();
		v.turnOff();
	}
}

All the years that I’ve spent analyzing and building IT systems have taught me that you just can’t “Leave it for later” without consequences. The “leave for later” happens throughout the project life cycle, a manager can choose to leave an important but hard to get stakeholder or an analyst can leave feasibility analysis of a requirement for later, a developer may choose code a vague requirement with assumption leaving clarification for later. The success probability of project, the amount of slogging required, client satisfaction and number of code bugs are all very closely related to the collective “Leave it for later” attitude of the team.

Anything left at any phase of project may go away for a while but eventually it’s going to come back with vengeance to bite at a time when its least expected. Mostly all of the ‘left stuff’ accumulates during project cycle and strikes near project delivery date because there is no date later than project delivery date!  Generally impact of “Leave for later” is directly proportional to the stage it was left for later. Anything left at the time of requirement has a potential of delaying or failing the project. Anything left at the time of design also has a potential of failing the project and delay but usually team avoids the failure by making last minute design patches, introducing design bugs, slogging or compromising the overall design of the system.  Any “leave it for later” done by developers usually does not have potential to fail a project and is resolved by slogging and introducing bugs.

Lot of project management manuals inculcate that anything should be left for later only if there is a proper justification for it and if done, should be tracked as project risk. My annoyance is specifically focused at “leave for later” at time of requirement and design by some analysts.  Most of these are not raised to manager they are either ignored or swept under rug by making unfounded assumptions. Understandably its difficult, time consuming and somewhat cumbersome process to find all the dots and then joining them together. It’s not always easy or possible to think to the lowest level of detail but it is something very important and best efforts should be applied to get it to a reasonable level.

Believe me even after giving it ones best effort and dealing with all the known issues some will still remain. One just need to be certain that at least all known issues have been taken care of.

Tag Cloud

Follow

Get every new post delivered to your Inbox.