How can I ensure that I have merged all my changes into my branches?

I’ve been working on a project recently that includes a few developers, a lot of code, and multiple different code branches.  The code has quick release cycles, and the problem I run into more often than not is pushing releases out the door that are missing bug fixes that have been committed to trunk, but have inadvertently not made it into the release branch.  For me, this is made more difficult because of the speed of development, multiple developers and multiple different branches to merge the fixes into.  This is most likely a common problem, with a myriad of different process based solutions, but for me, it’s got to be quick, and it has to not impede my frantic work flow.

In general, solutions that work to help me have to be largely transparent and heavily automated.  That is why I came up with an idea to solve this problem of forgetting to merge in critical pieces of code that have been committed to trunk.  Here is an example layout of a source tree:

trunk/
branches/product-version-1.0
branches/product-version-2.0
branches/product-version-3.0
branches/product-version-4.0
branches/product-version-5.0
branches/FeatureDevelopment

In this example situation we have a stable trunk, and a branch dedicated to each major version of the product as well as one branch used to perform feature development. The feature development branch,  in some cases, is not stable. When either bug fixes, or features are ready to be committed they are done so in logical units that are well tested and documented with commit messages. Then comes the task of merging all of those commits to all of the other branches that need the updated code.  How does one keep track of all of those commits by multiple developers working on the project, and ensure all of them make it in?

The Solution

In my case I use Basecamp heavily to manage Project To-Do’s, Documents, etc.  Because of this my solution makes use of Basecamp to create a To-Do each time code is commited as a mechanism to remind me before I do a release to ensure all of the code has been merged.  Because I have limited time and energy to manually create To-Do’s each time I commit something, or tell all of my developers to do the same, this has to be automated.  Luckily Basecamp comes with a handy Basecamp API.  Since I don’t feel like using curl to make all of my requests and handle responses myself I’ll make use of their Ruby wrapper to do the XML preparation and HTTP response handling for me.

With this wrapper script I will be able to create my own script that will take the necessary information from the SVN commit message and create a To-Do.  The next question is how to make sure this script is called automatically when code is committed to SVN.  Enter the concept of the SVN post commit hook.  If you’ve ever created an SVN repository or inherited one you’ve probably seen the hooks directory.  To learn more about it take a look at the Hook Scripts section here.  Basically every time a commit is made to the SVN repository, it will invoke a script.  In that script we will invoke our ruby script that will create the To-Do in Basecamp.

So, now we have our Basecamp API wrapper script, we know we’re going to add to SVN’s post commit hook script to call it, now we need to know how we’re going to get the information out of SVN as it relates to the commit message, and possibly who the committer was.  That is where we’ll use the svnlook utility.  Svnlook will give us a lot of information relating to a specific revision.  Here is an example of some of the information that can be gleaned from svnlook: author, changed, date, diff, dirs-changed, info, log.  That is just a subset of the options available to use with svnlook, but are the most pertinent in this case.

Now comes the actual script itself that will use svnlook and the Basecamp API to create the To-Do.  In my case, I want to only create a To-Do if I’m not merging code, and my continuous integration engine Continuum is not building a release for me.  The code is pasted below.

The Code

#!env ruby
#
#  add_commit_todo.rb
#  AddCommitTodo
#
#  Copyright 2010 Paul Codding
#  All rights reserved.
#
#  Released under the BSD license.

require File.dirname(__FILE__) + '/basecamp'
require 'time'

class AddCommitTodo
  PROJECT_NAME = "Your Project"
  TODO_LIST_NAME = "Your To-Do List that could be named 'Commits To Be Merged'"
  BASECAMP_URL = "yourcompany.basecamphq.com"
  BASECAMP_USER = "yourusername"
  BASECAMP_PASSWORD = "yourpassword"
  BASECAMP_USE_SSL = true
  SVNLOOK_PATH = "/usr/bin/svnlook"

  def initialize(repo_path, revision)
    if (repo_path && revision)
      @repo_path = repo_path
      @revision = revision
      @session = Basecamp.establish_connection!(BASECAMP_URL, BASECAMP_USER, BASECAMP_PASSWORD, BASECAMP_USE_SSL)
    else
      usage()
      exit(1)
    end
  end

  def usage
    warn "Please specify the repo path and the revision... e.g. 'ruby add_commit_todo.rb /home/paul/svn/project 100'"
  end

  def add_todo
    # Determine the To-Do content using svnlook
    svn_log = `#{SVNLOOK_PATH} log #{@repo_path} --revision #{@revision}`
    svn_log.strip!
    puts "[D] SVN Log output for revision: #{@revision} - '#{svn_log}'"
    if svn_log.include?("erge") || svn_log.include?("maven")
      puts "[!] Not adding To-Do as this is a Merge operation"
    else
      # Find the correct project
      Basecamp::Project.find(:all).each { |project| project.name.eql?(PROJECT_NAME) ? @project = project : nil }
      Basecamp::TodoList.all(@project.id, false).each { |todo_list| todo_list.name.eql?(TODO_LIST_NAME) ? @todo_list = todo_list : nil }
      puts "[*] Adding merge To-Do to list: '#{@todo_list.name} (#{@todo_list.id})'"
      @todo_item = Basecamp::TodoItem.new(:todo_list_id => @todo_list.id)
      @todo_item.content = "[ r#{@revision} ] - #{svn_log}"
      @todo_item.responsible_party = "c#{@project.company.id}"
      @todo_item.notify = true
      puts "[*] Saving To-Do Item with content: '#{svn_log}'"
      @todo_item.save
    end
  end
end
act = AddCommitTodo.new(ARGV[0], ARGV[1])
act.add_todo

The script can be downloaded here.

Ways to Improve

There are several things that could be added into this code, like the ability to automatically close out To-Do’s as the code is merged, or also create a Message in Basecamp to notify the team when bug fixes have been merged, etc.  There are also a number of other examples of Basecamp related post commit hooks here and here.

#!/usr/bin/ruby
#
#  connection_notifier.rb
#  ConnectionNotifier
#
#  Copyright 2009 Paul Codding
#  All rights reserved.
#
#  Released under the BSD license.
require 'rubygems'
require File.dirname(__FILE__) + '/Growl'

class Connection
  include Comparable
  attr_accessor :protocol, :local_ip, :remote_ip, :local_port, :remote_port, :id

  def initialize(connection_entry)
    pieces = connection_entry.split(" ")
    @protocol = pieces[0]
    @local_ip = pieces[3].split(".")[0..3].join(".")
    @remote_ip = pieces[4].split(".")[0..3].join(".")
    @local_port = pieces[3].split(".").last.to_i
    @remote_port = pieces[4].split(".").last.to_i
    @id = pieces[4].split(".").join()
  end

  def (other_connection)
    @id  other_connection.id
  end

  def to_s
    "#{@protocol} connection established from #{@remote_ip} to port #{@local_port}"
  end
end

class ConnectionNotifier
  GROWL_APP_NAME="Connection Notifier"

  def initialize
    @ports = Array.new(1024)
    @ports.fill { |port| port += 1}
    @connections = Array.new
    @growl = GrowlNotifier.new(GROWL_APP_NAME,['Ruby Connection Notifier'],nil,
              OSX::NSWorkspace.sharedWorkspace().iconForFileType_('rb'))
    @growl.register()

    # Pre-populate connections array so we're not automatically notified about
    # existing established connections
    check_for_new_connections(false)
    build_list_of_listening_ports()
  end

  # Poll for changes in netstat output
  def poll
    while true do
      sleep(2)
      check_for_new_connections(true)
    end
  end

  # Check for a new connection in the output of netstat
  def check_for_new_connections(notify)
    netstat_output = `netstat -na`
    for connection_entry in netstat_output do
      if connection_entry.include?("ESTABLISHED")
        connection = Connection.new(connection_entry)

        if connection.local_port != nil && @ports.include?(connection.local_port.to_i) \
          && !@connections.include?(connection)
          @connections << connection
          if (notify)
            @growl.notify('Ruby Connection Notifier', 'Connection Established',
              connection.to_s)
          end
        end
      end
    end
  end

  def build_list_of_listening_ports
    netstat_output = `netstat -na`
    for connection_entry in netstat_output do
      if connection_entry.include?("LISTEN")
        connection = Connection.new(connection_entry)
        if !@ports.include?(connection.local_port)
          @ports << connection.local_port
        end
      end
    end
  end
end

conn = ConnectionNotifier.new
conn.poll

I’ve created a launch agent configuration for the script as well so it can start when you login. The configuration and source code is located in the ConnectionNotifier.zip file.

To run the script, either execute the script directly, or place the com.paulcodding.connection_notifier.agent.plist file in your ~/Library/LaunchAgents directory and log in again. Just note that you need to update the path to the script within the agent file before it will work. Once the script is running and you attempt a connection from an external machine, or the localhost you should see a growl notification like the image below.

Growl Notification

If you are using lazy loading rather than eager loading for obvious reasons, and you’re trying to access associated objects in your view code, this error can be a bit difficult to resolve. Luckily there is the OpenSessionInViewInterceptor, or the OpenSessionInViewFilter. These solutions will keep the hibernate session open long enough for the view to render what is needed before it is closed, thus allowing you to access your lazy loaded associations without raising an exception. The detail of what this functionality provides can be found on the Hibernate website here.

Now, how does one get this to work in their application? Well, I’ve chosen to illustrate the basic steps necessary to implement the OpenSessionInViewInterceptor. You will need to modify 2 files. These being your applicationContext.xml, and YourApplication-servlet.xml files. We’ll start with the applicationContext.xml file. You will need to place the following snippet after you’ve defined your sessionFactory bean.

<bean id="openSessionInViewInterceptor"
    class="org.springframework.orm.hibernate3.support.OpenSessionInViewInterceptor">
    <property name="sessionFactory">
        <ref local="sessionFactory"/>
    </property>
    <property> name="flushModeName">
        <value>FLUSH_AUTO</value>
    </property>
</bean>

What does the FLUSH_AUTO do? Here is a quote from the javadoc for the interceptor:

“This interceptor will by default not flush the Hibernate Session, with the flush mode being set to FlushMode.NEVER. It assumes that it will be used in combination with service layer transactions that handle the flushing: the active transaction manager will temporarily change the flush mode to FlushMode.AUTO during a read-write transaction, with the flush mode reset to FlushMode.NEVER at the end of each transaction.”

We use the FLUSH_AUTO to automatically flush the session in this case because I am not using transactions.
Now comes the edit to your YourApplication-servlet.xml file, or whatever file you’ve used to define your URL Mappings. Add the following snippet to your URL mapping bean.

<property name="interceptors">
  <list>
    <ref bean="openSessionInViewInterceptor" />
  </list>
</property>

And that’s that, redeploy and you will no longer see that dreaded exception.

Sitemesh Integration

What do you have to do to integrate Sitemesh with your newly created Spring application? Well, here is the list of prerequisites:

• Download Sitemesh from OpenSymphony.
• Build the sitemesh-2.x.jar using ant.
• Copy the sitemesh-2.x.jar to the WEB-INF/lib directory of your web application

Now we can move on to integrating Sitemesh into your web application. To use sitemesh with your application only 1 file needs to be edited amazingly enough. Open up your web.xml file and add this snippet:

<!-- Sitemesh -->
<filter>
	<filter-name>sitemesh</filter-name>
	<filter-class>
		com.opensymphony.module.sitemesh.filter.PageFilter
	</filter-class>
</filter>

<filter-mapping>
	<filter-name>sitemesh</filter-name>
	<url-pattern>/*</url-pattern>
</filter-mapping>

So, now what? The next step is to setup the directory structure and files necessary to make things work. Refer to this page on the OpenSymphony site detailing the creation of your first decorator, and creating the WEB-INF/decorators.xml file. I’ve included a copy of my main.jsp file for reference:

<%@ taglib uri="http://www.opensymphony.com/sitemesh/decorator"
	prefix="decorator"%>
<%@ taglib prefix="spring" uri="http://www.springframework.org/tags"%>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
	<head>
		<title><spring:message code="application.title" /></title>
		<decorator:head />
		<%@ include file="/WEB-INF/view/includes/style.jsp"%>
		<%@ include file="/WEB-INF/view/includes/js.jsp"%>
	</head>
	<body>
		<div id="container">
			<%@ include file="/WEB-INF/view/includes/header.jsp"%>
			<div id="internalWrapper">
				<%@ include file="/WEB-INF/view/includes/links.jsp"%>
				<div id="content">
					<decorator:body />
				</div>
			</div>
			<%@ include file="/WEB-INF/view/includes/footer.jsp"%>
		</div>
	</body>
</html>

So, how do you get your Controllers to spit out decorated pages? Easy, just create a jsp that will be used as the content of the <decorator:body /> tag in the main.jsp file. When you’re returning your ModelAndView just refer to that jsp like the example below:

public ModelAndView list(HttpServletRequest request,
			HttpServletResponse response) throws Exception {
	List<DomainObject> userList = userService.findAll();
	request.setAttribute("userList", userList);
	return new ModelAndView("list.jsp", null);
}

Here is an example snippet of my url mapping for reference as to where the jsp is located:

<prop key="/users/*.html">userController</prop>

In this case my list.jsp is located in the users folder under my web root.

So, isn’t this easier than Tiles?

The first project involves the use of Growl, the notification system written for OS X, and snort. Snort is a great little IDS that I’ve been working with for quite some time. Ever since I checked out growl, I’ve been really excited to get something going that utilizes it. Luckily the project with snort is perfect for it. In short the project will use ruby to monitor the snort IDS alert logs for alerts and use Growl to send a notification to the user. Growl does have ruby bindings…hence the choice of ruby. I could use Objective-C, the only problem with that is that I don’t know the language . So, ruby it is. I will keep everyone posted on the progress/caveats.