Anthony Broad-Crawford

I recently blogged about how I was looking to participate in solving the social network profile interoperability problem.  I am both proud and excited to say I am now a part of Dataportability Healthcare Task Force.  Currently we are in the preparation phase.  However, look for some white papers beginning relatively soon.

It has been about five weeks since we elevated our fist release of our new platform coded in Ruby on Rails.  Looking back, my teammates and myself couldn’t be happier with making the switch over to Rails.  For the most part, it has continued to amaze us with how quickly we can add new functionality.  Additionally, the open source community has continued to provide valuable plugins for our application.  Between the technology stack and the community, we have been able to accomplish a lot with little time.

However, like with every technology stack, not everything is perfect.  Additionally, we learned as we went, so naturally we made mistakes.  Now, as we get ready to complete development on our second major release for our product, we have begun to incorporate some lessons learned into both our design patterns and coding practices for our team.  As we do this, I thought it would be nice to publish them here.  This would allow other development shops to both learn from our mistakes, and provide additional feedback as to how we might solve them differently.

Since there are several areas for me to talk about within this topic (Active Record, RJS, Plugins, Models, etc), I am going to do this over several posts.  This will allow me to go into detail on each specific area, rather then cram it all into one post.

That being said, I am going to first discuss what we have found out in regards to Active Record.  This has been our biggest pain point, and we have found some great simple solutions, and utilized some amazing open source tools to help us improve performance.

Active Record, and ORM’s in general are amazing to code against.  The simplicity they provide by encapsulating both the relationship, the mapping, and CRUD operations for the data types in your domain will make development easy.  However, they also tend employ a load on demand, and load everything paradigm when interacting with the database.  If you are not careful, these can have some serious performance implications in you application.  For us, these were the two biggest offenders.

Load on Demand

The load on demand problem typically rears its ugly head when used in conjunction of an enumeration.  This is typically referred to within the Rails community as the “N + 1″ problem.  The code snippet below shows an example of how you could easily create it

 <% @registered_users.each do |registered_user| %>
<%= registered_user.degrees.map(&:title).join(', ') %>
<% end %>

In this example, you will see that for each user in the system, we are making a call to retrieve the degrees they have associated.  As the number of users in the systems grows, so will the number of requests to the database.  Over time, this will begin to slow your page down to a crawl.  Thankfully, Rails and Active Record provide a solution for this problem.

For requests like the one listed above, you need to also include the associated degrees when initially retrieving the users from the system.  This way, a join will occur in your database to bring back all the necessary data in a single call, rather then multiple.

 @registered_users = User.find(:all, :conditions => "registered = true", :include => [:degrees] %> 

Keep in mind, that you need to monitor to actually see if the join generated by the include option to your models find method actually improves performance.  Joins prior to ORM implicate performance, its’ no different with an ORM.  So, keep in mind complicated nested includes might not actually have a performance benefit.

Retrieve everything, all the time

Active Record defaults to a retrieve everything paradigm until told otherwise.  Again, like load on demand, this makes development a snap.  A simple find statement retrieves you a fully hydrated object.  However, this can lead to unnecessary data been retrieved from the database.  We found that by leveraging the built in select statement within Active Record usually solves this problem

 @registered_users = User.find(:first, :select => "first_name, last_name") 

Using the select statement will retrieve only the data you specify.  This will allow you to keep the data retrieval as light as possible.  We have since moved to always specifying exactly what data you are retrieving through the select statement.  This speeds up the retrieval and adds a little extra documentation to the find.

Performance Monitoring

Through the course of our development we have found the use of several tools to help monitor our applications performance.  We typically do this two ways, the first being in our local development and test environments on a single page request basis.  The second being aggregate data pulled from our production boxes.

We monitor the single page performance in the development and testing environments using FiveRuns Tuneup application.  This plugin installs a div at the top our your page that upon mouse over will provide diagnostics information on page you are currently viewing.  We use this as our first line of defense for monitoring performance.

Additionally, we use http://github.com/wvanbergen/rails-log-analyzer/wikis to analyze our log files.  This provides aggregate data and will identify your top ten poorly performing pages.  Both have proven invaluable.

Don’t forget the database

At the end of the day, if you don’t let Active Record lull you into forgetting that there is a database out there, you should do fine.  We simply retrieve only what we need with selects and minimize database queries with includes and have seen a tremendous reduction in time spent at the database per page request.

It seems obvious, however, when we were juggling learning a new technology stack, and sitting behind the beautifully simple API’s provided by Active Record, it was easy to let these obvious mistakes enter our code base.  Fortunately, they were easy to fix.

If you have any additional Active Record improvements, suggestions, or points of interest, please post as a comment.

Hiring is a very difficult thing to do.  Finding someone with the right mix of passion, skills, and personality is extremely difficult.  I have found that it is even more difficult when running a start up as your margin for error is absolute zero.  At a start up, the stakes are extremely high, the work demand is extreme,  and you don’t have money or time to make mistakes, especially hiring mistakes.  You need to build and maintain a highly tuned machine.  The impact of hiring someone who cannot pull their weight, has a giant ego, or is only along for a paycheck will be devastating.  I’ve seen it set teams back weeks if not months.

Fortunately, I haven’t experienced anything like that as of yet.  As my team has grown, each new edition has been great. Each person brings missing skill sets, great personality for start up culture, and a passion for our craft and what our company does.  However, I am currently filling three open positions on my team right now.  A developer, a designer, and a tester.

Starting the hiring process again has got me thinking about what I look for specifically in a developer.  These aren’t in any particular order, nor am I discussing any weighting of one vs the other.  Additionally, not having one of these qualities doesn’t instantly remove you from potentially getting hired.  This are just some of the things I look for and why.

Passion

First and foremost, the person must be passionate about what they do.  When hiring a software developer, computer science should be their religion.  I want to know that when they wake up in the morning, this is what they look forward to doing.  Their night stand should be stacked with Computer Science books, and their browser history should show the usual suspects.

This is important because at a start up you work a lot of hours.  I typically tell someone during their interview that joining a start up isn’t a career decision, it is a lifestyle decision.  Currently my team typically averages somewhere between sixty and eight hours a week.  If software development isn’t your passion, it isn’t something you could do morning, noon, and night, you are most likely going to wash out.

Polyglotism

Seeing that the person understands the language that our technology is currently built upon is a plus.  However, I would much rather see a few languages under the persons belt.  Specifically, a good balance of dynamic, static, strong, weak, and functional.

I find this important because it shows the person is most likely stronger across the board.  Additionally, I want to know, if we adapt a newer technology or language, as it is a better fit for a particular job, the person isn’t going to be totally out of their element diving into new and uncharted territory.

Independent Projects

A few years ago, I probably wouldn’t have stressed this as much.  However, after seeing the difference this quality usually makes, it is high on my list.  It could be anywhere from participating in local user groups, to actively developing on an open source project.  I want to see that you actively participate in a broader community of your peers.

Data structures and algorithms

This is an important one, at least for me and my team.  We are currently building a large scale distributed system that deals with very large data sets.  More importantly, because we are a search engine, our ability to slice and dice that data in extremely fast times is critical.  Without have a solid foundation in data structures and algorithms would be an extreme disadvantage.  Typically I look for the usual suspects …

• Graphs
• Trees
• Linked lists
• Advanced Sorts
• Big O

Confidence and Humility

Everyone at a start up has a lot of responsibility on their shoulders.  I need to know that you will have the confidence to tackle very difficult problems and solve them in short time frames.  Additionally, you need to be able to ask for help, collaborate, and communicate.  Finally, you need to check your ego at the door.  Right now we don’t have anyone with the title of “lead”, “senior”, “architect”.  We are all engineers working together towards a common goal.

Diversity

It’s important to be more then just a development rock star.  This is important because of where we our in our start up.  As we grow, specialization will be more important.  However, right now, we need more then just a developer.  Having experience in development, infrastructure, databases, and security is incredibly beneficial.  Obviously finding someone with all of those is extremely rare.  Typically I look for a few of those skills in a candidate.  Specifically, where the team is currently lacking. Right now, having someone who has a strong back ground is security would be a major plus as the other categories are currently covered by someone on the team.

Wrap up

That’s really about it.  If I think of anything else I will update the post.  If anyone else has qualities they look for, please feel free to leave a comment.  Additionally, if you are looking to join a start up that is working with Ruby on Rails and building a professional networking platform and search engine for the medical community, please feel free to contact me.  My email address is anthony [ at ] within3 [dot] com.