Building Dev Teams: A Panel Discussion

September 13, 2016 Business

My friend Sumeet Jain put together a cool panel discussion about building development teams.

A lot of talk about hiring and training developers, and building tech company culture, is trite. Or sometimes it's completely contrarian. Some say you should hire only people you'd want to hang out with. Others say you shouldn't care about hanging out with them at all.

Sumeet wanted to find a middle ground, where there is real debate to be had on what makes teams productive and fun to be on.

Instead of trying to ask questions, he came up with six statements and asked the panelists to respond to them. The idea was to encourage opinions and controversy.

I'll present each statement and summarize my notes from the ensuing arguments. The panelists didn't disagree as much as Sumeet may have hoped, but the format led to a deeper discussion.

Note: anything in quotes is paraphrased, not verbatim.

Statement 1: Reviewing a candidate's GitHub profile is impractical, as no one is going to spend enough time reading their code to get a meaningful impression.

Kevin Berry, the technical crew chief at motorsports startup RaceNote: "I look at GitHub for 30 seconds, give or take. Instead of looking at the code, I try to get a feel for a person's projects and languages."

There are massively diminishing returns to looking at GitHub profiles. While there's an obvious and visible difference between the profiles of a major open source contributor and a recent graduate, it would take an enormous amount of time to parse out differences between two similar candidates.

If GitHub is impractical, what strategies do work for quickly sussing out a candidate's potential?

The highest efficacy comes from the referral, according to Berry. When he hears about a candidate from a trusted source, there's very little vetting he needs to do.

Rod Smith, a partner at the development agency Volano Solutions, was adamant that cover letters, or their equivalents, are much better indicators than code, especially for would-be consultants. (Emails that come with resumes attached are a modern form of the cover letter, he said, and even better is a face-to-face pitch.)

Raw tech ability can be taught to a greater degree than can relational skills. For an agency, where a developer is constantly working with clients to determine requirements and evaluate solutions, an in-person intro or a great email message are better bets than a carefully-crafted GitHub portfolio.

Statement 2: Programming puzzle questions are only useful if the interviewer is thoroughly trained in administering them–often these questions do more harm than good.

Yes. "Tricks are stupid."

That's not to say there is no code-based thinking taking place in their interviews.

Tony Noecker, CTO at hosting startup Flywheel, likes having developers walk through a problem out loud, so he can get a sense of how they think.

Berry went further: he doesn't mind imposing stressful situations on his candidates to see how they'll respond. (After all, every developer will be under fire on the job at one point or another.)

Richart admitted running candidates through the infamous FizzBuzz puzzle, but he emphasized that this is more of a weed-out activity. He stressed that writing code on the spot is never a make-or-break activity.

Smith likes to find out whether candidates can troubleshoot. Although this is a notoriously difficult skill to test in a contrived environment, it's not going to be any easier without some kind of coding exercise.

When in doubt, it's best to assign a small project, resources permitting. Certainly evaluating a custom piece of code will go much further than parsing a GitHub profile.

Richart, with the perspective of the only panelist at an old-guard company, seemed pleased that "reverence is fading" for the programming puzzle.

Statement 3: Quality and clarity of code–including up-to-date documentation, tests, and application setup instructions–have a direct impact on a team's ability to hire, as only the most experienced developers will be able to navigate a poorly maintained and documented codebase.

Here's where some disagreement began to surface. A few panelists thought this statement was oversimplified, yet with a kernel of truth.

For example, as the statement itself implies, the situation is different for experienced and emerging developers. For the former, the expectation is that she'll have no trouble diving into the deep end.

When hiring a junior developer, it's best to show them to clusters of low-hanging fruit in the codebase. Picking off bugs, perhaps, or writing the missing docs and tests for a stable module.

Berry and Noecker, the two startup co-founders, said that the assumptions in this statement are unrealistic. In a fast-moving environment, documentation can be an unaffordable luxury.

However, what you can have is readable code, hopefully, and strong mentorship channels, something that RaceNote and Flywheel are known for.

Berry posed the question, "are lives at stake?" Referring to Richart's Air Force contracting work, he said that for most hustling companies, hiring without "up-to-date documentation, tests, and application setup instructions" is the only way. It's simply too expensive otherwise.

And yet, there is a difference between a codebase that is improving and one that's getting more tangled and indecipherable. Even if your code is "bad", it should at least be getting better, or a company is not doing any favors for new hires.

Statement 4: Most companies that hire only senior developers do so primarily because they don't know how to train people.

Richart and Berry pointed out that while this may be the case, it's doubtful that companies that only hire senior developers know why they do it.

Berry defended the posture, since senior devs are better. Again, though, in his world, he can't only hire them if he's strapped for resources.

Smith advocated a mix of experience, for two reasons. First, there are pitfalls to senior developers, like the risk of overarchitecting and Not Invented Here syndrome.

Second, assuming you hire well, the juniors and mids are up-and-comers, learning from the seniors while making their bones. All else equal, a senior developer who came up through a company's ranks is more valuable than one coming in off the street.

Another practical benefit for small teams is the ability for developers to do more together than they can alone. Two developers are far better than one, according to Berry, although he advised employers hiring their first developer to go for experience. They can find the right junior developer later on.

Statement 5: Assuming cultural fit, it is better to hire a less experienced developer in a couple weeks and train them than it is to leave a position unfilled for 3+ months waiting for a senior dev.

Berry was vociferous on this point, saying that if he knows for sure that he needs a senior dev to solve a specific problem, he's going to wait.

Nobody else felt strongly either way, and it felt as though the junior-senior debate was running out of steam. Certainly this statement was among the more context-dependent. Sometimes a top talent is needed, but sometimes a product must be shipped quickly. Ah, trade-offs.

Statement 6: For any well-educated developer, the specific language or framework needed on the job is not a meaningful hiring criterion.

This was completely uncontroversial for the panelists.

Smith led the way, insisting that attitude and aptitude are the primary characterists of a good hire, and Richart quickly agreed.

Berry suggested that languages are more important for junior developers, using the example of coding boot camp graduates. An Omaha Code School alum is intimately familiar with modern web application stacks, and can quickly contribute to a team needing that type of work.

Such a student, however, might be at a disadvantage at Grumman, where the toolset is wide-ranging. Berry made an interesting point that traditional four-year Computer Science students would do better in that situation.

There was some ensuing discussion on the merits of code schools and universities. All agreed that there are differences between candidates in breadth and depth of knowledge, so seemingly alike hires can require different mentorship approaches.

An employer should not examine candidates in a vacuum, but rather match them with job requirements. Richart might need a Java expert for a particular project, but Noecker may have more room to coach up a lesser-experienced developer with big potential.

Tony went on to say that ultimately the type of work a potential hire is excited about and their desire to move up the ranks is more important than any language or framework they specialize in.

Lessons

  1. Don't rely on indirect or contrived ways (interview exercises, reading code) to evaluate programming skill–talk to someone who has worked on projects with a candidate, or have the candidate work on a project for you.

  2. Context is everything. Do your homework. Know the type of candidate you need to hire.

  3. Have a codebase candidates want to work on. Have people candidates want to work with.