There will always be a need for hardcore programmers
this video has been whirring around in my head ever since I posted about it : http://vimeo.com/36579366 it's titled inventing on principle, taking the principle that creators need an immediate connection to their creation and running with it, saying that when that's not true, invent a solution. Reduced to developer speak, he says it's necessary to reduce iteration time down to nothig so it allows moments of inspiration through the immediate feedback.
I think it's a great talk, go watch it if you haven't already. It's inspiring.
But, the thing that's just stung me though is the idea that although it's great to work with these tools that allow less experienced programmers to create wonderful things, and let non-programmers get the feedback they need, there's still always going to be a need for highly experienced or _hardcore_ programmers.
The reason why is hidden in the talk.
"... and to a large extent the people we consider to be skilled software engineers are just those people that are really good at playing computer."
Now, let that mull. Let that fester. Let the idea that a good computer programmer, a good software engineer, someone that's good at writing code or debugging stuff, is in effect, someone who's good at just knowing what a computer will do when it runs the code.
Knowing : Not figuring it out step by step, not referencing online documentation to find out what function foobar() does in this context, but just knowing.
Knowing is a whole different world to being-able-to-figure-it-out.
Although his tools are wonderful, there are areas of development where it's very hard to make a tool that works that way. If you work in games, then you''ll possibly come across some area or another where side effects cannot be seen in real time, such as anything relying on player input, or some other completely random process like network connectivity.
So there will always be a place for the programmer who can play computer, just like there will always be a place for the architect that can see the materials and stresses in his drawings, or the psychologist who can feel the response of their words without needing test them on their patient.
There doesn't need to be a segregation of those who create from those who play computer, but as it turns out, there usually is. Even if it's only in the sense that an individual is either in creation / debug mode. In my understanding, the best games developers are the ones that can combine the power of internal simulation for both the design and implementation stages.
But this post was about hardcore programmers, and how we will always need them. I think I've already made a convincing case that there will always be something horrible out there that can't have instant feedback (like checking if an e-mail server can support >4gb files), or bug that has a 0.001 probability of happening. In those cases we still need our hardcore programmers, but there's another area where the truly hardcore shine, its the same place the designers shine when they're given their 0 second iteration time.
Hardcore programmers, the programmers that know, shine in new frontiers. They think of the side effects that others miss.
Take as an example, a good programmer who chooses to use some container over another because the order of complexity is good for the task at hand. This is a good programmer because they consider what they know about the algorithm, about the data, and apply a solution based on some knowledge of both.
Then take as a counter example, a hardcore programmer who chooses to not use an existing container at all, and instead uses what he knows of the data to think through the requests, the assignments, and generates an on the fly container that is nothing much more than a lookup that resides in the same cache-line as the data that will be needed if the initial lookup indicates it will be needed.
The difference is that the good programmer is good, gets the job done, doesn't write bad code, his code is readable and understandably good. People can see that he's a good programmer because his code conforms to best practice guidelines, and his code is easy to read.
The difference is that the hardcore programmer sees the machine bare it's guts to him, notices latency on instructions at such an innate level that they are always thinking about the critical path through a function, watching for unnecessary branches, estimating the probability of the ones taken and always thinking about where the data is and what values it has as much as the good programmer is thinking about quantity and big O notation.
The difference can be seen as one of depth of understanding, but more often than not, it manifests as a difference in approachability. the hardcore programmer doesn't seem to be a good programmer on initial inspection by those that aren't clued up, but at least he'll always be required. The hardcore programmer usually has to prove themselves by results rather than how they got there.
The same is true of the hardcore architect. Other architects can come along and change "just one thing" and break the balance or strength of a building because they don't see the scope of the change that the hardcore architect can see easily.
take this tragic example: http://en.wikipedia.org/wiki/Hyatt_Regency_walkway_collapse
And this is happening all over your code when you hire good programmers and hold back the hardcore programmers because they're not following guidelines. Just because people aren't dying yet doesn't mean everything is okay.