Archive for the ‘Programming’ Category
Again, due to the email friendliness of Posterous, have posted it over there: http://protoiyer.posterous.com/setting-up-my-linux-box-and-ruby-on-rails-dev
If at all you want to add comments, kindly do it over there on Posterous. Thanks.
Due to the email friendliness of Posterous, have posted it over there: http://protoiyer.posterous.com/
“Most programmers have only a vague notion of how competent they are at what they do for a living” — Steve Yegge
“Experience comes from practice” — Andy Hunt
I thought I was done and dusted having wrote that previous redux post about how to be a better programmer. But my good friend Subru had posted a comment that made me (as it does most of the time) take notice and do some thinking and research about the importance of study and practice in the career of programmers. I did that since it has a direct impact on the topic I have been harping about — the talent of the programmers like me, or the lack of it.
Subru had wrote that reading a few books need not necessarily make you better. He said that observing and interacting with senior craftsmen would make you a better programmer. It is precisely this attitude and observation that I wanted to do some thinking about. I, based on my personal experience, believe that it is better to read the books than observing the masters. The reason being that you never know whether the maestro is actually doing the right thing or more importantly, it is the right thing for you at the stage or phase of your learning or career. Plus, what if the meastro hasn’t read anything in the recent past and so is not in touch with the latest developments in the field?
I had this doubt, nay, conviction, partly because I had the good fortune of reading about Shu Ha Ri which was an appendix of the great Alistair Cockburn book Agile Software Development: the Co-operative Game, and partly because I believe Subru was wrong about the books. Shu Ha Ri teaches you that a beginner (the Shu phase) needs a framework or a rigid set of rules to follow, plus constant feedback to get through the initial learning phase. In the intermediate level (the Ha phase) one masters the rules, and learns all the tricks of the trade. In the advanced phase (the Ri phase) one needs to actually forget the rules or transcend the rules and make one’s own rules of the game.
Ok, so what am I harping about here? Well, just the fact that with practice anyone (who is not physically or mentally invalidated to even attempt the task at hand) can move from being a novice to a master. All it takes is dedication and some knowledge of how to travel the path of mastery. I am not saying this. Hear it in Kathy Sierra’s (of the Head First Java and Head First Design Patterns fame) words here. Having experienced first hand how Apple and myself could actually become far better programmers than what we were when we started out, I can vouch for it. I am not talking about the linear progression in talent that people who don’t read books enjoy. In our case, the progression was non-linear, if not exponential, and it was made possible by the simple fact that we studied (read) and we practiced what we studied — him probably more than me.
In fact some of the best minds in our industry believe what Kathy was conveying passionately. Steve Yeggey, one of the best bloggers I have read, has analyzed this topic in-depth, and later came up with an article titled “Practicing Programming” for people like you and me (the average programmer) to reflect upon and work upon to becoming a better programmer. He wrote both the articles a few years back when he was still with Amazon, and now he is with Google.
He writes in the first essay:
Bob (our average programmer) knows this guy Joe who’s just amazing. Joe’s like the best programmer Bob’s ever known…He’s a natural at it. One of them whiz kids…
In Bob’s view of the world, there are essentially three programmer skill levels: folks learning how to program, folks like Bob who know how to program, and the inevitable whizzes, but they’re few and far between. There are always a few whizzes out there, the ones who used to be child geniuses or whatever…
Bob has no incentive whatsoever to try to improve his skills: He knows he’s not as good as Joe. But Joe’s great on account of his genes, not because he practiced or studied more than Bob did, back in school. Obviously Bob can’t compete with people who were kid geniuses, and he shouldn’t exert himself unduly on their account.
It is a great rant and a rich source of stuff for us to reflect upon. At least I came back knowing more about myself and realizing I had been that Bob at many a time, and probably still am a Bob in many ways.
This is reinforced by the works of two authors I respect very highly — Andy Hunt and Dave Thomas (of the Pragmatic Programmer fame). Dave, for instance, thought deeply about the importance of study and practice. He went on to devised a set of exercises, named Code Kata, for exerting the programmer’s brain — stuff for us to chew upon to become better at what we do. And finally, here is his take on the various phases of learning and skill acquisition which is similar to the Shu Ha Ri theory, but based on the art of karate.
Of course, the real spark for this article was this great post by Jeff Atwood: “Programming: Love It or Leave It” earlier this week. From there the trail of reading (thanks to Google) led me to this rather controversial article about how to become a better programmer by not programming. I don’t think, for once, the author is 100% right. Yes, Bill Gates is right, but the point is people can improve and many people really do. As Kathy says, of course, that won’t be enough to be the gold medal winner at the Olympics, but we can be the street, council, district, state or national champions, at least.
So, the point is, IMHO, it is a far safer better to depend on books rather than trying to observe a mentor, for the simple fact that at least I have bumped into only a handful of people in India in eight years from whom I could learn something about programming. Of course there are lots of people to observe and learn about people management and the art of maintaining relationships or soft skills, but that is beside the point of this post. I accept that perhaps I am not good enough to have worked at a Google or a Microsoft, or that I was perhaps exceptionally unlucky (not to have bumped into more mentors) but the point is 90% of the software developers are not working at Google or Microsoft either. And people like me too have a right — and a duty — to improve, right?
As Peter Norvig says in the article Teach Yourself Programming in 10 years, to be really good at anything including programming requires lots of time, effort and dedication, and more importantly, we all know that if the pioneers of the Design Patterns movement hadn’t read Christopher Alexander’s work(s) on architecture, there probably would have been no Design Patterns movement. And if I hadn’t read those great books or those great blogs, there wouldn’t have been this post either!
Apple implants the programming virus in my brain
Also, I wouldn’t have started my journey of becoming better had my good friend Apple not read the K&R book and made me both feel small and admire his skills when he wrote this sometime in 1999 in my notebook:
while (*t++ = *s++);
That, for the uninitiated, is the succint way of copying a source character array into a target character array in C. Apple wrote this after asking me to write a program to copy an array, and the best I could think of was to write half a dozen lines or so to achieve the same, and without using pointers. That was the moment when my journey to get better actually started. Even after that moment, I was the guy who did his C++ project in C (at NIIT), since I couldn’t quite understand what this fuss about using objects was all about (and I was good at C thanks to the K&R book)! And it required reading The C++ Programming Language by Bjarne Stroustrup in 2000 to make me see the light, at last, and luckily, I never turned my back at lapping up a great programming book since then!
To wrap up, I would rather continue reading than either sitting idle or just (waiting for and) watching the right mentor. And I would ask every developer interested in becoming better to do the same.
Oh, and yes, lest I forget…Happy new year and thanks for reading yet another long post!
"Software development productivity would skyrocket if the least effective 30% were fired tomorrow" — Neal Ford
It was in the December of 2005 that I first wrote about how to be a better programmer. At that time I was into my fifth year as a programmer, and though I wanted to write what I am writing today, I decided to narrow down my scope to non-technical issues. I wrote about code readability and code maintainability that day. As important as these issues were (are), I guess there is a larger issue that must be addressed — the technical skill set of the programmers. Then I wrote about (nay, linked to) the fatherly advice to new programmers by Chuck Jazdzewski.
I am now into my eighth year as a programmer, and unfortunately, the rate at which the industry has understood the importance of the quality of code or for that matter about issues like code readability or code maintainability is pretty dismal. This might be justified by arguments like "in service industry, deadline is more important". But people just don’t understand that in the rush to push the software out of the door, the quality of the code suffers, which directly affects how quickly you can ship the second or later versions. To make matters worse, people who can really understand the code at a deep level, and who are really interested in programming, is a minority, and I guess it would remain that way for the foreseeable future.
The triggering point for writing that piece three years ago was a mail from my counterpart in UK, who wanted the team to ensure that they are putting comments properly, and (though he didn’t explicitly mention the words, he provided an example that pointed to writing) comment at the level of intent. I am not sure how many people read that post (considering that I find it difficult to tell people that I blog; except to my close friends and family), but at least from that day, I made it a point to tell my team members the importance of writing code the proper way, and when required, show them how it is done. The triggering point for writing this has been yet another series of events. I thought it was time I write this down, so that a) I can refer back when I want to, and, b) pass this on to anyone to whom I feel like preaching in the future :).
The first and the most important aspect to become a better programmer is to have something called interest or involvement. Half the crowd I meet are programming just for the sake of doing some work. Programming, unfortunately, is not similar to doing most other jobs where you learn and master something once, and you can carry on without learning anything new for quite some time, and in some cases, for years altogether (or for life). Careers that spring to my mind include Government Service, Accountancy, Banking, Teaching, and certain professions like certain streams of Engineering and Law. Of course, in each of these cases, there is something new to learn every year (or every few years), but compared to the frequency at which things change, and the effort involved to keep ahead of the curve, they do appear trivial and manageable. [I know this is going to be a controversial perspective, but being a Mechanical Engineer and being someone who has a passion for teaching, and who dreamt about doing M.Tech and becoming a professor, I hope I know what I am talking about here]. Anyways, I don’t mean any kind of disrespect to any of the professions. I am just pointing out the fact that you don’t have to explicitly sit down and read a few books on your subject to remain good in many a profession out there.
In programming, what you know today is of little or no use two years down the line. I remember reading way back in 2002 about an interesting incident narrated by David Chappell in his book, Understanding Dotnet, First Edition. Chappell was taking a session on DCOM (if I remember right) at Moscow, when someone in the crowd got up and asked him why is that Microsoft is changing the technology every few years, and how he is finding it difficult to cope up with that. Chappell told him, as a matter of fact, that if he finds the change difficult to cope with, he is perhaps in the wrong profession. I wish we could make more programmers understand this simple fact – the fact that in this profession one can’t stop learning. In fact the number of programmers who have read any of the top 25 books on programming or software would be less than 10%. This might be an example of the Sturgeon’s Law (that 90% of people will never care and 10% of people would cover up for them in most professions).
To make matters interesting the IT industry doesn’t quite teach them that programming is important, and I live in a society, where if I tell them I am programming in the eighth year of my career, people think I am a (lowly) programmer still, perhaps because I don’t have what it takes to be a manager or a team leader who mostly takes care of team management related activities. The trend is to learn programming at the level of writing if..else, for..do and yes, playing around a bit with something called objects (without really understanding what they actually are and what they are meant to be), and do programming for four or five years, all the while remaining at that basic level, never bothering to go out of the way to improve oneself, and jump on the management related bandwagon at the first possible instance.
This is where some of the most successful companies around the world stand out, and it is not a coincidence that most of them are run by people who were great programmers, and who still love and do programming. The best example would be Google and I can never forget the beauty of this story and what it tells you about the company (or the story of how Sergey Brin wrote an application to test Android, and the application was about using the accelerometer in the device to determine the time an Android phone was in the air when it is tossed up; the point is there are very talented programmers at the top who are really interested in coding in spite of being so successful), though there are quite a few others like Microsoft, Martin Fowler’s ThoughtWorks, Joel Spolsky’s Fog Creek Software, and Steve McConnell’s Construx Software. Construx has published the Construx Professional Ladder for its employees who aspire to move up the corporate heirarchy (the ladder), and the best part is that it ensures that those who grow to be the Team Leads or Managers, are people who are widely read, and who have understood the nuances of programming, and can still solve the problem, if the programmers are not able to solve it.
To put matters in perspective, check this recommended reading list for a Level 10 programmer at Construx. Please note that Level 10 is not the Architect or the Technical Leader. It is the programmer with some experience ("College graduates will generally start at Level 9. Experienced developers may start at Level 10"). That might appear a bit over the top by your or my standards, but that tells you what separates the great companies from the merely good or average ones — it is the quality of the people. I think it was Kent Beck (in Extreme Programming Explained, Second Edition) who said something along the lines of "a more talented team will almost always beat an average or less talented team, no matter how disciplined and process oriented the average or less talented team is, and you can’t do much about it". A cruel (for me, at least) example of that in the wild is watching the Germans end up as the runners up (or loosing semi finalists) at the World Cups in 2002 and 2006, and in Euro 2008. With a bit more talent (or a bit more botch-up by the opposition, they could have won any of those events).
To be a good programmer, one needs to have a good grasp of the fundamentals of programming (and there is no better language to learn this than C followed by C++, though that unfortunately is out of fashion these days). This must be followed by learning a modern mainstream programming language like Java or C#. But if you start with either Java or C#, you would miss quite a bit on really understanding pointers and objects at a fundamental level. This must be followed by learning, reading and eventually mastering OOAD, UML, Refactoring and Design Patterns. It will be really useful if one can find the time to explore and learn dynamic programming languages like Python or Ruby, as they bring a different perspective to programming, thus enriching one’s knowledge and understanding.
Yet another mandatory aspect going forward would be exposure to Agile methodologies (don’t think I am crazy, but there is a big crowd out there that doesn’t understand what is agile). Even if the team is not 100% agile, there is a lot to learn from the agile community. If anyone has any doubt about agile, the only thing to keep in mind that the top guys of the Design Patterns movement are also the people who were the forefathers of the Agile movement — people like Ward Cunningham, Kent Beck, Eric Gamma, Martin Fowler, Alistair Cockburn, Dave Thomas and Andy Hunt to name a few that I can recollect over the top of my head. Since each of them are titans in their own rights, there must be something in it for us to learn.
The best part of this learning, IMHO, is that it leaves one in such a strong position to negotiate most of the stuff that comes one’s way – a bit like how Dravid could easily and with grace negotiate the unplayable deliveries from the Donalds, McGraths, Pollocks and Waqars of this world, in his pomp, in spite of not having grown up facing them or anyone of their calibre. This was made possible by the fact that in spite of not being a genius like Sachin or Lara, he kept on working on his game, improved every day as a batsman by putting in the effort, and thinking about his game. [I wrote this without being aware of the match saving 100 he made at Mohali].
The best resource that I am aware of to learn the basics of the aforementioned stuff , that I am aware of, is listed below. These books helped me immensely in my development as a professional, and I hope at least some of you can and will find them useful.
Programming Language basics:
The C Programming Language by Kerninghan and Ritchie (the K&R book)
The C++ Programming Language by Bjarne Stroustrup (it is a tome, but worth every minute of your time)
The Java Programming Language by James Gosling et al (a beautiful book)
Thinking in Java by Bruce Eckel (or the unpublished C# version freely available on the net)
UML Distilled by Martin Fowler
Refactoring by Martin Fowler
Applying UML and Patterns by Craig Larman
Object Oriented Analysis by Peter Coad and Edward Yourdon
Object Oriented Design by Peter Coad and Edward Yourdon
Design Patterns by Gamma et al. [the GoF book; I am yet to read this as it is in C++]
Head First Design Patterns by Eric Freeman, Elisabeth Freeman, Kathy Sierra, Bert Bates
Design Patterns Explained by Alan Shalloway and James R. Trott
Taking the skills to the next level:
Code Complete by Steve McConnell, Second Edition
The Pragmatic Programmer by Andy Hunt and Dave Thomas
Software Craftsmanship: The New Imperative by Pete McBreen
Books on Agile:
Extreme Programming Explained: Embrace Change, Second Edition, by Kent Beck (there are two editions and it is fascinating to read both back to back :)
Agile Software Development: The Cooperative Game, Second Edition, by Alistair Cockburn
There are no shortcuts to mastery in any field, and software is no different. Unless you read some of the above books, you wouldn’t even know what all you miss (in terms of knowledge and skills missing from your toolbox). Again, this must be supplemented by some real hard work at the PC — programming, trying to implement and refine what one has learned. As the old saying goes, there is no substitute for practice.
Next, since we live in a society, we do need to work upon our soft skills, and people interaction skills come right at the top of the skills to acquire. One of the best resource to learn from this would be How to win friends and influence people by Dale Carnegie, as attested by both Jeff Atwood and Steve McConnell. In fact, if I remember right, McConnell writes something along the lines of "it must be made a mandatory book for every member of the team". How true!
Internet is such a free and ever available source to learn new stuff. In addition to reading MSDN or other technology oriented sites, read the great programmer’s blogs and learn from the masters. Let Google Reader be your best friend in this attempt. It has been for me, for three years now.
Update [14-Jan-09]: cleaned up the essay a bit and included a few more links in the story.
“Delphi has a developer population of approx 1.75 million users worldwide in 2006. Delphi has 15% IDE marketshare overall (7.7% ranked out of 100%) and is the #2 Windows IDE and the #4 IDE overall (C++Builder is #3 and #5
…and you thought Delphi was not big/good!!
It is just that not many people are aware of it. No, you can’t use it to create an ERP application (perhaps), but it is pretty darn useful for 80% of the commercial software work carried out in most of the companies, and it actually helps you to create it faster.
Powered by ScribeFire.
I have been a passionate Delphi developer for the last 5 years and I can assure you that there is some truth in what I read somewhere: Once you start using Delphi, every other IDE and language appears inadequate!
Now, before you start baying for my blood, let me put on record that I am some one who learned and programmed in Fortran, C, C++ (including programming using templates/generics way back in 2000-01), and then did some basic J2EE programming during 2000-01, and then programmed Dot net using C# when it was still in early Beta (late 2001-early 2002), and even after it was officially launched, and so I know what I am talking about.
Simply put, there is nothing that even comes close to the thrill of using Delphi for development. That will be similar to saying: Simlply put, nothing even comes close to browsing using Mozilla Firefox! Hope you get the gist of it :).
The best news in the last few years for the Delphi community is this: Borland DevCo is in the process of recreating the magic of Delphi by launching the Turbo product range [Delphi for Win 32, Delphi for DotNet, C# and C++] that is available for free. You can download it from here.
There is a lot of renewed interest from the community, and there are lots of free introductory and advanced tutorials out there too. More info available here at the blog of Allen Bauer, Chief Scientist, Borland DevCo.
Now, hear this: Evans Data conducted a Spring 2006 IDE developer survey, and know what? The long dead and buried Delphi was rated first (yes, you read it right!) in the most important “Compiler/Interpreter” and “Debugger” category! More info from the horse’s [David I, who is the Vice President, Developer Relations and Chief Evangelist of Borland DevCo] mouth here.
Of course, this is not meant to take anything away from the obvious bitter truth that, like Macs, Delphi is a third or fourth rate citizen in the world of programming, languishing behind the products from the behemoths like Mircrosoft, IBM/Sun. The good old days of Delphi being the “big new thing” probably will never be back, but one can wish for a momentary glance or two at glory! It has almost the feeling of a classic Asterix story — Gauls v/s Ceaser(s)!
powered by performancing firefox
I have been working as a programmer for a little over 5 years now, and it was with great interest that I read the article What is a Professional Programmer, written by Sarah George, and hosted at DeveloperDotStar Magazine.
I think I can say that I have been on the right path so far when I evaluate myself on the various parameters she has mentioned, viz., trustworthiness, teamwork, leadership, communication, constant updating of skills, an interest in minimizing risks and accountability.
However, there was another theme running right through that article, the theme of “how much Domain knowledge you need to do good Software Development?”. It was triggered by this comment by Mario Van Damme, to which Andy Tegethoff posted a great reply. But Andy didn’t just stop there. He followed it up with this wonderful post on this topic.
I have been of the opinion that Domain Knowledge just cannot be the be-all and end-all of Software Development. Yes, one needs to know enough of the domain to make sense of what one is doing, and what the client is saying. However, the current trend in the Software market in India (and probably elsewhere) is this: an average candidate with average-to-good domain knowledge is a better bet than a good/great programmer with almost zero domain expertise. During screenings and interviews, I have often experienced this line of thought in action. The argument is something like “programming anyone can do[!]; but to get guys with domain knowledge is very difficult”.
So, one is considered a better/worthy candidate if one is having previous working knowledge in a domain. And sometimes, the programming skills takes a clear back seat since this new guy is average-to-good at domain. Of course, domain is where the money is (meaning you get paid more).
What these recruiters forget is the basic fact that you are recruiting people for doing software development, and not to look after your domain/business. In fact, as established by Extreme Programming school of thought, it is much more valuable to have a true blue Domain guy (the actual Customer/Client or someone representing him) in the programming shop with good programmers than to have an army of domgrammers — short for domain-oriented-programmers. After all, most of the time, you just can’t become great at a domain in a short span (say 1-2 years) without sacrificing your programming skills. Of course, I am talking about normal people here. As the rule goes, “there will be exception to all rules, except this one”!.
In one of my previous projects, there was this culture of “anyone can do programming; understanding the domain is what it is all about” logic. I was really amused and shocked at that line of thought, but thankfully, I never subscribed to that school.
Towards the end of my first year in that project, we found out that the most critical module of that project was causing a hell of a lot of problems. We also found that there were serious design flaws and hell of a lot of bugs in that module. Luckily, I was learning Refactoring, UML and Design Patterns at that time, and my design skills were good at that time (or at least I thought so :). During this time I was looking after that buggy module and so was spending lot of time fixing bugs and logic-flaws, that just should not have been there in the first place.
I took it upon myself to set the house right, by re-designing the core of that module, and doing enough refactoring to change the design. After “some” struggle, I managed to convince my boss(es) that this is the only way to do it and that I can really do it. Remember, I was still considered one among the juniors in the team.
By this time, client was also breathing down the bosses’ throat to solve the highly critical bugs in that highly critical module. So, at the end of it all, the boss(es) agreed to allow me a free hand in doing what I wanted to do.
I was able to come up with a new design, and then refactor the existing code to fit the new design framework. It was not refactoring actually — though I used it a lot — since it involved a change in the behaviour; after all I had to weed out the bugs. During this process (which took me about 3 months) I had to talk to the client scores of times, communicate the design to the team, do lots of coding and teaching, explain the design to the boss(es) and client etc. I thoroughly enjoyed the experience.
I also understood that no matter how hard I try, I can never match a pure domain guy’s depth of knowledge. Also, it is not useful too, since I can easily learn from them. In the process I also learned that it is pretty hard for a software guy to be as good as a real domain guy, who is breathing the domain day in and day out.
More importantly, had I fooled myself into believing — during my initial days — that learning a few hundred terms and definitions of the domain will make me a better programmer, I would never have been able to come up with the new design.
Just thought will post about one of my core beliefs — that technical knowledge is a hundred times more important for a programmer than domain knowledge.
But of course, the fact remains that I am paid and will continue to be paid less than a hot shot domgrammer or Business Analyst. But that is ok, since I get to do, what IMHO is, the most challenging task of converting what is required into a great design/software that provides the solution.
Of course, Dan Read offers his point of view here on this topic, and as always, it oozes with pragmatism and common sense. But yet, it is difficult to miss the message that in 90% of Software houses, domain knowledge is only that important.
Thanks for reading thus far. It will be really great to get some feedback/comments. Have a great week ahead.