Becoming a better programmer
 

Hi,

So, I went from a very basic level of skill (essentially having only made it part-way though programming tutorials) to a "not bad" programmer in the space of a little over a year. I have been working on a project, in Python.

I have a reasonable idea of data structures, I have read a little on design patterns and some "advanced"/"pro" books on programming in Python.

Can anybody recommend what to do/read next to become a better programmer?

In particular, what's bothering me is that I'm almost certain that I couldn't design something that was really big and complicated. Also, I know that a lot of the algorithms I use are slow and inefficient. Looking at code on github, some stuff can be really nice and elegant and clearly designed and put together by somebody much better at this than me. I know the above is vague and not too specific, but, any suggestions?

I think it is not a simple issue. You can be advanced or professional from one point of view and novice from another point. The most important thing is to learn new techniques and to be precise (perfect?). As you wrote your codes are not so efficient, so try to make them better, try to find your own mistakes and failures. And practice, practice, practice. I think all of us have different knowledge and you also will have your own "style". The others will give you a lot of books to read and they are certainly useful, but you need to find your own way. Probably it will take a long time, years - or forever? And also some more practice....

Study and do many exercises ;)
A good thing is find different way to resolve a problem. Try, try and try.

Copy them :)

No, I'm not joking. Studying existing solutions is very important.

Head First Design Patterns (http://amzn.com/0596007124) is a good book to get you thinking about object-orient problem solving.

The SICP, which I have not worked through myself, is a fantastic one to get you into solving problems with a functional approach.

Python is a multi-paradigm language, so both books are applicable.

In all seriousness: Get a job.

Programming is like swimming ... you can't learn it with books. You need to get into the pool. The sooner the better.

How to become a better programmer?
 

Practice, Practice, Practice, Practice, Practice, Practice, Practice, Practice, Practice.

And communicating with fellow programmers of course.
Cheers.

First learn C++

Python is a very bad language for "big and complicated", especially when the programmer doesn't have a lot of experience in "big and complicated".

Even in C++, there is no substitute for experience. Sometimes the best way to get experience is: Do it wrong and then do it over. Sayings like "measure twice, cut once" are great for carpentry and many other fields. But they don't hold up in software engineering.

If you don't have enough experience to design something correctly, no amount of thought or structure (or reading and research) will cause you to design it correctly. You can spend far more time designing than coding and still have a completely garbage design.

This is definitely not the right advice for every situation, but it is often correct and rarely used: Start coding early. Plan on getting most of it wrong. When you get into trouble, make sure you understand why.

The total time to do it wrong and then design it right is often less than the time to design it right in the first place. If you are being paid for the work, you need an unusual amount of trust and flexibility from your boss to get away with this. Otherwise you might be boxed into keeping the initial bad design and layering on a mass of patches. If you are your own boss, doing it wrong before designing it right can work very well.

It also helps you think modularly from the beginning. When you are working on a piece you understand, but expect to throw away the surrounding pieces later, ask yourself what it would take for the current piece to be kept. Does this chunk of functionality represent a natural chunk of the total design, such that you might throw out the surrounding pieces without changing the boundary of this chunk?

If you're limiting the discussion to desktop apps then yes, that's true. It is, however, absolutely not true for website development. There are a large number of big and complicated websites written in Python (addons.mozilla.org being one of them), and a lot of Python-quality (read: very good) libraries to support building architectures for this level of scaling.

And for the record, I agree with the following completely:

Btw...
Garda, since you were asking for resources, pyvideo.org is an important one.

As sundialsvcs noted, get involved. That of course can be easier said than done. So try and invent a meaningful project that can be achieved at home?

And remember that "big and complicated" is a series of "small and bite sized" problems/challenges. Part of being a good programmer is the ability to break a task down into component parts. Work on each part, test it and then build it into the bigger project.

That is only true for a very skilled and experienced programmer.

Most software engineers break big tasks down into the wrong component parts.

Doing each part well is necessary to do a big project well, but not sufficient.

I have seen (and been forced to use) a lot of formal design methods that try to deal with that big problem of software engineering. But none of them are actually helpful. If you pick a hard enough project, getting the modularity correct is critical and it is also an art, not a science.

Some of us (sorry for the ego, but I'm trying to be honest here) are naturally good at it and most are not. But at any level of native skill, experience can make a big difference (while formal design methods generally don't).

Finding clean boundaries between sections of a problem is the most important job for an engineer working on complicated tasks. That requires the more basic skill of knowing the difference between a "clean" boundary and a bad one. I don't know how to teach or even describe that skill. You need to find it yourself.

I would bypass the entire (non-)issue of "which programming language to use," and/or whether one is better-than the other. (I find the entire notion to be irrelevant.) The point, for me, is this: You need to write programs ... lots of them. Programs that other people actually use and can critique.

Over time, you will constantly find yourself in "shops" that have standardized on this-or-that programming tool(s). You can never predict which one(s) will be next.

Bingo! That's probably the first time I've heard that said, even though I've believed it (and even proven it, at least within my own circumstances) for many years. I think this is particularly true when there are significant elements of the problem that are unknown. This might be something like how to program a particular device, how the software fits into a larger system or integrates with other software, or it might be as unassuming as using a new OS or programming language. If the problem is simply how to produce another instance of something you've already done numerous times (another web site, another accounting/inventory/other business package, etc), then johnsfine's rule might not apply.
Thank you, johnsfine. Unless someone else has already taken credit for the concept, I vote that we coin it the Johnsfine Rule.
--- rod.

You are certain because you haven't tried(?)
Figure out a domain in which you are interested e.g. web designing/networking/databases etc.
Pick up an existing `open source`software which fascinates you w.r.t the domain in which you
are interested.
Do NOT study its code right then. Study its functionalities.
Pick up its two simplest looking functionalities and implement them.
Then post you code for review on LQ or here.
Study the people's responses and dwell on what you could have done differently and why.
Last of all study the code of that software to see how they have done it.

Now, you have three designs (solutions) to the same code: your own, other reviewer's, and the
programmers of that software.
IMO, this approach will make you learn how to think.

I think it is much better to have a review of a design, than of a completed project. Important issues should be caught as early as possible in the process, as it makes no sense to put in code what should be done completely differently or not done at all. A code review should be to assess whether it meets the requirements as designed, as well as a possible a code QA.

--- rod.