What are the steps you follow from that sweet moment you have an idea for an awesome program to that even sweeter moment you write its last line of code?
I want to see how other people go through this process. I think I'm doing it wrong, because my code often ends up looking messy after adding features upon features that I didn't think of originally.
Posted Dec 18, 2009, 2:23 AM UTC
sometimes wank before the final deployment
Would you mind to create a cheat sheet for your workflow?
This workflow evolves naturally.
First do what stinkypyper does for 10 years
When your accurate sizings are ignored and you're asked to deliver a 5,000 hour proect next week, start multiplying all of your sizings by 15. Add a kickoff meeting wank.
When you get 100% unit test code coverage on a 2 month project that never gets deployed, replace unit testing with beer drinking.
When the legal department shits all of your project when you're 3/4 complete making it utterly stupid and requiring a major rewrite, add pre-commit wanking hook (checkins must have comments AND wankage)
When your 9 month project is released to a pilot of 10 users and then canned, add tf2 and reddit. Add wanks for deployments and retrospection meetings.
When you become a sr. developer since you're the only idiot still working 12 hour days and you suddenly find you attend meetings 8 hours a day with morons and code 1 hour per week, get backdoor access to the defect tracking system and purge frequently.
You'll feel a lot better about your job AND you'll find that no one notices.
Oh yeah. And one more thing. Whenever someone says "we don't have to do that because we're Agile" you have to drink a shot.
I'll get a UML diagram out to you ASAP.
I start with overenthusiasm and an unworkable and hacky first draft. Then I proceed to overanalysis. From there I move on to the painfully slow process of trying to write some tests to flesh out my half-baked overengineered crap to replace the hacky beginnings. Then I throw all that away in a fit of pique. Actually I'm too chicken to throw anything away, I usually just hg init project2 next to project. Mild panic sets in as I realize it's never going to work and I'm a terrible programmer. Then I start again with something simpler that's closer to my first pass, but a little more sane. Then I write some tests. Then when I like how the api is going, I stop writing tests and start coding more and more, amazed to be writing something that I might want to actually use myself. Then I realize that I've lost my test coverage and panic and self-doubt return. Then I start a new project. I guess I'm not much help. Sorry.
I start with overenthusiasm and an unworkable and hacky first draft. Then I proceed to overanalysis.
Hegelian programming!
I was not expecting a philosophy (edited was economics. for some reason I thought you were referencing engels, and it still made me laugh) joke here :)
So, basically, you're like everyone else, but you need a little bit more confidence?
Well its tough when you have to balance all this with pushing a boulder up a never ending hell-mountain.
For my personal projects, the process is:
It's important not to skip step 4. I clean up and refactor constantly. Otherwise, the cruft just piles up until it strangles the codebase.
In the past year, I've learned a metric shit-ton about compilers by doing basically that:
"Holy crap, compilers look impossible! But tokenizing seems pretty simple. I could probably write a little program that would break a string into tokens... <code>. Man, parsers look hard. But I've already got this stream of tokens now, so it's really just... <code>. OK, now I've got an expression tree. Hey, to evaluate arithmetic, I can just write this little recursive function..."
Works surprisingly well.
thanks
I wished I listened to instinct the did the refactoring bit more often.
Then again, I never wrote anything long enough that a rewrite would be a hell...
All of this is predicated on the notion that I have a clear idea of what the program actually has to do. Also, the program has to be of a certain amount of complexity for this, um, process to apply.
At some point, I come to understand that I now know what the design is. This is when I set up the wiki and start writing. Also, it's at this point that it starts being useful for me to talk to other people about what I'm trying to do.
Now I'm ready to start work on the production version, or as often turns out to be the case, the real prototype. My, ah, workflow for that:
From here on, the code/test/release process gets repeated over and over again until enough features are implemented that the program can be deployed. At this point, I discover where the real limits of my continuous integration process lie. Like, yes, I need the program to check its version against the version in the database at startup, because this particular customer needed a sysadmin but instead they hired a guy who likes playing with computers.
As far as "that even sweeter moment when you write its last line of code," that moment isn't sweet at all. It's usually a post-deployment bugfix that, surprisingly, is not followed by another one.
My workflow would be considerably different if I didn't have to do all the development myself.
Here is the process I have taken on an iPhone game I am writing:
This is the exact workflow I use! Except I skip step one and stop after step two.
(Sorry, I may be a little pessimistic :))
Discussing programming with programmers is like discussing botany with plants.
(Tuesday is Margarita debugging day)
basically same deal here:
so you never release anything?
Software development is like clay pot, an organic art that slowly turns into a beautiful vase. Experience is what tells you what materials to start with and the perfect mixture. And as an artist, you will never right your last line of code. The last line will always be in your mind, wondering what would've been.
And now seriously, many projects have elegant design up until that 90%. The other 10%, you better have branched, because that code is never making it back to main. The 90% is built with passion, the 10% within budget.
write/read requirements
proof of concept work for new ideas or technologies that are new to me
project setup / source control setup
design / UI prototyping
UI design and implementation (top layer) a bit of mocking here of lower layers
End user documentation (first draft)
Data layer / hardware layers and unit tests for them (bottom layer)
Business logic / unit tests for this (middle layer)
Putting it all together
Integrated testing
Revising documentation
Send to QA (again even if it's myself.)
Fix bugs (repeat last 2 steps.)
maintenence
There's probably more there. Most important to me is doing the UI and getting approval (even if just from myself) before writing the code. I find that it keeps me focused on usability and prevents compromises on the part of the application that people have to look at and work with every day.
I think that you are mixing two different things. One thing is the scope of what you want to do, and the other thing is the workflow you use to make things done.
Maybe the process you use could work if you know where you want to go, and that means defining clear milestones
Hobby projects tend to be much more difficult to complete precisely because they are hobby projects, so you will be avoiding the boring stuff and just paying attention to the cool features, however most of the time the boring stuff is required.
Last line of code. That never seems to happen for me. It can always be made better with more functionality. I guess I live in the world where my specifications and requirements are unlimited and change and morph over time.
I usually bounce around a lot (I have no formal training). I focus on solving a problem, and then clean up the code for that problem. I then branch to a new problem, and find the method to combine the problems together. It's been tough for me as I switch from using a strong IDE for C# to no IDE writing perl scripts, and it has been an adjustment.
Yeah, I don't have any formal training either.
I know what you mean about the last line of code. I have made very few programs that I am 100% satisfied with. With all the other ones, I will keep coming back to it until I become disinterested and abandon it.
Last line of code or not, there is a end point where you stop fucking with it, send it out, and then fuck with it after that.
Yes, there are commit points, but no finish points.
Comments first. Write out the whole program in plain English comments. Gradually fill in the details until you know exactly what code goes where, then fill in the code and change your comments to answer "why did I do that?" better than they answer "what am I doing?".
Hardcode values everywhere until you get the basic functionality right. Once your hardcoded values work, replace them with the properly populated variables (whether it's from user input or values passed along throughout the program).
When you think you're done, take a break for a couple hours, then come back to your program and re-read it. Any time you find code that takes you some effort to figure out what it does or why, put in a comment.
When you think you're done, find annoying bits that you copy/pasted, and see if they can be made into functions or even classes of their own.
Write out the whole program in plain English comments
This has saved me many times. I remember there was one particular section of code in our engine that was very confusing. It was comprised of a huge nested if statement and made my eyes go cross-eyed. This section was victim to feature creep which is understandable as to how it got that way.
When looking at the code, I studied and gathered a full understanding of it. I then wrote, on paper, what the code did in plain English what it did. Then it became clear to me what needed to be done to optimize and debug it.
You just can't go wrong when you do that.
REPL ;-)
It depends on what the program does. I write HID and graphics drivers at work which would have a different work flow than a full program. The drivers, the way I see it, are just libraries or API's when it comes to actual development. So, I use the same workflow with static/dynamic libs.
So, for drivers I do this:
1.) Write out the basic uses for the device
2.) Break down these uses into primitive operations on the device.
3.) Write an interface for the driver based on these uses
4.) Write a test program to call the interface. This is usually just a straight forward console program.
5.) Implement the firmware's primitive operations.
6.) Implement the interface.
7.) Optimize.
8.) Debug
For full programs(Personal and at work):
1.) Write out the programs basic needed objects/libraries given that I have a full understanding of the programs goals.
2.) Any missing libraries are developed with the steps given above(give or take a couple of steps).
3.) Write out a state diagram of how the program will operate.
4.) Test out the program's basic operations and workflow via console or basic GUI.
5.) If the program has a GUI, implement the GUI and call any necessary delegates, state-calls, etc.
6.) Optimize
7.) Debug
Edit: I don't know how to count to 8 apparently.
If we actually knew how to properly evolve a system, wouldn't we have a political economic one that actually worked? Dijkstra seemed to recommend working from core definitions. The expression of which denotes a complete program, given the totality of the situation.
I usually design/build the ui and then start coding the classes for the relevant objects, trying to keep everything as modular as possible. But once you've done it a bunch of times you'll be able to easily map object relationships etc in your head. If doing it for a client i'll clearly define the scope before doing anything.
I start by making some data structures(I like statically typed languages for this reason). Then I write a bunch of code that doesn't compile or work, this gets me in the mind set and brings up a bunch of things I haven't previously thought about. Then I go for a walk/ride/sex/something away from the computer which gives me time to think and pattern match against random input to see if I can find a better solution to my problems. Next, I either throw out all my code or if it's workable I'll fix it until it compiles, then I'll test it until it works.
Write function that forms main part. Test in REPL. Fix. Continue for the rest, building up as you go.
The first thing I do is identify the hardest parts first. I write small test programs to make sure those problems can be solved. You have to get started on those the earliest and make sure you can pull it off, plus it gets you thinking early about these issues.
Next, if it has an interface, I build a basic, but fully functional one. I will even hard code some data in it or add some light background code so you can see how it feels. This the part end users will see and care about the most, it also helps you identify anything you might have missed and gives a very deep understanding of what it has to do. Next, I model any data. I don't really do it formally unless I have too, I can do it my head fairly quickly from experience. From there it is coding time. This will usually go smooth since by this time I have a very good idea of what I have to do, an interface -- if necessary -- to work with, and all the major problems solved. Since I have a lot of the pieces done, and have had time to lead up to them, I have had the time to think about how to best compose the software in my head, so things fit together real nicely. Next, I make sure to unit test the fuck out of the software, and do lots of final testing.
My best advice to you, don't jump in right away. That is always the biggest mistake. Every time you jump in too quickly into something, and then try and hammer it out quickly, you make stupid mistakes. You always end up not doing your research and writing code that already exists in some library some place, using the wrong API, wrong tools, etc... Problems that would not have happened had you given yourself breathing room, done your research, and talked to the right people.
As for you problem, it's called "feature creep", and again, it's from jumping in and not thinking clearly. Had you given yourself some more breathing room in the early stages, those "features" would not "creep" up on you. You need to focus less on the number of features. Your job as a programmer is too make peoples lives easier with simplicity and automation, not harder with clutter and confusion. You need to focus on your core features, and then build out secondary ones as the market and your customers dictate. Otherwise, you are wasting all of your time on one endeavour endlessly maintaing features no one ever uses.
Thank you for the well thought-out response!
I do tend to jump in too quickly into projects, or skip the boring parts, thinking that all the code I need is already in my head.
I like your approach. It seems like you have some nice people to work with. My boss basically insists that I work on the interface first, before anyone has a clear idea of why or what. She distrusts developers who work on hard problems first because they show little progress.
BTW, too != to :)
Spot on!
Great response, very similar to my approach. Also specifications from the customer change so don't get upset about it, try to accommodate it. Don't do "what if" programming, there is a balance between a design that can do everything and a design that is too narrow.
YES, "what if" programming is very bad. I have done that though in situations where it was highly probable that the thing I was programming for would happen. Though it has worked out I would not do it again and don't recommend it.