Pencils Down

This weblog is about my experiences in software development

Browsing Posts tagged Software Development Process

That is the title of a recent article in the NYT.  The author talks about his experience with New Guinea natives who refused to camp under dead trees.  The reasoning being that there is a 1 in 1000 chance of one falling over, I sleep under trees every night, if a tree lands on me it will kill me, the mathematical likelihood is 100%.

The author uses another example of taking a shower (hence the name) and it’s likelihood for older individuals.

I started to think about computer stuff.  If you think about some of the systems you have developed or used you may have similar concerns:

  • that disk drive that has a 0.01% failure rate
  • the expected occurrence of a divide by zero
  • neatly computed mtbf for that system delivered

All of these fall into the same case:  with large numbers any percentage greater than zero will cause a failure.

Currently working on student course selection software.  It has been painful dealing with the number of special cases that have to be hacked into otherwise pretty good looking code.  Why on earth do we care about these odd cases?

Using the above logic, “any expected failure will occur” in this case with a large number of students over a period of time, it becomes clearer – the system will fail.  It is just a matter of time.

Recently had a code review for a project.  All the normal class diagrams, interactions, SQL code: the works.  Went pretty smoothly.

The only glitch was someone non-technical attended the room that I had invited as optional, really out of courtesy so they would know the review is occurring.  The Problem was how do you describe some standard programming topics to someone who is not a programmer:

  • What is an interface?
  • What is an abstract class?
  • What is an implementation, extension?
  • How do you find if there are no records that meet your criteria?
  • What is a cache?
  • How do you pick out the records you want from all the others?

At first I tried to give some real life analogies to the concept involved.  As the review progressed and got inherently more technical I realized there really is no way to bring a non-programmer up to speed on how code works.  We live and breathe in this abstract reality that starts out with a user but quickly dissolves into something else.

I know the person was trying to understand and I was trying to give something they could relate to.  I am just not sure the gap can be easily traversed.  Feels like I failed.

In depth view of tumblr here.  Short overview:

– 15 BILLION page views a month

– 500 MILLION page views per day

– Some parts of architecture running at 40 THOUSAND requests per SECOND.

Humbling.   Of course the author sounds as calm as can be.

I have a long term client that wants to rework their web site.  We tried a designer I had worked with, but they weren’t happy with the result.  Probably too much part-time and indirect contact to convey their needs to the designer.

So, they hired a designer on their own.  After selection I talked with the guy to make sure it wouldn’t just be a PSD as the output: he agreed.  Once the design was agreed upon he didn’t just give me the PSD, he gave me a bunch of PNG files that he cut from the PSD.  How nice.

Now we are going into week two of the process trying to get HTML and CSS out of him.  I don’t think he has ever done this before.  He is good at coming up with nice pictures with the right color schemes and fonts and the like, probably an art background.  He is just lost coming up with the parts of the design you need to actually develop a web site – HTML, CSS and Javascript for any widgets/special effects in the design (there are a few).

Maybe I am expecting too much?  Every other designer I have worked with has produced the initial HTML… layouts.  Maybe I have  just been lucky?

At my current workplace we are automating a lot of tasks that were done using Oracle Forms to perform under a web interface. Sounds good so far.

One of the forms invokes some tens of thousands of lines of SQL code on the backend touching a large number of tables and invoking a good number of stored procedures. In order to automate this someone on the web team has to understand all that code. Again, from a programmers point of view, doable – might take a while.

Ok, now how to explain the size, scale of the problem to someone who knows nothing about programming?

I threw out the idea of going to the library, reading a novel that was several hundred pages long and having to understand exactly what happens to every character portrayed in chronological order.

Thinking about that later I wonder if I was very far off:

  • A typical page of a novel has 50-60 lines which equates to several hundred pages.
  • A line of SQL does not correspond well to a line in a novel, but given the power of the syntax it may have as much meaning
  • The chronological importance is important in both cases, more so for the SQL.

Maybe it’s not such a bad analogy to use.  Maybe even the PHB might get it:)

Small Companies

No comments

Interviewed at a small company.  They have a hold on a market niche for the last n years.  They have mostly been together for the last n years.  The prototypical lifestyle company.

Now one of their own is leaving.  All of that assumed understanding over the years is showing up as a lack of communication and follow-through.  The developer leaving has been off the range for years:

  • home grown web service – doesn’t like SOAP
  • obscure database – has a neato auto-maintenance feature, but nothing else
  • no ORM – it works much faster without it.  developed his own higher-level constructs like Expression builders.
  • really extensive app server – but there are just customers, orders, etc… in there.  what is all this code doing?
  • ignored the selling product – it uses MySQL, how yucky.  it’s mostly a Swing app, how uncouth.

Worst of all management had no idea what this guy was doing.  When they asked about interfacing some 3rd party app – he said sure, and started coding something homegrown.  When they asked about integrating the selling product – he said sure, and started coding the next homegrown section.  Piece after piece after piece.

What a nightmare.

Visualizing Code

No comments

A long-term client contacted me about a couple issues with their site.  Very common place coding should have been done correctly with my eyes closed – download document (from PHP) and authorize.net recurring payments.

I dreamed up all kinds of reasons that the issues existed.  This has usually been helpful in the past – try to come up with different scenarios before looking at code.  Try to visualize what the code is doing.  Got trained to do that somewhere along the line.

Once I looked at the code it was just wrong.  There was test coding around the edges that hadn’t been removed.  Fixes in place in 10 minutes versus time wasted visualizing.

Maybe I would be better off jumping into code asap?

Similar story with another client.  Should be mostly a re-skinning project.  Mind-numbing change 1000 references to widget x to widget y.  Estimates given are based on common practice – how would someone normally code such a thing?  what kinds of re-use would be involved?  how to account for configuration changes?  Again, the visualization thing.  Finally look into the code and your head explodes.

I should know better.  I have been programming for a long time.  I think people generally do a good job but pressures are always in place to get it done faster.  Hence the name of this blog (See About).

What is your approach?

I think we have all evaluated a proprietary product, whether it’s something from Microsoft or a company you never heard of, but has the product you need.  The basic idea is to reduce risk.

  • How long in business? profitable?
  • How popular is the product?
  • Size of company?
  • Release schedule?
  • Bugs/issues?  How quickly resolved?

But in these times it is likely that an open-sourced product does what you need.  There are likely even several versions from different vendors available.  Some vendors very large and well known, others one-man shows.  You search on google for the product and get 500,000 hits.  What do you do now?

The idea is still to eliminate risk.  I think you have different concerns though:

  • How long in business? profitable? – don’t really care.  there are plenty of people out there working in this solution, you have the source, you can get someone else to fix it.
  • How popular is the product? – probably not an issue.  yes, this particular vendors implementation varies, but quite a number of people have experience working on this
  • Size of company? – a question of long-term ability.  again, there are quite a number of other people out there who can help
  • Release schedule? – it would be great if you saw a steady stream of development, but this is unlikely.  the spec defining the open-source tool has probably been around for a while and there really shouldn’t be much of a need for updates
  • Bugs/issues?  How quickly resolved? – probably when first released there was some furious coding, but after a while it should be stabilized

So, how do you do this?

I think you can look at it like hiring a remote contractor:  You need someone to help out with implementing a well-known solution and want them really to go away afterward.  Maybe call them once in a while if you turn on some other feature in your product and don’t remember what steps are needed.

As such, some concerns you want to know:

  • Responsiveness
  • Experience in the area
  • How much other (not in the product you care about) work do they have?
  • Any skills matching up to your developers (Java, C#, database, etc…)?
  • Are they using the open-source code in a retail product they are selling?  Self-interest is always the best motivator

How do you evaluate open source software?

We have all been on projects where management decides to change resources, time and/or features of an ongoing project.  The old chestnut used to be these are the 3 things you have control over, pick 1.

Thinking about this I looked at what would need to happen if we changed one variable:

  • If you reduced Resources then you would need more Time or less Features
  • If you reduced Time then you would need more Resources or less Features
  • If you increased Features then you would need more Time or more Resources

But thinking about this, changes in Resources usually make no difference in the short term.  Adding another developer does very little until the developer comes up to speed.  Depending on their expertise this can take a while.  So, what if we treated Resources as a constant?

We end up with a relationship like: Resources = Features over Time (R = F/T).  Sounds really pretty nice.  Kind of makes sense.  Can be looked at as a productivity measures, quality measure, etc… as well.  Again you don’t expect the productivity of your team or their quality level to change much in the short term.  Walking through the cases above we get relationships to still work.

I think this is a nice clear mathematical argument even the most demanding manager can understand.  Oh, you want to increase Features this late in the project.  That means we need more Time – see here…  Yes, we can increase our productivity, but not in the short term (phase) of this project.

Failure Rates

No comments

We bought a Sony World Tour game for someone this Christmas.  This is one of the most popular games this past season.  Typical out of stock problems for a popular seasonal item.

It never worked.  Tried replacing various components about 6 times.  Finally returned the whole thing to the store.

At the store one of the clerks mentioned, “Oh, yeah.  Those don’t work a lot.  Be thankful you didn’t buy an XBOX.”

This prompted my curiosity, “What do you mean?”

The clerk said, “Oh, the XBOX’s fail about 75% of the time”.

Being in software I was a little flabbergasted.  “How can that be?  These are extremely popular items; that YOU sell.”

“I just work here.”  Realized he had revealed too much and proceeded to hush up for the rest of the transaction.

Being in commercial software I have lived through the horrors of bugs in software that persist well past the FCS date.  Some of these can be fatal, most are doggedly consistent errors in logic that just mess things up.

Maybe I am being stuck up, but the game software DOES NOT START!  This is not even close to an FCS quality level.

Do game software writers not work in the same realm as the rest of their brethren?  Are they immune from the QA/QC police?  Management/marketing breathing over your shoulders about typos and nits they found over the weekend?  Heaven forbid your application does not start, boot, begin correctly for them in whatever strange configuration they manage in their household.

The more you think you understand something the less you actually know.