Expectations

I can’t decide if one coworker is ridiculously literal or what. I’m trying to convince myself the guy isn’t stupid, because you don’t have a programming career if you’re stupid but still… The best I can manage is that he’s mostly worked in a field where what he gets for specs is spelled out to the last crossed t and dotted i. Which does not work with something that’s grown like Cthulhu on steroids.

Take an interface to the Evil Third Party. It includes the ability (sort of) to do what is known as a Single Sign On – you log into my employer’s software, click a button and a new window opens with you automagically logged into the Evil Third Party’s software. To do this, someone has to associate your login for the Evil Third Party to your user account with Lovecraft, Inc (okay, that’s not their real name, but there are times when it feels that way).

So. You, being a good little programmer, are told that the Evil Third Party remote login feature supports the characters A through Z, a through z, 0 through 9 and a smattering of others including dash, dot and the at symbol. You are also told that Lovecraft, Inc wants to tell the users when they press the magic button if their login has characters outside the list. Do you:

  1. Block the software from saving any logins with characters that aren’t in the list as well as block the magic button if somehow someone manages to save something with “bad” characters?
  2. Get a second list that was being sent to Evil Third Party with a message to say “You told us your remote login worked with these characters and it totally doesn’t” and block only those characters from saving logins, but still block the magic button when the login has bad characters?
  3. Confuse both lists, fail to understand every attempt to clarify things, ask for reproduction steps when the tester tells you she can make the software save characters the DATABASE doesn’t understand, and then wonder why the tester is trying to put a head-shaped dent in her desk?

If you said number 3, congratulations. You are why I periodically harbor fantasies involving lining the streets with impaled programmers.

I gave myself a time out before I acted on said fantasies. Or asked said programmer if he was just trying to piss me off because I found some of the hidden places where everyone had missed a field size change and he didn’t like me pointing to lines in the code. (Three attempts later, he still hadn’t found the one missed size change I know is there. But he doesn’t want me “criticizing” his code, so he’s going to keep getting it back with “the field is still being truncated” until he works it out himself. His choice. I’d have simply said “here. This procedure, line 25. Change the declaration for this property to have a length of 99. Is there anywhere else that this thing could be hiding?”)

I started the code dives because this software is so convoluted changing something in one place is no guarantee you’ve actually done what you thought. After the fourth… fifth…. tenth time an issue bounces back because I’m seeing the same end result, I get irritated and so does the programmer.

Am I being so unreasonable here?

FacebookTwitterGoogle+Share

Another week, another round of madness

On the work front, they’re actually trying to get everything on the staging server there before the end of the week so I have – HORRORS! – almost a whole week to test the integration. Whodathunkit? Even better, I got into the nitty-gritty of the Team Foundation Server Report Server and built a report that should save the integration crew (both of them) about 8 hours collating the issues, changesets, and files.

Yes, files. For my sins I work with classic ASP, the web model that was supposed to have died over 10 years ago. Not in my world it didn’t. So of course, nothing is compiled or anything so sophisticated. Deployment means copying files around (because nobody’s managed to get the system automagically syncing the source control and the servers yet) and running scripts on databases. It’s still way better than things were when I first started this job a little more than a year ago – there wasn’t any source control and there wasn’t a test server either. Just a dev server that spent most of its time broken because of the stuff being worked on and the live server. Now things are a bit cleaner (and I can actually test things without stepping on people’s toes, asking them to patch something so I can do something else, testing live, or any of those other fun things) we’re tracking work in something a bit less awkward than fifty different excel spreadsheets (I’m not sure exactly how many there were – I never counted) and bit by bit we’re building something more or less systematic and functional.

On the home front, we have allergy central, The Husband is recovering from an argument with gout and can walk again, and the cats alternate between being demanding little bitches (yes, the boys too) and snuggling up and looking like fuzzy little angels. When they’re not on one of us demanding snuggles.

Oh, yes. And weekends are never long enough. Ever.

Missing the Point – the tester edition number eleventy bajillion

This afternoon was a classic example of missing the point of having a staging server. You know, the one that is mostly a mirror of the production environment which, being one of those web applications that thousands of customers consider mission critical, isn’t ever allowed to be down. The one that is supposed to have code and database changes intended for release migrated to it a week in advance so that any problems with integrating the several hundred bajillion lines of interesting code that have evolved over the years into a tentacly mess to rival Cthulhu himself can be found well before the targeted deployment date so that pushing changes to the live servers goes as smoothly as humanly possible.

That one.

What actually happened was this: integration started after lunch today. For a deployment tomorrow at oh god am. It finished shortly before I usually leave. So instead of several days to test the targeted items and maybe do some regression in the critical areas (trust me, full regression is the kind of dream that makes winning the lottery look like something that happens to you on a daily basis) I have in “official” work time maybe fifteen minutes. Naturally, I stayed until my balance started to go, and the developers were still arguing with the Wonderful Source Control that doesn’t understand the difference between a change and a file with a change so dragged in a bunch of other changes that aren’t going out at oh god am tomorrow morning.

Ladies, gentlemen, and others, the point of having a staging server is not so you can have two nervous breakdowns for each deployment instead of just the one when your integration crashes and burns on the mission critical live environment. The point is so you can do this strange thing called planning and have everything ready and tested against something that was just like the live environment until you did the integration. So you know (within reason – this is a black art we’re dealing with) that you won’t have any serious issues when you do deploy.

Oh, and when your one and only tester is narcoleptic? It’s a really, really bad idea to leave it all until the evening before you deploy. Just saying.

Why Testing is Hard

I spent most of today rerunning the same tests over and over and over and over – trying to get a fix through where each round of fixes broke something new. No, this wasn’t the developer’s fault. He’s dealing with a horribly “interesting” codebase with even more “interesting” legacy implementation choices.

Take a large web application with multiple customers. Users can be linked to one or more customers. The user has the login but they can’t to anything without picking which customer they’re working with. Naturally, there’s a linking table that connects users with customers. So far so good.

When you’re doing stuff with users, you can edit ones who are already set up with the current customers, you can create completely new users (who will link to the current customer) or you can link an existing user to the current customer. Still not so hard, right?

Right… Now, add the user identifier for the Evil Third Party Interface. Which must be linked to both the user and the customer. So, it gets added to the linking table because that’s the logical place for it. And now the fun begins. Because the person who is logged in can also have the identifier for the Evil Third Party Interface, and who gets to see what is controlled by privilege, to make sure this one new thing did what it was supposed to, I needed to check all sorts of things:

  • I’ve got the Evil Third Party thingy, I’m a god user, and I change something else in my user record.
  • I’ve got the Evil Third Party thingy, I’m a god user, and I change my Evil Third Party thingy.
  • I’ve got the Evil Third Party thingy, I’m a god user, and I change something for someone else who has the Evil Third Party thingy.
  • I’ve got the Evil Third Party thingy, I’m a god user, and I change something for someone else who doesn’t have the Evil Third Party thingy.
  • and so one…

There ended up being rather a lot of these lovely combinatorial options just to check the addition of one simple field. The poor developer was as frazzled as I was by the time we finally got them all doing what they were supposed to do.

And this, Ye Who Do Not Test, is a pretty simple situation. Believe me, I’ve seen worse. I live worse. The last time someone asked me why I (the lone tester for a team of 10) don’t do much regression testing, I showed them my half-finished regression guideline wiki. Which is now somewhere in the order of 300 pages, all of them with multiple cross-links because one small change here can affect things in half a dozen other places, and the validation rules need to be seen to be believed.

Brainless checking is easy. Any monkey can do it. Testing? That’s hard. Figuring out the combinations in the first place is hard enough. Working out which ones you need is harder.

Long Day

I go to work early. Partly because I start to fade around 2 or 3 even when I sleep disgustingly late, and it makes sense to get in as much that needs thinking and a functioning brain before my body decides “hell no”. Unfortunately that means if I’ve got to stay late – like today when a release went out after 5pm (gotta love live web stuff. Can’t take it down when someone will be using it. Even if the “someone” is the company’s internal people who could work around things so long as we did it in a quiet time) so I didn’t get home until nearly 7. Yay for 12 hour workdays. Not.

It’s also my Mad Genius day. This means I write a post a day or two before and set it up to post early in the morning. When I get home in the evening I respond to comments. Today’s effort was a bit of an expansion of my thoughts on three of the panels I was on at Ravencon. Next week I’ll probably ramble about one of the other panels I did, unless someone in the writing world does something spectacularly stupid that I can rant about.

And on that note, it’s time I went to bed.

 

Quiet Day

Quiet day at work today – I spent a good chunk of it decommissioning my collection of Windows XP virtual machines and setting up a series of Windows 7 virtuals to replace them.

Why do I need so many, I hear you ask? Well, if you want to test a website properly, you need multiple versions of Internet Explorer, and that means multiple systems. So now I have a system with IE 8 (third largest slice of the employer’s customer base), one with IE 9, and one with IE 10. My machine is now on IE 11 (largest slice of the customer base), and also runs Firefox and Chrome. Then there’s the Macbook that lives on my desk to test Safari. Chrome/Safari is the second biggest slice of the customer base, so Chrome and Safari need to get tested as well.

It’s quite remarkable how many bugs are specific to one or two browsers. Even more fun are the “bugs” which are actually the browser working as designed, but our software exploits a fault in the IE family…

Oh, yes. Browser compatibility testing is fun. I am so grateful I don’t have to test for IE 7 compatibility as well. And I am terrified by the users who are still running with Windows 2000 and IE 6. If they ever report a bug they’re probably in for a very nasty shock.

Little Things Matter

Also known as “Fun with testing software, episode the infinity-somethingth”.

Today at work the helpdesk calls were coming in fast, all from the customers using the venerable (but stable and more importantly usually working) PC payroll program, complaining that their new FTP passwords (they’re reset periodically for security reasons) to our systems weren’t being accepted. The plot thickened with the discovery mid-morning that pasting the new passwords worked just fine. Keying them failed. By late afternoon, the truth was discovered (and with it, the list of who had not moved over to the newer fully-encrypted FTP module)… The venerable software, written to work with an equally venerable mainframe system that does everything in all caps naturally uppercased everything that went in. Including – ta-da! – the FTP password. Which is – of course – masked, so users couldn’t see that it was busily uppercasing itself.

Such a little thing, and such angst it caused. Yes, now everyone in the programming and test team knows why our passwords were always in all caps in the past.

Lesson learned: do not neglect the little things, for they shall trip you when you least expect it and you shall spend much time going “WTF?” before you figure it out.