Archive: September 2015

Some More Reasons Why You Don’t Have Good Automation w/ Update on Generic UXD Legos Source Code Packages

Update on the example source code packages for UXD Legos:

I am writing the sample page objects to demonstrate how to use the API and its components. I have refactored a lot of things since I presented the API at the conference. The configuration service implementation has been refactored to auto-initialize itself by reading in all the available configuration profiles. It has also been updated to be more robust in a multi-threaded environment.

I have significantly re-worked the model for controls on UI components. The new version is more flexible. Controls are treated as separate entities from the components they control. Since some interactions, like dismissing a UI component, are actually achievable by multiple controls, it makes sense to do it this way. My original design didn’t robustly handle this. For example, a chooser dialog can be submitted, or cancelled, or closed via three different controls. All three actions result in the dialog becoming hidden from view, but my abstract model for components which can be toggled visible and invisible didn’t handle the fact that there could be multiple controls that dismiss a component from view. The new model can handle all three of those scenarios in a generic way. I am working out a good interface for the builder of the UI component to handle specification of all of its controls. Each type control I modeled has a specific builder implementation for it, which is great if you are just focused on specifying and building a control, but not so great if you just want to specify and build the whole component, along with all of its possible controls. It’s clunky to have to interact with more than one builder. I think I have found the right design for this, and I am implementing it right now, along with the sample page objects.

I think, in addition to delivering the C#, Python, and Ruby versions of this API, I will also need to write an updated slide show about it.

And now on to Why You Don’t Have Good Automation:

I am grumpy today. I had a meeting this morning to discuss the automation strategy for a major product. We ended up talking about how the test suite is growing in size by 5-15 tests a day and I pointed out that I have a GIANT bottleneck in the form of inadequate infrastructure for executing Selenium test suites. It was revealed that this is a big problem for many other teams as well. There is no company-wide Selenium grid system available. We also do not have the green light for using Sauce Labs. I was under the impression that other teams had their own private Selenium grids which they jealously guarded because they are a limited resource and building a comprehensive Selenium grid is a big task. If you read my previous post on this subject, it’s not particularly easy to acquire the necessary resources and IT support to build one. I suspect that the reason we do not use Sauce Labs is that we would need to scrub all data in the test environments of personally identifying information. Which… will probably not happen anytime soon, so let’s backtrack to Square One: there is a giant bottleneck related to the lack of test infrastructure for running large suites of Selenium tests. It’s just crazy that there are tests that could be running with every build, that are not running because we only have the infrastructure for getting quick results for the highest priority tests.

If your company, like my company, spends a lot of time recruiting engineers with Teh Awesome Mad Skillz, but they do not budget resources to handle all those awesome automated tests, they are wasting a lot of money and a lot of talent. They would be better off hiring a cheap team of off-shore manual testers to run the tests manually. Otherwise, you have to force those amazing engineers to run tests manually in lieu of writing automation you can’t support in order to ensure that your products are tested. It’s expensive to hire development engineers to run manual tests all day. Plus, it pisses them off and then they leave for jobs where they get to write code.

The other issue that came up is that my tests take a long time to run because we have _TWO_ whole test environments available for an engineering organization of hundreds. TWO. WHOLE. ENVIRONMENTS. And they are unstable because there are too many people using them. Hence, all my tests have to do a lot of checking in order to capture failures related to environment craziness as opposed to real failures so that they can be re-tried until a verifiable true pass or fail can be determined. Other teams are running destructive tests all day long on these environments, so I can make absolutely no assumptions about the availability of test data on the system. Therefore, I have to make each test method search for test data that meets the necessary conditions for the test first.

The end result is that the bottleneck in Selenium grid infrastructure is doubly painful because the terrible test environment situation requires lots of retries and querying that would be unnecessary if each engineer had an isolated environment only they were using. Tests take more time to write because of all the extra overhead of dealing with the environment, so I’m not as productive as I could be. Not that it matters because…. Square One: Inadequate selenium grid infrastructure to run all the tests.

To summarize:

Automation is not magic dust that is gathered by tiny little fairy hands and deposited at your doorstep overnight for free. It takes a lot of planning and infrastructure. So, you like… need to invest in it and stuff. If you don’t, you won’t have good automation. You will have shitty automation that mostly functions as an Automated Smoke Test Teddy Bear that you can cuddle up to at release time for that warm feeling of false security that it gives you after forcing your resentful automation engineers to run manual tests until they are ready to grab the nearest fork and dig out their eyeballs from the sheer boredom of it all.

Share This:

Why You Don’t Have Good Automation w/ Update on UXD Legos Source Code Packages

Update on the example source code packages for UXD Legos:

I’m almost there! I could not resist the temptation to refactor the code from my presentation to handle weaknesses I was hoping no one would notice at the conference. The design for Accessible/Dismissable components didn’t really address the components which are accessed and dismissed by a combination of hovering and clicking on accessor and dismisser WebElements. That was my secret plan all along — present far too much content and speed through slides full of complicated Java source code to prevent anyone from comprehending it enough to notice that my abstract interaction model for these components was lacking something.

Putting together this example package really highlights just how much I overshot the bounds of a reasonable amount of content for a 45 minute presentation. Not only is the amount of source code way too much, there were too many different design concepts to cover. I think I could have easily broken this up into three 45 minute presentations. Sorry again, conference-goers who sat through my presentation. We’ve all had those college professors who did this to us, right? Start nice and slow with something simple and digestible for the first 10 minutes, then jam a semester’s worth of content in the next 30 minutes. The good news is that there’s no final exam at the end of this.

The task of providing example source code in other languages is going to be a longer term effort. Some of the design tricks I used are reliant on concepts and features that are really Java-centric, and more specifically, Java 8-centric. It will be an interesting exercise in figuring out how to translate it to C#, Python and Ruby.

Now on to the actual subject of this post.

I am grumpy today. I work for a large, multinational technology company. I came here after working a couple of start-up gigs at small companies with less than 25 employees. When I took this job, I was SOOOOOO glad to be joining an organization that I was sure would have resources I could only dream about at the cash-strapped employers I had been working for. It hasn’t been anything like that. In my last job, where the CEO regularly cursed out creditors over the phone for not realizing just _who_ he was when they dared to call him and ask for repayment of funds they should be glad to not be getting from someone as important as he was, I had three monitors. I could have my test plan open on one screen, a JIRA screen open on another and my Eclipse IDE open on a third. All I had to to was ask the IT guy (and there was only one of them, so I knew who to talk to), and it happened. When I wanted a VM, I asked the same IT guy and it was done within a day.

At my current employer, I have been trying to get a second monitor since I started working here 3 years ago. Every time I go through official channels, I am told that it is not company policy to provide a second monitor. Apparently, increasing employee productivity via inexpensive and trivial perks like additional computer monitors is just not one of those things they want to concern themselves with. I am now reduced to thieving the second monitor one of my former team members somehow finagled through unofficial channels. It’s just sitting there at his now-empty desk, left behind when he transferred to another team a few weeks ago. If I were feeling bold, I could take both his monitors and hide one in my file cabinet as a backup in case one of mine died.

When I want a VM, I have to enter a ticket in an impossible-to-navigate system, fill out a long questionnaire and…. wait. For a really long time in particular if I want a Windows VM. I requested some VMs before the conference because I had these big ambitions of building this amazing selenium grid with TWO WHOLE NODES during the Selenium Grid workshop on the first day of the conference. I couldn’t install anything on them because the process to grant me sudo privileges failed to work for my VMs. Before the conference, I entered a ticket about this problem in the vain and naive hopes that it would be resolved before the conference started.

When the ticket sat untouched for over a week, I resorted to looking for people who might tangentially be related to this part of the IT support staff to pester via email about solving this problem. As the conference grew near, I finally got a response from someone who said that it wasn’t their job to deal with this even though his name is listed on the web page for requesting sudo permissions on lab machines. He forwarded the email to someone else, who solved one problem for me, but not the remaining issue I had with adding a repository to apt-get so I could install Java on my hub VM. It worked perfectly on my two node VMs. My subsequent emails inquiries failed to generate a response. Today, I got an email from the help desk system, almost a month after I entered my original ticket complaining that I had no sudo privileges on my VMs. Apparently, the ticket was finally assigned to someone today.

So, this first edition of Why You Don’t Have Good Automation has to do with the demoralizing effect of tedious and silly barriers to the acquisition of reasonable resources to do a good job. If you make your employees feel like they are extras on the set of Candid Camera, Kafka Edition as they engage in seemingly futile efforts to achieve small gains in the face of an absurd and meaningless bureaucracy, you will not have good automation. You probably won’t have any at all.

Share This:

Selenium Conference 2015: Portland Oregon

I bought this domain and paid for a web hosting account right after the end of the Selenium 2013 conference in Boston. Life intervened in the form of 60 and 70 hour work weeks and a second child. This year, I wanted to tackle my long-standing fear of public speaking and submit a proposal for the Selenium 2015 conference to present an API I had developed for building generic representations of common UI components.

I was shocked when the proposal was accepted, and then I was terrified because I hadn’t written any slides. I scrambled together a very source-code heavy presentation that quickly became too long for the 45 minutes I had asked for. So lesson one from this experience is that you should have no more than one slide per minute of a presentation and even that is pushing it. A more leisurely presentation would be one slide per 2-3 minutes. Lesson number two is that source code should be limited unless you’re doing a workshop-style presentation.

My husband, Dan, and my children accompanied me to this conference and we drove from Massachusetts from Portland over four days. Dan and I had never been on a cross-country road trip, so it seemed like a good time to do this. Lesson three is that you should budget five days for a cross-country road trip with two small children. We didn’t make the hotels early enough on two nights to take the kids to the pool to relax and decompress from the long ride and we had to get up really early to drive the next day.

We took the kids to a local amusement park in Portland with some lax safety standards that allows kids under four feet to ride things most parks wouldn’t even dream of allowing them to get on even with an adult accompanying them. Of course, the first thing my 3-year-old son wanted to ride was the craziest thing he could get on as long as he was with an adult. Which was me because Dan would never consider riding anything like the Rock-O-Plane. It consists of a steel cage which you can rock insanely while you are spun around on a Ferris wheel-like structure for about 4 minutes.

I hadn’t ridden anything remotely like this since high school, and the last time I did, I puked on the ride. When the Zero-G sensation set in on the first rotation, I had my first Oh Shit! moment. There was no getting off this thing until it was over. When my son glanced at me sideways after the first two rotations with a nervous giggle and an expression that said, “Should I freak out now? Let’s see if Mommy is freaking out.”, Mommy started screaming ‘Whee!” at the top of her voice because there was no way I was going to provoke hysterical fear in my 3-year-old because I was too wimpy to handle a carnival ride that an adventurous 10-year-old would find tame. I figured I might as well get used to this because I will be riding even scarier things until he gets over the height limit for riding these things alone. Which will be a while because he is really small for his age.

The feeling I had before I started my presentation was exactly like that. Oh Shit! What did I get myself into? Why did I do this? What was I thinking? It wasn’t unlike the moment when the real labor pains set in when my son was born. But, unlike the Rock-O-Plane and this presentation, I sidestepped that whole excruciating labor business with a quick epidural. Not that having a needle jammed into my spine was not terrifying, but I could hear the muffled screams of other women in the hospital and there was no damn way I was having any of that. But, there’s no epidural for public speaking that is professionally acceptable. I am not Kanye West, after all.

So, I did it. I slammed through way too much source code. I lost my place in my slides with an errant swipe of the fingers. I tried to show a web page example which didn’t display on the projector because I had not set up my computer to handle 2 monitors. People’s eyes were probably glazing over. Lesson four: Don’t use a computer with an OS you are relatively new to to do a presentation. Lesson five: See lessons one and two.

I couldn’t see the audience. There was a bright light shining in my face the whole time. So, apparently, I was losing them with the over-whelming source code tidal wave and I was  not able to see this when it was happening, but I could feel it anyway because I knew that I would be overwhelmed if I were in a presentation with this much unfamiliar source code :-/. My sincere apologies to the people who sat through my presentation. You got to experience first-time speaker syndrome. Hopefully, the release of the source code with some examples will make up for that. I am working on putting that together now. And to address the concerns that it was just all too Java, I will try to implement these ideas in Python, C# and Ruby in my free time or when the boss isn’t looking.

I love code and I wanted to share something that had speed up my page object development process by light years in the hopes that it would help others do the same. It’s my ugly baby and I adore it. It’s definitely prettier than endless if-then-else clauses to handle silent failures of native Selenium actions in certain environments and lots of repeated boilerplate code to implement the same components over and over again. See you two years from now at the next Selenium conference in the States? Don’t worry, I probably will not work up the courage to do this presentation thing again. I have crossed it off my bucket list for now, so I’m good with that.

Also, Utah, Wyoming, Idaho, and Montana are gorgeous. I can’t wait to do a cross-country road trip through the Southwest now. Portland is the nicest city I have ever visited. It has displaced Cambridge, MA as my ideal urban place to live. It is the fastest gentrifying city in the country, so it will probably be as crowded and over-priced as Cambridge very soon.

Slides from my Presentation:

Video from my Presentation: (Sorry if it is overwhelming/boring. I was just trying to get through the material and not freak out):

Share This: