The first two articles are here:  Part 1   Part 2

Original article: iPhone Tech Talks: Part 3 – Libraries and Performance

We’ve posted the 2009 schedule for the Indianapolis Workshops on Software Testing. We’re looking for people willing to share experience reports and people who just have a general interest in the various topics and would like to attend and ask questions.

Here’s a summary of the lineup:

February 26, 2009: Presenting test results

March 27, 2009: Test-driven development

June 27, 2009: Test design and development

August 28, 2009: Testing mobile devices

September 26, 2009: Automation and performance logging

October 29, 2009: Time management for testers

November 21, 2009: Managing your focus while doing exploratory testing

If you’re interested in attending any of the workshops, please drop me a line at mike@michaeldkelly.com.

I sent him an email saying that I would be interested in the March and October sessions. Perhaps you have some interest in the same/other sessions?

Original article: Testing Seminars

I’ve edited liberally, so here goes…

Workshop on Regulated Software Testing

May 15, 2009

The WREST is coming to Indianapolis again in 2009. The topic for the workshop is “Beyond Scripted Testing.” You can learn more about the workshop here.

…

Major Questions of Interest

How can we better test software in regulated environments while retaining processes and documentation that will pass regulatory audits?

How can we prove the value of implementing progressive test approaches within the context of the necessities of regulation (FDA, SOX, CMM, etc.)?

Workshop on Performance and Reliability

April 16-1716-18, 2009

Through much arm twisting and whining, the WOPR 12 organizers are hosting WOPR next year here in Indianapolis. The theme for the workshop is “Resource Monitoring During Performance Testing.”

You can find more details here.

…

The ultimate goal of performance testing is identifying and fixing performance bottlenecks before production deployment. While a large percentage of bottlenecks are ultimately traced to the application code, they may also occur in servers, system software, and network components. Critical to identifying their source and diagnosing their root cause is monitoring the “bellwether metrics”. Let us define “bellwether metrics” as the “key indicators of future developments or trends”, and thus the minimum set of system vitals that we should be measuring for each hardware and software component.

Thoughts

The process for applying to share an experience report was pretty easy. Just have some relevant experience and know enough about it to share. Come up with a presentation. Kind of like a brown bag really. If either of these two are similar in size to IWST, they are enjoyable because you get to know the people at the meeting. If you’re jazzed about this kind of stuff, you should totally do it, for both the knowledge and the experience.

Original article: Indianapolis Conferences

The first point the presenter made was that the percentage of time that you spend on portions of the development cycle is quite a bit different for a successful iPhone app than most software applications. He showed a diagram that looked way cooler than this:

Difference between normal development and iPhone development

What this means is that you need to spend a lot more time in the design aspects of your app throughout the life cycle than you normally would. This set the tone for the rest of the presentation.

He defined “design” as mapping out requirements, doing the art and layout, and figuring out how core features and navigation are going to work. The cycle for developing a new product should typically go: familiarize, conceptualize, realize, finalize. Executing these well will take your application from average to excellent.

Familiarize

One solid suggestion was to read through Apple’s Human Interface Guide. Apple has spent a lot of time thinking about how people will interact with the iPhone and iPod Touch, so you don’t need to duplicate their work. What’s more, there are many conventions that you will be unaware of and unwittingly break. For example, Apple intends all buttons to have rounded corners, so if you have something that’s not a button, don’t make it rounded. Likewise, buttons should look like buttons because users have a model of what a button looks like and what it does.

The presenter reiterated considerations voiced in the general session, such as thinking about where the user will use the application and considering general properties of the iPhone and how it differs from the web and from a desktop computer.

One significant suggestion was to design for the thirty-second use case. You need to consider that the user wants to open your app and get what they need within thirty seconds. If your app has a bunch of meaningless screens or has poor performance, it will significantly hinder your ability to help them. If it takes twenty seconds to navigate where they need to go, you only have ten seconds to please them. If you can save any time by remembering or determining data (zip code for searching for restaurants, for instance), you should do that. It takes a lot of time and mental effort to type things in on the iPhone, and it would be very annoying if the user has to type something in multiple times.

The Pareto Principle must be invoked at least once per usability discussion, so here goes. You should design for the 80% use case. Make the most common functions be dead simple. With the limited screen space that you have, consider eliminating ancillary functionality. Make your apps hyper-specialized and super-focused. Apps that do everything do nothing. I liked this approach because it reminded me of something that the folks at 37signals would say (review forthcoming.)

Conceptualize

The key to conceptual design is to define a solution not a collection of features.

Consider the example the speaker presented: you are an interior decorator and you learn about the iPhone. You see something that makes you think of creating an interior decorating app. Your mind whirls at how this will make your life so much better. You grab a piece of paper and a pen, and scribble as fast as you can. “Wow, I can take pictures and store them in an album, and then upload it to my Mac. I can generate textures and hold them up to the wall. I can use the iPhone as a level to make sure that my wall hangings are straight. I can use this same app to make estimates of how much a room will take to redo.” The list grows to a page, then two pages.

This is exactly the wrong way to do an iPhone app.

You need to have a very lean feature set. If you start with an idea and brainstorming, or if you are modeling your app off of an existing app, you already have too many features. You need to prune.

A great way to prevent featuritis is to create an app definition statement. This statement summarizes the purpose of your app, and every key design decision that you make should be based off of it. With this much importance, it’s critical that all of the stakeholders buy into it. An example of an app definition statement is “Easy to use digital photo sharing for iPhone users.” Note that this definition statement clearly identifies who will be using the app and what they will be doing with it. Less is more with respect to features. Condensing your app is helpful for team morale and cohesiveness.

Realize

It’s important to understand your application type to make the correct design decisions. Apps live on a continuum of [serious..fun] and [entertainment..tool]. If you have a serious tool, you need the UI to be out of the way as much as possible. The user is just trying to get something done quickly, so there should be few frills. If you have a fun tool, you can get away with more graphics, but not at the expense of functionality. In other words, users will tolerate some distractions. If you have a fun entertainment app, it should be interaction driven and intensely graphical. This would mostly correspond to games. There also exists serious entertainment. This concept might seem silly at first, but could be something like a movie player. In this case, you would want the app to be more data-driven, and have a minimal UI.

These are the extremes, so you should consider to what degree your app is serious or fun and made for entertainment or utility. In conjunction with the app definition statement, this will help you decide what design elements should be used for your application.

As far as aesthetics go, you should remember to avoid having too much stuff on the screen at one time. Ask yourself: does this really need to be on the screen? Could it be hidden instead? Also: what’s important right now? Could this be shown later or on a different screen? Avoid layout cramming and make sure that your padding is consistent throughout the app. This is a professional touch that will separate you from the crowd. You should use built-in controls where appropriate because users are familiar and they are well tested, which saves you a lot of time. If you have options that aren’t commonly changed, put them in the global options file so you don’t have to keep showing them.

The biggest message to me of this section was to do paper prototyping or “iterate with paper.” Many fewer resources need to be expended to create a paper mock-up of the screens you are going to have, and getting the flow down is critical to the end user experience. You need to solve for all of the edge cases up front and see where your metaphors break down. Doing this on paper lets you experiment with several ideas and see which one fits your target user the best. One recommendation that made a lot of sense was to actually Xerox your iPhone or create a stencil to create a template that has exactly the right dimensions.

The presenter demonstrated how several apps evolved through time and it was clear that spending time with design was crucial to get their apps working optimally.

iPhone specific

This section is pretty specific, you can skip it if you aren’t interested in some nitty-gritty details. However, there’s one more section after this one.

For navigation, the standard is to move from left to right as you progress through the app. You should have a navigation bar with a title to give users an idea of what they are looking at. There should also be a back button. You should provide controls to filter data down.

A de facto standard is to have any kind of element addition be in the top right-hand corner of the screen. The same is true of edit controls.

Although you should have a title for context, you should avoid breadcrumbs and home buttons. Breadcrumbs are not scalable due to screen real estate, and both breadcrumbs and home buttons provide the possibility for a state change that is dangerous to the user’s mental model. If they accidentally tap one of the links in the breadcrumbs or hit the home button, they will become very confused and frustrated. This works for the web, but not on the iPhone, based on what Apple has seen in apps.

When you use lists and pickers, try to group similar items if possible. Associate an icon, as the eye typically sees icons first, then words. If you do have icons, try to make them minimalistic. If you have multiple screens with tables, try to switch the formatting up a bit to let the user feel like they are making progress. One more note on pickers: they are optimized for 12 items, which happens to be three swipes of a finger.

Finalize

You can (and should) enhance your application with photos and imagery. This aids in quick recognition of your app. Your icon is really key, as it is the first thing people will look at when they are evaluating whether your app is worthy or not. One thing that I probably never would have thought of is that users actually appreciate expensive-looking textures. If you can get some mahogany or marble or polished chrome textures, these will make your app look more valuable than other apps with standard backgrounds or textures. Don’t go all out and then skimp in the finalizing section. While your app will have great value, people often judge the book by its cover.

You can add excitement with animations, but be sure not to go overboard. You want them to give feedback so they are functional. A great example is how the iPhone home icons jitter when you are trying to move an application around. Functionally, this shows the user that they have entered a different mode than the standard mode. Emotionally, they see that some of the icons are excited because they might get to move to the front page, and some icons might be shaking because they are worried that the might get deleted! So it is fun and entertaining and also gives insight into the current state of the system.

I think that communicating application modes is a very important topic. My own example is when you start typing something in Microsoft Word but you keep overwriting characters that they have already typed. Because the ‘insert’ mode is not shown well, you might have no idea what the heck is going on, and can be extremely frustrated or even lose data.

Once you release, you can’t rest on your laurels. You should keep iterating and use customer feedback if you can get some. Better yet, if you can somehow track which features users use and how they use them, you can use this data to improve it empirically. That’s something I just thought of. But overall, you need to make sure that every release you have is high quality. Users don’t have patience with this kind of stuff. They will just uninstall your app without waiting for updates.

Let’s summarize: familiarize, conceptualize, realize, finalize. One more post to go–and that’s a lot of -izes!

You can see the first part of this series here.

Original article: iPhone Tech Talks: Part 2 – Interface Design

I took like fourteen pages of notes, so I will probably break it up into a few posts. This one, which gives an overview, one about usability (I was writing the entire hour and a quarter session) and one about performance and using different libraries.

What’s in it for you?

Hmm… Well, maybe you are doing or planning on doing some kind of iPhone development. That would be nice.

But I was talking with someone who has the G1, and the features are pretty much the same aside from native multi-touch, so most of the design principles and considerations will be present there. Indeed, any mobile device has at least some of the constraints that you need to think about when creating an iPhone app: usability, market reach, battery life, screen real estate, interrupts, performance, and more.

I would say that this was my first foray into mobile development, so it was definitely an eye-opener. If you have developed for a mobile platform before, then some of these comments will be old hat, but there still might be some gems in there.

Overview

The session lineup can be seen here. I figured that I had a moderate understanding of Room A or could read about it online, so opted to stick in Room B because the C room sessions just didn’t really seem all that interesting. You could mix and match, but conveniently the ones that I wanted were in the same room. I did sit in on the “Using Advanced Web Technologies on iPhone” session, but it was after lunch and not that interesting, so I ended up just going to the other room after about forty-five minutes. It was a lot more engaging. It’s interesting making decisions with very little information on what the actual session will be about. I’d be interested to hear if anyone else has rules of thumb for figuring out which sessions are going to be useful, or what kinds to avoid.

So everyone went to the general overview session at the beginning of the day. They started out with some “get pumped” videos with the standard Apple music, showing off some popular apps like MLB.com and Urbanspoon. Then they gave a business update. The iPhone platform is very vibrant, with four million iPhones purchased in the first 200 days, 13 million to date (not counting iPod Touch), and over 200 million App Store downloads in only 104 days. These are impressive numbers that seem to imply that if you create a good app, you won’t have trouble getting it into people’s hands. The iPod Touch mentioned earlier is significant because it also runs App Store apps and has the same features (even screen dimensions), save for things like phone capability. So that’s another thing to consider.

Next they gave an overview of ingredients for success with an iPhone project. If you have a product in mind, it’s recommended that everyone who will be giving input has at least some working experience with using the iPhone. This makes sense, because there are some pitfalls if you don’t consider what you have to work with. There is no mouse, so there are no rollovers or scrollbars. The metaphor of clicking doesn’t apply because you are actually tapping on the screen. The mouse offers continuous input, while the finger is discrete. The interface is quite a bit different because of user expectations and use conditions. Text input is accomplished with a soft keyboard rather than the traditional one. The iPhone only runs one app at a time.

These are things that are clear if you use the iPhone on a regular basis, but imagine working with people that have never used it before. There would some difference in ideas that would not be easy to overcome.

Also, you want your app to feel like it fits the iPhone, so you need to consider things that are already built-in: microphone, camera, location awareness, ubiquitous internet access.

Typically you should envision that your user will be using the iPhone under a table in a poorly lit room at about arm’s length. This severely alters your approach to designing the user interface and visual elements. The screen needs to have high contrast. There was a discussion about whether white text on a black background was better than black text on a white background. The thought was that you should consider where your user will typically use the app, and go from there. If they would typically use your app in a poorly-lit environment, then you should make it white on black. Conversely, if your app is only used out in daylight, consider using black text on a white background for better contrast with the environment.

As a programmer, you need to handle interrupts gracefully. Because the iPhone is a phone, you should expect it to ring at any time, and should test the behavior frequently. You need to save the state of your app somewhere, cut out your audio, pause the game, and so forth. When the phone call or other interrupt is done, you should also resume in an orderly fashion. Just unpausing the game when the phone call is over will lead to your user being unhappy.

Attributes of a successful iPhone app

Your app should be delightful. This means that it is inviting, intuitive, engaging, exciting, and enabling. It should be a pleasure to use, and is something you would consider remarkable. If your users are to remark positively to others, it must be remarkable.

Your app should be innovative. It should be revolutionary, inspirational, or fresh. This doesn’t mean that you have to create something new, just do it better than what is out there in some way.

Your app should be designed. This takes support from all levels of your organization, and works best with small teams.

Your app should be integrated with the iPhone’s many rich features. One of the strengths of the platform is connecting to other services, so your app should probably be connected.

Your app should be optimized for performance. They made a suggestion that you automate performance tests so that when code is checked in, the performance tests automatically run. If the test fails, the build should break so that the offending code can be cleaned up for better speed. This was a nice idea, and I had not heard it before. But with the importance of performance, it’s almost as important to get it fast as it is to get it right. So this matches up well with continuous integration. Consider it ‘continuous performance.’

Other miscellanea

Over lunch and between the rapid-fire sessions, I actually ran into many people that either had ties to Indianapolis, or actually worked or lived there. So that was pretty exciting. I figured that I would be the only one. There were others from Bloomington, etc.

I enjoyed the tech talks, although I didn’t really enjoy the driving in Chicago. It’s quite different beast from Indy. There are like horses and bikers and pedestrians and crazy taxis all over the place.

They gave out a free shirt that had some interesting packaging, which you can see here.

Original article: iPhone Tech Talks: Part 1 – Overview

The first experience report probably had the most relevance to things that I have directly worked with. In the production Rails project that I worked on, we used Selenium for automated integration testing as verification of the development process. Dave Christiansen covered using Selenium to do automated integration testing on a Rails app. However, what was noteworthy to me about the approach that he took was that instead of writing and executing these tests only in the test phase of the development process, they were written and executed throughout the development lifecycle. He wrote a wrapper to generate RSpec tests from the Selenium IDE, refactored to DRY up the code, and then had the team write their own integration tests as they were developing. Since the tests were written in RSpec, they would actually be run whenever code was checked in (continuous integration.) After awhile, they built up enough helpers to stop really using Selenium at all except for replicating bugs.

This seems like a nice contrast to the approach that we took for two reasons. First, integration tests would automatically keep up with the development process because the team could run these tests within their own environment. If an integration test was broken, it would be immediately visible. Contrast that to having to rewrite tests several days after the structure changes for no apparent reason. Second, they ran a headless version of the Selenium Grid, which had the effect of getting their integration tests run in about five minutes. I remember this process taking quite a bit longer for us. Part of this was likely due to supporting IE6, IE7, and Firefox, but it seems like this still could have been sped up.

But overall, it’s nice to take the integration testing process and decentralize it. This might not work with all types of projects. Obviously having the developers in charge of their own integration tests seems like conflict of interest, but I suppose that solid reviews and other testing would reduce defects. But depending on the project, I can see that allowing the development and test stages to work more closely together is a plus.

Lunch was a nice experience, it was fun hearing about some things that people have done in the past and how that affected their current views on software development and testing.

Recurring themes

It seemed like people liked using Ruby for data munging and actual testing. Over half of the presentations given were using some form of Ruby for development, testing, or tools. You could probably get away with using Perl or something for this, but you can also probably pick up most of Ruby in a couple of days if you have used Perl before.

There was emphasis on abstracting tests to the largest degree possible to facilitate agility of development. This made a lot of sense to me, because one of the common pain points people expressed was having tests that were very brittle. When the application changed, the tests had to be scrapped. Considering testing to be a valuable software architecture practice in itself makes more sense than seeing testing as isolated.

Ideas and technologies

One interesting idea given in the “Open Season” part of each presentation was the idea of thumbnails for quick manual content tests. Instead of specifying a test that used screenshots or some sort of brittle method, instead print out an expected screenshot of each of the screens and the current screenshot on a single sheet of paper. Someone can then easily scan a whole page of these relatively quickly, as humans are naturally good at picking out differences between images and determining whether they are important or not.

Exploratory testing and pair testing were topics that were mentioned a few times, and I had never heard of them before. James Bach was suggested as a resource for this and pair testing. His site contains some newer articles that deal with these subjects.

Technologies discussed included:
Ruby and Ruby on Rails
Selenium
Watir
RSpec
soapUI
Groovy
Hammer/QTP

Overall thoughts

I really appreciated the constructive nature of the conference. Everyone was very knowledgable and had something to contribute to the process. All questions were asked kindly and there was a feeling of building on things others had said. There was a perceptible energy in the room during the concluding processes. People said that they were fired up to go to work on Monday and solve a problem that they had been facing using new approaches and technologies.

Something that stood out to me was the vast differences in terms that people had for testing practices. Obviously I am not an expert at testing, so there were some word hangups that I had. For example, the decision to call something a unit test versus an integration test was seemingly person-specific, if they used those terms at all. I can see how having a good stream of communication with team members, especially new ones, is very important in testing. Obviously this applies to software development in general.

While I have never been a dedicated tester before, I enjoyed the problems that people presented and their creative approaches to solving those problems. I think that I walked out of the conference having a lot more respect for software testers because of the challenges they expressed, both technical and organizationally.

One meme that I took away from this was the pirate heuristic: When you are out of ideas, steal someone else’s. I was out of ideas, and hence stole this one myself.

Original article: IWST Summary

At least one interesting thing that I learned was ProtoXAML, which is a way to take prototypes and make them seem a bit more flawed so that customers are more likely to give you good feedback. It was interesting to see this perspective, and might be helpful with some client interactions we have.

Original article: Architecting Scalable and Usable Web Applications