Macintosh Software Testing Resources
mactester.com: information about testing Macintosh software on the modern UNIX-based OS X (that's "ten", not "ex"); some token materials remain available for Classic Mac OS. Intended audience: new and experienced Mac software testers, developers, tech writers and other development project team members. Be a champion for a great user experience. Develop, test and document great Mac apps.
Last updated: 2 August 2006 23:07 MDT
Tidbits
- Reminder to Mac developers: since StuffIt Expander no longer comes installed with Mac OS 10.4.x, don't assume the user can open .bin, .hqx, .sea, .sit, .sitx files.
- If your Mac app is a Universal Binary, you might check out Apple's Mac OS X Universal Logo Program
- Wondering what systems Mac power users are using? Check the Omni Software Update Statistics page: OS version, preferred language, installed RAM, CPU/speed/count, and other useful system info about Omni users' systems
- Unsanity Smart Crash Reports: allows developers to add crash report submission capabilities to their apps. Users include: Now Software, Karelia Software/Sandvox, TextMate, Cyberduck and Rogue Amoeba.
- The Transition to Intel-based Macs: An Introduction For Testing and QA Engineers: make sure to test file format and Rosetta environment compatibility
- Need a powerful and free text editor? Check out TextWrangler, little brother to the highly-regarded and even more powerful BBEdit. (Coming from serious Mac developers, of course it's already available as a Universal Binary.)
General Mac information
Information for Mac developers
Developer events
Finding Mac developers/jobs
Reference
OS X-specific
Classic Mac OS
- TN 1090: Mac OS 7.6
- TN 1102: Mac OS 8.0
- TN 1121: Mac OS 8.1
- TN 1142: Mac
OS 8.5
- TN 1163: Mac OS 8.6
- TN 1176: Mac OS 9
- TN 2010: Mac OS 9.1
- Mac OS 9: Available Updates links to 9.0.4, 9.1, 9.2.1 and 9.2.2 info, downloads
- AppleCare Document: 25114 OS 8, OS 9 hardware compatibility chart
- CarbonLib version notes (1.0.4, 1.2.5, 1.3.1, 1.4); may be of some value. For those dealing with the numerous possible versions of installed CarbonLibs, a more thorough (and recent) reference (including versions 1.0, 1.0.1, 1.1.1, 1.5, 1.6) would be appreciated.
- TB39 - Appearance Versions
- Technical Q&A: NW 64 Open Transport Versions
- Creator code registration
- TN 1126: Checklist for Building Applications and Extensions
- TN 1132: Version Territory
- TN 1134: The Preferences Problem
Tools/utilities
- Apple's System Profiler - comes installed on all systems. Tip: system_profiler is the command line version
- PListChecker
- OSX AppleScript GUI scripting: System Events application with support for controlling an application's GUI via scripts - select menu items, push buttons, enter text into text fields, and more
- Redstone Software: Eggplant platform-independent automated GUI testing tool
- Mactracker, a freeware app (donations appreciated), displays system information for all known Macs: find out CPU, RAM and other info. Thanks, Ian!
- AppleSpec online database of Mac hardware specs
Bug databases
(really, it's much better than storing bugs on scraps of paper, in a text file or an email folder; invest a few hundred dollars to improve your bug management workflow)
- Off-the-shelf
- Seapine Software's TestTrack Pro cross-platform bug database server runs on OS X, as well as Windows, Linux and Solaris. Well-designed software with workflow and state change smarts; view the Extended Defect Workflow document. An older TestTrack Workgroup client runs on Classic.
- Brew your own with...
- Other
- Bugzilla, an offshoot of the Mozilla Web browser project. This is a Web-based bug database. Requires much patience and a fair amount of Unix sysadmin knowledge to assemble all the pieces, install and configure the server. The TestTrack folks do make a pretty good case for buying an off-the-shelf product, rather than spending time and effort trying to assemble a whole bunch of free software (and basing a very important part of your software business on it).
Documentation/testing
- Apple Help | An Introduction to Apple's HTML-based Online Help System
- From "The Elements of User Interface Design" by Theo Mandel
- All textual aspects of the interface should be designed by those with writing skills. All writing is an art, including writing system and program documentation and messages... establish the proper tone of voice in messages and prompts. It is important to assign no blame for errors or problems...
- Use terms that users can understand, rather than system or developer terms
- From Aqua Human Interface Guidelines: Has all user-visible text been reviewed by a professional writer?
- From the excellent "Testing Computer Software" (Second Edition) by Cem Kaner, Jack Falk, Hung Quoc Nguyen: How Testing Documentation Contributes to Software Reliability
- You'll find more bugs than you expect. Surprisingly many bugs show up when a competent tester thoroughly checks the manual (or help) against the program... A full test of the manual takes about an hour per three to five pages...
- It's an important source of real world combination test cases
- Bug reports arising out of documentation testing are particularly credible... These are things many people will do. These are errors that magazine reviewers can publish without fear of correction... either the manual will change or the program will change.
- Use the program exactly as the manual says.
- Check every statement of fact and every obvious inference from the stated facts, instructions, and suggestions. The manual is the product's specification, and the customer's final place to check whether the program is working correctly.
- A style guide can be used to document proper (and improper) usage of terms - spelling, capitalisation, trademarks, copyrights, voice, tense, and so on - to help ensure consistency in all company materials (software, documentation, Web, catalogs, brochures, etc.). Even if you don't have a technical writing department, or even a tech writer, consider starting your own style guide. Apple Publications Style Guide 2006 (196 pp., 872K PDF) is an excellent reference.
- Proofreader's marks (PDF)
- Boulder Writers Alliance (Colorado) resources: links to professional organisations, online dictionaries and other valuable information for tech writers.
Creating an installer
Tools
Consulting
Installer suggestions
- Use meaningful filenames for the media, content
- Don't install a bunch of files on the desktop. Always enclose files in a folder.
- Give user option to select target folder/disk
- Test for sufficient disk space and tell user how much is required if there's not enough free space
- Extensions
- Test for the presence of required extensions; don't give the appearance that the app is going to work if some of the components are missing.
- If a required extension is on the disk but disabled, perhaps your installer can enable it and restart; don't tell the user to reinstall from disk if all s/he needs to do is enable an extension or two.
- If a newer version is required, alert the user.
- As a convenience to the user, consider licensing necessary Apple software components to install with your app.
Access to variety of machines, OS versions for testing
- Borrow, barter for services or product; user groups as points of contact
- Apple's ADC Compatibility Labs in Cupertino, California, USA. If you are a Premier Member of ADC, your membership includes access to the testing facilities at Apple's headquarters (Room With A View, Project X and Field of Dreams).
- Local/regional software testing lab facilities
- Rent locally or ship from out-of-town
Outsourcing your Mac software testing and/or development
- Absolute Quality Inc. (Maryland, USA and United Kingdom)
- Critical Path Software (Oregon, USA)
- iBeta (Colorado, USA)
- mactester.com, maintainer of this Web site (Colorado, USA)
- Parallel Software (Illinois and California, USA)
- Product Quality Partners, Inc. (California, USA)
- Recommended Test Labs, Inc. (California, USA)
- VeriTest
- Sys.Test Labs (Colorado, USA)
- Test Partners Ltd. (UK)
- Also see Apple's Product Testing page
Beta testing
When your application has been thoroughly tested, you may offer a limited beta release to select individuals, and later a general public release. Don't confuse professional software testing with beta testing. There are companies, such as Centercode, that will help developers set up a beta test program. Refer to Joel Spolsky's Top Twelve Tips for Running a Beta Test (March 02, 2004), which includes these points:
- Plan 8-10 weeks for beta cycle
- New builds at least 2 weeks apart
- Plan at least 4 beta releases
- Most beta testers will try the program when they first get it, then lose interest
- Even with a beta application process, only 20% of testers will actually send feedback
Getting the word out
- The Cult, user groups. Excerpts from "The Macintosh Way" by former Apple Evangelist and Apple Fellow Guy Kawasaki (Scott, Foresman and Company ISBN 0-673-46175-0)
- Make sure that The Cult uses your product. The Cult is tightly-knit and vocal, and it recommends what it uses. Make sure that The Cult - power users, developers, luminaries, dealers, press, and analysts - use your product by selling it to them near cost, exchanging for their products, or if you have to, giving it away.
- Kill the white space. The Cult wants to make informed purchase decisions by gathering and reviewing in-depth product information. Therefore, your brochures, advertising, and point-of-purchase materials should contain an abundance of technical information, not white space and yuppie lifestyle depictions.
- Promote test driving. If you've got a great product, then a sample package... or a demo kit containing a limited version of your product can significantly increase sales. If you've got a bad product, samples can be good for your competition...
- Don't try to fool user groups
- Never wear a tie. Wearing a tie prevents closeness to a user group. The user group perspective on ties is, "When you ask a Tie a technical question, he has to find a T-shirt to get the answer." (A note to company presidents who are not engineers: take a T-shirt when you visit a user group.)
- Get straight to the demo
- Don't concentrate on only the big groups
Finding Macintosh User Groups
Announcing, distributing your freeware, shareware
Selling your software on the Web
Software awards (:-)
Avoid bad publicity (:-(
- Bad reviews - do your best to avoid this sort of publicity:
- "Program is Profoundly Flawed... assuming you could get past the bugs, import limitations, and clunky interface. Aspiring desktop publishers on a budget should look elsewhere... barely usable in its current form... Let's hope the next version builds on this foundation and fixes the dire shortcomings... Inability to import popular text and graphics formats; amateurish templates; can be unstable..."
- "Other aspects... are less wonderful. The shipping version was clearly produced in haste; several of the... files wouldn't open, and the awkward dimensions of the shoddy manual make it difficult to use. Instead of a modern HTML-based help system... offers balloons left over from its System 7 incarnation. The HyperCard-based utilities are more nostalgic than impressive in the age of the Web"
Other business advice for developers
- The Do's and Don'ts of Shareware - Sanford Selznick
- Part 1: the foundation upon which to build your emerging software enterprise.
- Part 2: writing and testing maintainable code, assembling a deliverable package, and marketing and distributing your software.
- Part 3: payment processing, distribution, and marketing of your new application. Solid information helpful to beginning and experienced developers alike.
- Media relations and other useful TidBITS
This Web page maintained by:Doug Grinbergs
Need help testing your Mac software and documentation? Please contact us for information about our services.