Testing is an important function of the software development process, no matter how big or small the development project. But not every company or developer has access to professional testing tools, which can run into hundreds and even thousands of dollars. The good news is that they don't need them, thanks to the tons of freely available open source software testing tools.
In simple terms there are two major approaches to testing software -- the manual way (a summer intern with a checklist) or through an automated program. With automated testing programs, you can spend a lot of money procuring these tools or distracting yourself from the task at hand by rolling out your customized automated testing software.
Instead, you could head over to sites like Open Source Testing (OST), QAForums, Open Testware, and others that catalog various testing tools and look for something that works for you.
"The largest category of open source testing tools," says Mark Aberdour, who manages OST, "is functional testing tools. This can cover a range of practices from capture-replay to data-driven tests, from Web application testing to Java application testing, and lots more in between." Aberdour, who before his current software development role spent 10 years on the other side of the fence in software testing and test management, says that the list of open source testing tools also includes many performance testing tools and test management/defect tracking tools, as well as a good number of security testing tools.
"If you include unit testing tools, then there are large numbers of tools for the more popular languages, particularly where test-driven development (TDD) is more popular," he says. There are several tools for testing Web and Java applications, but "as is the way in open source," Aberdour says, "if there's an itch, someone will scratch it, so there are tools available for all manner of obscure needs." OST lists testing tools for languages such as PHP, Perl, Ruby, Flash/ActionScript, JavaScript, Python, Tcl, XML, and so on. "The list is probably bigger than I've brought together on OST, and TDD practitioners should head over to testdriven.com, which has more focus on that area."
So how do they compare with the expensive proprietary tools? "In some cases, very well," says Aberdour. He points out WebLOAD and OpenSTA as examples that hold up well in the performance testing market -- no surprise there, since they were both originally commercial tools. Underlining his point, Aberdour says, "You have tools like Wireshark, which is huge in the security market, and Bugzilla and Mantis in the defect tracking sector. In functional testing there are a number of really great tools (Selenium, Abbot, Jameleon, jWebUnit, Marathon, Sahi, soapui, and Watin/Watir to name a few) with strong feature sets."
According to Aberdour, in addition to the tool itself, the advantage of the commercial tools is often their integration with an automated software quality (ASQ) suite, and of course an established company behind them, which will feature in a lot of people's selection criteria. Here again things are looking good for open source testing tools. Aberdour points out the RTH test case tool which integrates with Watir, HttpUnit, JUnit, MaxQ and the commercial WinRunner. Even bug trackers like Mantis and Bugzilla integrate with functional testing tools and test case tracking tools.
When it comes to reliability and accuracy, Aberdour says the classic open source arguments apply. The popular tools are tested and used by a high number of people, and many of the tools have evolved large and sustainable communities with many people feeding back on quality-related issues. "For many of these tools, innovation is high and release processes rapid. Yet it is not a free-for-all -- the code base may be open, but write access to the repository is closely guarded, and developers have to earn the right to commit."
According to Aberdour, the main issue for people using the tools is not whether their need is served, since in most cases you will find a tool to do the job, but whether the tool is mature enough to invest in. "Test automation is a major undertaking that has high costs in terms of upskilling your test team and creating test scripts, etc., and people need to know that the tool is a good investment, whether or not there is a license fee to pay. People will be asking if the tool is going to be around in five years' time, what levels of support are available, how good is the feature innovation, bug fixing, release schedules, and so on." Aberdour thinks there is a big gap in selection and evaluation support and paid technical support services, which is what he wants to focus on next with OST.
Of course there are strong and sustainable communities around the more popular tools that provide excellent support, but Aberdour says the market isn't at the point yet where there is a lot of commercial support available. Yet there are some examples of support companies evolving around open source products, and a lot of the product teams will hire themselves out. "In a commercial sense this market is still quite young, but it will mature, and when the product integration, community maturity, and commercial support are right, it has the potential to be highly disruptive. Gartner reckons the ASQ market is worth $2 billion a year, and while Hewlett-Packard and Mercury absolutely dominate, there are hundreds of smaller proprietary vendors at real risk from open source disruption. Things are certainly moving in the right direction."
Open source testing tools offer great performance and are a bargain compared to proprietary testing tools. The lack of a formal and consistent support structure might work against some tools being used to test mission-critical apps. But if you are an open source developer or with a software development company pondering over your testing budget, spend some time checking out the testing tools Web sites and forum boards. You might save yourself some serious bucks.
Since I put my workstation, which is at the same time the MythTV server / backend streamer for the family into a new case and added another TV card, I had some issues.
No matter in which PCI slots I used these cards (two analog ones, a V-Stream Terminator with SAA7133 and a Hauppauge Win-TV with BT878 chip sets), they changed device numbers during boot time. So sometimes card 1 was /dev/video0, sometimes card 2 had that device number. Additionally, the BT878 card changed its /dev/dsp number. Not good if after each reboot (and they happen sometimes, because I wanted to dual boot into something newer than Etch for instance), you had to stop the mythtv-backend, change TV-Card settings accordingly, and then to restart the backend again.
This shouldn’t normally happen with PCI cards, but well, different timings, the moon phases, some weird things you could never analyze without real expensive equipment, … after all, a PC is only human as well, right? Hm.
I had read about udev already, but never actually cared much for it, not even to make rules for hot-pluggable stuff like USB sticks and so on. However, now I had to read, so this is what I did. Interesting, really. But for my special case here, a short intro on the MythTV Wiki should also give enough information for those whose /dev/video0 and /dev/video1 are swapping.
For the impatient: find out more than you want to know with just using
udevinfo -a -p $(udevinfo -q path -n /dev/video0)
(or video1 for your second card), then use part of those informations for writing a rule for udev. This is what I ended up with:
# this file was generated by wjl on Sat Apr 19, 2008
# to give persistent names for video and dsp devices to MythTV
#
# Add these symbolic device names to the mythbackend setup,
# and they should not change anymore (be persistent).
#
# let’s start with video:
KERNEL==”video[0-9]*”, ATTR{name}==”saa7133*”, SYMLINK+=”video-saa7133″
KERNEL==”video[0-9]*”, ATTR{name}==”BT878*”, SYMLINK+=”video-bt878″
#
# and then create a symlink for the bt878 DSP:
KERNEL==”dsp[0-9]*”, SUBSYSTEM==”sound”, DRIVERS==”Bt87x”, SYMLINK+=”dsp-bt878″
After that, as already commented in the rule file itself, use these symbolic names in the TV Card Setup of your MythTV backend. And that’s it. After the next boot, you (and MythTV) should be using these symbolic names instead of /dev/video0 or /dev/dsp1.
We need a new conversation about infrastructure. That's what Linux, the Net, and half a million FOSS (Free and Open Source Software) applications require. Because they're all part of a new infrastructural system that cannot be explained in terms of the old one — especially since infrastructure itself is not well understood. Or rather that it is too well understood in a general way, though not in the specific.
Look up "infrastructure" on Google and you get over a hundred million results. Relatively few pertain to what infrastructure meant in the first place, which the American Heritage Dictionary places about eight decades ago:
Usage Note: The term infrastructure has been used since 1927 to refer collectively to the roads, bridges, rail lines, and similar public works that are required for an industrial economy, or a portion of it, to function. The term also has had specific application to the permanent military installations necessary for the defense of a country. Perhaps because of the word's technical sound, people now use infrastructure to refer to any substructure or underlying system. Big corporations are said to have their own financial infrastructure of smaller businesses, for example, and political organizations to have their infrastructure of groups, committees, and admirers. The latter sense may have originated during the Vietnam War in the use of the word by military intelligence officers, whose task it was to delineate the structure of the enemy's shadowy organizations. Today we may hear that conservatism has an infrastructure of think tanks and research foundations or that terrorist organizations have an infrastructure of people sympathetic to their cause. The Usage Panel finds this extended use referring to people to be problematic, however. Seventy percent of the Panelists find it unacceptable in the sentence FBI agents fanned out to monitor a small infrastructure of persons involved with established terrorist organizations.
Linux and the Internet are infrastructure, in the sense that they qualify for the American Heritage Dictionary's #1 meaning: "An underlying base or foundation especially for an organization or system". Infra is Latin for under, and you can find Linux under countless applications on devices that range from supercomputers to cell phones to wireless picture frames. Linux is also the platform of choice for many (perhaps most) sturdy and popular Web sites and services, including Google's search and Amazon's S3 and EC2 (to name too few among way too many).
And you'll find the Internet under Linux. If we're stacking up connected people and devices, nothing could be more infra, more fundamental, than the Net itself. In fact, the Net is so infra that two legacy forms of widespread commercial infrastructure — telephony and television — now run on it as well.
A couple of months ago I got a couple of wonderful birthday presents. My lovely geeky girlfriend got me two Western Digital 500 GB SATA 3.0 drives, which were promptly supplemented with a 3ware 9550XS 4-port hardware RAID card. Immediately I came up with the idea for this article. I had just read up on mdadm software RAID so I though it would be perfect to bench mark the hardware RAID against the software RAID using all kinds of file systems, block sizes, chunk sizes, LVM settings, etcetera.
Or so I though… As it turns out, my (then) limited understanding of RAID and some trouble with my 3ware RAID cards meant that I had to scale back my benchmark quite a bit. I only have two disks so I was going to test RAID 1. Chunk size is not a factor when using RAID 1 so that axis was dropped from my benchmark. Then I found out that LVM (and the size of the extends it uses) are also not a factor, so I dropped another axis. And to top it off I discovered some nasty problems with 3ware 9550 RAID cards under Linux that quickly made me give up on hardware RAID. See The trouble with 3ware RAID below. I still ended up testing various filesystems using different blocksizes and workloads on an mdadm RAID 1 setup, so the results should still prove interesting.
Hardware used
I ran all of the benchmarks on my home server: A HP ProLiant ML370 G3. Here are the specs:
Dual Intel Xeon 3.2 Ghz with hyperthreading, 512 KiB L2 cache and 1 MiB L3 cache each (giving me four CPUs)
1 GiB RAM
Adaptec AIC7XXX SCSI RAID with four 36.4 GiB disks in RAID 5 and two 18.2 GiB disks in RAID 1
3ware 9550SXU 4-port SATA 3.0 RAID with two 500 GiB disks in JBOD, running in mdadm software RAID 1
Debian GNU/Linux 4.0 (Etch)
Yes, it's probably a bit overkill for a home server :-) When doing write speed benchmark, the files were read from the RAID5 unit which can read at about 150 MiB/s, much faster than the 3ware mdadm RAID 1 is able to write. That means that the RAID5 read speed should not have affected my benchmarks.
The trouble with 3ware RAID
The hardware RAID card has been giving me a fair bit of trouble. I bought it because I needed a SATA card that was compatible with PCI-X. Most of the "cheap" software RAID cards out there only support regular PCI, which is too slow. As soon as you start using three or four drives you hit the limit of the PCI bus. My server supports 64bit PCI-X at 100 Mhz which is more than fast enough. Sadly I couldn't find a simple SATA card with PCI-X support so I bought the expensive but high quality 3ware 9550SXU instead.
what's brewing in x86.git for v2.6.26?
too many topics to list them all - there are 884 patches from 74 authors
at the moment.
a few highlights:
- 4096 CPUs support. (Yes, such big boxes exist, and they run Linux.)
- mmiotrace feature: trace accesses to hw components to help figure out
how they are programmed.
- kmemcheck feature: Valgrind for the native Linux kernel in essence -
detects access to uninitialized memory.
- ftrace plugin for sysprof
- fixed StackProtector security feature (these fixes were too intrusive
for v2.6.25)
- lazy FPU allocation/speedup - offloaded/large FPU/SSE state support
- SMP-boot, mpparse, DMA ops unification
- PAT support - first step towards phasing out MTRR's for cache
attribute control
- enable GBPAGES - faster TLB misses on CPUs that support it
- debug helper: view kernel pagetable layout via debugfs
- lots of paravirt work, unification and cleanups
- generalized bitops - they are faster and smaller.
- tons of code cleanups either via the unifications or via explicit
cleanup patches - the metrics [via checkpatch] look like this today:
Boot SplashYesterday the Fedora Project released a "Preview" of Fedora 9. Today Ubuntu released a "Release Candidate" for 8.04 "Hardy Heron" and the openSUSE team released openSUSE 11.0 Beta 1. Since my preferred Linux distribution for the desktop is Fedora, I've been keeping up with all of the test releases. What follows is some commentary about my experiences with the Fedora 9 Preview including an image gallery. I'd like to encourage MontanaLinux users of other distros to write up their experiences with their preferred distributions.
Firefox 3 beta 5
Fedora 9 Preview - the media
Some time ago the Fedora Project decided to follow Ubuntu's lead... and started offering a number of single LiveCDs. They offer a GNOME-based LiveCD as well as a KDE-based one. Since the Fedora Project decided to focus on making it easy for anyone in the community to create their own "spins" a number of other LiveCDs have cropped up including one that is XFCE-based. I downloaded the GNOME LiveCD, the KDE LiveCD, and the DVD media which is NOT a Live media. Since my home computers have 2GB of RAM or less, I just use the i386 flavor rather than the x86_64.
As a film trilogy, on this howto we'll finish our dvd foto album with cinelerra. As you can see cinelerra is very powerfull, without 3d transitions ( for now ), but with an user interface that can make everythings possible. But on this tutorial we'll can see "only" all step to make our dvd.
Before continue a couple of tricks: If you have an opengl 2.0 ( as the new nvidia cards with proprietary driver ) you can use the x11-opengl to speed up preview Settings > preferences > playback > video driver.
If are not one this lucky users you must go to Settings > preferences > performances and enable "use backgroud rendering", in this way cinelerra make a motion jpeg sequence as preview, mind that this require disk free space ( if you not have lot of space, you can set less jpeg quality with tool simbol near to Video: )
VirtualBox is a general-purpose full virtualizer for x86 hardware. Targeted at server, desktop and embedded use, it is a professional-quality virtualization solution that is also Open Source Software.
VirtualBox runs on Windows, Linux, Macintosh and OpenSolaris hosts and supports a large number of guest operating systems including but not limited to Windows (NT 4.0, 2000, XP, Server 2003, Vista), DOS/Windows 3.x, Linux (2.4 and 2.6), and OpenBSD.
The below video by EdgyRootStudios demonstrates how to run Microsoft Windows XP on Linux Ubuntu Edgy Eft 6.1.0 using VirtualBox.
In my next three articles I'll profile three native Linux software synthesizers (a.k.a. softsynths). I'll introduce their basic synthesis architectures and program operations, then I'll guide my readers briefly through the process of creating a new sound for each synth profiled. Our voyage begins with Nick Dowell's Analogue Modeling SYNTHesizer, better known as amSynth.
amSynth Synthesis Basics
The amSynth Web page advertises that the program "... provides virtual analogue synthesis in the style of the classic Moog Minimoog/Roland Junos". If you've never heard those instruments, check out some of the excellent demonstrations of the MiniMoog and Juno G/60/106 synths on YouTube to get an idea of what kind of sounds amSynth sets out to produce.
The Roland and Moog synthesizers are considered the classic analog synths, with highly characteristic audio palettes resulting from unique implementations of the fundamental design for all similar subtractive synthesizers. An oscillator supplies the primary audio signal source by cycling a waveform such as a sine or sawtooth wave at a designated frequency. The output from the oscillator is filtered to subtract unwanted frequency ranges, then the filtered signal is fitted with an amplitude envelope, modulated by an LFO (low frequency oscillator) and optionally further processed with effects such as reverb and distortion. A simple process, but the possibilities increase greatly when two or more oscillators provide the audio source.
There are many new and innovative features brewing within the
X.Org development community right now -- among the many are Gallium3D, the TTM
memory manager, and MPX (Multi-Pointer X) -- but one of the features that has
risen towards the top of the list and delivers visible benefits to the end-user
is kernel-based mode-setting. As implied by its name, kernel mode-setting involves
moving the mode-setting code for video adapters from the user-space X server drivers
into the Linux kernel. This may seem like
an uninteresting topic for end-users, but having the mode-setting done in the
kernel allows for a cleaner and richer boot process, improved suspend and resume
support, and more reliable VT switching (along with other advantages). Kernel
mode-setting isn't yet in the mainline Linux kernel nor is the API for it frozen,
but Fedora 9 shipping next month will be the first major distribution carrying
this initial support. In this article we're looking more closely at kernel mode-setting
with the Intel X.Org driver as well as showing videos of kernel-based mode-setting
in action.
There is currently work underway in porting the
Radeon driver to do kernel mode-setting (the driver is named radeon_ms), but for Fedora
9 the only X.Org driver with this support is the open-source Intel driver. Support
for the kernel mode-setting interaction isn't in the mainline xf86-video-intel
driver, but the Fedora 9 Intel driver is based upon the intel-kernelmode branch.
The intel-kernelmode branch is in turn based upon the intel-batchbuffer branch.
Intel-batchbuffer is also the branch that contains the initial DRI2
support, TTM memory management, Render
improvements, and other bleeding-edge advancements. Other drivers will turn
to using kernel mode-setting once the API is stabilized and can be found in the
mainline Linux kernel. On the Mesa/DRM side is a modesetting-101 branch.
Suspend and resume support is improved with kernel mode-setting
as the kernel no longer relies upon external resources for restoring the graphics
adapters. With the process now being in-kernel, it's able to restore the mode
automatically and more quickly. Likewise, virtual terminal switching is also improved
as a result. Kernel mode-setting will also allow for an improved debugging experience,
as this will eliminate the "hard hang" and make it possible to display
a graphical error message (think the "Blue Screen of Death" for Linux). This technology leads to a flicker-free boot experience
by only needing to set the video mode once, instead of turning on and off when
starting the boot process (in the case of Fedora, with Red Hat Graphical
Boot) and then properly initializing the device when the X server has finally
started and loading the GNOME Display Manager. Kernel mode-setting has been one
of the items on Keith Packard's (Intel) list of features
for a happy Linux desktop.
The kernel mode-setting components ship by default with Fedora
9, but the Intel kernel mode-setting isn't enabled by default. In order to make the switch, i915.modeset=1 must be set as one of the kernel command
line arguments through GRUB on boot or modifying /boot/grub/grub.conf.
Until kernel mode-setting is used by default, this command is necessary to let
the system know to use this method as opposed to mode-setting via the user-space
X driver.
Back
Open source testing tools target varied ta...
