藍色情懷

Network Test Automation with Mac OS X and Tcl

If you haven't been introduced already, the Tool Command Language, Tcl, is a secret weapon used by large network hardware corporations to test their devices. It's an extremely flexible scripting language that has been ported to nearly every operating system in existence, and that now includes Mac OS X.

As wonderful as it is, Tcl isn't perfect all by itself. That's why I'm also going to discuss Expect, which is an extension of Tcl that allows interactive automation to your Tcl scripts. For instance, using Expect you can automate telnet sessions, database queries, and file transfers.

For some reason, Apple didn't include Expect in its operating system release. Not to worry, Expect has been ported to Mac OS X, and I'll walk you through the install of this handy extension.

Utilities that you will need

First I'll introduce you to a few utilities that I recommend you install on your test network workstation. Nearly everything is available on the Net -- most of it from those diligent code porters involved with the open-source sourceforge.net project.

With the current economic trends, your manager will be pleased to know that you set up an awesome workstation using mainly open-source applications. The focus here is creating an automated test environment running on Mac OS X. However, the same concepts will apply to any Unix workstation -- thanks, again, to open-source efforts.

Rootless X on Mac OS X

Figure 1. XFree86 running in rootless mode on Mac OS X desktop. (click on graphic for larger image)

To begin, I recommend that you install Rootless X.

Torrey Lyons has released a rootless version of XFree86 for Mac OS X. I provided installation instructions for X on X in a previous O'Reilly article, Installing XFree86 on Mac OS X . Torrey's rootless XFree86 is available from sourceforge.net (X on X). XFree86 will soon become indispensable as you put together your test automation system. The fact that your Mac OS X machine is running rootless allows you to run the XFree86 desktop and the Mac OS X desktop in the same window. Prior to this, you were required to toggle between the two desktop environments. See Figure 1.

The examples listed later in this article will also run in the Mac OS X console window utility. But in my opinion, life is a lot easier using the X Windows environment with multiple terminal windows opened. This feature comes in handy when you're testing multiple network devices simultaneously, such as in a network system test environment.

Emacs on Aqua

Many editors are now available for Unix environments; the two most prevalent are vi and emacs. The Emacs-on-Aqua utility is a complementary tool to run with the rootless XFree86 environment. The Emacs utility executes in Mac OS X environment but allows you to easily edit files in the Darwin file system.

Emacs contains most of the features you will require for editing programs in the Darwin environment. But Emacs can be difficult to use. The hidden keystroke commands are particularly annoying. For example, you must use control (^)x^s to save a file. The Aqua-on-Emacs port does, however, provide a nice menu bar to assist you if you're not familiar with the editing environment.

Tcl and Expect for Mac OS X

Now that you've set up your rootless environment and have your editor installed, we can fire up Tcl.

Tcl is an installed package on Mac OS X machines. However, Apple didn't include Expect, or Tk (X11 tool kit for Tcl). Don't ask why. Not a real problem though. Michael Peters has provided an excellent port of Expect 5.32 which requires Tcl 8.4a2 -- a revision different from Apple's. You'll need to download and install these two compressed packages to start using Expect.

The downloadable file is a StuffIt .hqx file. Once downloaded, the image decompresses to a mountable disk image. You will need to use the Disk Copy utility in the Applications/Utilities folder to mount the .img file.

The mounted image is an installation package that you will need to double-click in order to start the installation. Remember, the new Tcl installation is required for the Expect installation.

Unfortunately, the Expect and Expect Tk (the graphical user interface and toolkit) portion of the package has yet to be tackled for Mac OS X. In a network test environment, we can survive with command-line scripts. But I know how excited your manager gets when you show them a GUI application. If you were adventurous enough to port Expect Tk, or know where a working image is hiding, please let us know in the O'Reilly TalkBack section for this article. In the meantime, we're command-line constrained.

Invoking Tcl

Tcl is a simplistic language that is string-based. You can invoked Tcl by executing the command tclsh from your terminal or console window. A simple "Hello World!" script looks like this:

Spongebob # tclsh
% set myString "Hello World!"
Hello World!
% puts $myString
Hello World!

This code snippet demonstrates how Tcl uses the set command to assign the string value "Hello World!" to the variable myString. Assigning a "$" in front of the variable, myString, instructs the interpreter that the script is going to reference the value contained there. In this example, the string value is "Hello World!". The puts command writes the value referenced to the terminal window.

Tcl and regular expressions

One of the true powers of Tcl, which often becomes a love-hate relationship, is the regular expression (regexp) command. This command provides the flexibility of unique pattern searching. For a simple example, let's say you're testing a voice gateway server and you need to locate a phone number, 555-1212, to match to an IP address list. You could use:

% set myPhoneList "555-1212  nautilus.ipvoice.com"
% regexp {([0-9]+)-([0-9]+)}  
   $myPhoneList matchFound prefix localNumber

Wow! What is that?

Welcome to the world of regular expressions. The regexp command has its arguments enclosed in curly braces. The funky ([0-9]+)-([0-9]+) is instructing regexp to grab all numbers with the pattern nnnnnnn-nnnnnn (hyphen-separated). The regexp command would identify that "555-1212" has this pattern and copy it into the variable matchFound.

% puts $matchFound
555-1212

We instructed regexp to search for two subpatterns. The actual pattern was hyphen delimited. So, we can grab the two subpatterns and place these into sub-variables.

% puts $prefix
555
 % puts $localNumber
1212

Okay, so what if we used an 800 number? What would our regexp look like? This is tricky. Let's take a look.

% set myPhoneList "(800) 555-1212  nautilus.ipvoice.com"
% regexp {(\([0-9]+\))?([0-9]+)-([0-9]+)} \ 
   $myPhoneList matchFound  areaCode prefix localNumber

You'll notice we had to escape the open and close parentheses, "(" and ")" because they are a part of the regexp syntax.

Now for a real-life application, how about using a regexp command on a Cisco router interface. The following data is from an IOS "show interfaces" command and is in the variable myRouterInt.

Ethernet1/0            121.1.1.2    YES manual up        up      
 

We could use:

regexp {(Ethernet|Fddi)([0-9])/([0-9]) .*(up|down) \
  $myRouterInt match int_type slot_id port_id int_status

This little snippet could tell us if an interface is up or down. We'll learn more about poking around inside a Cisco router in the complementary article, "Cisco Router Management Using Tcl on Mac OS X." Examining the above little piece of code reveals a wealth of Tcl regexp coding practices. Such as, how to grab subpatterns from an input string and split the subpattern matches into sub-variables. Regular expressions are tough to master. So don't feel too bad if they look a bit complicated at first glance.

Expect

Even though Tcl is a powerful scripting language, it has its shortcomings. For instance, from a Unix environment I may want to FTP to another machine and retrieve a file. In Tcl, I could use the socket library and write an FTP script from the socket layer up. The other option is to have available a Tcl-based toolkit that handles interaction and spawning of processes. The toolkit with these goodies is our friend Expect, coded by Don Libes in 1990.

Expect provides all the hooks a system test engineer could desire. Most network devices provide either a console interface, an HTTP server, or both.

For example, Expect allows you to exercise a device to test a console parser. To exercise a device, one would use a script that would simulate typical or even stressful operations. A network device shouldn't crash because someone was using your device's console interface. Expect can exercise a console rather nicely. Another application is generating network traffic. Perhaps you wish to generate background traffic, such as file transfers, and then check the files for integrity after the transfer. Expect can also handle e-mailing out the reports after your script has completed. That ought to put a smile on your manager's face.

An Expect scripting example

So what's all this talk about automating a console login? Let's take a look at a scripting example for automating an FTP remote login. This should shed some light on what Expect offers. Here is our Expect example script from the file ftp_drvr.exp.

proc ftp_login { host username password } {
  upvar spawn_id spawn_id
  
  spawn ftp $host
  send "$username\r"
  expect -re "331 Password required for $username" {
   expect -re "Password:" {
     send "$password\r"
   }
 }
}

This example demonstrates an Expect procedure -- remember it's Tcl syntax -- to login to a Unix-style FTP server. As mentioned, Expect provides the components for spawning a process. The spawned process ID is stored in Expect as an integer. The upvar command is not a typo. The spawn_id, as explained by Don Libes in his book, "Exploring Expect," has a scope local to the calling procedure. The Tcl upvar command pushes the scope up to a global variable named spawn_id.

The next item you'll notice in the script is the Expect command (not Tcl), expect. The expect command handles the interactions with the command prompts of the application, such as an FTP server login prompt. If the expect criteria is met, which is typically pattern matching, the command in braces is invoked. Here we see the user name and password submitted to the FTP server.

The driver script for this procedure is as follows,

set host "skynet"
set username "spongebob"
set password "ixnay123"

set spawn_id [ftp_login $host $username $password]

Another Tcl caveat: Note the open and close braces syntax is used when calling a Tcl procedure. To call this script, you can execute the following command-line statement.

Spongebob# expect ftp_example.exp

Did you notice we didn't use the tclsh interpreter? We used the Expect interpreter instead. A standard practice when using Expect scripts is to test the scripts in the interpreter environment and then make the script an executable. You can do this by adding the following command on the first line of your script.

#!/usr/local/bin/expect

Be sure and give the file the correct executable rights.

chmod 777 ftp_example.exp

Now try it out.

./ftp_example.exp

Now you're a system test hacker!

Tcl/Tk a fork in the road

Tk, the X11 toolkit for Tcl, has recently surfaced as ports for both Darwin and Mac OS X. The latter version surfaced on October 15, 2001, as a programming effort by Jim Ingham at Apple Computer. Tcl/Tk for Darwin is available from the Fink project. Fink is an installation tool for adding and removing packages to Darwin. The Fink utility requires a Unix-savvy user. At times Fink can be clunky and may require manual intervention. On the flip side, Jim Ingham's Tcl./Tk for Aqua package is easy to install, but is native only to the Mac OS X environment. Keep in mind that the Expect package is not compatible with either of these installations.

Tk for Aqua

Packages are being ported to Aqua and Darwin in droves. I have installed both the Fink Tcl/Tk and Mac OS X Tcl/Tk package, macOSXTk8.4a4.tar.gz. I found the latter package is the one I use the most. And it is also the easiest package to install. Packages I typically use, such as QuickTime for Tcl/Tk will be ported to Mac OS X soon.

Installation of Tcl/Tk for Aqua

Installation of Jim Ingham's package is simple. Download the file macOSXTk8.4a4.tar.gz and decompress it. Two parent folders are provided, one is Applications, for the wish.app, and the other parent folder is titled "Library". The Library folder contains the Frameworks folder. Open the Frameworks folder and select all the contents and drag them over to the ~/Library/Frameworks folder. You can drag the wish.app file over to the Applications folder if you wish. Now just double-click on wish.app and you're up and running.

What now?

Admittedly, Tcl takes a lot of getting used to. At first, most people from a systems programming background tend to get a little frustrated. I know I was. After a couple of Tcl coding projects, my C and C++ instincts mellowed out. Just tackle a simple project first. The FTP code I provided would easily give you the self-inflicted frustration you desire. Just play around with the spawn_id. Once you conquer these obstacles, you'll be amazed at how easily you can hack out a Tcl or Expect script. You can manage an entire large system test network with Tcl.

As for Expect, I found it complements Perl rather nicely. An Expect script can launch and take statistics of your device under test. The results can be handed off in Perl for posting on an engineering web server. Wow, all this stuff on your Mac OS X system! And the doubters said there was no useful software out there for us yet ...

Michael J. Norton is a software engineer at Cisco Systems.

Return to the Mac DevCenter.

Copyright © 2006 O'Reilly Media, Inc.