Test Manager User's Guide
A computer automated testing system

John Doroshenko

Table of Contents

Overview

This document describes a system for automated testing of students using a DOS personal computer. Before trying this, or any other software, the user should back up their data to protect it from loss. DISCLAIMER: The material in this document, and the programs described, are presented with no expressed or implied warranties of merchantability or fitness for a particular purpose.

The system consists of a number of programs. These are testman.exe, makekeys.exe, taketest.exe, testprin.exe, and makedisk.exe. Students run taketest with tests prepared by an instructor. Results of each test are accumulated on the floppy disk containing the test file. After a number of tests have been taken, the instructor runs testprin to see the final scores and print detailed reports of each testing session (to be given to students.)

Makekeys is a program which converts the instructor's key (answer) file to a kfc file (with extension kfc.) Taketest uses the kfc and tst (test) file to test the student. Testprin reads the log file produced by taketest and can print individual, or all the results, to the PC's lineprinter. Makedisk is a program which can be used to prepare a student's test disk.

Testman is the Test Manager program. This Windows based program can be used by the instructor to create, edit, and test tests. It can invoke the other programs to help manage the testing process.

System Requirements

The minimum requirement for a test-taking PC is an 8086 DOS PC with a floppy drive. All programs except testman can be run on the test-taking PC.

The testman program requires Microsoft Windows 3.1, which requires atleast an 80286 processor.

The recommended setup is for the instructor to run testman from Windows, and prepare tests. Testman will call on the other programs to aid in the process

Installation

Important: While some testing was done on these programs, it is basically a beta system. Test the system with a small test file prior to using it in a classroom setting. I can only provide this "as-is" at this time.

To download a compressed ZIP file containing the programs and example quiz download TESTM301.ZIP. Then uncompress the ZIP file and proceed with the instructions below.

After installing the program files, as described below, proceed to the next section to install the icon for testman on your Windows desktop.

Installing Test Manager files

Running the simple automatic install.


From the dos prompt, type 
	b:setup
and press enter (replace b with a if your 3-1/2" drive is a.)

Manual installation

* To hard disk


Create a destination directory where you'd like to store the testman programs and data files. Copy all the program and files from the installation floppy to this destination directory.

* To floppy disk


You will need atleast a 1.2 MByte floppy drive to install all the testman programs. A 5-1/4" or 3-1/2" High Density floppy is sufficient to store the program files and provide you with enough space for a small number of test files. If you do decide to use a floppy, you should frequently check the free space on the floppy. Note that if you run the programs from a floppy, instead of a hard disk, they will take longer to load. It is recommended you install to a hard disk instead of to a floppy disk.

Windows icon setup for testman


You will need to manually install the icon for testman on your Windows desktop.

Select the frame of the Windows GROUP into which you want to install the testman icon. Select FILE then NEW from Program Manager. Make sure Program Item is checked, then type the path to the testman installation directory. ie., if you used the floppy setup procedure to install testman, you should enter 

	c:\testman\testman
If you chose to use a floppy to hold the testman files, and enter (ie.) 
	a:\testman
to install the icon, then the floppy must be in the drive when you click on the testman icon, or when you try to run any of the other programs.

You can also start testman by choosing RUN (and ie., c:\testman\testman) from Program Manager.

Running the Programs

The DOS programs


Most of the programs that are part of the Test Manager system can be run directly from the DOS command line. These programs are makekeys.exe, taketest.exe, testprin.exe, and makedisk.exe. Of these, only taketest.exe is meant to be run by the students. The other programs can be run by the instructor, from the command line, or from Test Manager.

Makekeys


Makekeys is a DOS program which reads in a key file, and outputs a kfc (encrypted key) file. When Makekeys is run, it prompts the user for the name of these input and output files. The format of a key file is simply one answer per line. Using an ASCII editor, or the editor built into Test Manager, your key file should be made to contain as many lines as you have questions in the corresponding testfile. The purpose to running Makekeys is to run a simple encryption on the key file. This encryption stores the answers in the kfc file in such a way that it won't be obvious to the student what they are should the file be displayed. The taketest program understands the encryption method used by makekeys, and can score the test using the reconstructed keys. When preparing the key file, begin in the first column of each line, and do not define answers which are more than a single character, number, or word.

A typical run of Makekeys might look like this:

Notice that the input filename ends in .key, while the output filename ends in .kfc. The prefix of the filename should be the same so that you can tell which encrypted key file goes with which answer key file. For the same reason, you should use the same filename prefix for your .tst files as you do for your .key and .kfc files.

Taketest


Taketest is the DOS program which interactively gives a test to a student and records results in a log file. Taketest is meant to be used by the students, but should also be used by the instructor during test preparation (for testing the test). Taketest expects an argument on the command line which is the name of the test that should be taken. For example, one would enter
taketest intro
to take the test named intro. For this to succeed, a file named intro.tst and a file named intro.kfc should be in the current directory, or (if run from Test Manager) in the same directory as the Test Manager program. All test files must end with the .tst file extension. The Test Manager executable should either be in the PATH line of your AUTOEXEC.BAT file, or in the current directory with the tst and kfc files.

Taketest test file format


Test files can be divided into two sections. The first is the parameter section, where some constants can be set for the duration of the test. These parameter commands may appear in any order, or be left out. They must begin in the first column of the .tst file. 
/pageburst nn	If there are nn (an integer) lines already
			on the output page, do a page break	
			before the next question.  This affects
				the appearance of the Report.  If 
the				command is omitted, there will 
be no				page breaks.
/hiddenlog	If this command is included, the log file
			will be written as a hidden DOS file.
/delaytime  n	Delay n seconds after each question.	
			Default is no delay.
/logfile PATH:filename.ext
			Log results to PATH:filename.ext	
			instead of the default (b:res.log).
/heading1 text string
			Place the text string at the top of each
			bursted page.  The text string is also	
			displayed at test startup.		
			(Put your institution name in heading 1.)

/heading2 text string
			Printed just below heading1.  (Put the
			instructor and class name here.)
			
After the parameter section, begin the test with the
	/begintest test-descriptive-text
command.  A question block contains a
	/question
followed by the text of the question (up to 20 lines), followed 
by the
	/end
line.  An additional command, /accept, can be placed in the 
body of the question (it is suggested you put it just before the 
/end ).  With a statement such as
	/accept a b c d e
you can restrict the answers actually graded wrong.  The 
student is given up to 3 chances to enter one of the listed 
answers before the question is marked wrong.  The /accept 
command can also be used for fill-in type answers containing 
more than one character.  ie.,
	/accept clinton bush reagan carter
After the last question in the .tst file, there should be an
	/endtest
command.

Testprint


Testprint is the part of the system which prints the reports. It should be used by the instructor. As Taketest is run, it creates or updates the logfile (ie., res.log) which is a file which stores transcripts of repeated runs of Taketest. When Testprint is run, it reads in the logfile and allows you to print transcripts of all the tests, or any single test run. After a series of tests are taken, so that res.log contains the results of about a dozen tests, you may want to print the result reports for these tests. To do this from the DOS command line type:
	testprin
and press the return key (since DOS only recognizes the first 8 characters of the file prefix, it doesn't matter if you type testprin or testprint.) If you did not use the default logfile, by including the /logfile command in the test file, you can enter the logfile name as a command line parameter. ie.,
	testprint a:res.log
will print the results logged in res.log on floppy drive a:.

Here is an example of a typical Testprint screen:

Each numbered line corresponds to a student's test that has been logged in the logfile. Each line is numbered to allow you to select a particular entry to print. You can also print all of the tests stored in the logfile by entering a for all. Usually you would want to print all of the test reports at once. However, if the printer develops a problem during output you can reprint selected test reports by entering the line number displayed in the Testprint menu. You can repeatedly enter the number for the selection you wish to print. When done, enter q to quit.

Testprint troubleshooting


If the printer is not ready, or runs out of paper or into other trouble, you will be asked for direction on how to proceed. If you can fix the problem, do so and enter r for retry. Testprint reads a log file which is written when the student takes a test. This file is updated as the student proceeds through the test. If the student reboots the PC, or there is a problem with the disk drive, then the log of the test being taken may not be completely written. Testprint will handle this case and not display a score for the incomplete test run. When testprint is starting, messages will indicate when something unusual about the logfile is discovered.

The results logfile is always at risk of damage from a malicious user (especially those unprepared for the test.) Using the /hiddenlog test directive makes the logfile more difficult to discover. It is recommended that you limit the number of tests logged in the logfile to under 20.

If Testprint issues an ERROR message, make sure to note the error number. Look for the error number in the table of ERROR CODES to see what went wrong. If Testprint doesn't like the results logfile for some reason, you can still get your results by just printing the log file.

Makedisk


Makedisk is a program which can prepare a test disk. This disk will contain a copy of the taketest program and the .tst and .kfc files for a test of your choosing. The program will repeatedly ask you for a test name, and copy the associated .tst and .kfc files to the floppy.

After copying the .tst and .kfc files for a given test, Makedisk sets the DOS attribute of the .kfc file to HIDDEN. So in addition to the encryption of the answers of the .kfc file, the file will not show up if a normal DOS DIRECTORY command is given.

A typical Makedisk run might look like this:

The example above shows a test disk being created on drive b:. A test with name example (comprised of files example.tst and example.kfc) is transferred to the test disk

Test Manager, for Windows


The Test Manager program is Testman.exe. Test Manager is a Windows program which can be used to create and edit test files (which have .tst filename extensions) and key files (which have .key extensions). Test Manager is able to call on the other programs to help prepare, test, and print reports of tests. Testman is intended to be a sort of IDE for creating tests. IDE stands for Integrated Development Environment. This means that most test creation functions can be controlled through Testman and it can be used to help write and score tests.

The Test Manager window:

This example shows the Testman program with a single test file, example.tst, loaded. You can load several different test files into different editor windows, and use CUT and PASTE between the windows. When you compose a new test, you can save it to the name of your choice. Most Testman actions can be initiated from the pulldown menu at the top of the Testman window. The button bar below the pulldown menu is a quicker way to initiate the same functions available through the pulldown menu. You can single click on the button to initiate the action. The bottom line of the Testman window displays a short description of the botton you've moved the cursor over.

Testman allows the instructor to:

An Example Test


The following simple worked example should help familiarize you with the Test Manager system. The example test used, called example, should be in the directory as the other Test Manager files.

Creating a test


From the Testman menu bar, select FILE, then NEW (or press the button on the speedbar.) For this example, we'll use a test that has already been entered into the system. To open an existing test, select FILE, then OPEN from the menu. Select the proper filename from the FILE OPEN Dialogbox (in this case it would be example.tst). You can also press the button on the speedbar, to open an existing test file. Once you're on your own, you can edit your old tests to create new ones, or prepare new tests from scratch. An example test template, for 10 questions, can be used as a starting point, and should be located in your installation directory called simple10.tst. You can save your test with a new filename by selecting FILE, then SAVE AS, from the menu or update the existing file with FILE, SAVE. The save operations are initiated from the speedbar by pushing the button.

While editing the file, you can use the buttons, or EDIT pulldown functions, to move text around within a single test, or between any Windows application (that supports the Clipboard.) The button can be used to reverse the last operation. You can also search through tests using the SEARCH menu items, or by pressing the buttons on the speedbar. The WINDOW menu items are used to arrange the open file windows within the Testman window. Printing functions can be accessed through the FILE menu or directly from the speedbar with the buttons. Once you've typed in your test you should save it to a new filename.

Creating the answer key


Open a new file and type in the answers to the .tst file. Type one answer per line, without putting any spaces in front of the first characters in each line. You can have both the .tst and key files open and displayed in the Testman window. After the .key file is entered, save the file with a .key filename extension (with the same filename prefix as the test it answers.) In out test example, you would save the key file as example.key (you should find example.key in your installation directory.)

Encrypting the answer key


Next we must turn example.key into example.kfc using the Makekeys program. The Makekeys program can be run by selecting Makekeys from the menu, or by pressing the button on the speedbar. Makekeys will prompt you for the input and output files. In this example you would type example.key for the input filename, and example.kfc for the output filename.

Testing your test


After you've written the test and encrypted the answer file you should test your test and answers. You do this by running Taketest. Taketest will ask you for the test name, if you do not enter one on the command line (the test name for our example is example). The Taketest program can be run by selecting Taketest from the menu, or by pressing the button on the speedbar.

Making a test disk


Once your test is complete, and you have tested it using Taketest, you can prepare a student test disk. When you run Makedisk, you enter the drive to copy the files to, and the name of the test (in our example, the test is example). The Makedisk program can be run by selecting Makedisk from the menu, or by pressing from the speedbar.

Printing the test scores


Use Testprint to print the scores recorded on the test disk floppy, once a number of test runs have been accumulated. When Testprint starts, you should enter the name of the logfile from which Testprint is to print results. The Testprint program can be run by selecting Testprint from the menu, or by pressing the button on the speedbar.

Error Codes


If something goes wrong, the test programs will issue a message conatining an error code before exiting. Look up these error codes in the following list to try to determine what went wrong. Since students will be running the TAKETEST program, they should be instructed to report any error codes to the proctor. 

Testman 3.0 Error Codes

100	Makekeys.exe missing.  Executable file is missing from 
	installation directory.
101	Taketest.exe missing.  Executable file is missing from
	installation directory.
102	Testprin.exe missing.  Executable file is missing from
	installation directory.
103	Makedisk.exe missing.  Executable file is missing from 
	installation directory.


Makekeys 3.0 Error Codes

200	Unable to open input key file for read.  Check that 
	spelling of the key filename is correct.
201	Unable to open output file to write encrypted keys.
202	Error writing encrypted string to output file.

Taketest 3.0 Error Codes

300	Unable to open the test file (testname.tst).  Check that 
	the .tst file exists.
301	Unable to open key file for testname.  Check that the 
	.kfc file exists.
302	Error reading keys from .kfc keyfile.  
303	Testfile is empty.  If, ie., the test name is testname (and 
	the test filename is
	testname.tst), this indicates that the contents of 
	testname.tst is empty.
304	The /delaytime argument in the testname.tst file is out 
of 	range.  Use a value less than 10 for the number of 
	seconds to pause after each question.
305	Incomplete contents of testfile.  File ended while 
reading 	parameters.
306	A user name (test taker name) less than 2 characters 
	long was entered.  Names of 2 or more characters are 
	required.
307	Problem reading test name from .test file.  Check the 
	/begintest line in the tst file.
308	Unable to open log file.  If a /logfile line was used in 
the 	.tst file, check that the destination directory exists.  If 
no 	/logfile line is used, then the log file is written to 
	B:RES.LOG (check that the floppy is present)


309	Unable to read log file attributes.
310	Unable to set log file attributes (in order to make the 
	results-file HIDDEN if the /hiddenlog option is used.
311	Unable to complete a write to the log file.
312	No questions were found in the .tst file by TAKETEST.

Testprint 3.0 Error Codes

400	TESTPRINT couldn't find the log file.  If you do not 
	enter a log file name,  TESTPRINT will attemp to open 
	res.log in the current directory.  Check		that 
res.log is in the current directory.  If you entered a 	log file 
name, check that the filename is correct.
401	A test log was being read from the log file when a new 
	test began (before the end of the last test was found).  
	This error could be seen if a floppy containing the log 
	file is removed before the test is finished, or the PC is
	rebooted before the completion of a test.

402	End of a test found without a beginning record.  There 
	was a problem writing to the results log file.
403	Extra data found in the log file, or results log file 
format 	not followed.
404	TESTPRINT aborts if it detects many errors in the log 
	file.  If the file typed on the TESTPRINT command 
line 	is not a results log file, this error will occur.  Check that 
	the results file name is correct.
405	Truncated end of test record.  The record written at the 
	end of a test, containing the score, was short or 
	damaged.
406	Choice entered to print is greater than the number of 
	tests in the log file.
407	Test number to print is greater than the maximum 
	number of tests supported.

Converted to html from Test Manager User's Guide.

From version dated 6-27-1994.