Frames, Arrays, and Files

How Things Work, The Big Picture

Normally, when a program (NS BASIC or even NewtonScript) quits, the information stored in the variables is discarded. This is normal, and actually good, since this frees up space in the relatively small area of memory set aside to hold variable data. This area is called the "Heap", and it's what you run out of when you get that annoying message box saying that there is not enough memory, and restarting may help. The other time that variable memory is discarded is when the Newton is restarted. How do you save information that you'd like to access again? The Newton records information that you want to retain in a special area of protected memory called a "store". This memory is protected from all but the most catastrophic of events, like hitting the Newton with a hammer, or losing all power (including back-up batteries). Some Newtons (like the 120) even save the information with no power, since they use Flash memory to store information. Every Newton has at least one store, the internal memory of the device. A storage card is just an external bit of memory. When you insert the card, the Newton recognizes the new memory, and treats it as a separate store. Yank the card out, and the store disappears. This is not unlike using a floppy disk or other removable disk on a computer.

We now have these big areas of memory that can be used to record information, but exactly how does the Newton organize that information? Everything placed into a store must be in the form of a Frame. Recall from last month's column that a Frame is a data container for one or more items of information. You can think of a frame as containing chunk of related information, like an inventory record, bank transaction, meeting, name card, or note. You can't just stick frames in a store, you must group them into bigger clusters called soups. Soups can contain as many frames as memory will allow, and they can be as similar or different as you'd like. That's why they're called soups: you can throw anything you want in, and it still tastes good! NS BASIC calls soups "Files". We call the frames within the file "records".

When you create or access an existing file, you can add new records, access existing records, modify records, and delete records. The file you create will exist on all stores that are available when your program is running. If you have a storage card inserted, then the file will exist on both the card and the Newton. The records you create will be placed on the card if "Store new items on Card" is checked in the card app. Otherwise they will be created on the Newton. You can read and modify records without concern for where they are physically located, but remember that any records on a storage card will "disappear" when that card is removed. This is just like the built-in applications.

Using Files In NS BASIC

We're going to create a simple filing application. This application will keep a record of your music collection. You'll be able to keep track of the CDs, Tapes, and (for us older folks) LPs you've got, and the ones you want to get. It's not going to be fancy at first, but it will grow into a full-fledged Newton UI program with time. We've got to start somewhere, so we'll start with the file used to store the collection. The first step is to think about what kind of information you need to keep for each item in the collection. This will help you to understand what the records stored in the file should look like. Here's my first try at identifying a record:

Music information
Group: (string)
Artists: (array of First Name (string), Last Name (string))
Title: (string)
Media: CD, Tape, and/or LP
Running Time: (time)
Tracks: (array of Name (string), Length (time))
Notes: (lots of text)
Cost: (real)

I'm sure you could come up with a better list of fields, but we'll go with this for now. Note that I tried to identify the data type for each field, and that some fields (like Artists, Media, and Tracks) seem to contain more than one value. Here's a frame that could be used to store one music information record:

{Group: "The Guppies",
Artists: [{LastName: "Meanie", FirstName: "Blue"}, {LastName: "Tick", FirstName: "Edward"}],
Title: "The Guppies, Unplugged",
Media: {CD: true, Tape: false, LP: false},
RunningTime: 56,
Tracks: [{Name: "Swim with me", Length: 26}, {Name: "Swim with me, Instr.", Length: 30}],
Notes: "This is one wet disc!",
Cost: 12.95}

We'll worry about exactly how you get this information from the user and into a record next month. For now, we'll just create this and some other records in our program, and write them into a file. Then we'll access the records. Then we'll all take a break until next month!

Files: A Three Step Process

There are three steps your program takes to use a file. Note that you can actually access many files at once within a single program, but for simplicity I'm just using one. The first thing you must do is create the file, using the CREATE statement. You actually only need to create the file the very first time you use it. After that, you can simply open the file using the OPEN statement. Because of this, a simple set of statements is usually used to create or open a file:

10 REM Music Information
20 OPEN MusicDb, "MusicFile", Title
30 IF FSTAT <> 0 THEN CREATE MusicDb, "MusicFile", Title

The OPEN statement attempts to open a file named "MusicFile" with the Key field Title. If it succeeds, the open file id will be placed in the variable named MusicDb. You use the file id (called a Channel) to perform GET and PUT statements. GET retrieves a record based on a key value, and PUT saves a new or modified record into the file. The Key field is a field that will have an index on it. You can locate specific records quickly using the key field, and retrieve all records sorted on the key field. The one limitation of the key field is that every record in the file must have a unique key field. We'll talk about that more in a later column, but for now we'll just select the Title field for the key, since titles are usually unique. The variable FSTAT is set by the various file statements. If the operation succeeds, FSTAT will be 0. So the code above translates into "Try to open the file, if that fails, then create the file." Since the file won't exist the first time, it is created. Every other time you run the program, the file will simply be opened.

The file is now open. Let's add some records by including them directly in the program:

40 rec := {Group: "The Guppies", Artists: [{LastName: "Meanie", FirstName: "Blue"}, {LastName: "Tick", FirstName: "Edward"}], Title: "The Guppies, Unplugged", Media: {CD: true, Tape: false, LP: false}, RunningTime: 56, Tracks: [{Name: "Swim with me", Length: 26}, {Name: "Swim with me, Instr.", Length: 30}], Notes: "This is one wet disc!", Cost: 12.95}
50 PUT MusicDb, rec
60 rec := {Group: "The Fish", Title: "The Fish Live"}
70 PUT MusicDb, rec
80 rec := {Group: "Hurl", Title: "Projectile"}
90 PUT MusicDb, rec

Note that line 40 is just one really long line, without any returns in it! You can create program statements of any length with NS BASIC. These six lines just add three records to the file. Note that only the first record has all the fields. This is not a problem in NS BASIC.

The next chunk of code prompts for a Title, and attempts to find the record with that title:

100 PRINT "Enter a title, or press return to stop searching:"
110 INPUT searchTitle$
120 IF searchTitle$ = "" THEN GOTO 180
130 GET MusicDb, foundRec, searchTitle$
140 IF FSTAT = 1 THEN GOTO 180
150 IF FSTAT = 2 THEN PRINT "I didn't find that record, here's a close one:" ELSE PRINT "Found it:"
160 PRINT foundRec
170 GOTO 100

This loop lets you enter a title, and then it searches for the matching record. GET attempts to locate the record in the file with the key value specified in search Title, returning the record in foundRec. FSTAT will be 0 if the key field matched. It will be 1 if all keys are smaller than the search value. It will be 2 if there was not an exact match, and the record with the next larger value will be returned. If the user pressed return without entering a search value, or if no record was found, the program uses a GOTO to get out of the loop. Otherwise, the record found is printed and the program loops back to the entry of the search value.

The final chunk of code just displays all the records in the file in sorted order:

180 GET MusicDb, foundRec, ""
190 IF FSTAT = 1 THEN GOTO 230
200 PRINT foundRec
210 GET MusicDb, foundRec
220 GOTO 190
230 END

Here we use an initial search value of "", which is smaller than every key, to get the first record. Line 190 checks to make sure a record was found. If it was, line 200 prints it out, and line 210 GETs the next record in index order. Note that no search value is used in line 210. Line 220 makes the program loop back to repeat the process until the end of the file is reached.

Sample Output

Here's a sample run of the program:

* run
Enter a title, or press return to stop searching:
? The Fish
I didn't find that record, here's a close one:
{Group:"The Fish",title:"The Fish Live",_uniqueID:1}
Enter a title, or press return to stop searching:
? The Guppies, Unplugged
Found it:
{Group:"The Guppies",Artists:<Array:2>,title:"The Guppies, Unplugged",Media:<Frame:3>,RunningTime:56,Tracks:<
Array:2>,notes:"This is one wet disc!",Cost:12.95,_uniqueID:0}
Enter a title, or press return to stop searching:
? Projectile
Found it:
{Group:"Hurl",title:"Projectile",_uniqueID:2}
Enter a title, or press return to stop searching:
?
{Group:"Hurl",title:"Projectile",_uniqueID:2}
{Group:"The Fish",title:"The Fish Live",_uniqueID:1}
{Group:"The Guppies",Artists:<Array:2>,title:"The Guppies, Unplugged",Media:<Frame:3>,RunningTime:56,Tracks:<
Array:2>,notes:"This is one wet disc!",Cost:12.95,_uniqueID:0}

Note that the search key, if only a partial match, sets FSTAT to 2. Also note that the records are sorted by the title field when they are accessed sequentially in lines 180-220.

Next Month

We still have a lot of information to cover for files, such as:
how to modify records,
how to delete records, and
what to do if you have a field you'd like as the key, but the values are not unique

We'll continue with files next time.

Undocumented Tip

The array data type is very powerful and useful. Using NS BASIC statements, you can create a single-dimensioned array using the DIM statement:

10 REM Arrays
20 DIM a[20]
30 PRINT a
* run
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

You can also create an array with an initial value simply by specifying the array literal in an assignment statement:

10 REM Arrays II
20 a := [1,2,3,4,5,6,7,8,9,10]
30 PRINT a
* run
[1,2,3,4,5,6,7,8,9,10]

You can use all of the NewtonScript Array functions to add array elements, remove array elements, and search arrays. Only a few are in the NS BASIC handbook. Here's a brief introduction and example of some of the more useful ones:

AddArraySlot (a, val) - Add a new element to the array in variable a, with the value val

Array( numElements, val ) - Create an array of numElements elements, each set to val

ArrayPos( a, searchVal, startIdx, testFunc ) - find the offset in array a for a value searchVal, starting at offset startIdx, using testFunc to compare (testFunc can be NIL for numeric equality testing, or something like getglobals().functions.strequal for strings, or even an NS BASIC function)

ArrayRemoveCount( a, startIdx, num ) - removes elements from an array, making it smaller

You can create multi-dimentional arrays using several techniques. For small arrays, just use an array literal with initial values in an assignment statement. This creates a 3x3 array:

10 REM Arrays III
20 a := [[0,0,0],[0,0,0],[0,0,0]]
30 PRINT a
* run
[<Array:3>,<Array:3>,<Array:3>]

You can access multi-dimentional arrays by using as many sets of [] as needed. For our 3-d array we can set values as follows:

a[0][2][1] = 5
a[2][1][0] = 10

A more general purpose solution to creating multi-dimentional arrays is possible. You can create arrays of any size. Here's an NS BASIC snippet to create a 20x10x5 array:

10 REM Arrays IV
20 REM create arrays of dimensions 20x10x5
30 DIM a[20]
40 FOR i = 0 TO 19
50 a[i] := array(10, nil)
60 FOR j = 0 TO 9
70 a[i][j] := array(5,nil)
80 NEXT j
90 NEXT i
* run
* a[5][5][3] := 100
* print a[5][5]
[NIL,NIL,NIL,100,NIL]
*