Go, PET, Let Hen - Curious adventures in (Commodore) BASIC tokenizing

22 pointsposted 18 hours ago
by masswerk
(masswerk.at)

7 Comments

OhMeadhbh

14 hours ago

This takes me back a few years. I spent HOURS writing BASIC programs to analyze other BASIC programs as a kid. My favourite PET trick was to hide the basic source by putting a comment (REM statement) at the beginning and end of the program. Then POKEing the address of the ending comment in the "next line" link in the first line. It turns out that when the interpreter was running the program, it didn't use the "next line" link, it just assumed the bytes following the current line were the beginning of the next line. But the LIST command //did// use the link. So you could get a program to run perfectly fine, but when someone did a LIST, the only thing they saw were the two comments.

I can't remember if this worked on the C64, but it worked on the 4016 and 4032's in our high school's computer lab.

jim_lawless

13 hours ago

You could do similar things on a C64 and other computers. You might try this out on a C64 emulator such as VICE.

10 REM NOTHING TO SEE HERE

20 PRINT "HELLO!"

POKE 2049,1

Run it. You'll see HELLO! LIST it and you'll continuously see line 10. If you try to LIST 20 the machine pretty much locks up.

Screen image is here:

https://jimlawless.net/images/remtrick.gif

(note that in the above image, you'll see two RUN lines ... it appears that I captured the screen as it was in mid-scroll... )

LocalH

11 hours ago

At least on the C64, you could also put a line containing REM shift-L in the program, and the LIST command would crash out when encountering it.

masswerk

10 hours ago

The problem being that the LIST routine should handle a comment like a string (which is how it is parsed and stored: a comment is essentially an unquoted string extending to the end of the line), but doesn't bail out of keyword expansion, whenever it encounters a REM token.

[Edit]

Coincidentally, a shifted "L" is PETSCII code 0xCC. Which is just one after the highest available token in Commodore BASIC 2.0 / V.2. (The last one being 0xCB, `GO`.) Therefor, a lookup into the keyword list will yield the terminating zero-byte, which probably causes the problem. (E.g., by defeating what was intended to be an unconditional branch instruction.)

(In BASIC 4.0 for the 40xx/80xx PETs, this is actually a valid token, namely `CONCAT`, which is expanded by LIST without further issues. Meaning, this kind of LIST protection can be broken by simply loading the program on one of the later PETs.)

jklowden

12 hours ago

Why do I remember that every C64 BASIC keyword was a 2-byte integer? A typing shortcut was to enter the first letter, followed by a "shifted" high-bit character. Every keyword was represented that way.

Variables were also 2-bytes, but ASCII. The user could enter a longer name, but only the first two characters were significant.

masswerk

11 hours ago

Yes, variable names are 2 bytes, in their stored memory location in RAM. As these must be 7-bit ASCII bytes, the sign-bits and their distribution over these two bytes is used to encode the type. And all simple variables take 7 bytes of memory in total, regardless, whether the remaining 5 bytes are actually needed to store the data or not.

  sign-bits   type (payload)

  0   0   ... floating point number (1 byte exponent, 4 bytes mantissa)
  1   1   ... integer (2 bytes)
  0   1   ... string (1 byte length, 2-bytes pointer to location)
  1   0   ... FN function (2 bytes pointer to BASIC, 2 bytes pointer to parameter variable)
In a program (the BASIC text), though, variables names are stored in full and in plain ASCII, at whatever length of characters.

LocalH

11 hours ago

Not every keyword could be abbreviated with only two characters. The linked article actually discusses this mechanism. Once tokenized, the keywords only took up a single byte.