<?xml version="1.0" encoding="UTF-8"?>
<!-- **************************************************************************
.... For copyright and licensing terms, see the file named COPYING.
.... **************************************************************************
.... vim: indentexpr=:
.-->
<?xml-stylesheet href="docbook-xml.css" type="text/css"?>

<refentry id="commodore-basic-detokenize">

<refmeta xmlns:xi="http://www.w3.org/2001/XInclude">
<refentrytitle>commodore-basic-detokenize</refentrytitle>
<manvolnum>1</manvolnum>
<refmiscinfo class="manual">user commands</refmiscinfo>
<refmiscinfo class="source">BASIC</refmiscinfo>
<xi:include href="version.xml" />
</refmeta>

<refnamediv>
<refname>commodore-basic-detokenize</refname>
<refpurpose>detokenize Commodore BASIC into PETSCII</refpurpose>
</refnamediv>

<refsynopsisdiv>
<cmdsynopsis>
<command>commodore-basic-detokenize</command>
<arg choice='opt' repeat='rep'><replaceable>filename(s)</replaceable></arg>
</cmdsynopsis>
</refsynopsisdiv>

<refsection><title>Description</title>

<para>
<command>commodore-basic-detokenize</command> converts a saved Commodore BASIC program, in tokenized (<quote>PRG</quote>) form, to human&#x2010;readable text in the PETSCII encoding.
It is a filter program, in the same vein as <citerefentry><refentrytitle>linenumber</refentrytitle><manvolnum>1</manvolnum></citerefentry> and <citerefentry><refentrytitle>vis</refentrytitle><manvolnum>1</manvolnum></citerefentry>.
</para>

<para>
If invoked with one or more <filename><replaceable>filenames</replaceable></filename>, it will read those files and detokenize them.
Otherwise it will act as a filter and detokenize its standard input.
All output is always written to standard output, with any error messsages written to standard error.
</para>

<para>
Line numbers are printed in decimal.
Tokens are converted to upper&#x2010;case human&#x2010;readable strings.
The tokenizations of Commodore BASIC versions 2.0, 3.5, and 7.0 are understood, as each is a subset of the next.
Commodore <emphasis>PET</emphasis> BASIC 4.0 added tokens to 2.0 in a way that is incompatible with the other two, and (depending from whether they use those tokens) programs might not be converted properly.
</para>

<para>
<command>commodore-basic-detokenize</command> inserts SPC between tokens to prevent detokenization creating code where one token runs into the next where it did not in the original program.
</para>

<para>
No additional processing is done.
No analysis is done on the token stream to see whether it is actually a valid program.
No transcoding of characters is done.
Characters are left in the PETSCII character set.
</para>

<caution>
This is not the language syntax nor the keyword set nor even the character encoding used by <citerefentry><refentrytitle>BASIC</refentrytitle><manvolnum>5</manvolnum></citerefentry>.
This will not usually make Commodore programs usable in <citerefentry><refentrytitle>basic</refentrytitle><manvolnum>1</manvolnum></citerefentry> or <citerefentry><refentrytitle>basica</refentrytitle><manvolnum>1</manvolnum></citerefentry>.
</caution>

<tip>
Output programs are native Commodore PETSCII text, complete with the PETSCII newline character, which is Carriage Return rather than the Unix convention of Line Feed.
To load or view the program in UTF-8 tools convert it with <citerefentry><refentrytitle>iconv</refentrytitle><manvolnum>1</manvolnum></citerefentry> <arg choice='plain'>--from-code=Latin1</arg>, <citerefentry><refentrytitle>petscii-to-ecma48</refentrytitle><manvolnum>1</manvolnum></citerefentry>, or <citerefentry><refentrytitle>petscii-to-wtf8</refentrytitle><manvolnum>1</manvolnum></citerefentry>.
See the <citetitle>basic Guide</citetitle> for a discussion of where each might be appropriate.
</tip>

</refsection>

<refsection><title>History</title>

<para>
This <command>commodore-basic-detokenize</command> was written in 2026, inspired by the author wanting to see how an old Commodore PET hangman game was written, without the clunky overhead of spinning up an entire emulator and trying to get a listing in a host operating system text file.
</para>

</refsection>

<refsection><title>Author</title>
<para><author><personname><firstname>Jonathan</firstname> <surname>de Boyne Pollard</surname></personname></author></para>
</refsection>

</refentry>
