All » Articles » Commodore 64 (7)
All » Downloads (10)

ai64: Batch archive extractor for IDE64 users

Posted by: Ferenc Veres on January 28, 2005 09:23:41 PM +00:00(62656 Reads)

ai64 allows you to convert complete directory structures containing c64 programs into IDE64 compatible copy of the whole structure. e.g. try running it on a complete copy of Digital Dungeon FTP Archive, before burning a CD for IDE64 usage. Most of the archive will be converted to .prg and .d64 files, which are very easy to use on C64! The program is licensed under GPL v3.

Latest version: made available GitHub in 2021: ai64_extractor

Demo Result

You can see a demo of what this program can make from the various zip, rar, zipcode, lnx, t64, d64 and more archive files on the c64.rulez.org FTP site.

400kb HTML file list:  ai64 demo result.

System Requirements

PHP-CLI Which makes it possible to run PHP programs from the command line. Normally part of all GNU/Linux distros, on Mandriva Linux it is the "php-cli" package.
GNU/Linux or Unix
Tested on Linux only. Windows compatible filename generation is now supported, Windows platform support is planned (never tested).
unzip Command line unzip to decompress the files.
unrar Command line unrar to decompress the files.
tar Command line tar to decompress the files.
gzip Command line gzip to decompress the files.
zip2disk
Zipcode to d64 converter by Marko Makela.
https://c64.rulez.org/pub/c64/other-OS/Unix/
cbmconvert
Marko Makela's converter masterpiece to handle all the c64 file formats (tested with v2.1.2).
ftp://ftp.zimmers.net/pub/cbm/crossplatform/converters/unix
c1541 Command line utility, part of Vice emulator (tested with v2.2).
https://vice-emu.sourceforge.io/
d64list
[Not needed if you use Vice c1541] This is a little C program I made years ago, included in ai64 download.
https://c64.rulez.org/pub/c64/other-OS/Unix/

What does it do?

The program will process a complete directory of C64 files convert all files to c64 usable format and copy them into a destination directory structure. Thus from all your ZIP, TGZ, etc files, you will get a big directory directly usable on your IDE64 or via PCLink cable, SD card readers, pendrive compatible cards like 1541 Ultimate, or on Ultimate 64 or a C64 Emulator.

Currently the following formats are handled (anything can be nested, e.g. a single prg in a d64 in a zip):

C64 files:
D64 extracted if 1 file only and BAM is valid. Kept as D64 otherwise. Hi-score files (if identified) are ignored if there is only 1 more file. Some BAM validation messages in names/header are taken into consideration.
T64 extracted to files (if there is a single file named "FILE" is inside, the name of the original t64 is used when saving it to prg.)
P00 extracted to normal files
PRG copied as is
Compressed files
ZIP extracted and re-processed
TAR extracted and re-processed
GZ extracted and re-processed
TGZ extracted and re-processed
RAR extracted and re-processed
LNX converted to d64 and re-processed
ZIPCODE (1!,2!..) converted to d64 and re-processed
Not copied (not usable on C64):
TXT (also extensionless README, 00INDEX)
DIZ, ME, NFO
COM, EXE
AVI, MPG, MPEG
PDF, DOC, DJVU
PNG, JPG, JPEG, GIF
DB  
INI  
DEL (D64 extraction dirt)
DIR (IDE64 dir type, cannot be copied)
LNK (cannot be copied to IDE64)
REL (cannot be copied to IDE64)


Files starting with dot are not copied (Unix hidden files, FTP site messages).

Not listed files: copied as is.

Support is missing for: BZIP2, LHA (lzh).

All the stored files are saved with characters which are readable on C64.  Filenames are converted to 16 + extension, IDE64 can handle this length (requires 0.9x or higher IDEDOS). Already existing files are not overwritten, the new file will get "-1", "-2".. index, thus running the program twice will create all destination files twice.

When the whole directory structure conversion is finished, ai64 will take another long walk. It will rearrange all the directories which contains more than 100 files, to make sure there are no more than 100 files in a directory (easier to handle on c64, MAN, etc). The created subdirs will be called "ai100-X", "ai200-X" and so on, where "X" means the first word of the first program in that dirctory. E.g. "ai300-blackjack".

Installation

Edit ai64 executable file if necessary:

  1. Customize the location of your php interpreter in the first line.
  2. Set the location of your d64list program or comment out that line to    bypass analizig d64 contents.
  3. Make sure the $tmp_dir is far from any important locations, because the  program will run thousands of "rm -rf" commands inside it (honestly, I create a "lion2" user just to run the program, you never know... especially while developing it.)  See ramdisk tip below.
  4. Customize the location of "cbmconvert" executable as well.
  5. Make sure the other decompress executables are on your path.
  6. There are some other configurations in the source's head.

Usage

  1. Collect the c64 stuff what you want to convert into a directory structure
  2. Run "ai64.php sourcedirname destinationdirname"

ai64.php [options] original_dir destination_dir

-s path/name Skip to this file before staring processing
(Use this to continue after something went wrong)
-x , Use ',' as file extension separator (default is '.')
-n 100 Number of maximum files per folder (default 300)
-v Verbose, list successfully processed files
-V Super-verbose, also list archives while processing them
-w Force windows compatible file naming (remove more chars)
-u Enable unicode chars like ↑ | (remove otherwise)
-t path Temp dir for extractions (default '/tmp/[USER].ai64/')
-h Print this help


If ".php" is not registered to your PHP interpreter, you may need to type  "php ai64.php" instead, assuming php.exe is on your PATH.

There is no warranty of any kind. So I advice again, to create a temporary user which cannot write your home directory, and run the whole conversion by that user!

Examples

ai64.php -x , -v -w c64/Demos c64stuff

Converts with comma as extension separator, windows compatible file names. Useful when copying to HD using FuseCFS (http://singularcrew.hu/idedos/).

ai64.php downloadedwares coolwares

Convers with dot as separator. Linux file naming conventions. Useful when burning a CD for use with IDE64.

Tips

Using Ramdisk or TmpFS

Since the program creates and deletes a LOT OF TEMPORARY FILES while uncomressing the archives, it is a good idea to choose a temporary dir (configred in the script) in RAM.

Using tmpfs (recommended)

mount -t tmpfs none /mnt/rd
chown lion2:lion2 /mnt/rd

Using ramdisk (more complicated)

Usually Linux distros have ramdisk by default, to find it see:

ls /dev/ram*
dmesg | grep -i ramdisk

The second one also displays the size, what you can configure with the "ramdisk_size=64000" (64MB) kernel parameter at boot time (bootloader config). I use the following small shell script as root, before converting:

mke2fs -m 0 /dev/ram0
mount /dev/ram0 /mnt/rd
chown lion2:lion2 /mnt/rd

Logging the errors

There is no option in the program to log the error messages, but it is very simple using standard Unix tool, "tee" (-a is for append, if needed).

ai64.php orig_dir dest_dir 2>&1 | tee -a errorlog.txt

Contact

Feedback and patches are welcomed, use the contact menu on this site. For updates look at this page.

Related links

C64 XLFiles - Download large archives convered with ai64
IDE64 website

C64.rulez.org FTP site (c64, plus/4 programs)
Utopia.hacktic.nl (c64 programs)

About me

Photo of me Ferenc Veres
web developer
about me

Commodore books
Commodore logo My C64 and Plus/4 book collection (Hungarian): Commodore könyvek
Featured article

Exisitng editors for text data DjVu files are quite limited, like for example DjVuSmooth. So I've implemented a new editor in JavaScript, that allows editing both the strucutre of the text (paragraphs, lines, words,...) and the coordinates of the text boxes by simply dragging with the mouse, features like create, delete, merge are also available.

My other websites
Categories