tinyDisk2: a floppy disk emulator

🇬🇧 Translation revision in progress… please take this article as is.

In a previous article, I told about the Apple II of RetrOfficina with the broken floppy drive. This is a pity, most of the software for this old computer was distributed by floppy disk, and even tape software may require specific BASIC or DOS version, loadable… by disk. Looking around for already made project, I found floppyEmu and wDrive, very nice but a bit expensive. So I decided to create my personal floppy emulator, with the aim of releasing it with an open source license.

In realtà— I started searching on the web for some free project, ending up to SDISK II. The author released both schematics and source code with open source license. I’ve tried to build it without success, but I didn’t put much effort: I was not totally confident with the hardware. Anyways, the firmware code was a very useful base to start with my project.

In tutto questo, I had no idea about floppy drive operating mode, men che mai Apple II one. First of all, I started reading some documentation to understand the electronic circuitry and the software prodocol.

The protocol - physical data organization

The soft plastic cover of a 5¼" floppy protects a plastic disk covered with a thin magnetic layer. All the information are digitally stored by applying a local magnetic field, che vi resta impresso. Conun processo inverso, reading is done by detecting the changes in the magnetic fields from the disk’s surface.

Both reading and writing are done through a couple of heads, very close to the disk’s surface, while the latter is kept spinning. Considering the disk surface divided with regular slices, bits are mapped using an appropriate encoding: bit 1 is represented with a polarity reverse in magnetic field, while bit 0 with an absence of magnetic field variations.

The magnetic field of the disk is flebile and can’t be directly digitalized: an amplifier circuit is needed to keep a constant signal amplitude. In this way the disk drive can handle both new disks, where field variations are “molto marcate”, or older ones. Unfortunately, a long sequence of repeated zeros gives a constant output and the drive would try to increase the gain of the amplifier, portando in evidenza il rumore di fondo e considerandolo come variazioni, quindi “1”. For this reason a bit sequence must not contain more than two consecutive zeros. This is a great limitation since we can’t transcribe a full text directly onto the disk: let’s say, the uppercase A letter has ASCII code 01000001, which is forbidden! But wait… there’s more! During reading operation, the head faces a continuous bit stream and the Drive II does not provide a mechanical way to detect the start of a valid segment. To overcome this latter problem another constraint is requested: a valid byte must have 1 as starting bit. Thus, the A is doubly forbidden. The correct aligment is ensured by a particular sequence of 12 bits, equal to 1111111100, that must preceede every valid segment. This particular code is called “self-synced FF”, since it makes the reding state machine aligned to the following byte.

As for modern 3.5" drives and hard drives, physical disk divided in concentrical tracks, and for each track data is segmented into sectors, così da poter gestire letture e scritture in gruppi più piccoli. The content of a track would be similar to something like this:

In spite of other competitors, the Apple Drive II used a very basic hardware. A price paid with more complex routines needed to handle the disk geometry.

For this reason, the software needs a way to distinguish between one or another sector, represented by a descriptor field, named address field, placed at the beginning of each sector.

The address field contains all the information to identify a specific sector, like “you are reading track number 2, sector number 7”. These values need just one byte each, but they can’t be directly written onto the disk (remember the constraints …..?) An appropriate encoding scheme is needed, named odd-even: you take the fixed sequence 10101010 and you replace the 0s with the odd bits from the byte to be encoded. Then, you take the same fixed sequence and you replace the 0s with the even bits from the byte to be encoded. You have now two bytes that have a starting 1 and do not have any consecutive 0. Of course… each byte will take twice the space, but this is a very basic encoding system.

Address field is always followed by data field, which contains the actual disk data.

Data is not encoded with the odd-even scheme, because it would require twice the space. Other encodings, with an higher efficiency, were adopted. They are called 5-and-3 and 6-and-2, but their implementation is outside of the scope of this article, you can go deep reading th Beneath Apple DOS. For the moment, it’s enough to know that a sector can store 256 byte, but the 6-and-2 encoding makes it grow to 342 byte.

Both fields do have an unique 3 character prologue, needed to distinguish them. The first two bytes are common for both fields, i.e. D5 AA, which are reserved bytes that cannot be used elsewehere.

As mentioned before, each field is preceeded by self-sync FF sequences. Beside their syncronization pattern purpose, these gaps are used as writing buffers.

Since writing relies entirely on the software routines without hardware support, it is done without any feedback, and the gaps are mandatory to avoid that a field being updated would overwrite a next sector.

The Apple II drive system is composed by two board: one inside the drive, named analog card, which has to convert analog signals in digital ones; the other board is inside the main unit, it is named interface card and it is responsible for the higher level coordination. Include, fra le altre cose, it includes a ROM with the bootstrap code and some chips needed to serialize e deserialize bytes between CPU and the drive. The aim of the TinyDiskII emulator is to mock the analog card.

The drive should stay idle until drive enable signal is enabled. To enable the drive means to request a read operation. During read status, all the field variations captured from the head will continuously flow through the read wire. The digitalization logic will clean up the dirty analog signal giving a squared digital signal. In a period of 4 µs, a pulse means a 1 bit, while no pulse means a 0 bit.

The drive is put in writing mode by enabling the write request signal. In this phase, the writing head is turned on and it overwrites any previous magnetic field. The new content is estabilshed by the “write” wire.

The position of the head is controlled by a stepper motor. One phase at time must be activated to perform a step. Track change needs two steps, probably to increase the precision of the movements. This means that there is a ghost track between two valid ones. These tracks are usually ignored, but some programs used them for anti-cheating purposes, which I’m not interested to emulate.

L’emulatore

The SDISK II was a good starting point for the implementation of the emulator. I decided to adopt the same philosophy using just a microcontroller. Disk images are stored on an SD card FAT formatted, then served on the Disk II interface.

In first place, only a command line serial interface was provided to choose image and check system status. Then, I added an OLED display to remove the ‘‘’la presenza di un secondo computer’’’.

I chose an AVR microcontroller - the same of the first Arduinos - because I have a development board with an XMega microcontroller. Moreover, the SDISK II was based too on AVR architecture and I used parts of its code as starting point.

The microcontroller, the SD card and the OLED display require a 3.3V power supply, while all the Apple II system works at 5V. The interface between the emulator and the Apple II ‘‘‘richiede particolare cautela’’’, not only because of the different operating voltages, but ‘‘‘standard logici non compatibili (LVCMOS vs TTL)’’’. An appropriate voltage adapter is needed, else the two parts would not understand each other.

For the first draft I used scavenged parts: cheap and slow logic level translators, some buffer to regenerate the signal from the emulator. The PCB version makes use of appropriate chip designed for different logic interface.

Lettura

There are many ways to store a digitalized copy of an Apple II disk. To keep it simple, I adopted the .NIC format, used by SDISK II too. This is a binary file containing raw disk bits, basically what the reading head is picking up from the disk. The data is ordered by growing sector and tracks, ignoring the invalid “ghost tracks”. In the figures that follow I have highlighted the structure of a .NIC file dump.

Including gaps, each .NIC sector is 416 bytes long. To keep each sector aligned to 512 bytes, a 96 byte zero-padding must be added. This choice was driven by the SD card specification: data is partitioned in blocks of 512 bytes and they can’t be partially read.

The microcontroller has not enough memory to fit the whole .NIC image in RAM. Instead, known the track and the sector to be served, the firmware requests a block from the SD card and, every 4 µs, fetches a bit and serves it on the write line.

This job can be entirely done in code, without the help of timers or DMA. The SDISK II makes use of this approach using some assembly routines.

In my implementation, I made use of a PWM timer peripheral to generate the reading pulses. This peripheral allows the generation of a square wave signal with a given the period and the duty cycle (here, the fraction of time in a period when the signal is pulled low).

Inside the microcontroller, this peripheral is just a counter with an output pin. There are two main register: the counter top value (which is the period) and a threshold value. While the counter is lower than the threshold, the output is kept to 0. When the counter goes over the threshold, the output is pulled to 1.

Suppose you want to generate an 1 but: you just need to program the peripheral’s period to 4 µs and a threshold value of 3 µs. On the other hand, by setting a threshold value bigger than the period, the match never occurs and you get a 0.

At the end of each period the software has to update the threshold according to the next bit. A busy wait is not needed since the PWM can generate interrupts, then the firmware has about 3 µs to fetch the the next bit from SD card and to program the threshold.

The Drive II serves the whole track as a continuous stream and has no knowledge of sectors. Most of the operations are charged to software so, when a certain sector has to be read, the software has to fetch all the incoming address fields until the desired one. This is very simple to be implemented in the emulator: while the drive enable signal is active, the firmware has to cycle all the sectors within the same track.

The track control is done through the four phase wires which, on the original drive, directly operated on the stepper motor responsible of the head positioning. The emulator handles the track change in the main loop without the aid of the interrupts, since phase signals variations are very slow. They can even be displayed on LEDs. The track control routine monitors the four phases and increments an internal counter accordingly. The track change can happen in any moment, even during a sector read. To avoid further complications, I decided to not interrupt a sector read and apply the track change only during sector reloads from the SD card. In the real world, data read during track change have to be discarded.

The track change is relative to latest position and has no direct feedback but the value read from address field. For this reason, the Apple II does a reset operation for each startup by moving the head towards the track 0. This is the explaination of the tickling motor sound that you can hear at the startup.

During the implementation, a first read experiment gave an interesting result. I had not yet implemented the track handling and, whatever was happening on the phase wires, the emulator served the first track only. The Apple II software was able to read sectors from it, then moving to the next track it noticed an inconsistency: the track in number in the address field was inconsistent. In this case, the software gives up with a forced head zeroing, hoping for a spurious track misalignment.

The worst timing constraint during read operation comes from the bitrate of the bits from a same sector, that is 4 µs. There is no hurry to serve a new sector, instead. Just to be clear, the Apple II routines do have a timeout that stops the reading if the sector is not served within a certain time. But this time interval is enough to do some of the housekeeping. To serve a new sector the firmware has to address the correct block from the SD card, which requires something more than 10 µs.

Like I said, my first attempt doesn’t make use of sector buffering. Data are read bit by bit from the SD card each 4 µs, then immediately served on the read wire. The 512 bytes from each block must be entirely read, so at the end of each block there are 96 bytes to be ignored. This increases the overhead time between the end of a sector and the beginning of a new one. For this reason i decided to make use of DMA peripheral to reduce dead times.

The DMA, or Direct Memory Access, is a microcontroller’s component able to manage data exchanges between memory and peripherals without the direct intervention of the CPU. In this particular case, the peripheral il the SPI managing the communication with SD card.

With the DMA, the overhead is reduced to just the technical time needed to setup the DMA itself, requiring the read from the correct SD card block. Then, it loads the sector’s content in RAM in about 300 µs. The firmware does not have to wait the complete buffer reading before serving bits onto the read wire, since the SD card communication is way faster than the Disk II protocol.

I stopped here with optimization of reading phase, since the choice of DMA was mainly driven by writing and formatting phase.

Writing

Two writing cases should be considered, the sector overwriting and the track formatting. Although they’re both writing operations, the former changes the data field content only, one sector at time, while the latter rewrites the entire track, including the address fields. The formatting has stricter timing constraints, so that first tinyDisk II did not support it.

Let’s consider the easiest case, the sector overwriting. We can make some assumptions:

• before starting to change the data onto the disk, the Apple II routine has to align over the correct sector. To do this, it waits for the matching address field;
• the address field is never overwritten, it is just read.;
• the byte 0xD5 is reserved, we have seen before that every field starts with this byte, and can be used as alignment marker;
• during sector writing, the track is never changed.

The approach used to convert write signal into a binary stream is quite simple. First of all, 8 symbols are needed to complete a byte. Each voltage level transition represents a 1. The 0 cannot be immediately detected, since it is represented with an absence of transitions in a slot of 4 µs. A timer must be set up, with a threshold value slighly bigger to keep into account eventual an clock jittering from the Apple II.

The Apple II routine always begin writing a sequence of self-synced FF, which can be discarded. The first useful information, and fundamental for the aligment, is the data field prologue, that is the 0xD5 byte. Got it, all subsequent bytes can be saved in a buffer that will be stored in the correct SD block, overwriting the old .NIC file. At the end, the Apple II comes back reading to align to another sector to be overwritten, if needed.

When multiple sectors has to be overwritten, my firmware can’t handle a new writing until the write back to SD operation is not ended. Luckily, this time is not so long and the Apple II is patient enought.

Obviously, the Apple II has a timeout to avoid an infinite loop if no address field matches the requested sector number.

The formatting is a quite different story. Since this operation redefines the disk geometry, in this case the track is entirely overwritten without pauses between sectors. Unfortunately, the microcontroller has not enought RAM to store an entire track - a single sector takes 512 byte of the 2kB available.

To solve this issue, I have implemented a pipeline using the single buffer available. Briefly, once detected a formatting request, the buffer is filled decoding the write signal, like sector overwriting. At the end, there is a small grace time represented by the gap writing. The firmware takes advantage from it to setup DMA, starting the SD write back. While the buffer is being emptied, the firmware can start reading the next sector, filling it again.

Even with a single buffer, is rare that the reading of the new sector clashes with the write back operation, overwriting a part of data not yet stored in SD. There is a x8 data rate factor between outbound SPI and inbound Drive II write signal.

The main issue lies in SD card write finalization: these memories accept a whole data buffer though SPI, then they require further time to operate physical erase and overwrite. This time changes with the model and the age of the memory module. This further delay gave me some headache, but it can be mitigated forcing sectors from a same track to be physically sequential on the SD card - i.e. requesting a defragmentation in .NIC files.

Final judgement and conclusions

The main goal to replace the original drive with an emulator was successfully acheived. I have validated it on the field trying some software: games, demos, office applicazions… I had no reading issues. The same for the writing, trying both saving some BASIC lines from DOS, and by arbitrarily change sectors content through the CopyII+ software. The formatting operation needs further investigations, since I’ve seen sporadic failures with older SD cards.

Despite the simplifications, that limit the emulation accuracy, the project was an interesting challenge. This housemade project was useful to learn something new. I’ve tried to sum up the project highlights without going too deep in details, hoping to intrigue someone. Whoever wants to explore the code, both for study, to improve it or to realize an own version, it is released with free software licence and can be found at the following link. Further links are available with more technical articles.

• Project repository
• Retromagazine [IT], an online magazine where this article was published
• SDISK II, project’s home page
• Beneath Apple DOS
• The Amazing Disk II Controller Card, Disk II controller described by the “Floppy Emu” author
• Understanding the Apple II