I should have mentioned this before, but the Amiga Forever CDs and DVDs are really pretty awesome. The original commodora-era videos, pre-installed games/environment, everything is done very nicely.
I have the 2008 Premium edition and have meant to post about it for a long time. I’ve got to break it out and check it out again. I don’t think I ever watched all the videos.
This isn’t by any stretch finished, but it does do what I’ve expected it to do.
It responds both to differences in phase (counter is set to 0), and differences in frequency (period is adjusted in the direction of the frequency error)
I did this at 3am last night, so there could be a couple bugs in there.
With all this being said, I’m not entirely sure that PLLs are actually required for good reading of amiga floppy disks. My regular “static interval” solution works about 95%-98% of the time. I’m going to come up with a list of problem disks and see if this solution works better/worse/otherwise.
I’ve used a read data window of 1us, which starts being centered on 2us, and is automatically adjusted as data comes in. This produces windows around 2us, 4us, 6us, 8us. I output the overall error which is the deviation from the center of the window as each pulse is received. I’d like to graph this error, but it doesn’t look like Xilinx’s iSim will export a particular output as CSV or whatever.
So, as I mentioned in another post, I added Jim Thompson’s phase-jerked-loop to the project. Phil had good luck using it with his DiscFerret, but it really didn’t help me. My problem disks are still problem disks. And the data returned as a result still have the same problem.
This PJL really didn’t help me.
http://www.analog-innovations.com/SED/FloppyDataExtractor.pdf
I expected the numbers to look a little more stable —- as if we are tracking them better, but I really just didn’t see this.
I’m planning on reverting my changes soon unless some more playing around will help.
I collected roughly 1,000 index pulses from the Sony MPF920-E with one common floppy inserted.
The motor speed variation was very very small. I am impressed as to the accuracy.
Across 967 index pulses, all were within 56 microseconds of each other. The average was 200.487ms. Most of the group were within 20 microseconds of the average.
The standard deviation is 9.46 microseconds.
(this was posted to Classic Computer mailing list, please disregard if you’re on the list. I think this is an important topic)
The last two nights I’ve been busy archiving some of my Amiga floppy collection. Most disks were written over 20 years ago.
On a sample size of about 150 floppies, most of them were perfectly readable by my homegrown usb external amiga floppy drive controller.
I paid very close attention to the failures or ones where my controller struggled.
Without sounding too obvious here, the time between the pulses (which more or less define the data) were grossly out of spec. The DD pulses should nominally be 4us, 6us, and 8us apart before pre-write compensation. Most good disks are slightly faster, and normal times for these ranges are:
4us: 3.2-4.2us. Many around 3.75us
6us: 5.5-6.2us.
8us: 7.5-8.2us
(notice margins around 1-1.3us)
My original microcontroller implementation was 3.2-4.2, 5.2-6.2, and 7.2-8.2.
When my current FPGA controller would have a problem, I’d notice that there were problems right on a boundary. So maybe pulses were coming in at 3.1us apart instead of 3.2. Or maybe 4.3 instead of 4.2. So I kept bumping the intervals apart, making a larger range of pulse times acceptable — the XOR sector checksums were passing, so I was likely making the right choices. The bits were ending up in the right buckets.
But as I went through some of these disks, I ended up with the difference between ranges(and basically my noise margin) being reduced smaller and smaller. Some to the point where an incoming pulse time might fall darn smack in the middle of the noise margin. Which bucket does THAT one go into?
My approach has been very successful(easily 95%+), but it makes me wonder about Phil’s DiscFerret dynamic adaptive approach where a sample of the incoming data defines the ranges.
Some disk drives and controllers might be faster or slower than others, and if you create custom ranges for each disk (each track?), perhaps you’ll have better luck.
New demo video for the FPGA version of the AFP!
Enjoy!
(45mb download warning)
Here are some new photos for the FPGA version of the AFP.
Click for full size.
Enjoy!
So tonight, I put my new FPGA implementation of the amiga floppy project to good use. I read some more of my collection of amiga floppies.
GREAT NEWS.
It’s working like a champ. As a matter of fact, I selected disks which I could not previously read with my microcontroller solution. I could read 90% of the previously unreadable ones. Most of the unsolvable problems were related with HD disks (which, based on my earlier posts, some drives handle better than others) Note this is just temporary until I try other drivers to read the disks — and try covering the HD hole.
I have better visibility on the PC as to the “problem” delta T’s. So pulses that are slightly too far apart, just on the boundary of what I consider to be valid, if I adjust my boundary accordingly, and now consider them valid, everything is peachy-keen. I want to add a real-time option in my java client to allow this to be adjusted on the fly. See problems? Adjust the slider, and problems go away. Pretty neat.
I didn’t have this visibility when the microcontroller was interpreting the delta T’s. The microcontroller had no easy feedback method to tell me what was happening. Having high-level debugging of the information on the PC makes this all possible.
Nice to see the software purring. There is still plenty of improvements to make, usability to be improved, etc. But its working very nicely.
With this new FPGA solution, certain tracks would result in what I call a “short read.” A short read is any received track that contains less than 28,125 delta T’s, aka pulse times. Given a certain capture time, there are minimum’s and maximum’s of the number of pulses on an amiga track.
If we have a 300 rpm drive, then its 60s/300 = 200ms per revolution. If the bitcells are 2us wide, then you have at most 200ms/2us = 100,000 bit cells. The one’s density is at max 50%(raw MFM ’10′), so this means every other cell would contain a 1. So 50,000 pulses, so 50,000 delta T’s. The minimum one’s density is 25%(raw MFM ’1000′), so 25,000 pulses. Now we read more than just one revolution of data, because we will very likely start reading in the middle of a sector. So instead of 11 sectors worth of read time, we actually need to read 12 sectors worth, to ensure we read the entire sector in which we started. This is 218.2ms of time minimum. We could potentially re-assemble data, using some type of circular buffer, but this is more trouble than it’s worth. I currently read 225ms of data.
225ms / 2us = 56,250 maximum, 28,125 minimum.
I had my FTDI chip, for the usb<->ttl converter, D2XX USB parameters setting the USB transfer size to 57000 bytes. This is definitely over and above what was needed. Or so I thought.
I bumped the transfer size from 57000 to 60032 (docs said specifically 64 byte multiples), and everything started working. I had already narrowed it down that the problem tracks were ones that had a high density, where there were lots and lots of pulses. So I knew the size of the track was related. I checked for FIFO overflow, and it wasn’t overflowing.
I’ve got to look when I have a free second, but I think my USB packet size is 4096 bytes. So 56250+4096 (some amount of padding?) = 60346. Uh-o, I better bump that to 60,352. I think the driver (or windows?) that maxes out at 64k transfer size, so I still have a little wiggle room.
Long and short is that it appears to be working much better. I was glad to find this bug with just a little brainstorming, and getting better visibility into my actual pulses count on the FPGA.
So I was wondering exactly how effective the XOR checksum that Commodore used for the amiga sectors was. If I read a sector, and perform the checksum, and the checksum matches the stored checksum, then the data is the same, right? Not always. I had expected it to be better, maybe not MD5 or SHA-1 level, but I expected it to be decent.
I had run into this paper a year ago when I was looking at my transfer checksums. But this certainly also applies to the amiga sectors, too.
Some good excerpts:
- the percentage of undetected two-bit
errors becomes approximately 1/k (k being checksum size), which is rather poor performance for
a checksum (i.e., 12.5% undetected errors for an 8-bit checksum, and 3.125% for a 32-bit checksum).
- The XOR checksum has the highest probability
of undetected errors for all checksum algorithms in this study…..
- There is generally no reason to continue the common practice of
using an XOR checksum in new designs, because it has the same software computational cost as an
addition-based checksum but is only about half as effective at detecting errors.
So I wanted to actually prove myself that XOR is that bad, looking at it from the amiga sector perspective, so I wrote a quick program to:
That paper lists the properties of an XOR checksum, and I wanted to compare results:
The two-bit errors is really the case that I worry about. If two bit errors occur in the same bit position, and the inverted oppositely(1—>0 and 0—>1), then it won’t be detected. So how often does this happen with random data? Paper’s author, Maxino, says 3.125% of the time. I can definitely confirm this. My testing shows 1.8 million hits over 56 million tries. There are some differences with Amiga data, but I think the results can be the same. I might load some example amiga sectors and try them.
Once the number of bit errors increase, the probability of them happening in the same bit position, in the same direction, and occurring evenly on each position goes down —- and therefore the overall chance of undetection decreases as well.
4 bit undetected errors happen 0.3% of the time.
6 bit undetected errors happen 0.04% of the time.
8 bit undetected errors happen 0.009% of the time.
I have other testing to do, including running burst error tests.
So for each sector, you really don’t want to see only two bit errors occur. A single bit error, sure. Multiple bit errors, ok, because we can detect them. You don’t want to think you have a good copy when it fact the checksum passing was a false positive.