FAT/FAT16 tables - finding the data

Having successfully found the root directory on our sd card, we now need to start actually reading the data back from it. This is the last little link in a chain of

• initialise the sd card
• find the lba (logical block address) by reading sector zero
• read the mbr (master boot record)
• calculate where the root directory starts
• find the file we're interested in
• read the file

The easiest way to find a file is to load a filename into an 11-byte buffer. The easiest format to use is the name, padded with spaces plus the extension, minus the full-stop; so wav001.raw becomes wav001[space][space]raw. Or, perhaps more accurately, WAV001[space][space]RAW (since FAT16 likes to store filenames in uppercase).

A root directory entry looks like this:

• The first eight bytes are the file name - FAT16/MSDOS only supported up to eight characters in a filename.
• The next three bytes are the file extension
• Then there's an attributes byte
• Followed by time stamps for the file creation/modified date.
• The next two bytes are the starting cluster for the data (note, cluster not sector)
• The last four bytes are the total file length (in bytes)
• Every root directory entry uses exactly 32 bytes

It's the first bunch of bytes we're interested in - the file name, starting cluster and total file length:

Now we just read the root directory, comparing the first 11 bytes of each 32-byte entry in the root directory to the bytes in our filename buffer and if they match, we've found the file we're interested in.

Once we've got our file, we need to work out the file-length (in bytes) and the starting sector for the first cluster of data.

Byte 26 of 32 is the least-significant byte of a two-byte word  (e.g. 0x02)

Byte 27 of 32 is the most-significant byte of a two-byte word (e.g. 0x00)

The starting cluster for the file in this example is therefore 0x0002

Bytes 28-31 represent the file length, starting with the least significant byte of a 4-byte value.

In this example:

Byte 28 = 0x43

Byte 29 = 0x5e

Byte 30 = 0x09

Byte 31 = 0x00

The total file length is 0x00095e43 which in decimal works out as 613,955

Looking at the file wav003.raw in Windows Explorer confirms that the file is, indeed, 613,955 bytes in size

Now previously, we worked out a whole heap of important things from our MBR (master boot record) including where the actual data started, and the number of sectors per cluster (in a FAT16 formatted disk, this is usually 32 sectors per cluster, making each cluster 16kb)

If we know which sector the actual data begins from, and which cluster to begin reading from, we can calculate at which sector the cluster begins.

unsigned long clusterToSector(unsigned long cluster){
     // a cluster is a complicated thing:
     // first there's an ofset to the master boot record
     // then some reserved sectors
     // then a fat table (maybe two)
     // then the root directory (fixed length)
     // THEN there's the first data block,
     // which is where we start reading our clusters from
     // BUT there's a catch - clusters start from zero not 2
     // so whichever cluster number we've been given, subtract 2
     
     unsigned il;
     il=cluster-2;
     il=il*sectorsPerCluster;
     il=il+dataStart;
     return(il);     
}

Great stuff!
Convert the cluster number (in this case, cluster 0x0002) into a sector (in this case, because clusters start from 2 - it's an anomaly of the FAT16 format - our first cluster is also the first sector where the data starts. We've already calculated this value)

"The actual data starts at sector 615 + 32 = 647"

If we jump to sector 647 and start reading back the data, we find that there is, indeed, data in there!
But with a file that's 613,955 bytes long, it's not all going to fit into a single cluster (one cluster is 32 sectors and each sector is 512 bytes, so that's only 32x512 = 16,384 bytes - 16kb)

So where's the rest of the data?
That's where the FAT table comes in!

Firstly, take the cluster number and double it. That's the same as bit-shifting the entire value one place to the left. In our example, our starting cluster was 0x02 so we double this and end up with 0x04
This tells us where in our FAT table to find the next (two-byte) cluster number.

Since the FAT tables themselves are written across a number of sectors, we need to convert this cluster_doubled value into a sector and byte offset, to read the actual "next cluster value" back.

Divide the cluster_doubled value by 512 to get the sector number
The remainder is the byte offset.
In our example, this gives us sectors zero, byte offset 4
So we want the first sector (sector zero) in our FAT table, fourth and fifth byte

Since the FAT table begins at sector 143, we add zero to this, open the sector (using our earlier functions) and read back all 512 bytes. When we get byte four, this makes up the least significant byte of a two-byte (16-bit) value. Byte five is the most significant byte.

In fact, when we open our sd card, read back bytes four and five from sector 143, we get
Byte 4 = 0x03
Byte 5 = 0x00

This tells us that the file continues at cluster 0x0003.
Using the same technique, we open and read the data from cluster 0x0003 (sector 647 + (3-2)*sectorsPerCluster = 647 + 32 = 679)

We continue reading data from the sector(s) and calculating the next cluster where the file continues until we've either read back the entire file (total bytes read > file size in bytes) or the FAT table returns 0xFF as the next cluster (this is a special response to say "no more clusters for this file").

This is summarised in the following function (remove comments around UART calls to see what the microcontroller is actually doing when calculating next FAT clusters).

unsigned char openNextBlock(){

     unsigned short iFATSector;

     unsigned short iFATBytes;

     unsigned short iy;

     unsigned short ix;

     

     //UARTPrintLn("opening next block of data");

     

     // the cluster_number_doubled is the location in the FAT table

     // where we can find the next block of data. If this entry is 0xFF

     // it means that there's no more blocks of data, otherwise the entry

     // is the next cluster where the file continues from

     

     iFATBytes=nextFatClusterDoubled & 511;

     iFATSector=nextFatClusterDoubled>>9;

     iFATSector=iFATSector+FATStart;

     

     //UARTPrintLn("look up next cluster in FAT ");

     //UARTPrint("sector ");

     //UARTPrintNumber(iFATSector);

     //UARTPrint(" byte offset ");

     //UARTPrintNumber(iFATBytes);

     //UARTPrintLn(" ");

     

     // check the FAT tables for the next cluster for the current file

     r=sdReadBlock(iFATSector);

     for(iy=0; iy < 512; iy++){

          r=readByte();

          if(iy==iFATBytes){ix=r; nextFatCluster=ix;}

          if(iy==(iFATBytes+1)){ix=r; ix=ix<<8; nextFatCluster=nextFatCluster+ix;}

     } 

          
     // close the currently open sector
     sdSecReadStop();

     nextFatClusterDoubled=nextFatCluster<<1; // this is the same as multiplying by two!


      
     //UARTPrint("next FAT cluster ");
     //UARTPrintNumber(nextFatCluster);
     //UARTPrintLn(" ");          
     
     if(nextFatCluster==0xFFFF){          
          // if we're at the end of the block, send the end of block marker
          //UARTPrintLn("no futher fat clusters to follow");
          return(0xFF);
     }else{
          
          // open the next sector
          iSector=clusterToSector(nextFatCluster);
          sectorCount=0;
          //UARTPrint("file continues at sector ");
          //UARTPrintNumber(iSector);
          //UARTPrintLn(" ");
          
          return(0);
     }
}

Reading and writing to a single sector on an SD card with a PIC microcontroller

Having got our sd card to respond to the SPI initialisation routines and "boot up" correctly, it's time to actually start reading and writing data to the card.
The first thing to understand is that an SD card is made up from a series of sectors. Each sector is 512 bytes in size (almost always for SD cards) and the SD card likes to read and write entire sectors at a time.

not to scale ;-)

Reading data is a case of sending the appropriate "command packet" - 6 bytes which are:

• the command byte (CMD17)
• 4 parameter bytes (the address of the sector to read)
• two CRC bytes (an accurate crc value is not actually necessary in SPI mode, but two bytes still need to be sent)

It is possible to read multiple sectors of data, one after the other, and use the sd card like one massive serial eeprom, but for now, we're going to just work one sector at a time (this approach is much better suited to working with FAT16-based files, which we'll discuss later).

The command value to read a single sector of data is CMD17. After sending the command, we have to check for an "ok" response from the sd card then wait for the "not-busy" response (any non-0xFF value).

Boolean sdReadBlock(UInt32 sec){
     UInt8 v;
     UARTPrint("Starting to read block ");
     UARTPrintNumber(sec);
     UARTPrintLn(" ");
     
     v =  sdCommandAndResponse (17, ((UInt32)sec) << 9);
     if(v & FLAG_TIMEOUT) {
          UARTPrintLn("start read block command 17 timeout");
          fatal(1);
          return false;
     }else{          
          do{
               v = sdSpiByte(0xFF);
          }while(v == 0xFF);
          if(v != 0xFE) {
               UARTLog("Read block response",v);
               fatal(2);
               return false;
          }     
          
          curPos=0;
          return true;
     }
}

This function accepts a single 4-byte parameter, the sector address to open, and returns true or false to indicate whether the card responded correctly.
It also resets a counter curPos, which will keep track of the number of bytes we've read from the current sector on the disk. When we've had exactly 512 bytes back from the disk, we'll have to "close" the current sector by reading back the two bytes that make up the crc for the previous 512-byte data stream.

void sdSecReadStop(){     
     
     if(curPos<512){
          UARTPrint("add stuff bytes to close sector - ");
          UARTPrintNumber(curPos);
          UARTPrintLn(" ");
     }
     
     while(curPos < 512){
          sdSpiByte(0xFF);
          curPos++;
     }
     
     // read back the two CRC bytes
     UARTPrint("CRC ");
     r=readByte();
     UARTByte(r);
     UARTPrint(" ");
     r=readByte();
     UARTByte(r);
     UARTPrintLn(" ");
     UARTPrintLn("End read sector");
     UARTPrintLn(" ");
     
}

Now we can open a sector and close it again, the last piece of the puzzle is to stream the data back from the sd card and do something with it! When we come to make our audio player, we'll use the data immediately - sending it to the speaker - but for testing, or for more general purpose use, we'll put the data into a series of buffers and use it later.

Now - here's a thing.
So far, everything discussed can be ported to another platform (AVR/Ardunio for example) but when it comes to storing the data in our internal buffers, there's a peculiarity with the PIC microcontroller - or at least, with most compilers that support using arrays. It may be the same with Arduino and/or other compilers, but we've always found that large arrays are often difficult to use.
This is because of the way PICs store data in RAM - it uses "banks" to keep the data in memory, and one array can't usually span more than one bank of data. In practice, this means a limit of about 90 elements in an array.

To keep things simple, we're going to store an entire sector's worth of data (512 bytes) in 8 eight arrays of 64 elements.

unsigned char wavData1[64];
unsigned char wavData2[64];
unsigned char wavData3[64];
unsigned char wavData4[64];
unsigned char wavData5[64];
unsigned char wavData6[64];
unsigned char wavData7[64];
unsigned char wavData8[64];


unsigned char readSector(){
     unsigned short cy;
     unsigned short cv;
        
     r=sdReadBlock(iSector);               
     for(cy=0; cy < 512; cy++){
         r=readByte();               
         UARTByte(r);    //write to serial for debugging
                  
         if(cy< 64){
            cv=cy;
            wavData1[cv]=r;
         }else if(cy<128){
            cv=cy- 64;
            wavData2[cv]=r;
         }else if(cy<192){
            cv=cy-128;   
            wavData3[cv]=r;
         }else if(cy<256){
            cv=cy-192;   
            wavData4[cv]=r;
         }else if(cy<320){
            cv=cy-256; 
            wavData5[cv]=r;
         }else if(cy<384){
            cv=cy-320;   
            wavData6[cv]=r;
         }else if(cy<448){
            cv=cy-384;   
            wavData7[cv]=r;
         }else if(cy<512){
            cv=cy-448;   
            wavData8[cv]=r;
         }
     }           
                                        
     sdSecReadStop();     
     UARTPrintLn(" ");
     return(1);
}

Now we can read a sector-ful of data, it's time to write some data back.
There are any number of ways you can get data into the PIC/microcontroller in order to write them to the sd card. We'll leave that side of things for another post - for now, here are some functions to write data to a specific sector on the card.

The command to write a single sector-ful of data to the card is CMD24
(you can write an entire stream across multiple sectors if you're using the card as a large serial eeprom chip, but this makes things difficult if we're going to work with FAT16 formatted cards in future).

When writing data, the 6-byte data packet consists of:

• single command byte
• four parameter bytes (sector to write to)
• single crc byte
• a single byte representing a token confirming the command request

Boolean sdWriteBlockStart(UInt32 sec){
     UInt8 v;
     v =  sdCommandAndResponse(24, ((UInt32)sec) << 9);
     if(v) {
          UARTLog("write block start response",v);
          fatal(1);
          return false;
     }else{          
          UARTPrintLn("write block started");     
          // keep track of how many bytes we've written in this sector
          // (when we hit 512 we should expect some extra bytes in the packet data)     
          bytesWritten=0;     
          
          // send the correct token for CMD17/18/24 (0xFE)
          // REMEMBER the token for CMD25 is 0xFC               
          r=sdSpiByte(0xFE);
          return true;
     }
}

Once the card has responded with an "ok" after starting to write a sector, the next bytes streamed to the card are written to the selected sector. We need to keep track of the number of bytes written - when we hit 512, it's time to close the current sector, read back the crc bytes, then open another - you don't have to write to the sequentially next sector every time, you can write to any old location if you like!

Boolean sdWriteByteToSector(UInt8 b){     
     UInt8 ix;
     r=sdSpiByte(b);          
     bytesWritten++;     
}


Boolean sdWriteToSectorClose(){          
     // finish closing the sector
     UARTPrintLn("writing stuff bytes to close sector");
     while(bytesWritten<512){
          sdSpiByte(0xFF);
          bytesWritten++;               
     }
          
     // send two CRC bytes
     sdSpiByte(0x00);
     sdSpiByte(0x00);
          
     // response should immediately follow
     // (for writing, we're looking for 0bXXX00101 data accepted)
     r=sdSpiByte(0x00);     
     UARTLog("write finish response",r);
     
     // now wait for the sd card to come out of busy
     while(r!=0x00){
          r=sdSpiByte(0x00);
     }
          
     UARTPrintLn("write finished");          
}



// write some data to sector numbered five:
void writeSomeData(){
    UARTPrintLn("Writing one block");
    ret=sdWriteBlockStart(0x05);          
    if(!ret){
         fatal(3);
    }else{

         for(iy=0; iy<=255; iy++){
              sdWriteByteToSector((255-iy));
         }                    
     
         sdWriteByteToSector(0x10);
         sdWriteByteToSector(0x11);
         sdWriteByteToSector(0x12);
         sdWriteByteToSector(0x13);
         sdWriteByteToSector(0x14);                    
                    
         sdWriteToSectorClose();                         
    }
}

That's pretty much it.
We can now read and write to an individual sector on the sd card.
There are commands for reading and writing to/from multiple blocks at a time, but we'll leave those for again; for FAT formatted cards, we need to be able to read data out-of-sequence and possibly even write files into different (non-consecutive) sectors.

How SPI works with an SD card

SD cards have two main operating modes.
Their default mode is high-speed through 4-bit wide port but we're going to be working with the "legacy" SPI (two-wire) mode.

In SPI mode, the master device (our microcontroller) talks to the slave device (the sd card) using a data and a clock line. Every time the clock line goes from low-to-high (or, if you prefer, from high-to-low - you can change this to suit the application needs) the receiving device looks at the data line. If it's high, it receives the single-bit value 1, if it's low, zero.

in this example, when the clock line goes from low-to-high (sometimes called a rising edge trigger) as denoted by the red vertical lines in the clock timing diagram, the state of the data line is converted into a value

The great thing about SPI is that it's not time dependent. Because the master device sends the clock line along with the data, it can be speeded up and slowed down (this is not possible using methods such as UART/serial, which has a fixed data rate; ie. the data has to be moved within a specific time period).

Using this clock-and-data method, we can send commands to the sd card, to tell it to do specific things. So we can send a specific value (the sd format sets out specific values to send for specific commands) to get it to reset, for example.

When an SD card has received and understood a command, it can remain in a busy state for quite some time. It is important to wait until the card has finished doing whatever you asked it to do, before blasting more data or commands at it. To do this, we poll the card (continuously ask it for data) until it gives us a "ready" token.

SPI is actually a data exchange mechanism. There's no difference between reading and writing bytes between the devices. As one device sends a byte of data, so the other transmits one. After sending a single byte of data from an SPI buffer another (possibly different) byte may appear in it's place - this is the byte that has been received.

So to read a byte from the sd card, we have to send it a byte too. It's normal to send either all zeros (0x00) or all ones (0xFF in hex is 255 in decimal, which is 11111111 in binary) for "don't care" bytes - so if you're just reading data from the other device, and it's not important what data you send, it's common to either send 0x00 or 0xFF.

Whenever a command is sent to an SD card, it follows a specific format:
There is a single command byte
There are four "parameter" bytes - data which tells the recipient how to perform the command requested
There is a single CRC (checksum) byte to prove that the previous bytes have been transmitted correctly.

To send data to our SD card, we need a couple of functions:

UInt8 sdSpiByte(UInt8 data){
     ssp1buf = data;
     while(!(ssp1stat & (1<<BF)));
     return ssp1buf;
} 


static inline void  sdSendCommand(UInt8 cmd, UInt32 param){     
     UInt8 send[6];
     
     send[0] = cmd | 0x40;
     send[1] = param >> 24;
     send[2] = param >> 16;
     send[3] = param >> 8;
     send[4] = param;
     send[5] = (sdCrc7(send, 5, 0) << 1) | 1;
     
     for(cmd = 0; cmd < sizeof(send); cmd++){
          sdSpiByte(send[cmd]);
     }
}

This provides us with a simple method of sending commands (and their parameters) to the SD card.

The first function simply puts a value into the hardware SPI buffer then waits for the SPI busy register value to go "not-busy" before returning the value it finds in the buffer (which is now the value received from the other device - data exchange remember!)

The second function actually sends commands to the sd card. Every command byte sent to an SD must have bit 6 set (so the device can recognise it as a command and not some data).
Bit 6 in binary is 01000000 which is 64 in decimal or 0x40 in hex.
So to make sure that every command byte has bit 6 set, we always OR the command byte with 0x40 (so when we send command zero, for example, it's actually transmitted as 0x40, command one is sent as 0x41 and so on).

But every time we send a command (and it's parameters), we have to wait for a not-busy response from the SD card. While the SD card is busy, it holds its "output pin" high - the data clocked out of it is always 11111111 (or 0xFF) So we build these little functions:

static inline UInt8  sdReadResp(void){
     UInt8 v, i = 0;     
     do{                
          v = sdSpiByte(0xFF);
     }while(i++ < 128 && (v == 0xFF));     
     return v;
}

static UInt8 sdCommandAndResponse(UInt8 cmd, UInt32 param){   
     UInt8 ret;          
     sdSpiByte(0xFF);
     sdSendCommand(cmd, param);
     ret = sdReadResp();     
     return ret;
}

This sdCommandAndResponse function sends a "dummy byte" to the sd card.
It then sends the command byte, followed by the 4-byte parameter value(s).
Next it calls the read-response function, which continuously sends the dummy byte 0xFF to the sd card, until the response back is not busy. When the response goes not busy, the response value is returned to the sdCommandAndResponse function. The response could either be "all ok" or it may be some kind of error code to explain why the command given could not be completed.

One last function to mention is the CRC generating function.
Strictly speaking, once we've told our card to work in SPI legacy mode, we don't actually need to generate the CRC values, but it's included here for completeness.

Note: We didn't actually create this function, we ported it from another sd card library for another platform:

static UInt8 sdCrc7(UInt8* chr,UInt8 cnt,UInt8 crc){
     UInt8 i, a;
     UInt8 Data;

     for(a = 0; a < cnt; a++){          
          Data = chr[a];          
          for(i = 0; i < 8; i++){               
               crc <<= 1;
               if( (Data & 0x80) ^ (crc & 0x80) ) {crc ^= 0x09;}               
               Data <<= 1;
          }
     }     
     return crc & 0x7F;
}

Before we can actually start sending data over SPI, we need to set up the PIC to use the hardware SPI peripheral. This means writing some values to particular registers in the chip. The names of these registers should be similar across different PIC models, but may not be exactly the same if you're using a different chip:

static void sdSpiInit(void){
     ssp1add          = 21;              //slow clock down to < 400khz
     ssp1con1     = 0b00101010;          //spi master, clk=timer2
     ssp1stat     = 0b11000000;          //CPHA = 0
}

The important registers here are the SSP1ADD (multiplier value) ad SSP1CON1 register.
When the last four bits of SSP1CON1 are 1010 this slows down the SPI clock speed - by how much depends on the value in the SSP1ADD register (and the actual speed, as in time taken to send each clock pulse, is dependent on the overall processor clock speed so will change with each chip model). When the last four bits are set to 0000, the SPI clock changes on every instruction cycle. When it's set to 1010, it changes on every x clock cycles, where x is the value held in the SSP1ADD register.

The SD card initialisation routine after powering up is:

• Set the clock speed to less than 400khz
• Hold the chip select line on the card low and send in about 80 clock pulses
• Pull the chip select line high to tell the card we're talking to it
• Send in the "soft reset" command (CMD0)
• Wait for the sd card to respond "ok" with the value 0x01
• Send in the "initialise card" command (CMD1)
• Repeat sending CMD1 until the card responds with an ok value of 0x00
• Set the sector size using CMD16 with a parameter 512 (each sector is 512 bytes)
• Turn of the CRC requirement by sending CMD59
• (all future transmissions do not require a valid crc value)
• The next time the card responds with an "ok" value, it has been initialised and we can ramp the clock up to full speed.

All of the initialisation routines have to be carried out at the relatively slow clock speed of not more than 400khz. So we need a couple of extra functions to enable us to set the clock speed and enable the chip select line:

static void sdClockSpeed(Boolean fast){
     if(fast){
          ssp1con1 = 0b11110000
     }else{
          ssp1con1 = 0b11111010
     }
     
}

void sdChipSelect(bool active){
     portc.3 = !active;
}

For debugging, we've written a simple "fatal error" function to let us know which part of the initialisation failed (if there are any problems). When a fatal error is hit, this function reports it, then puts the microcontroller into "sleep mode" so that the program flow is permanently interrupted.

void fatal(UInt8 val){          //fatal error: flash LED then go to sleep
     UInt8 i, j, k;

     for(j = 0; j < 5; j++){     
          for(k = 0; k < val; k++)
          {
               delay_ms(100);
               P_LED = 1;
               delay_ms(100);
               P_LED = 0;
          }
          
          UARTLog("Error",val);
          
          delay_ms(250);
          delay_ms(250);
     }
     
     while(1){
          asm sleep
     }
}

By using all these functions together, we can now write our sdInit routine:

Boolean sdInit(){     
     UInt8 v, tries = 0;
     Boolean SD;
     
     SD = false;
     
     sdSpiInit(); // initialise SPI
     sdClockSpeed(false); // slow clock     
     sdChipSelect(false); // CS inactive          
     
     for(v = 0; v < 20; v++) {
          sdSpiByte(0xFF);     //lots of clocks to give card time to init
     }

     sdChipSelect(true); // CS active
     
     v = sdCommandAndResponse(0, 0, true);
     if(v != 1){fatal(2);}

     v = sdCommandAndResponse(1, 0, true);     
     for(int i=0;v != 0 && i<50;++i){
          delay_ms(50);
          v = sdCommandAndResponse(1, 0, true);
     }
     if(v){fatal(3);}
     
     v = sdCommandAndResponse(16, 512, true);          //sec sector size
     if(v){fatal(5);}
     v = sdCommandAndResponse(59, 0, true);          //crc off
     if(v){fatal(6);}
     
     // now set the sd card up for full speed
     sdClockSpeed(true);                    
     return true;
}

If this function successfully returns true, the SD card has been successfully initialised and the SPI lines set to run at maximum speed for the fastest (spi-based) data transfer.

Talking to an SD card from a PIC 16F1825 microcontroller

Following on from our earlier post, we're continuing with our create-an-audio-module project which reads wav file data from an SD card and plays it through a speaker.

So far, we've written the basic framework, using Sourceboost C so now it's time to start fleshing it out.
Before we actually start making any sound, we're going to make extensive use of the UART/serial comms to report back at every stage of the data transfer, so we can pinpoint any problem areas. We're aiming to read an entire file off the sd card and stream it's contents back to the PC over serial.
Once we've done this, we'll modify the code to send the data to a PWM module to actually play the sound.

First up, with any Sourceboost project, we need our main() function.

void main(){
     unsigned char response;
     Boolean ret;     

     init();
     delay_ms(50);     
     ret = sdInit();
     if(!ret){
          UARTPrintLn("SD init failed");
          fatal(1);
     }else{
     
          setupFAT();
          response=openFile(1);
          if(response){
                         
               readCluster();
                              
               while(response!=0xFF){
                    response=openNextBlock();
                    if(response!=0xFF){
                         readCluster();
                    }else{
                         UARTPrintLn("end of file clusters");
                    }
               }
          }
          
          //just to prove we got here!
          UARTPrintLn("Done");
               
     }

     while(1){}
}

The main function pretty much speaks for itself, since it's calling a whole heap of other (hopefully appropriately named) functions:

Firstly, we initialise the microcontroller.
This involves setting the registers to define how all our different multi-function pins are set up. One of the great things about the PIC range of microcontrollers is their versatility - each pin can have multiple functions and be used for a variety of different things. Unfortunately, this can also mean hours and hours of headaches, as it can make debugging quite difficult.
So in the main, we should always call an initialise routine after powering up, to put the microcontroller into a "known state". In Oshonsoft, this includes commands such as

• CONFIG PORTA=OUTPUT 
• ALLDIGITAL 

etc.
These BASIC commands are simply macros to put specific values into specific registers on the chip. In this program, we're going to to that manually, following the datasheet.

void init(){
     osccon                    = 0b11110000;     //32 MHz clock     
     intcon                    = 0b10000000;     //ints enabled, all sources masked
     apfcon0                    = 0b01000000;    //MOSI on RA4
     trisa                    = 0b11001100;      //out on 0, 1, 4, 5, in on
     trisc                    = 0b00000010;      //portc.1 is our spi input
     ansela                    = 0;              //no analog inputs
     anselc                    = 0;              //no analog inputs
     option_reg               = 0b10000000;      //no weak pull-ups, timer0=osc/8
     wdtcon &= ~(1<          
     UARTInit();
     curPos=0;     
}

The first register value OSCON sets the PIC to internal oscillator, 32Mhz.
The INTCON register is where we set up interrupts (bit 7 is the global interrupts bit)
APFCON0 is the alternative pin configuration register - we've put the SPI data out (MOSI = master out, slave in) onto pin RA.4 - on this particular chip the SPI out pin can go on different pins if required.
TRISA is the register where we set which pins are inputs and which are outputs. A 1 makes the corresponding bit an input, zero makes it an output (so bit 7 of the TRISA register maps to pin 7 on PORTA).
ANSELA and ANSELC are the analogue registers for ports A and C - setting these both to zero is the same as the Oshonsoft command ALLDIGITAL.

Once we've got the chip up and running, and in a known state, we allow a few milliseconds just for all the internal voltages to settle and allow any multi-function pins to take the state requested.

Next, we call the initialise SD card routine.
This function will return a value to say whether or not the SD card started up correctly. If it didn't we report an error back and stop the code executing there (there's no point trying to read data back if the SD card isn't responding as we expect it to!)

If the SD card does respond to say it's set up correctly and ready to receive commands, we then call a function to read the FAT16 tables on the card. This involves finding the boot sector on the disk, finding the location of the actual FAT tables themselves, finding the location of the root directory on the disk and calculating where the actual data starts.
This is a long and complicated process and will be explained fully over a number of blog posts! It's hard going, but worth sticking with - being able to read and write data in a PC readable format is very very useful.

The setupFAT function returns a value to say whether or not the PIC was able to make sense of the contents of the card, and if it did, we open a file and start reading clusters of data from the card.
As long the file consists of more data, the function will keep calling and reading data back.

But already things are getting a bit confusing - clusters, sectors, boot sectors..... it's time to look into how FAT works

A short interruption for a demo

Here's a quick movie showing our final audio player actually being used in a final product.
The PIC has been programmed to read a value from eeprom, increase it by one, rollover back to zero once a certain value is reached, and use this value to decide which sound to play.
In doing this, we can get our alarm clock to play a different sound each morning!

The original alarm clock mechanism has been kept the same - when the alarm time is reached, the clock mechanism pulls a line low. We use this as the ground for our board, so that it is switched on an off by the alarm clock mechanism (the mechanism keeps the line low for about 15 minutes after passing the alarm time). By doing this, we're conserving the batteries, since the entire circuit is only every powered for about 30 minutes each day (once for 15 mins in the morning, once in the evening, unless the alarm on/off switch has been moved to disconnect the ground rail from the circuit).

SD card clock edge problems?

There's something still not right with our SD card initialisation routines.
We've suspected for a while that there's something going on with the clock edge for reading/writing responses to/from the SD card.

Our logic probe appears to be displaying what we expect to see, but in code, we're never trapping the correct responses back from the card. For example, we're looking for 0x01 from the first init routine but now we're getting none 0xFF responses, but they're not 0x01 from the card.

The probe says the card is returning 0x01 but the PIC is getting some other value, so we hacked in a bit of debugging code that we can watch on the "output" window:

// send the cs line low to send commands
spi_assert();

// send the initialise command
r=sendCommand(CMD_GO_IDLE_STATE, 0);
if (r != IN_IDLE_STATE ) {
      state=r;
      sendClocks(2, 0xFE);
      sendClocks(2, state);
      spi_deassert();
      return SD_RETCODE_NOT_IDLE;
}else{

      // do rest of init code here

}

The resulting output looked like this....

The logic probe says we're being sent 0x01 from the SD card but when we echo the incoming value back onto the SPI line, we get 0x7F.

So we're expecting (and the probe says we're getting) 0b00000001
And the PIC says it's getting 0b01111111

However, comparing these to the timing graph, the 0x01 actually comes in during the transfer of the byte 0xFE (our error code to say we've had a non-0xFF response from the card). So the PIC has already decided that it's time to raise an error, although the probe says we've had an 0xFF response.

It looks like the response from the card 0x00 followed by 0xFF is being merged by the PIC - it's taking the last bit of 0x00 and appending the following 7 bits of 0xFF - giving us 0b01111111 (or 0x7F in hex).
So somewhere in this little lot, the responses from the SD card look like they're being read on the "wrong" edge of the clock pin.......

SD card send/receive still not quite there

After breathing a massive sign of relief once the SD card starting responding to SPI commands, we thought it'd be relatively straight forward to get data into and out of the card. We've already worked with NAND flash memory (in our case AT45DB041D eeprom chips), and stream-reading/writing data, so it shouldn't be too difficult....

We're still having problems getting a reasonable response from the SD card.
Everything points to the clocking edge being either incorrect or incorrectly configured, but it's just not making sense! We got an ok response from the SD card after initialising it. So now we send command 18 and wait for an ok before streaming the data back from the card:

void testReadData(){
      // send command 18, param x10 (start multi-block read from address 0x10)
      // and wait for an ok response
      sendCommand(CMD_READ_MULTIPLE_BLOCKS, 0x10);
     
      // what follows should be data?
      r=SPISendAndReceive(0x00);
      r=SPISendAndReceive(0x00);
      r=SPISendAndReceive(0x00);
      r=SPISendAndReceive(0x00);
      r=SPISendAndReceive(0x00);
     
}

But here's the weird thing - after we've successfully initialised the card, we get some sort of noisy signal back from the card after sending the second byte from the (redundant) CRC check.

So that we can see where our init routine ends, we send a single byte 0xC0 to the SD card once we think the init routine has successfully completed. This can be seen clearly in the screenshot below:

What is unusual is the noisy signal coming back from the card. The logic probe decodes it as 0xFF but there are definitely peaks and troughs occuring in the signal in between the clock cycles.

Here, we're sending a command to the SD card, to initiate a multi-block read so we send a command byte 18, followed by the sector to begin reading from (we chose 0x10) and a CRC value of 0x95 (the CRC is not actually used when in SPI mode, but must be included as part of the data "packet"). In the screen grab above, we seem to be getting a noisy signal back from the SD card while sending the sector address to read from.

The send command function includes a check for an ok response before returning:

unsigned char sendCommand(unsigned char command, unsigned long param) {
      r=SPISendAndReceive(0xFF); // dummy byte
      r=SPISendAndReceive(command | 0x40);
      r=SPISendAndReceive(param>>24);
      r=SPISendAndReceive(param>>16);
      r=SPISendAndReceive(param>>8);
      r=SPISendAndReceive(param);
      r=SPISendAndReceive(0x95); // correct CRC for first command in SPI
      // after that CRC is ignored, so no problem with
      // always sending 0x95
      r=SPISendAndReceive(0xFF); // ignore return byte
      return waitForNot(0xFF);
}

But we never seem to get a non 0xFF response back from the card.....

Initialising an SD card with a 16F1825 PIC microcontroller

After nearly two days of headache and heartache - and thanks in no small part to the brilliant Arduino/AVR code from Chris McClelland- we finally managed to successfully initialise an SD card with our PIC microcontroller.

Before we stuff it up and it stops working, here's some demo code:

#include <system.h>
// ##################################
// using a 32 Mhz internal oscillator
#pragma DATA _CONFIG1, 0x0804
#pragma DATA _CONFIG2, 0x1DFF
#pragma CLOCK_FREQ 32000000

unsigned char arrayMarker;
unsigned char arrayPointer;
unsigned long currSector;
unsigned char state;
unsigned char r;
unsigned char sample;
unsigned char playSample;

// ##################################

#define TOKEN_SUCCESS 0x00
#define TOKEN_READ_CAPACITY 0xFE
#define TOKEN_READ_SINGLE 0xFE
#define TOKEN_READ_MULTIPLE 0xFE
#define TOKEN_WRITE_SINGLE 0xFE
#define TOKEN_WRITE_MULTIPLE 0xFC
#define TOKEN_WRITE_FINISH 0xFD
#define IN_IDLE_STATE 0x01
#define ERASE_RESET (1<<1)
#define ILLEGAL_COMMAND (1<<2)
#define COMMAND_CRC_ERROR (1<<3)
#define ERASE_SEQUENCE_ERROR (1<<4)
#define ADDRESS_ERROR (1<<5)
#define PARAMETER_ERROR (1<<6)

#define CMD_GO_IDLE_STATE 0
#define CMD_SEND_OP_COND 1
#define CMD_SEND_CSD 9
#define CMD_STOP_TRANSMISSION 12
#define CMD_READ_SINGLE_BLOCK 17
#define CMD_READ_MULTIPLE_BLOCKS 18
#define CMD_WRITE_SINGLE_BLOCK 24
#define CMD_WRITE_MULTIPLE_BLOCKS 25
#define CMD_APP_SEND_OP_COND 41
#define CMD_APP_CMD 55

#define SD_RETCODE_SUCCESS 0x00
#define SD_RETCODE_NOT_IDLE 1
#define SD_RETCODE_ERROR_INIT 2
#define SD_RETCODE_ERROR_READ 3
#define SD_RETCODE_ERROR_NO_DATA 4
#define SD_RETCODE_ERROR_READ_CAPACITY 5
#define SD_RETCODE_ERROR_WRITE 6
#define SD_RETCODE_ERROR_NOT_READY 7
#define SD_RETCODE_TIMEOUT_ERROR      8


unsigned char SPISendAndReceive(unsigned char b){
sspbuf=b; //send data
while(!(ssp1stat & (1<<BF)));
r=sspbuf;            
      return(r);
}

void sdClockSpeed(bool fast){     
      ssp1con1 = (ssp1con1 & 0b11110000) | (fast ? 0b0000 : 0b1010);
}

void spi_deassert(){
      // pull the /cs pin high on the slave
      porta.1=1;
}

void spi_assert(){
      // drop the /cs pin low on the slave
      porta.1=0;
}

unsigned char waitForNot(unsigned char response){
      unsigned short i;
      unsigned char byte;
      i=0xFFFF;
      byte=response;
      while(byte==response && --i){
            byte=SPISendAndReceive(0xFF);
      }     
      return byte;
}

unsigned char waitFor(unsigned char response){
      unsigned short i;
      unsigned char byte;
      i=0xFFFF;
      byte=response+1;
      while(byte!=response && --i){
            byte=SPISendAndReceive(0xFF);
      }     
      return byte;
}

void sendClocks(unsigned char numClocks, unsigned char byte) {
      unsigned char i;
      for ( i = 0; i < numClocks; i++ ) {
            r=SPISendAndReceive(byte);
      }
}

unsigned char sendCommand(unsigned char command, unsigned long param) {
r=SPISendAndReceive(0xFF); // dummy byte
r=SPISendAndReceive(command | 0x40);
      r=SPISendAndReceive(param>>24);
      r=SPISendAndReceive(param>>16);
      r=SPISendAndReceive(param>>8);
      r=SPISendAndReceive(param);
r=SPISendAndReceive(0x95); // correct CRC for first command in SPI
// after that CRC is ignored, so no problem with
// always sending 0x95
r=SPISendAndReceive(0xFF); // ignore return byte
      return waitForNot(0xFF);
}

unsigned char initialiseSD(){
      unsigned char i;
     
      // initialise the SPI module on the PIC
      ssp1add            = 0x19;                        //slow clock will be 32,000,000/100 = 320khz
     
      //ssp1con1.ckp=1;                              //spi idle clock high (bitx)
      ssp1con1      = 0b00111010;                  //spi master, clk=timer2, spi_clck idle high
     
      //ssp1stat.cke=1;                              // cke=bit6
      //ssp1stat.smp=0;                              // smp=bit7
      ssp1stat      = 0b00000000;                  //spi mode 3?
     
      sdClockSpeed(false);      //      load the SD card in low speed <400khz mode
     
      // hold the cs line high when powering up to enter spi mode
      spi_deassert();
     
      // power up the sd card and allow time to stabilise
      porta.0=1;
     
      // with CS high, send 256 clocks (32*8=256, 1.024ms @250kHz)
      // with DI low while card power stabilizes
      sendClocks(32, 0x00);
     
      // with CS high send 80 clocks (10*8=80, 0.32ms @250kHz)
      // with DI high to get the card ready to accept commands
      sendClocks(10, 0xFF);
     
      // send the cs line low to send commands
      spi_assert();
     
      // send the initialise command
      if (sendCommand(CMD_GO_IDLE_STATE, 0) != IN_IDLE_STATE ) {
            sendClocks(2, 0xFE);
            spi_deassert();
            return SD_RETCODE_NOT_IDLE;
      }else{
     
            // Tell the card to initialize itself with ACMD41.
            sendCommand(CMD_APP_CMD, 0);
            r = sendCommand(CMD_APP_SEND_OP_COND, 0);
     
            // Send CMD1 repeatedly until the initialization is complete.
            i = 0xFFFF;
            while ( r != TOKEN_SUCCESS && --i ) {
                  r = sendCommand(CMD_SEND_OP_COND, 0);
            }
     
            sendClocks(2, 0xFF);
            spi_deassert();
     
            if ( i == 0x0000 ) {
                  sendClocks(2, 0xFD);
                  spi_deassert();
                  return SD_RETCODE_TIMEOUT_ERROR;                 
            }
     
           
            // for debugging, turn off the LED/card
            porta.0=0;
           
            // we should now be in true SPI mode
            sdClockSpeed(true);
      }
     
}

void preloadTimer1(){
      //------------------------------------
      // pre-load timer1 with 65172 (0xFE94)
      //------------------------------------
      // 1/22050 = 0.00004535147
      // at 32mhz, fosc=8,000,000
      // and 8m/22050 = 362.812
      // so we want our timer1 to only count up to 363
      // timer1 is a 16-bit timer (0-65535) so we need to
      // preload it with 65535-363 = 65172

      tmr1h=0xfe;
      tmr1l=0x94;
}     

void startMultiBlockRead(){
      // send the spi commands to start a multi-block read from the sector value
      // held in the variable currSectorValue
     
}

void initialise(){

      osccon=0b11110000;            // oscillator=32mhz
      delay_ms(50);                   // wait for internal voltages to settle
      state=0;
      playSample=0;
     
      intcon=0b11000000;            // global interrupts enabled and peripheral interrupts
      apfcon0=0b01000000;
      trisa=0b11001100;            // out on 0,1,4,5 input on 2, DNK on 3
      trisc=0b00000010;            // all portC lines are output except pin1 (RC1=spi in)     
      ansela=0x00;                  // turn off analogue pins on portA
      anselc=0x00;                  // turn off analogue pins on portC
     
      option_reg=0b10000000;      // no weak pull-ups

           
      //-------------------------------------
      // initialise the SD card
      //--------------------------------------
      initialiseSD();
     
}

void nothing(){

      //----------------------------------------------------
      // start a multi-block read from the appropriate place
      //----------------------------------------------------
      currSector=1;
      startMultiBlockRead();
     
      //------------------------------------
      // start a pwm carrier signal @ 125khz
      //------------------------------------
      // Disable the CCPx pin output driver by setting the associated TRIS bit.
      // (for CCP1, the output pin is RC.5)
      trisc.5=1;
     
      // Load the PRx register with the PWM period value:     
      pr2=0x00;      // this will change every 1/22050th second, start at zero
     
      // Configure the CCP module for the PWM mode by loading the CCPxCON register with the appropriate values:
      // Load the CCPRxL register and the DCxBx bits of the CCPxCON register, with the PWM duty cycle value.
      pr2=0x3F;                        // this is the value to create a 125khz carrier (from datasheet)
      ccp2con = 0b00001100;       // puts the CCP module into PWM mode     
      ccptmrs = 0x00;             // all comparitors are using PWM based on timer2
     
      // Configure and start Timer2/4/6:
      t2con = 0b00000100; // start timer2, no scaling

      // Enable PWM output pin (CCP2 out = RC.5)
      trisc.5=0;
     
      //-------------------------------------------
      // interrupt on timer1 22050 times per second
      // ------------------------------------------
      // set up timer1 as a 16-bit timer, count on instruction cycle (fosc/4)
      t1con=0b00001000;      // set bit zero to 1 to actually start the timer
      // set up the actual timer1 interrupt:
      pie1=0x01;            // timer1 rollover interrupt is bit zero
      intcon=0x192;      // global interrupts (bit7) + peripheral interrupts (bit6)     
      preloadTimer1();
     
}

void playNextSample(){
      // dump the next sample from the buffer to the PWM module
      //pwm=sample;
      ccpr2l=(sample>>2);
      ccp2con.1=sample.1;
      ccp2con.0=sample.0;
     
      // read the next byte from the SD card
      sample=SPISendAndReceive(0x00);
     
}

void main(){
      initialise();
      while(1){
            if(playSample==1){
                  // drop the current sample to the PWM module
                  playSample=0;
                  playNextSample();                 
            }
      }
}

void interrupt(void){
      // bit zero of pir1 is tmr1 overflow interrupt
      if(pir1.0==1){
            // clear the timer1 interrupt flag
            pir1.0=0;
            // preload timer1 to get the next interrupt to occur in a timely manner
            preloadTimer1();
            // set the flag to play next sample (background task)
            playSample=1;
      }
}

With a microSD card connected in an SD card holder, using RA0 to power the card through a transistor and using RA1 as the chip select line, with the SPI lines are per the hardware peripheral (SPI_Clock = RC0, SPI_MISO/data in =RC1, SPI_MOSI/data out=RC2/RA3) here's the output from the logic probe from boot-up:

There's also the start of our PWM/timer1 interrupt code in there, just so that we have a global interrupt handler. The PWM control is already coded up, but was mostly stripped out to allow us to focus on getting the SD card working.

And thanks to Chris McClelland, it looks like it is!
Thanks again Chris.
You're ace!

SD Card SPI timing graphs

This whole SD-card-access-via-SPI is proving to be a bit of a headache.
Just to make sure we're not having a timing issue (cards need to be interfaced at < 400khz until properly initialised) we've even resorted to bit-banging the SPI and introducing delays between pin wiggling on the clock line.

The frustrating thing is that despite numerous attempts, we're still getting nothing back from the SD card. We found a cut down FAT16 library for Hi-tech C compiler, and tried converting that back to SourceBoost with limited success.

Using this hacked-up code, we did manage to get something back from the SD card:

The response we're looking for is 0x01. This is visible on the probe output (above) and the probe software correctly identifies it as 0x01 but for some reason, our code doesn't interpret it as such. It may be because the clock cycles - complete with stretching where the crazy library-based code tries to re-sync the clock line (and, it looks like, it fails!)

At least we're getting something back from the SD card, so we can presume it is actually powered up!
However, following the SD guide, sending the command bit 0x40, then 4 bytes of 0x00 and a CRC checksum of 0x95 should, after a few clock cycles, see 0x01 being returned from the SD card.

Sadly, we never see this...

#include <system.h>

//config1 = 0x0044      //68 =4+64
//config2 = 0x1DFF      //7676 = 1+2+4+8+16+32+64+128+256+1024+2048+4096

#pragma DATA _CONFIG1, 0x0804
#pragma DATA _CONFIG2, 0x1DFF
#pragma CLOCK_FREQ 32000000

typedef unsigned char byte;
unsigned char r;
unsigned char goSlow;
unsigned char spiIn;
unsigned char spiOut;

// ################################################################################

void toggleClock(){
      portc.0=1;
      if (goSlow=1){delay_us(10);}
      portc.0=0;
      if (goSlow=1){delay_us(8);}
      portc.0=1;
}

unsigned char sendAndReceiveSPI(unsigned char in){
      unsigned char tmp;
      tmp=0;
      spiOut=in;
      spiIn=0x00;
     
      if(spiOut.7==1){ porta.4=1;}else{porta.4=0;}
      toggleClock();
      if(portc.1==1){spiIn+=128;}
     
      if(spiOut.6==1){ porta.4=1;}else{porta.4=0;}
      toggleClock();
      if(portc.1==1){spiIn+=64;}
     
      if(spiOut.5==1){ porta.4=1;}else{porta.4=0;}
      toggleClock();
      if(portc.1==1){spiIn+=32;}
     
      if(spiOut.4==1){ porta.4=1;}else{porta.4=0;}
      toggleClock();
      if(portc.1==1){spiIn+=16;}
     
      if(spiOut.3==1){ porta.4=1;}else{porta.4=0;}
      toggleClock();
      if(portc.1==1){spiIn+=8;}
     
      if(spiOut.2==1){ porta.4=1;}else{porta.4=0;}
      toggleClock();
      if(portc.1==1){spiIn+=4;}
     
      if(spiOut.1==1){ porta.4=1;}else{porta.4=0;}
      toggleClock();
      if(portc.1==1){spiIn+=2;}
     
      if(spiOut.0==1){ porta.4=1;}else{porta.4=0;}
      toggleClock();
      if(portc.1==1){spiIn+=1;}
     
      return(spiIn);
}

void assertCS(){
      porta.1=0;
}

void deassertCS(){
      porta.1=1;
}

void initSDCard(){
      unsigned char i;
     
      // pull the SD card power pin low
      porta.0=0;
      delay_ms(1);
           
      // pull all the other pins low - they should be low
      // during the SD card power-up cycle
      portc.0=0;
      portc.1=0;
      portc.2=0;
      porta.3=0;
      porta.1=1;      // hold CS high during power up
      portc.0=1;      // hold clock high during rest
     
      // power up the SD card
      // (this is also the trigger on the logic probe)
      porta.0=1;
     
      // clock a few pulses to clear buffers etc.
      deassertCS();
      for(i=0; i<10; i++){
            r=sendAndReceiveSPI(0x00);
      }

     
      // enable the CS on the SD card
      assertCS();
                       
      // send the reset command (0x40 | 0b10000000)
      r=sendAndReceiveSPI(0x40);
     
      // send four blank/null bytes
      for(i=0; i<4; i++){
            r=sendAndReceiveSPI(0x00);
      }
      // send the CRC/checksum
      r=sendAndReceiveSPI(0x95);
           
      while(r!=0x01){
            r=sendAndReceiveSPI(0xFF);
      }
     
      // turn off debugging
      porta.0=0;
}

void initialise(){
     
      osccon=0b11110000;            // oscillator=32mhz
      intcon=0b11000000;            // global interrupts enabled and peripheral interrupts
      apfcon0=0b00000000;            // alternative pin function - bit6: SDO1 function is on RA4=1, RC2=0
      trisa=0x00;                        // all outputs on portA
      trisc=0b00000010;            // all portC lines are output except pin1 (RC1=spi in)     
      ansela=0x00;                  // turn off analogue pins on portA
      anselc=0x00;                  // turn off analogue pins on portC     
      option_reg=0b10000000;      // no weak pull-ups
     
      goSlow=1;
}

void main(){
      initialise();
      delay_ms(10);     
      initSDCard();
}

void interrupt(void){
      // bit zero of pir1 is tmr1 overflow interrupt
      if(pir1.0==1){
            // clear the timer1 interrupt flag
            pir1.0=0;
            // preload timer1 to get the next interrupt to occur in a timely manner           
      }
}

Once we get the basic SD initialise working, we'll look at putting it back onto the SPI hardware (bit-banging is soooo last year) but for now we're just trying to get the SD card to even acknowledge it's presence!

PIC 16F1825 pinout for SPI/SD card interface:

SPI_CLOCK:  RC0
SPI_MISO (in): RC1
SPI_MOSI (out): RC2 / RA3 (can be changed in firmware)

We're using RA0 to turn the SD card on through a FET
SPI_CS (chip select): RA1

Sourceboost SD Card reader

We're having a bit of a time of it at the minute, just trying to get our PIC16F1825 to correctly initialise an SD card. It's not helped by the fact that we're using a new chip we've never used before, in a language/syntax we've not used for a long time, with a compiler we've never seen, using hardware peripherals we've only ever simulated in software. These last few days have been a very steep learning curve!

At last night's BuildBrighton meeting, Jason spent a good few hours going over all our registers and code to see if there was anything obviously wrong, but found nothing. But we couldn't even get the SPI hardware module on the PIC to operate, let alone send data using it!

Jason has a Scanalogic Logic Probe/Analyser which has proved invaluable in finding out what's going on at a pin-wiggling level. Using it, it took just a few minutes to realise that one of our fuse settings was incorrect and that instead of sending data over SPI, we were outputting the PIC clock signal onto the (multiplexed) SPI_OUT pin. D'oh.

With the Scanalogic Probe in place, we were able to actually see pin activity as data was exchanged between the devices

Using this helped us identify where things were initially going wrong and after three hours of getting nowhere, we finally manged to see SPI data being sent over the pins. Now our problem is, there's nothing coming back from the SD card!

In the image above, the blue line is the SPI clock. The yellow is the SPI data going from the PIC to the SD Card. Red is the trigger pin (used to start the probe reading the data) and the green is *supposed* to be the data coming back from the SD card.....