Preparing For The Big Move And Finding Inspiration On The Road

This week I have been making numerous trips to my new home that I will be moving into in a couple of days.  The number of trips so far has now reached number nine.  NINE!  This seems a bit much but will in the long run be a blessing on moving day.  Nearly everything that can be boxed and moved has already been done.  So all the big pieces of furniture, tools, scrap wood, half finished projects and remaining odds and ends will be what is left to move this coming Saturday.  This is the first time I have moved in almost 33 years so to go to a new house is a big adjustment for me.  Not including all the work that this has been to get ready for the move. 

  The plus side to all of this is a better home, a new chapter for my life, a whole new town and the Tinker's Workshop will be bigger and better once it is set up.  The new Tinker's Workshop will be a vast improvement over the original shop so I am looking forward to exciting projects that have been on the drawing table for a VERY long time just waiting for the right time to start building them. All of this is both exciting and scary at the same time.  It will all work out I have no doubt and so I continue to push forward on the prep for the move and the day when I can get back to building in my new and improved workshop. Just the setting up of the new Tinker's Workshop will be good for at least a few good posts on how to set up a sweet workshop. It will be fun for me as well as informative and entertaining for you.  So check back and I'll keep you up to date on my progress.

   Also with all my traveling this week from the soon to be old house to the new one I came across this billboard on my route that I just had to post a photo of for everyone to see, chuckle at, and possibly inspire you to follow your dream.   This is what I saw.  Live your dreams and you won't go wrong.  Keep tinkering!

"Official" Poll: Do you think robots will dominate the world

iRobot, fact or fiction? Cast your vote today!

Do you think robots will take over the world?
	Definitely!
	Maybe, it's possible.
	No.
	I hope not :(
pollcode.com free polls

Playing catch with a humanoid robot? No problem!

Usually playing catch is played between two people. As of now, catch can be played with a robot. Developers at Disney Research Labs have created a humanoid robot that is capable of throwing and catching a ball with its hand. To start, a camera locates the ball, then the ball's destination is predicted. Once that is done, the robot's hand is positioned in the ball's predicted destination. Resulting in the robot catching the ball. To see a demonstration, click the video above!

Salt and pepper shaker robots available on Amazon.com

Too lazy to pass the salt or pepper? Don't worry, there's a bot for that. These two little bots are able to do just that. Designed with clever clockwork, these robots are capable of strolling across the table to deliver you salt and pepper. A tiny twist of their inner-workings and they're off to the other side of the table. The white one carries the salt and the black bot carries the pepper (obviously). If your interested in purchasing these robots, then visit amazon.com . Anyways, enjoy the demo video above!

Large format vinyl cutter at BuildBrighton Foison C24

Last night was BuildBrighton night (in fact, almost every night is a BuildBrighton night these days, but Thursdays are the "traditional" come-along-and-hang-out night) and as is usual, come the witching hour, cool things start to happen.

This video was taken at around ten past midnight, after we spent a couple of hours restoring the large format vinyl cutter that we donated a few months back. It's a Foison C24 and has been used about three times in it's lifetime. Hopefully, now it's in it's new home and working, and more people have access to it, it'll get used quite a bit more.

As you can see, it's quite a whizzy-fast machine. Because it's designed to use a drag knife, it takes corners quite slowly, but really flying down the straights!

For testing, we replaced the drag knife with a fat-tipped Sharpie and drew on some copper board. Of course, it's only a matter of time before we're drawing PCBs directly onto the copper board, by-passing the whole press-n-peel stage of etching ;-)

Entertaining Destruction For My Thanksgiving

  As most of you have already seen on my blog I posted the loss of my garage space that was next to my house almost two months ago now.  A major fire had taken place in my garages that I had been renting and were destroyed along with some great projects, my motorcycle and Mini Cooper.  This as you can imagine has turned my world upside down to say the least.  
  Today was a good day at the Tinker's Workshop as the repair work was completed on my damaged home and the garages were torn down to top off the day.  I am here to say that I am glad the garages are finally being knocked down and removed as I have had to look at the devastation for these past two months every day I walk outside.  Not a nice thing to see or be reminded of over this time.  So without further rambling here are some photos I took today to show you how the mess is being cleaned up. The guy with the Bob-Cat was a real artist.  He could make that thing get up and dance. 

Here is a shot of what I have had to look at for the past couple of months.  Depressing to say the least.

The guy in the orange sweatshirt is the owner of this pile of rubble.  I would not want to trade places with him for anything.

Another shot of my garages or should I say the makings of another pile of rubble.

TIMBER!

Going,

going,

going,

going,

gone.

Strange to see the garages gone after looking at them for almost 33 years.  End of an era and lots of projects built in them over that time.  The really good thing now with the building knocked down is that it makes my home look that much better and I can even see down the street now from my kitchen window!  Improves the look of the entire neighborhood too. So there is a plus side to it after all.  With my moving to my new house in the coming weeks hopefully I can get this place sold in short order and move another step forward in my life.

  Yes this will be a Happy Thanksgiving for me here at the Tinkers Workshop and with this building (now rubble) being cleared away it's a very good start to the holiday weekend.  I hope you have a good turkey day as well!

DipTrace for home etching, gerber export, 3d views and milling outlines

The more time we spend with DipTrace, the more we like it, here at Nerd Towers.
We first came across it while designing PCBs for our miniature guitars last year - changing over from ExpressPCB because we needed to generate gerber files to have some circuit boards manufactured online.

It's easy to use, has a massive library of components, but uses an approach similar to Eagle; when placing components in the schematic, you have to choose the "correct" component with the appropriate footprint while you're drawing the schematic. We much prefer the ExpressPCB method: draw a schematic then link it to the PCB designer - so long as the number of pads in the PCB layout software matches the number of pins in the schematic, you can use pretty much any component footprint to represent any component in the schematic.

As a result, we still turn to ExpressPCB whenever we need to "knock out" a few boards for home etching.
But since we've had to design some more boards for manufacture (for a recent alarm clocks project) we've returned back to DipTrace to generate more gerbers.

And we've found that DipTrace has some pretty cool features - even for the homebrew enthusiast. Firstly, you can create homebrew boards easily and - using the ExpressPCB method - print to a PDF file using CutePDF and flip in Inkscape, reading for printing onto some press-n-peel.

Of course, DipTrace can export to Gerber (so the file(s) can be emailed for manufacture by a professional PCB fabricators). But it also has a few other handy export options.

We've already discovered the brilliant 3D view:

The great thing about this 3d generator is that you can place components so that they "hang off" the board. Not many tools let you do this. In the example above, we've created our audio board but cut the dimensions down, so that the SD card holder is fixed at one end. There's no need to waste copper board under the rest of the sd card holder - not all PCB layout tools allow you to create a board outline smaller than the components on it!

As well as 3D, and creating toner-tranfer images, DipTrace supports exporting to DXF.
Why is this so impressive? Well, if you've a CNC router, you can get it to draw around the outside of each of your pads and traces:

Wow! Now that's cool. Anyone with any experience of using Mach3 or similar CNC software knows that it's relatively easy to get from a DXF drawing to some g-code which we can use to drive the CNC router and - rather than all that nasty etching with stinky chemicals - we could rout out the PCB board from a sheet of copper-clad board.

All these output options from the same PCB layout. DipTrace is looking like a really useful bit of kit. And perhaps something we should persevere with for a little while. After all, up to 500 pins, it's FREE as well.

Etching heater

Etching with Ferric Chloride is a messy business. So the quicker you can get it over with the better - and for a few reasons. One way to decrease the time taken to etch is agitation: basically stir the mixture or dunk the board in and out of the etchant.

The problem with this, of course, is more opportunity for spillage. And with ferric chloride being such a nasty staining solution, you don't want it splashing around the place.

Another way to speed up etching is using heat. Ferric Chloride works best at about 50 degrees. But sitting the solution in a bath of hot water only increases it's temperature by a few degrees at best. So we tried one of these things - it's called an etching heater, but really it's just a fancy aquarium heater!

It's like a super-duper tropical aquarium heater in a sealed glass case- complete with temperature control on the top, so you can set the thermostat to anywhere between 40 and 60 degrees: perfect for ferric chloride etching! Just search ebay for "etching heater". This one cost less than �15 including delivery to the UK and took about 8 days to arrive.

We put about a litre of ferric chloride solution (made up quite strong, with a full 500g packet of crystal dissolved into about 1L of warm water) into a sweetie jar. This should allow us to etch rather larger boards if we need to. Just stand the heater in the solution, switch on and back away. After about five minutes, the entire thing is up to perfect etching temperature.

(heating the solution is easy - plop in the heater and switch on!)

A relatively small board can now be etched in about two-to-three minutes, rather than the 15-20 minutes it has been taking recently, when etching "cold" (as the winter nights draw in, it does get colder in the nerd cupboard, so ambient temperature is about 16-20 degrees).

The great thing about etching quickly is that because the board is in the solution for a lot less time, the chance of under-cutting is much less (under-cutting is where the etchant eats away at the copper under the toner-transfer mask because the board is left for too long in the solution). By etching hot and quickly, we get really sharp, clear etching, even with thin traces. 

In the photo above, we've managed to get traces down to 0.2mm and they've etched perfectly. When etching "cold" we would rarely go below 0.38mm trace size to avoid over-etching the tiny thin traces.

Here's the board cleaned up, with a micro sd card connector soldered in place, ready for populating. In short, 

DipTrace and the amazing 3D render

I recently had to get some PCBs made up for a little project, so had to leave my favoured ExpressPCB behind and use DipTrace again, in order to generate some gerber files for manufacturing (normally we'd just etch our own boards, but time is tight with this one, so they're on a three-day turnaround from a UK-based manufacturer).

I was looking for a gerber viewer when I came across Jason's post about a web-based 3d renderer: https://plus.google.com/u/0/105009608886388132613/posts/1oST39owVAZ

Now that looked exciting. I uploaded the gerber files generated from DipTrace and eagerly waiting for the image to appear..... nothing. Whether it was the gerber file, my lack of 3d graphics hardware in my little notebook or something else, I don't know. I do know that other people commented that they couldn't get it to work either. So no 3D renderings for us.....

Then I spotted this little icon on the toolbar:

So what would that do....?

Ta-da! A perfectly rendered (and populated) PCB. I'm not quite sure how to get the components to sit down in the board properly (seriously, who solders their components so they stand up like that?) but as a quick overview of how your final board could look (especially now we're moving away from through-hole and trying to use SMT wherever we can) it gives great results!

We might just give DIPTrace another go.
Familiarity and laziness have kept us using ExpressPCB. But these 3D renderings do add a fresh dimension to documenting projects (-no- pun intended) so maybe it's time to give it a whirl. The freeware version is quite restrictive with only 300 pins allowed. But for simple, little projects like this one, that's more than enough!

DC blocking capacitor

We've had a few questions about the in-line capacitor on our audio player board. This is the capacitor between the PIC pwm output pin and the gate pin on the transistor amplifier. At first glance, it seems a bit counter-intuitive: we're "blocking" the output from the PIC to the speaker. What's it there for?

Well, here's the thing - the circuit will actually work without the capacitor in place.
Go ahead, try it. You can connect the output from the PIC directly to the gate on the transistor, and you'll still hear sound. It might not be quite as loud, but it'll still be an amplified signal. So what does the "blocking capacitor" do?

Consider what's happening - we're using PWM to drive a speaker.
It's not the most efficient use of the speaker, since we're not generating negative voltages, so we're only every driving the speaker "forwards" using a positive voltage. But let's ignore that for a moment!

Our PWM is creating an "average" voltage over a particular time period, to move the speaker by a certain amount. Now most digitally recreated sound waves go from minus the maximum amplitude (let's call it -5V for simplicity) and plus maximum amplitude (e.g. +5V). When there's no sound/signal, there's no voltage 0V.

But because we don't have negative values in our PWM , we're having to use a value 0-255 to represent the full range from -5 (in this example) to +5.

So -5 (the lowest possible value in the wav file) becomes PWM value zero (the lowest possible value in our PWM output). And +5 (the maximum value in the wav file) becomes PWM value 255 (the highest possible value in our PWM output).

So where the audio file is normally zero (half-way between max and min) our output PWM value is actually 128 (half-way between 0-255). When there is no audio signal, our PWM output is 128, or half-duty cycle, or (on average) 2.5V output.

Now driving 2.5v continuously through the coil in our speaker is going to cause it to heat up and over time, eventually burn out the speaker. So what we do is put a capacitor between the output pin and the gate (which in turn controls the amount of current going into the speaker).

So now, while our audio is silent, there's nothing actually coming out of the other side of the capacitor.

• When the wav signal falls, the capacitor discharges  and creates a high output.
• When the wav signal rises, the capacitor starts charging up and the "output" side falls low.
• The output from the capacitor lags the input by a single sample width.

This means that while the signal is fluctuating - ie. we're using an AC signal - the capacitor allows the signal to "pass-through" albeit one sample width behind the input. When the AC signal becomes DC (a period of sustained silence, for example) the output from the capacitor drops to zero and no voltage is passed into the speaker. The "blocking capacitor" acts like a filter - stopping DC current but allowing AC signals through.

This is a very crude method of driving the speaker, but one that works.
The downside is that where a very quiet audio signal is being played, with large gaps of silence, or very quiet background noise, the speaker is hardly active at all, and you can hear all the other "noise" in the line (the PWM carrier signal, for example).

We can hear this at the start of the song being played in the earlier video.
For the first ten seconds or so, the background sound of rain just sounds like a digital noise and you can hear the hum of the PWM signal. This is because parts of the audio signal are being treated as "rippling DC current" rather than a true AC signal. However, once the audio signal amplitude increases, a large AC signal is generated and the blocking capacitor allows all of the signal to pass into the speaker and the audio clarity improves.

"Cereal Killer," the automated cereal maker

If your too sleepy in the morning to pour yourself a bowl of cereal then this robot is here to help. Built out of Legos and controlled via a micro controller, this robot is perfect for when you wake up on the wrong side of the bed. Whether your too lazy or tired to make yourself breakfast, this bot will take care of it. To see how it makes breakfast, check out the video above. Other than that, thanks for reading!

The Gremlins Have Hidden A Lot Of Stuff Over The Years!

  A lot of things have been happening over the past few days here at the Tinker's Workshop.  In an earlier post I let you know that I will be moving very soon to a new location in Wisconsin due to the loss of my garages at my current home.  Through all of this a lot of people behind the scenes have been busy pushing through all of the paperwork that goes on with buying a new home and while this has been happening I have been packing everything that I can and started dismantling the workshop. 

   Just the workshop alone has been a huge undertaking as I have had to do this all by myself.  A lot of work had to be done just in this portion of the house to get ready for the upcoming move.  I thought it interesting to see the workshop once again as I had found it almost 33 years ago.  Needless to say I stuffed a lot of tools into a very small space to make the workshop actually usable.  It has been surprising to a lot of people over the years as to how I managed to create what I did in such a small shop. To compare the way it looks now with everything dismantled and scattered about to how it looked beforehand take a look at the photos I took today and the Tinker's Workshop title photo when it was all together.  Quite a difference. 

A portion of my living room is slowly being swallowed up by over 30 packing boxes already filled with almost all the small items that come with moving my home and the workshop hand tools, miscellanous parts, hardware and power cords.  I'm just happy that I am not also moving belongings of four or five more people.

The workshop is slowly being dismantled and the space cleared of years of hidden dust, dirt, and lost screws and nuts.  The gremlins manage to hide a lot of missing screws and nuts over 33 years.

  This pile of parts was my CNC machine.  I was able to take it apart in sections and only had to disconnect one stepper motor cable for the move.  This will make things easier to move and reassembly of the machine an easy task.  I already have plans to update the CNC machine to improve the setup on the  cutting bed to a full T-slot table.  Possible making this out of aluminum.  I also have some ideas on how to level the cutting table easier and quicker than the original desing. I'll this update to the machine once I get this big move completed and the shop set up once again. 

This photo shows the work table and the work bench taken apart ready for moving day.  Every little thing that I can do now will save valuable time when moving day gets here and I can head to the new house and workshop space.  The current floor space of the workshop had one area that is 12' x 12' and another area 6' x 6'.  A grand total of 180 square feet of usable floor space.  Very small to say the least. The new workshop will have a floor space that is 13'6" x 20' plus a storage area that is 12'9" x 24'4".  This comes to 580.25 square feet of usable floor space.  Well over three times the work and storage space of the current shop.   I love it!

  I received word today that my loan for the new house and ultimately the new home of the Tinker's workshop has been approved!  Another call came today from my realtor to see if I could show my current home to a prospective buyer.  So as I said earlier a lot of things are happening here at the Tinker's Workshop.   Can't wait for it to all come together.  Check back again for more updates as I post them. 

Different speakers sound different

It's almost so obvious as be incredible!
Here's our PIC-based wav audio player with a cheap 15p speaker from Farnell. The difference between this and the speaker on the alarm clock is amazing! We've tried all kinds of different ways of amplifying the output but the best/loudest to date is this 16 ohm, 700hz, 90dB piezeo speaker with a humble BS170 FET transistor.

The audio in this example is much clearer than with the earlier speaker (and louder too).
Perhaps this is because rather than put the speaker in a box, we're holding it to create a cavity behind the speaker cone. You can definitely hear the speaker gain more definition between hanging loosely and being "encased in flesh"!

Interestingly, the best sound came when we created a tube effect rather than a closed box shape with the hand holding the speaker. Whether this only applies to hands, or to enclosures in general, we've yet to see.....

The latest Call of Duty features robotic warfare

Today, Call of Duty Black Ops 2 was released all across the nation. In the newest copy/paste Call of Duty game, robots play a huge role. For example, the cheesy storyline is when the enemies takes control of America's robots, or as the trailer states, "gets the keys." The setting is also in 2025, which gives us a clue that these robots could be right around the corner. In the latest trailer, two quadruped robots and multiple quadrotors are shown. These two robots easily resemble BigDog. In addition, FPS Russia was showing off the quadrotor drones. Even though those types of robots can't carry a gun, I'm sure it won't be long until DARPA reveals one. Thanks for reading and check out the video above to see the robot-friendly trailer!

Good News For The Tinker's Workshop

  If you have been following my blog at least over the past couple of months you know what I have been going through here trying to put the pieces together after the major fire destroyed my garages and a lot of great projects.  This is now in the past and good things have been happening rather quickly because of the what has happened.
  I now have my house up for sale and already have found the new house and future location of the improved Tinker's Workshop.  The Tinker's Workshop will be online as long as I have a following that wants to see what I am up to with the projects that I work on.  So this move will be huge for the me to start over after the fire and a plus for the blog site.  Lots of great projects have been on hold for years simply because I did not have the room in the workshop to start construction or have the place to store what I want to build.  So with all of this in mind here is the good news about the new place for the shop and my home.
  The new workshop that I am planning will be three times larger than the one that I currently have in my house now.  (As my workshop in my basement now is only about 12 feet square.  VERY small.) That is plus number one.  Secondly in the new location for the first time in my life I will have my own 24 x 24 foot two car garage.  This opens up all kinds of possibilities for future projects that as I've said have been on hold for years.  Like building a full sized electric three wheeled car that I built a model of a couple of months ago on my Makerbot 3D printer. (See earlier posts of this project). Or building a custom built teardrop trailer which would look excellent at any camp site. (See Blender 3D Creations link on this blog).  Or a custom built composite fiberglass micro catamaran sailboat. I also have several projects that will be perfect for my new 1/4 acre back yard.  My current back yard is only 20 feet square so 1/4 acre  will be like living on an estate!  It will be great to be able to build another big project at the new location and not bump into something every step of the way.
  So out of all the hassles with the insurance company and losses that I have had to endure during the past couple of months there will be some really good stuff that will come out of the mess.  At this point the smile has returned once again to my face and the prospect of getting a chance to work on projects that yesterday I thought were only long lost dreams give me great hope for the future of the Tinker's Workshop and my sanity.  Yes I do believe someone is watching over me and also is smiling! As always I will never stop tinkering and so should you.  Stay tuned for more updates on the move and new projects that will be in the works.

ABB's new "Dragon" is the fastest industrial robot to date

The last time I wrote a post about a fast robot, it was related to the soda can challenge. Yet again, ABB has succeeded in making a fast and agile robotic arm. Dragon, as they call it, is currently the fastest industrial robot in the world. It's designed to make computer mice efficiently and productively. If you want to see this robot in action, then check out the video above.

FAT/FAT16 tables - finding the data

Having successfully found the root directory on our sd card, we now need to start actually reading the data back from it. This is the last little link in a chain of

• initialise the sd card
• find the lba (logical block address) by reading sector zero
• read the mbr (master boot record)
• calculate where the root directory starts
• find the file we're interested in
• read the file

The easiest way to find a file is to load a filename into an 11-byte buffer. The easiest format to use is the name, padded with spaces plus the extension, minus the full-stop; so wav001.raw becomes wav001[space][space]raw. Or, perhaps more accurately, WAV001[space][space]RAW (since FAT16 likes to store filenames in uppercase).

A root directory entry looks like this:

• The first eight bytes are the file name - FAT16/MSDOS only supported up to eight characters in a filename.
• The next three bytes are the file extension
• Then there's an attributes byte
• Followed by time stamps for the file creation/modified date.
• The next two bytes are the starting cluster for the data (note, cluster not sector)
• The last four bytes are the total file length (in bytes)
• Every root directory entry uses exactly 32 bytes

It's the first bunch of bytes we're interested in - the file name, starting cluster and total file length:

Now we just read the root directory, comparing the first 11 bytes of each 32-byte entry in the root directory to the bytes in our filename buffer and if they match, we've found the file we're interested in.

Once we've got our file, we need to work out the file-length (in bytes) and the starting sector for the first cluster of data.

Byte 26 of 32 is the least-significant byte of a two-byte word  (e.g. 0x02)

Byte 27 of 32 is the most-significant byte of a two-byte word (e.g. 0x00)

The starting cluster for the file in this example is therefore 0x0002

Bytes 28-31 represent the file length, starting with the least significant byte of a 4-byte value.

In this example:

Byte 28 = 0x43

Byte 29 = 0x5e

Byte 30 = 0x09

Byte 31 = 0x00

The total file length is 0x00095e43 which in decimal works out as 613,955

Looking at the file wav003.raw in Windows Explorer confirms that the file is, indeed, 613,955 bytes in size

Now previously, we worked out a whole heap of important things from our MBR (master boot record) including where the actual data started, and the number of sectors per cluster (in a FAT16 formatted disk, this is usually 32 sectors per cluster, making each cluster 16kb)

If we know which sector the actual data begins from, and which cluster to begin reading from, we can calculate at which sector the cluster begins.

unsigned long clusterToSector(unsigned long cluster){
     // a cluster is a complicated thing:
     // first there's an ofset to the master boot record
     // then some reserved sectors
     // then a fat table (maybe two)
     // then the root directory (fixed length)
     // THEN there's the first data block,
     // which is where we start reading our clusters from
     // BUT there's a catch - clusters start from zero not 2
     // so whichever cluster number we've been given, subtract 2
     
     unsigned il;
     il=cluster-2;
     il=il*sectorsPerCluster;
     il=il+dataStart;
     return(il);     
}

Great stuff!
Convert the cluster number (in this case, cluster 0x0002) into a sector (in this case, because clusters start from 2 - it's an anomaly of the FAT16 format - our first cluster is also the first sector where the data starts. We've already calculated this value)

"The actual data starts at sector 615 + 32 = 647"

If we jump to sector 647 and start reading back the data, we find that there is, indeed, data in there!
But with a file that's 613,955 bytes long, it's not all going to fit into a single cluster (one cluster is 32 sectors and each sector is 512 bytes, so that's only 32x512 = 16,384 bytes - 16kb)

So where's the rest of the data?
That's where the FAT table comes in!

Firstly, take the cluster number and double it. That's the same as bit-shifting the entire value one place to the left. In our example, our starting cluster was 0x02 so we double this and end up with 0x04
This tells us where in our FAT table to find the next (two-byte) cluster number.

Since the FAT tables themselves are written across a number of sectors, we need to convert this cluster_doubled value into a sector and byte offset, to read the actual "next cluster value" back.

Divide the cluster_doubled value by 512 to get the sector number
The remainder is the byte offset.
In our example, this gives us sectors zero, byte offset 4
So we want the first sector (sector zero) in our FAT table, fourth and fifth byte

Since the FAT table begins at sector 143, we add zero to this, open the sector (using our earlier functions) and read back all 512 bytes. When we get byte four, this makes up the least significant byte of a two-byte (16-bit) value. Byte five is the most significant byte.

In fact, when we open our sd card, read back bytes four and five from sector 143, we get
Byte 4 = 0x03
Byte 5 = 0x00

This tells us that the file continues at cluster 0x0003.
Using the same technique, we open and read the data from cluster 0x0003 (sector 647 + (3-2)*sectorsPerCluster = 647 + 32 = 679)

We continue reading data from the sector(s) and calculating the next cluster where the file continues until we've either read back the entire file (total bytes read > file size in bytes) or the FAT table returns 0xFF as the next cluster (this is a special response to say "no more clusters for this file").

This is summarised in the following function (remove comments around UART calls to see what the microcontroller is actually doing when calculating next FAT clusters).

unsigned char openNextBlock(){

     unsigned short iFATSector;

     unsigned short iFATBytes;

     unsigned short iy;

     unsigned short ix;

     

     //UARTPrintLn("opening next block of data");

     

     // the cluster_number_doubled is the location in the FAT table

     // where we can find the next block of data. If this entry is 0xFF

     // it means that there's no more blocks of data, otherwise the entry

     // is the next cluster where the file continues from

     

     iFATBytes=nextFatClusterDoubled & 511;

     iFATSector=nextFatClusterDoubled>>9;

     iFATSector=iFATSector+FATStart;

     

     //UARTPrintLn("look up next cluster in FAT ");

     //UARTPrint("sector ");

     //UARTPrintNumber(iFATSector);

     //UARTPrint(" byte offset ");

     //UARTPrintNumber(iFATBytes);

     //UARTPrintLn(" ");

     

     // check the FAT tables for the next cluster for the current file

     r=sdReadBlock(iFATSector);

     for(iy=0; iy < 512; iy++){

          r=readByte();

          if(iy==iFATBytes){ix=r; nextFatCluster=ix;}

          if(iy==(iFATBytes+1)){ix=r; ix=ix<<8; nextFatCluster=nextFatCluster+ix;}

     } 

          
     // close the currently open sector
     sdSecReadStop();

     nextFatClusterDoubled=nextFatCluster<<1; // this is the same as multiplying by two!


      
     //UARTPrint("next FAT cluster ");
     //UARTPrintNumber(nextFatCluster);
     //UARTPrintLn(" ");          
     
     if(nextFatCluster==0xFFFF){          
          // if we're at the end of the block, send the end of block marker
          //UARTPrintLn("no futher fat clusters to follow");
          return(0xFF);
     }else{
          
          // open the next sector
          iSector=clusterToSector(nextFatCluster);
          sectorCount=0;
          //UARTPrint("file continues at sector ");
          //UARTPrintNumber(iSector);
          //UARTPrintLn(" ");
          
          return(0);
     }
}

Another video interruption - serial audio with a PIC 16F1825

While writing up how SD cards work, and reading files using FAT16, we've also been developing our little audio board a bit further. We got as far as playing audio files from our SD card and fitting them to the alarm clocks for a tester project the other day.

We've now taken the project one stage further, and introduced serial/uart support.
So rather than play a file immediately, you can send serial commands to the audio board.

• # - send back a list of all files found in the root directory
• | - pause playback
• > - resume playback 
• \ - half current volume (max four times)
• / - double current volume (to max)

At any time you can send in a filename over serial.
If the filename is in the format xxxxx.yyy (where xxx is any string up to 8 characters and .yyy is a full stop followed by a three-character extension) the chip will search the root directory for a file with the same filename and extension.

If a file is found, it begins playing immediately.
If the chip is in the middle of playing a wav, the current wav file is stopped, the current sector is closed, and the chip plays the new sound from the start.

Note - our video is blurry (again) thanks to wonderful camera-phone technology! And trying to hold the speaker in one hand, the phone in the other, and typing at the same time ;-)

The PWM carrier signal generates a hum which is much more noticeable in the video than in real life, but in a future revision, we'll look to remove this using a low-pass filter or capacitor across the speaker terminals.

At present we're playing raw (headerless) PCM wav files.

The frequency is hard-coded to 22khz and all sounds are played in mono at 8-bit. This requires the sounds to be prepared before copying them to the SD card. A future revision will be able to play different types of wav files, but for now, we've a single-chip, low-cost, low component-count audio player board which works with all SD cards we've tried it with!

Robot swagger

Even though Japan has robots like Asimo, the US has robots with swag.

Understanding FAT tables

While it's possible to simply read and write data to and from our SD card serially (treating the card as one big eeprom chip) the most useful way to use the card is to format it using FAT16.
This will allow the card to be removed from the microcontroller and the data either read back or written to using a PC. This is far more useful that having to use some dedicated hardware to stream read/write data to/from the card and report it's finding back over UART/serial.

FAT is quite a complicated structure and needs careful understanding; it's very easy to get lost and end up reading chunks of data from the wrong sector and mangling files - especially if files on the card are frequently added or deleted and the files end up fragmented (i.e. written over a number of non-consecutive sectors).

To get started, download a hex editor (we used HxD) and format your sd card to FAT or FAT16 (not FAT32) format.

Here's your first gotcha:
When you mount an sd card into a hex editor to view the entire file contents, the hex editor usually starts from the LBA (logical block address). It will often call this sector zero.
But on the SD card, this may not actually be physical sector zero. If you use the previous card reading functions and write out the contents of sector zero over serial, you'll see it doesn't look like a boot record - it's mostly zeros. Sector zero usually just contains information about where you'll find the actual master boot record (and where the hex editor will call sector zero).

We found the easiest thing to do was use the UART tool on the PicKit2 software to view the serial data as it was read back from the sd card. We used the "save to disk" option and then read through the file after it had been saved as a text file. You can use whichever serial-based tools you're happy with for debugging at this point:

The last two bytes of the entire sector were 0x55 and 0xAA
This is the FAT signature to say that we're reading a FAT-formatted drive. We just double-checked that our code was working and actually reading back sensible data by looking for this signature at the end of the sector-ful of data.

Most of our sector zero data was zero - 0x00.
Having read up a little about FAT tables and file layouts, we were expecting a bit more than this - we were told to expect things like the OS name/type, volume label and so on, but nothing here looked anything like it. But that's because we weren't looking at "logical sector zero" - the first sector in the file structure - we were looking at physical sector zero.

So our first job was to find the logical block address LBA.
In amongst all the zeros, there was a little bit of data, starting at byte 447.
This is the data describing the actual drive itself - sectors, clusters, heads and cylinders (used in older drives). The data we're after is the two bytes at 454 (0x1C6) and 455 (0x1C7).
This is the sector number containing our master boot record MBR.

Another gotcha coming up - where data is written using two or more bytes, the order of bytes is back-to-front to what you'd expect: the least significant byte comes first. So in our example, we've got 0x87 followed by 0x00. But this actually translates to 0x0087.

Using the same routines as we did for reading sector zero, we get the PIC to read back the contents of sector 0x87 (135 decimal).

Straight away we can see that this has some useful information. Already, we can see the name MSDOS5.0 being spelled out. So what about all the other junk in there? Here's how the boot record is laid out:

• Jump instruction (3 bytes)
• Operating system (8 bytes)
• Bytes per sector (2 bytes)
• Sectors per cluster (1 byte)
• Number of reserved sectors (2 bytes)
• Number of file allocation tables (1 byte)
• Max number of possible root entries (2 bytes)
• Small sectors (2 bytes)
• Media type (1 byte)
• Sectors per file allocation table (2 bytes)
• More stuff we're not particularly interested in

The bits of information we're interested in are the bytes per sector (11 bytes into the record), sectors per cluster (13 bytes in) number of reserved sectors (14 bytes in) number of FAT tables (16 bytes in) the maximum possible number of entries in the root directory (17 bytes in) and the number of sectors used by each FAT table (22 bytes into the record).

Looking at our log files, we find entries 11 and 12 are 0x00 and 0x02 respectively.
But we have to remember that data is stored back-to-front, so the bytes per sector value is actually 0x0200 which is 512 in decimal. This looks like a sensible value so we can assume we're correctly reading the data back.

Sectors per cluster (a single byte) returns 0x20 which in decimal is 32. Again, a sensible-looking value and one we'd expect (32 sectors of 512 bytes = 16kb per sector, which is the FAT16 standard).

Number of reserved sectors is important for us to work out where the FAT tables and data actually reside on the card. Reading bytes 14 and 15 we get the (don't forget to flip them around) value of 0x0008 - i.e. there are eight reserved sectors following this boot record.

Usually FAT tables are duplicated to help provide a means of data recovery should one table become corrupted. Although very few people ever use the second table (and in fact, should either of the tables become corrupt, the host machine should stop reading/writing to the sd card and report a FAT mismatch error) it still takes up space and needs to be accounted for when working out where the data starts on the disk. Byte 16 is 0x02 - ie the FAT table is duplicated once; there are two working copies of the FAT system on our card - again, the expected value.

Maximum possible number of entries in the root directory is important - we can work out where the root directory is, and the data is stored immediately after it, so we need to know how many sectors the root directory takes up. Bytes 17 and 18 tell us that the max number of records in our root directory is 0x0200 or 512 in decimal.

The number of sectors used by each FAT table is in bytes 22 and 23.
In our case, we have the value 0x00ec or 236 in decimal.

From all this information we can deduce:

• Our boot record is at sector 135 (in your hex editor, this is sector zero)
• After 8 reserved sectors, the FAT tables begin at physical sector 143 (in your hex editor, these would start at sector 8 - we're going to stick with physical sectors to avoid confusion, but be aware that your sectors will be offset in the hex editor)
• Each FAT table is 236 sectors in size
• There are two FAT tables
• Our root directory starts at sector 135 + 8 + (2*236) = 615
• The root directory can hold up to 512 entries. Since this is FAT16 (each entry uses 16 bytes) this means our root directory takes up 512/16 = 32 sectors
• The actual data starts at sector 615 + 32 = 647

Our SD card already contains a few files.
Since we're going to (eventually) be playing audio, we put some RAW audio data on there (files imaginatively called wav001.raw, wav002.raw etc.) and a short text file called FAT16.txt

From our deductions, we run our "read-sector" routines on sector 615 and see what we get:
(your card may have different offset values for the boot record, reserved sectors, FAT table sizes etc, so use the equations above, but substitute with your card's values to get the actual sector to read from - your card may not have the root directory starting at sector 615. In fact, we're not sure that ours does yet.....)

Holy cow Batman, it looks like we've found our root directory........

Reading and writing to a single sector on an SD card with a PIC microcontroller

Having got our sd card to respond to the SPI initialisation routines and "boot up" correctly, it's time to actually start reading and writing data to the card.
The first thing to understand is that an SD card is made up from a series of sectors. Each sector is 512 bytes in size (almost always for SD cards) and the SD card likes to read and write entire sectors at a time.

not to scale ;-)

Reading data is a case of sending the appropriate "command packet" - 6 bytes which are:

• the command byte (CMD17)
• 4 parameter bytes (the address of the sector to read)
• two CRC bytes (an accurate crc value is not actually necessary in SPI mode, but two bytes still need to be sent)

It is possible to read multiple sectors of data, one after the other, and use the sd card like one massive serial eeprom, but for now, we're going to just work one sector at a time (this approach is much better suited to working with FAT16-based files, which we'll discuss later).

The command value to read a single sector of data is CMD17. After sending the command, we have to check for an "ok" response from the sd card then wait for the "not-busy" response (any non-0xFF value).

Boolean sdReadBlock(UInt32 sec){
     UInt8 v;
     UARTPrint("Starting to read block ");
     UARTPrintNumber(sec);
     UARTPrintLn(" ");
     
     v =  sdCommandAndResponse (17, ((UInt32)sec) << 9);
     if(v & FLAG_TIMEOUT) {
          UARTPrintLn("start read block command 17 timeout");
          fatal(1);
          return false;
     }else{          
          do{
               v = sdSpiByte(0xFF);
          }while(v == 0xFF);
          if(v != 0xFE) {
               UARTLog("Read block response",v);
               fatal(2);
               return false;
          }     
          
          curPos=0;
          return true;
     }
}

This function accepts a single 4-byte parameter, the sector address to open, and returns true or false to indicate whether the card responded correctly.
It also resets a counter curPos, which will keep track of the number of bytes we've read from the current sector on the disk. When we've had exactly 512 bytes back from the disk, we'll have to "close" the current sector by reading back the two bytes that make up the crc for the previous 512-byte data stream.

void sdSecReadStop(){     
     
     if(curPos<512){
          UARTPrint("add stuff bytes to close sector - ");
          UARTPrintNumber(curPos);
          UARTPrintLn(" ");
     }
     
     while(curPos < 512){
          sdSpiByte(0xFF);
          curPos++;
     }
     
     // read back the two CRC bytes
     UARTPrint("CRC ");
     r=readByte();
     UARTByte(r);
     UARTPrint(" ");
     r=readByte();
     UARTByte(r);
     UARTPrintLn(" ");
     UARTPrintLn("End read sector");
     UARTPrintLn(" ");
     
}

Now we can open a sector and close it again, the last piece of the puzzle is to stream the data back from the sd card and do something with it! When we come to make our audio player, we'll use the data immediately - sending it to the speaker - but for testing, or for more general purpose use, we'll put the data into a series of buffers and use it later.

Now - here's a thing.
So far, everything discussed can be ported to another platform (AVR/Ardunio for example) but when it comes to storing the data in our internal buffers, there's a peculiarity with the PIC microcontroller - or at least, with most compilers that support using arrays. It may be the same with Arduino and/or other compilers, but we've always found that large arrays are often difficult to use.
This is because of the way PICs store data in RAM - it uses "banks" to keep the data in memory, and one array can't usually span more than one bank of data. In practice, this means a limit of about 90 elements in an array.

To keep things simple, we're going to store an entire sector's worth of data (512 bytes) in 8 eight arrays of 64 elements.

unsigned char wavData1[64];
unsigned char wavData2[64];
unsigned char wavData3[64];
unsigned char wavData4[64];
unsigned char wavData5[64];
unsigned char wavData6[64];
unsigned char wavData7[64];
unsigned char wavData8[64];


unsigned char readSector(){
     unsigned short cy;
     unsigned short cv;
        
     r=sdReadBlock(iSector);               
     for(cy=0; cy < 512; cy++){
         r=readByte();               
         UARTByte(r);    //write to serial for debugging
                  
         if(cy< 64){
            cv=cy;
            wavData1[cv]=r;
         }else if(cy<128){
            cv=cy- 64;
            wavData2[cv]=r;
         }else if(cy<192){
            cv=cy-128;   
            wavData3[cv]=r;
         }else if(cy<256){
            cv=cy-192;   
            wavData4[cv]=r;
         }else if(cy<320){
            cv=cy-256; 
            wavData5[cv]=r;
         }else if(cy<384){
            cv=cy-320;   
            wavData6[cv]=r;
         }else if(cy<448){
            cv=cy-384;   
            wavData7[cv]=r;
         }else if(cy<512){
            cv=cy-448;   
            wavData8[cv]=r;
         }
     }           
                                        
     sdSecReadStop();     
     UARTPrintLn(" ");
     return(1);
}

Now we can read a sector-ful of data, it's time to write some data back.
There are any number of ways you can get data into the PIC/microcontroller in order to write them to the sd card. We'll leave that side of things for another post - for now, here are some functions to write data to a specific sector on the card.

The command to write a single sector-ful of data to the card is CMD24
(you can write an entire stream across multiple sectors if you're using the card as a large serial eeprom chip, but this makes things difficult if we're going to work with FAT16 formatted cards in future).

When writing data, the 6-byte data packet consists of:

• single command byte
• four parameter bytes (sector to write to)
• single crc byte
• a single byte representing a token confirming the command request

Boolean sdWriteBlockStart(UInt32 sec){
     UInt8 v;
     v =  sdCommandAndResponse(24, ((UInt32)sec) << 9);
     if(v) {
          UARTLog("write block start response",v);
          fatal(1);
          return false;
     }else{          
          UARTPrintLn("write block started");     
          // keep track of how many bytes we've written in this sector
          // (when we hit 512 we should expect some extra bytes in the packet data)     
          bytesWritten=0;     
          
          // send the correct token for CMD17/18/24 (0xFE)
          // REMEMBER the token for CMD25 is 0xFC               
          r=sdSpiByte(0xFE);
          return true;
     }
}

Once the card has responded with an "ok" after starting to write a sector, the next bytes streamed to the card are written to the selected sector. We need to keep track of the number of bytes written - when we hit 512, it's time to close the current sector, read back the crc bytes, then open another - you don't have to write to the sequentially next sector every time, you can write to any old location if you like!

Boolean sdWriteByteToSector(UInt8 b){     
     UInt8 ix;
     r=sdSpiByte(b);          
     bytesWritten++;     
}


Boolean sdWriteToSectorClose(){          
     // finish closing the sector
     UARTPrintLn("writing stuff bytes to close sector");
     while(bytesWritten<512){
          sdSpiByte(0xFF);
          bytesWritten++;               
     }
          
     // send two CRC bytes
     sdSpiByte(0x00);
     sdSpiByte(0x00);
          
     // response should immediately follow
     // (for writing, we're looking for 0bXXX00101 data accepted)
     r=sdSpiByte(0x00);     
     UARTLog("write finish response",r);
     
     // now wait for the sd card to come out of busy
     while(r!=0x00){
          r=sdSpiByte(0x00);
     }
          
     UARTPrintLn("write finished");          
}



// write some data to sector numbered five:
void writeSomeData(){
    UARTPrintLn("Writing one block");
    ret=sdWriteBlockStart(0x05);          
    if(!ret){
         fatal(3);
    }else{

         for(iy=0; iy<=255; iy++){
              sdWriteByteToSector((255-iy));
         }                    
     
         sdWriteByteToSector(0x10);
         sdWriteByteToSector(0x11);
         sdWriteByteToSector(0x12);
         sdWriteByteToSector(0x13);
         sdWriteByteToSector(0x14);                    
                    
         sdWriteToSectorClose();                         
    }
}

That's pretty much it.
We can now read and write to an individual sector on the sd card.
There are commands for reading and writing to/from multiple blocks at a time, but we'll leave those for again; for FAT formatted cards, we need to be able to read data out-of-sequence and possibly even write files into different (non-consecutive) sectors.