Wednesday, June 7, 2017

Taken by a Strange Mood - Is any project really "finished"?

I'm back.  I can't stay away.  I can't leave well enough alone.

First things first, I need to catch you guys up.  There was one more major modification that I needed to make to the Pi/fan interface to make everything actually work.  You see, when I purchased the fan that is now permanently glued inside the pi case which is also permanently glued inside the TV chassis, I didn't take enough care to note that it pulled something like 250 mA when operating.  So my nice brilliant fan control script worked, but because I had the fan wired up to the 5V GPIO pin, it would draw its 250 mA from the Pi computer itself.  What this resulted in is emulator and playback slowdown whenever the fan came on, because the Pi would automatically throttle back its clock speeds when it detected an undervoltage due to too much current being drawn.  Long story short, my fan was pulling too much current from the rest of the pi, and I had to wire in a separate 5V power source to power the fan alone.

I used a voltage regulator connected to my extra 12V pins that come off of the TV power supply. (If you'll recall, I have 4 pins on that connector, 2X 12V and 2X ground).  I had a lot of problems with intermittent fan function after doing this, and extra loud fan, and it all came back to a wiring problem with these 4 pins.  So I had to remake the connector with better crimped terminals so I could get consistent connection.

So, fan being powered technically by a separate source, my power issue when fan on problem went away... mostly.  I'm not sure exactly what is going on (because I can't take voltage measurements while the thing is in "real world" use), but every few seconds I'd still get my undervoltage signal as well as emulator slowdown/stutter.  It was never consistent, and completely intermittent.  My power supply is rated at 5V, 3.5 amps, so it should be adequate, but I have no way of testing it to verify that it can actually supply this.  Ultimately, I decided to ignore the warnings and disable the throttle back at undervoltage.  I added these lines to my /boot/config.txt file.


These lines disable the on screen undervoltage warning icon, and ensures that turbo mode stays on, as well as increasing some network performance to ensure we get good streaming playback from my media server.

So, that gets you caught up to where I am today.  The TV has been in use and deployed in the kids play room for months, with hardly a hiccup.  I've been doing everything I can to optimize the software experience.  The Pi boots into Kodi 17.3 (as of writing), with PlexKodiConnect addon installed.  This allows me to use my Plex smartphone app to queue up and play movies on this TV, just like a chromecast.  We also use the Kodi remote app Kore, which allows us to also share youtube videos to this TV from a smartphone without having to navigate the on screen display.  I'm patiently waiting for the official Plex for Kodi addon to mature enough to the point that I can use it exclusively (still needs the plex companion, and support for Plex addons before I can do this).

I've also been messing around with the RetroPie half of this device.  Specifically, I'm trying to find a frontend for all the roms on the server that's quicker than EmulationStation.  ES is fine for small libraries, but it takes a long time to startup if you have large ones.  I tried out Mehstation, but found that scraping artwork and game info to be too much of a chore.  Now I've settled on Attract-Mode, and have almost finished getting things set up.  Its functional, but not pretty.  I have a lot of missing artwork that doesn't make any sense to me.  EmulationStation scraped artwork beautifully, and it also filled in intellegent human readable names for all my MAME games, and it properly scraped artwork. Neither Attract-Mode or Mehstation matched it, and I'm not sure what's going on.

Anyway, while working on this software problem I ran into the last limitation caused by my hardware setup.  I had the wife accidentally turn off the TV while I was working on it remotely, and then I had to wait 5 minutes or so for the supercapacitors to drain down before I could power it back on. A minor annoyance, but one that I've determined to fix.

I read an article about hooking up a proper on-off switch to the pi, and how one guy accomplished an auto power-off after pi shutdown using a secondary microcontroller.  Before I knew it I had one on order.  It comes tomorrow.  The plan is to sense a pin on the pi that will go from high to low at shutdown, and then use a transistor to cut the power from the capacitors.  This will effectively "unplug" the Pi after it has finished its shutdown process.  I'll then sense another pin on the microcontroller connected to the power supply that will turn the transistor "on" at power on, allowing the pi to boot.

I wish I had known about this solution before, because with this setup I actually could have used an off the shelf phone battery backup instead of my supercapacitors, but we're here now.  I'll post another blog with pictures once I'm done.  Until next time...

Tuesday, February 21, 2017

Taken by a Strange Mood, Part 5 - The shutdown problem

We're going to call this blog post "The Shutdown Problem".  The Raspberry Pi, though small in stature and large in utility, is in fact a full linux computer.  That means that all the things that come with a full linux computer apply, included the need to shut the computer down before pulling the plug on the power.  This isn't normally a problem because most normal people just have their pis plugged into a wall outlet until they've actually done a manual shutdown.  I don't have this luxury.  My Pi is powered by the TV itself, and when I hit the off button on the TV, so goes the Pi.

This is a problem.  In my setup, there is no guarantee of a safe shutdown.  This TV is ultimately going to live in my kids' playroom, so they're just going to be hitting the off button when they're done with it.  If I don't do something about this problem, we'll eventually get file system corruption, and I'll have to start the whole process over again.  Fortunately, there is a solution.

I'm going to take this moment to apologize for not having this blog post up sooner.  I believe its been nearly a month or more since my last post.  Its been that long because its taken me that long to get my elegant solution actually working.  I went through many trials, more errors, and intermittent success before working out all the kinks.  And now that I have, I can officially call this project "finished".

So, where to start?  I guess we'll define the specific problem.  When the power goes off, my pi goes off, without a shutdown.  How do we fix it?  Shutdown at power down.  But we can't complete a shutdown at power down because power is already down.  We'll need some power stored to keep the pi alive during shutdown, and we'll need a way to sense when power has been cut in order to initiate the shutdown automatically.

Solution#1:  In line rechargeable battery.  You know? One of those big secondary batteries that you get to charge your iPhone when you don't have a wall to plug into.  Fully packaged, simple, easy.  It also won't work.  When a Pi shuts down, it isn't doing a true shut down, its doing a system halt.  Effectively the same thing, but different enough to give me trouble.  Once a pi is halted, the only way to unhalt it and cause it to boot is to pull the plug, or hit the reset pins.  Unfortunately, I didn't know about the reset pins until after I had already permanently glued my pi into the TV.  I couldn't reach them.  So since we can't boot without pulling the power, that means I wouldn't be able to start up the pi again until the battery completely died. With a battery the size of the ones available, even the smallest ones, that could take hours, if not days or weeks.  We need a better solution.

Solution#2, provided by the nice folks at the adafruit forums: create a supercapacitor circuit and place it in line with the power supply.  I'm not going to go into the nitty gritty of capacitors, but suffice it to say that they are capable of storing energy, and then releasing it when the charging voltage is removed.  This is a much better solution because capacitors charge relatively quickly, and more importantly for my application, they also discharge relatively quickly.  After I did some math and estimations I determined I needed about 10 farads worth of capacitance, at 5 volts.  Oh, and the capacitor also needed to be able to discharge with a current of at least 1.5 amps.

I jumped onto the internet, to get buying.  I found one that I thought was perfect.  A single 5V capacitor that could store like 15F.  I bought it.  It came in.  I wired up my circuit.  I plugged it all in.  I let it charge.  I pulled the plug. It didn't work.  I double checked the data sheet: this capacitor was only capable of supplying 35 miliamps.  Back to the internet.  This is when I hit a snag.  I could not find a single capacitor rated at 5V, 9-15F, and also 1.5 amps or more.  But I did find capacitors at 2.7V, 30F, and like 9 amps (it was alot, more than I would draw).  This complicated the circuit, because now I'd need two capacitors in series.  Caps in series add their voltages, just like batteries.  But they also half their capacitance. (Actually, its a formula. 1/C = 1/C1 + 1/C2 + ...)  The other problem with using more than one capacitor is that, due to manufacturing tolerances, no two capacitors are exactly alike, and if they are different enough they will be imbalanced, and eventually one or both of them will break.  So I needed a balancing circuit.

Behold My Circuit Diagram. Yes, I know my diode symbol is backwards.
Another problem with capacitors is that when they are completely discharged, they look like a short circuit to a power supply.  And most supplies freak out when they think they've been shorted.  Some burn out.  Others just cut their supply so as to not burn out.  This is what mine does.  How to get around this?  You have to put a small resistor in front of the capacitors so that it no longer looks like a short.  It has to be small because the resistors themselves reduce the amount of charging voltage available to the caps, thus reducing the maximum charge I can put into the caps.  So 1 or 2 ohms is sufficient.  Unfortunately, the smallest resistors I had at the time were 100 ohm.  And I discovered that this was way too much resistance.  What's a guy to do?  I took all 11 of my 100 ohm resistors, twisted their ends together, and put them in parallel.  This produced an equivalent resistance of just under 10 ohms.  This worked much better, but was not ideal.
Actual Circuit.  The two large caps are connected, the smaller one isn't.

The other problem with these resistors and capacitors was that if the resistors were just put straight into the charge lane, they'd also be in the way of the discharge lane, and reduce discharge voltage, giving me headaches.  The solution to this is to put a diode in parallel with the resistors.  The diode prevents current from skipping over the resistors when charging, but allows current to flow around the resistors when discharging.  You see, capacitors discharge in the reverse direction of charging.

So, after much trial and error, I finally had a circuit put together that actually worked.  Mostly.  See how my large caps are bent over in the picture.  Turns out this caused me a world of pain that I wasn't able to correct until today.  You see, when I bent the pins on the caps like that, I accidentally damaged whatever internal connections there were.  Damaged, but not broken.  So they worked... sometimes.  It was very frustrating, and I blew up 2 different voltmeters while trying to figure out just why I was having the trouble I was having.  Anyway, here's the final circuit in place.

I'm skipping over a huge part of this, because its less interesting because it worked the first time, every time.  The method for sensing a power down was super easy.  I picked of a "voltage regulator".  The specific regulator I got takes a 5-15 volt signal, and converts it to 3.3V, which is the logic level of the raspberry pi GPIO pins.  I connected the regulator straight to the original 12V supply (before my 5V converter), put a large (1000ohm) resister in series with the 3.3v signal, and connected it to a GPIO pin.  Then, I have my fan control script watch that pin, and whenever it goes from on to off, we initiate a shutdown.  Here's how it looks.  You can also see my fan control transistor in this picture too.

That was the easy part.  Here's how the whole thing looks now that its done.

So, it was quite a struggle, but I persevered, and I was all done.  Or so I had thought.  After I tested it, I put everything back together, hooked it up, and it worked great.  All the emulators running on the Pi ran without a hitch, and when I killed the power I could verify from the activity lights that it kept alive through the shutdown.

And then one day, it didn't.  I was getting some software set up, and I noticed that the standard raspberry pi low power warning was constantly popping up on my screen.  "Huh", I thought.  I didn't think my little circuit would pull that much current, especially not after it was charged up.  I turned off the power and checked the activity lights.  Dead.  Nothing.  No shutdown.  Something was broken.  Turns out, I broke the capacitors when I bent the connectors, because the bend point was actually just inside the cap, instead of outside.  Oops, rookie mistake.  I bought two more this week, and just got finished installing them.  They charge up faster, discharge consistently, and no more low power indicator.  Thank goodness.

I hope you all enjoyed this little technical journey.  Here's some pictures of the finished product again.

If you want to see ALL the pictures from this build, I've linked them on a google plus album here.  Thanks for following!

Monday, January 2, 2017

Taken by a Strange Mood, Part 4

What do you do when you've permanently affixed a loud fan to your new homemade smartTV?  How do you quiet it down?  How about with lower fan speeds thanks to the new software PWM module in RPi.GPIO for python.

Do what now?

So we have all these extra pins on the Raspberry Pi. These pins are known as GPIO or "General Input Output Pins".  We can do some cool things with these pins in software.  One of the things we can do is, with a python script, control whether a pin is a sensor for 3.3v, or if we're sending a signal of 3.3v on each pin.  There's also another cool thing you can do, and that is send a variable voltage across a pin through a neat little trick called Pulse Width Modulation (PWM).  PWM emulates a variable voltage by pulsing the maximum voltage (3.3v in this case) at a high frequency.  In theory this will allow me to control my fan speed through code.

I say in theory though, because what I have is a 5 volt fan, and the GPIO pins only output 3.3 volts.  I tried hooking up the fan to the GPIO pins directly, but 3.3v apparently isn't enough to get the fan spinning.  It NEEDS a 5 volt pulse to go.

Well, now what am I supposed to do?  I can't control the fan directly because I can't pulse a high enough voltage to get it to spin up.  And I can't control the 5v pin, because its connected directly to power.  I can't interface with it through python code.  Enter the Transistor.

How transistors work is beyond my knowledge.  All I know is how to wire one up to do what I needed it to do. Basically, when wired properly, a transistor works kind of like a relay switch.  It has 3 pins, a base, a collector, and an emitter.  When a voltage is applied to the base, it connects the collector pin and emitter pins.  When the emitter is ground, and the collector is my fan in series with its 5 volt supply, it acts as a switch that I can control with a 3.3 volt pin.

Back to the internet!  I ordered a pack of PN2222 transistors, some 5050 LED breakout PCBs, and some assorted wire for me to solder to.  I followed along with this tutorial in order to figure out what needed connecting where.  After some fine wire work, soldering and clipping of legs, we have my finished transistor module:

All that remained was putting it into the case. Unfortunately I don't have any pictures of what it looks like in the case.  I'll be diving in later to make another modification later this week, so I'll try to get a picture then.  Suffice to say, it's become a bit of a mess.  BUT, I have the ability to control my fan speed programmatically through python.  BEHOLD MY CODE!

Sorry for the pictures.  If you need to copy it for your project I have it here.  There are a couple of extra lines of code in there for the next phase of this project, which I will detail in part 5.

So a basic explanation is that we import the modules we need at the top, followed by initializing some constants like what pin is connected where, how much I want the fan speed to step up or down as a percentage (that's your duty cycle, percent of time the pulse is on vs off).  Then we define a few functions to get the temperature in a usable format, write temperature and fan speed to a file, and then run the meat of the code in a loop.  Every cycle, we check the temperature.  If its above my set max temperature, we'll increase the duty cycle by the "step" amount.  If its below the minimum temp we decrease the duty cycle.  This arrangement allows the temperature to sit between the low and high level with the slowest fan speed possible, minimizing fan noise.  Then we wait (sleep) for a period of time and do the check again.  I can tell you, after having run this for a number of days now, it works great, and its very slick.

In part 5, we'll talk about what happens when you hit the off button, and what SHOULD happen, and how we make what SHOULD happen actually happen.  Seeya later.

Taken by a Strange Mood, Part 3

As I intimated at the end of Part 2, there were problems that extended this project.  Sure, I had a working embedded Plex image running on the Pi, but I had more ambition for that.  I downloaded and burnt the latest RetroPie image to an SD card and got that started.  The original plan was to have 2 SD cards: one for games and emulation, the other for movie watching.  Then I found out that Kodi could be installed under retropie, and that there was a brand new Plex plugin for it.  Once I discovered that, I stopped using the Plex image altogether, and have stuck with retropie.

This is not a blog post about setting up retropie, however.  It went pretty standard as long as you follow along with their instructions on their page, so I won't address that.

No, the problem I started seeing almost immediately is app crashes, freezing, and unexpected behavior.  The IC on the Raspberry Pi 3 is rated at a max temperature of 80' C.  When I checked my temperature, even at idle, I was running mid 70s.  Yipes.  Too hot!

Even though the RPi shouldn't need active cooling under normal circumstances, I was not operating under normal circumstances.  Ambient temperature inside the TV was affected by the TV electronics heating up, so the pi could not passively cool effectively.  I needed a fan.  At 5V, 0.2A max, this fan seemed ideal.  So I bought it.  It can plug directly into the 5V rail of the GPIO on the pi, and connect to ground. There are 2 pins on the GPIO that seem like they were designed for just this, pin 4 and 6.  All we need to do now is open up my case, and install the fan.  Sounds easy, right.  Well it would have been easy, except that I had ALREADY permanently affixed the thing inside my TV, so I couldn't remove it to get the top of the case off, or do anything else much with it.

Fortunately, the case didn't snap closed, and there was just enough wiggle room for me to be able to fit the small fan into the gap I could create between the top and bottom of the case.  Here's the other problem: no vents in the case.  Time to get out my drill!
I picked the closest sized coring tool I had in my drill bit set, lined it up, and went to work.  Even at slow speeds the drill didn't so much cut through the plastic as it melted through.  My plastic chips were all melted together, and some of them are still stuck inside the case.  Now came the hard part, getting the fan in there, and also mounting it to the inside of the top half of the case.

Case with fan hole cut.
I ended up using two small allen wrenches as feelers to position the fan under the hole.  Got another JB weld mix going, and then tried to hold it in place for 15 minutes. That didn't happen.  I was shifting the fan all around as I tried to hold it steady.  I needed a more permanent temporary way to hold the fan in place while I waited for the epoxy to cure.  In came my good old friend, Masking Tape.
Pull up, apply masking tape, hope it doesn't fall

Now that I had the fan in place, I needed to do something about the metal plate that covered the Pi here.  You see, a fan doesn't do you very much good if it doesn't have any access to ambient air to blow.  I sized up the location of my fan relative to the edges, and laid out my new hole pattern I was going to drill.

Hole Pattern

Drilled holes.

Holes in plastic cover
I think I messed up the label.

By the time I was finished drilling holes, my JB weld finished setting, and it was time to bring it all together. I plugged in the fan into the two pins I mentioned before  Closed the whole thing up, and turned it on.  Fan spun up.  It sounded like the blades were brushing something, because it was loud, and getting louder.  Even completely covered up by the case, running at 10k and brushing something with each blade on every revolution, this thing sounded like a small dustbuster.  This was No Good for a media smart TV.  I opened the thing up again to see if I could find what we were running into.  Turns out that this particular fan doesn't keep its wires away from the fan blades very well, and the blades were brushing the wires.  This took lots of finagling over several attempts to get the fan where it wasn't hitting itself when it spun up.  But even then the sound was very loud.

I checked my temps, and this thing was running cool.  I want to say it was less than 55' C under full load with the fan fully spun up.  So it was doing its job, just at a audible level I was not happy with.  Remember how I used JB weld again, and this thing was now a permanent resident?  Yeah, time for some more engineering.

I almost forgot to mention.  Before all of this I was getting a lot of noise out of my speakers.  Anytime something would be busy on screen, or there was disk activity, I'd hear a little noise.  Turns out the AV cable I used wasn't of the highest quality, and in addition the way I had my power supply connected meant that my Pi wasn't getting properly grounded.  I fixed that (mostly) by adding a wire from a ground GPIO pin to the case.  No, that solder joint didn't hold, but the wire is sandwiched between the side plate and the top plate, so it stays in place.  And my noise was reduced significantly (but not totally).  Too bad that AV cable was now a permanent resident.

In part 4, you get to find out how I dealt with my obnoxiously loud fan.