A little corner of the Empire on the web.

15 July, 2013

Kindle Weather Display pt4 Putting it all together

By this point, if you've followed the previous posts in this series, you should have a server that's downloading weather info from a weather service, and converting that into a Kindle friendly PNG image, and you should have a Kindle that's been jailbroken, has working USB Networking (and SSH access) and has the Kite application launcher installed on it. Congrats, now let's put it all together and finish this crazy thing off.

Setting up the Weather Display on the Kindle


Find the "kindle" folder in the master.zip file you downloaded previously, or get the files from the "kindle" folder on GitHub.

Copy the "init-weather.sh" file into the "kite" folder on the Kindle. Then create a new folder called "weather" and copy both "display-weather.sh" and "weather-image-error.png" into it.

Open up the "weather\display-weather.sh" in a text editor (the same one that understands Unix line endings from above) and change the line that contains "http://server/path/to/weather-script-output.png" to the web server and file location that you set your weather script up on. When you save the file, make sure that you select Unix style line endings (this will probably be an option on the Save As dialog). Safely Remove and unplug your Kindle.

Now you're ready to test everything, you should see a new "PDF book" on your Kindle's screen called "init-weather.sh" if you don't, then switch your Kindle off and back on again. Select that and your screen should flash a few times and then either show your current weather, or show a warning sign and "Weather Image Unavailable".

Either of those mean that everything's working on your Kindle and you've succesfully run a custom script on it, if you get the image unavailable message then check the web address you've put in for the picture is correct.

One other possibility here is that when you click the init-weather.sh your Kindle puts a loading circle in the top-left corner for a moment, and then flashes back to the Home screen, this probably means that either init-weather or display-weather have been saved with Windows line endings (some decompression/un-zipper apps "helpfully" translate the line-endings for you when you extract text files, yours may have done this) just open the files up in a text editor and make sure that they're saved with the Unix line endings option select and try again.

One thing you'll notice when you've successfully got the weather scipt running is that there's only one way to exit it, and that's to hold down the power button on your Kindle for 20 seconds or so and wait for it to reboot. While that's fine when everything's up and running it can get a bit annoying while you're just testing things (like getting connected to your web server, or playing around with the look of the weather page), one alternative is to drop a test-weather.sh file into the "kite" folder containing:

#!/bin/sh
/mnt/us/weather/display-weather.sh

This should show the weather display on the screen for a moment, and then dump you back to the Kindle's home screen.

Set up Cron On the Kindle

This is it, we're nearly there now, just need to set the Kindle up to regularly check back with the server for an updated weather picture. Cron is how we do that.

Firstly, if you've never used Cron before, read up on Cron at wikipedia and follow links to further info if needed. I can also recommend searching for something like "crontab calculator" or "crontab generator" and using one of those to help build up your cron line, or running a search for "cron" on SuperUser.com.

Now you need to SSH into the Kindle again, you know the drill by now, switch on USB networking on the Kindle, plug it in and reconnect to it with Putty.

Once you're SSH'd into your Kindle, mount the file system for editing:
mntroot rw
Edit the CRON file in vi (a command line, text mode, text editor) (unbelievably *nix people like this thing and even fight religious wars over how much better it is than other text mode editors, like it's still 1980)(http://en.wikipedia.org/wiki/Vi)
vi /etc/crontab/root
A list of the jobs in the root cron file should appear:
*/15 * * * * /usr/sbin/checkpmond
*/15 * * * * /usr/sbin/tinyrot
*/60 * * * * /usr/sbin/loginfo tmpfs
*/60 * * * * /usr/sbin/loginfo localVars
*/60 * * * * /usr/sbin/loginfo memusedump
*/15 * * * * /usr/sbin/loginfo powerdcheck
All we need to do now is add the display-weather.sh script onto the end, mine's set to run every 6 hours, starting at just after midnight.
Try 10 */6 * * * /mnt/us/weather/display-weather.sh



Move the cursor to the end of the final line (using the arrow keys), press "i" to switch into "insert" mode, press Enter for a new line, and then type in your cron line, press Escape to leave Insert mode, and then press ":w" to save the file, and then ":q!" to exit vi. There, wasn't that straightforward and obvious! For more help with vi commands and a tutorial, consult your favourite search engine.

Your crontab file should now look something like:
*/15 * * * * /usr/sbin/checkpmond
*/15 * * * * /usr/sbin/tinyrot
*/60 * * * * /usr/sbin/loginfo tmpfs
*/60 * * * * /usr/sbin/loginfo localVars
*/60 * * * * /usr/sbin/loginfo memusedump
*/15 * * * * /usr/sbin/loginfo powerdcheck
10 */6 * * * /mnt/us/weather/display-weather.sh

And then you restart Cron, to read the new job details, by entering:

/etc/init.d/cron restart

Which should give an output like:
[root@kindle root]# /etc/init.d/cron restart
system: I cron:def:stopping crond
system: I cron:def:starting crond
[root@kindle root]#

Note: while you can set the Cron repeat interval to very small amounts of time when testing, I really wouldn't advise setting it to repeat any more often than every 5 minutes, as to get in there and change Cron's settings you need to wait for the Kindle to come on, turn on debug mode, turn on USB networking mode, SSH in and adjust the timings, all before the weather job kicks in again. That's not a quick process, you'll want to give yourself a few minutes.

Unplug, switch USB Networking off again, run init-weather.sh and enjoy your weather display.

And we're there!

The finished job, a Kindle showing today's weather
Some atypical English weather, showing on my Kindle.

Useful references and acknowledgements:

Thanks to Matthew Petroff for the inspiration and for putting in all the hard work to get all of the original scripts together and get this working on a Kindle in the first place.

Todo

Thoughts for future changes to this:
  • Wind - being a sailor, the wind conditions can be a useful thing to know, a couple of the weather feeds include this, either by default or as an option, so it should be possible to include it with some tweaks to the vbscripts, a tweak of the SVG template, and some new icons.
  • New icons - while the icons included in Matthew Petroff's original collection are clean and clear, I'm quite fond of the old BBC weather icons - they're what I grew up with - a member of the design team for the BBC's new visual style has made a set of icons in the style of the old classics, available here, it shouldn't be much work to swap these in.
  • Two other personal projects that I'm planning to do at some point are learn PowerShell and rebuild my 2003R2 based home server into Server 2012 whilst recreating most of WHS's functionality, as part of this I may rewrite the server-side VBscripts as PowerShell scripts.

Update 07/2016: download links updated now that project files have been moved to GitHub

  1. Introduction
  2. Setting up the server
    • Weather Services
    • Getting the Image Tools
    • Scheduled Tasks
    • Download the Scripts
  3. Setting up the Kindle
    • Jailbreak your Kindle
    • Setting up USB Networking (1)
    • Setting up USB Networking (2)
    • Installing Kite on the Kindle
  4. Putting it all together
    • Setting up the Weather Display on the Kindle
    • Setting up Cron on the Kindle
    • Useful References and acknowledgments
    • Todo

12 June, 2013

Kindle Weather pt3 Setting up the Kindle

Back to the first post in this series.

Setting the Kindle Up


I'm doing this with a black Kindle 4 (non-touch) running Kindle v4.1.0, some of the following steps could change if you have a slightly different model (grey Kindle 4's may have older older firmware which give you more jailbreak options, they may also have the built-in SSH server enabled), I believe that most of this should work on a Kindle 3 (aka Kindle Keyboard) but I haven't tried that, so do it at your own risk.

Jailbreak your Kindle

I used the Universal method from here: MobileRead.com Kindle4NT Hacking.

Get SSH access to your Kindle (USB Networking) - attempt number 1


Plug in the Kindle and create a blank file named ENABLE_DIAGS in the Kindle's USB drive's root, "Safely Remove" the USB cable and restart the Kindle (Menu -> Settings -> Menu -> Restart).

It should reboot into diagnostics mode.

While you're in the diagnostics mode, plug the USB cable back in again and go to USB networking: Misc individual diagnostics -> Utilities -> Enable USBnet -> Exit. This will enable network access to your Kindle via USB.




Windows may fail to autodetect everything, if so you'll probably need to install a specific driver to access the Kindle over USB networking. This .inf file should do the job (I'm running Windows 7, Windows XP users need to do slightly more than this, good luck).

Once you have your driver set up, you'll need to configure the settings on your new network interface, set them to: IP 192.168.15.201 Subnet 255.255.255.0




Open your favourite SSH client (assuming that you have such a thing), I'm using Putty.

Try running Putty root@192.168.15.244 if that connects you're gravy, and your Kindle has a SSH server, if not and you get a "Connection Refused" then you're like me, so carry on.



Get SSH access to your Kindle (attempt 2, USBNet hack)


Download kindle-usbnetwork-xxx.zip from http://www.mobileread.com/forums/showthread.php?t=88004 extract it and copy update_usbnetwork_XXX_k4_install.bin onto your Kindle's USB drive. Safely Remove, unplug, then go into Settings, Menu, Update your Kindle. USB Networking should now be installed.

Restart your Kindle (not strictly necessary, but the Readme's recommend it).

Plug your Kindle in, and double-check the settings in D:\usbnet\etc (you'll need to use a text-editor that understands Unix line endings, Notepad won't work here, Notepad++, TextPad and others will work). In particular make note of the IP addresses, by default mine were set to:
HOST_IP=192.168.2.1
KINDLE_IP=192.168.2.2

On a Kindle 4, it's recommended to check that USE_VOLUMD is set to "true"

Following the README_FIRST.TXT from the usbnetwork zip:

  • Safely remove and unplug your Kindle,

  • You'll need to be in debug mode to run private commands, so, on the Home screen, bring up the
    search bar (by hitting [DEL] on devices with a keyboard, or the keyboard key on a K4, for example), and enter:

    ;debugOn

  • And now we can enable usbnet:

    ~usbNetwork

  • If you don't need to enter any more private commands, switch debug off.

    ;debugOff

  • Now, to actually connect to the device, we'll need to bring the shiny new USB network interface that the kernel prepared for us.

  • Plug the Kindle back in, wait for Windows to detect it, your Kindle should now be detected as something like a "RNDIS/Ethernet Gadget" or "CDC Ethernet Device network adapter" by Windows.

  • Then go into network settings, find the new network connection, and set the IPv4 settings to IP address 192.168.2.1 (assuming that's the HOST_IP that was set in your Config file above) and a subnet mask of 255.255.255.0.

  • To test it, open a command window and Ping 192.168.2.2 which should come back with a reply.

  • Now try Putty again:
    putty root@192.168.2.2

  • Hopefully this time you should get a connection and a warning message about the server's "host key". If so, say "Yes" to add the key to your cache, and then you should find yourself at a password prompt. A blank password works here, so you can just press Enter.
Congratulations, you now have SSH working. Careful, you can do a lot of damage here!

Install Kite (Kindle Application Launcher)

Now you need to install Kite, this lets you easily run custom programs on your Kindle. I tried installing Kite using the update package, with no luck, so here's installing it using the script method. First, download the kite.gz file from http://www.mobileread.com/forums/showthread.php?t=168270 and copy the kite.gz file onto the USB drive of your Kindle. If your Kindle is currently in USB Network mode, then you'll need to switch back to USB storage mode to copy the Kite installer onto it, as the usbnetwork command is a toggle, you can issue:
;debugOn

~usbNetwork

;debugOff
Once you've copied the files on, switch back to USB Network mode by issuing the commands again.
;debugOn

~usbNetwork

;debugOff
Plug your Kindle back into your PC again and test connectivity with a PING (depending what you've been doing in between, Windows may detect your Kindle as a new device again. If so, re-set the IP settings back onto the network adaptor as above, I went been to France for the weekend in between setting up USB Networking, so it happened to me). Connect to your Kindle's shell with Putty
putty root@192.168.2.2
Issue the commands to uncompress, and then run the Kite install script:
  • Mount the Kindle file system, type:
    mntroot rw

  • Move into /mnt/us:
    cd /mnt/us

  • Unzip the kite.gz file, and run the install:
    gzip -d kite.gz
    sh kite

Kite should now be installed, unplug your Kindle and then reboot it (Menu, Settings, Menu, Restart). As the reboot happens you may see a kite appear in the picture of the boy reading under a tree, or the land behind the tree may just flatten out, either are good signs. Once it's rebooted plug it back into your PC and check if there's a "kite" folder on the Kindle, if not create one (you may need to move/delete a file called "kite" first).

Next: Setting up the Weather Display on the Kindle


  1. Introduction
  2. Setting up the server
    • Weather Services
    • Getting the Image Tools
    • Scheduled Tasks
    • Download the Scripts
  3. Setting up the Kindle
    • Jailbreak your Kindle
    • Setting up USB Networking (1)
    • Setting up USB Networking (2)
    • Installing Kite on the Kindle
  4. Putting it all together
    • Setting up the Weather Display on the Kindle
    • Setting up Cron on the Kindle
    • Useful References and acknowledgments
    • Todo

Kindle Weather pt2 Setting up the server

Back to the first post in this series

Setting up the server


Matthew Petroff's original scripts assume that you're using a Linux server, they're made up of a BASH shell script (that won't do anything on a Windows box) and a Python script (that needs an extra install of 3rd party software to get it working on Windows), as well as relying on a couple of small tools that either aren't available to us, or don't work as well on Windows.

So to get things compatible with pretty much any Windows box, I've rewritten the shell script as a batch file and the Python script as VBScript, both of those should just work on anything from Windows 2000 upwards, and I've found some replacements for the small tools.

What you'll need for this section:

Create a new folder somewhere on your server (preferably outside of your web server's root) and download and extract the zip file into it. This should give you three folders, one of which is called "Server-VBScript", and in there are three folders called "ServerNOAA", "ServerWU" and "ServerMet" inside each of these you'll find an SVG, CMD and a VBS file. Now you need to decide which weather service you'll be getting your forecasts from, they all make their data freely available for non-commercial usage but have different coverage, may differ slightly in their forecasts of different areas and provide slightly different info, whichever one you want copy that service's folder into the new folder on your server:

  • NOAA - the US Government's NOAA's National Weather Service. The major downside (for me) is that they only provide forecasts for the USA.
  • WU - Weather Underground, run from a US University, they provide worldwide coverage, and in multiple languages
  • Met - Her Majesty's Government's Meteorological Office (The Met Office) forecasts. They only cover the UK (this is what I'm using on my Kindle)

Pick whichever weather service you want, and feel free to delete the other folders if you don't want them.

Weather Services


Depending which weather forecast service you pick, you'll need to find your location code, and possibly sign up for a (free) API key to use it, I'll go though these:

NOAA
All you need is to find the latitude and longitude of your location, and change these in the long URL inside the ServerNOAA\weather-script.vbs file using a text editor, also note that the final parameter in that URL "&Unit=e" sets it to the temperature in Fahrenheit, changing it to "&Unit=m" will output the temperatures in Centigrade.

WUnderground
You will need to sign up for a developer account at Weather Underground, and then you can create a (free for the usage level that we'll be putting it to) API key to access the weather reports. Once you have that you just need to work out your city's code, acceptable formats are:
CA/San_Francisco | US state/city
60290             | US zipcode
Australia/Sydney  | country/city
37.8,-122.4       | latitude,longitude

Then open up ServerWU\weather-script.vbs file using a text editor, find the line with the long URL in it and replace "APIKEY", "COUNTRY" and "CITY" with your API key and location.

Met Office
As with WUnderground, you'll need to sign up for a free API Key to use the Met Office's forecasts, once you have that you can visit the following URL to get a list of the location codes that you can use:
http://datapoint.metoffice.gov.uk/public/data/val/wxfcs/all/xml/sitelist?key=API_KEY
Take the 6-digit location code from there, and your API key, and plug them into the long URL in the ServerMet\weather-script.vbs file using a text editor.

If you've got everything set up right, then when you double-click the VBS file it should create an SVG file in the same folder, showing your graphical weather forecast. You can view this SVG file with most web browsers.

Getting the Image Tools

(Side-story, looking for a suitable replacement for the RSVGconvert *nix tool to do command-line SVG to PNG conversions I went straight to my old, faithful ImageMagick, but every version of that and every option I tried seemed to choke on the embedded icons, eventually I came across Inkscape and discovered it had a decent command line interface, so here we are)

Next download Inkscape SVG editor and extract it into an Inkscape folder in your server's weather folder.

Then download PNGCrush and save the file in your server's weather folder.

Your folder structure should now look a bit like this (the "public" folder won't have been created yet):


Then go into the folder for the weather provider you're using, open the CMD file and edit it to reflect the paths of all these tools, and change the references to the "public" folder to point to the location of the folder within your web server's root that will be serving up the PNG image.

Now try running the CMD file in your chosen folder by double-clicking it, and with luck after a few seconds your 'public' folder should now contain a PNG image that looks a bit like this:

typical English weather

Scheduled Tasks


Finally you'll need to decide how often you want your weather image to update and set up a Scheduled Task in Windows to run the relevant CMD file as often as you want it. Personally I'm going to have my Kindle updating its weather display 4 times a day, so I'm setting my scheduled task to run every 6 hours.


Download the Scripts

  • GitHub project: all files for the project browsable online, and individually downloadable
  • Entire project master.zip (72KB zip compressed file): if you don't know what a GitHub project is, use this link to download everything needed in one go.


Update 07/2016: files and download moved to GitHub, all previous updates merged in.



Next: Setting up the Kindle.


  1. Introduction
  2. Setting up the server
    • Weather Services
    • Getting the Image Tools
    • Scheduled Tasks
    • Download the Scripts
  3. Setting up the Kindle
    • Jailbreak your Kindle
    • Setting up USB Networking (1)
    • Setting up USB Networking (2)
    • Installing Kite on the Kindle
  4. Putting it all together
    • Setting up the Weather Display on the Kindle
    • Setting up Cron on the Kindle
    • Useful References and acknowledgments
    • Todo

09 June, 2013

Kindle Weather Display (from Windows)

I admit it, I'm interested in the weather. What can I say? I'm an Englishman and a sailor, I need something to talk about, and some way to know whether it's shorts weather, or if I need to take a brolly to work. Add in the fact that I love tinkering with gadgets, and have a mini server box sitting in the corner of my living room that spends large parts of the day doing very little, then I was very interested when I ran across Matthew Petroff's Kindle Weather Display blog post.


The only problem was that although he has all of the files available for download from that page, and a barebones hint on how to use them, it assumes a certain familiarity with Linux and its tools, and assumes that you're using a *nix server for the web server, and a *nix machine to do the install from, much of the Kindle hacking needed as a pre-req makes similar assumptions. I've barely touched Linux in the last 10 years (why deliberately make life hard for yourself when there's better choices out there?), was planning to set this up on my Windows Home Server box, and to do it all from my Win7 laptop. Clearly I was going to need to rewrite a few things, and work a few things out.

This is my tale of how that was done, so that you may read it, and do the same yourself.

To accomplish this you're going to need a few things that most people apparently don't have lying around their homes for some reason:

  1. A Kindle to dedicate as the display - while you could use the same Kindle that you use to read in bed at night, or on the train in the morning, that really won't be very convenient. Beware also that we're going to be hacking the Kindle and doing some (potentially harmful to the Kindle) things that will not be covered under your warranty. That said, if you hunt around you can find Kindle 4's for a fraction of their retail price on eBay, the one I'm using is an ex-shop demo unit that I got for a hair under £20.
  2. A Windows server (or at least a Windows box, doesn't necessarily have to be server hardware or a server OS sku) that will be switched on all the time, with an "always on" internet connection (the server itself doesn't need to be accessible from the internet). I have an old Tranquil PC SQA-5H that I use to run my home services, so that'll do me.
  3. A web server installed and running on your Windows server - all we need is a way to serve static PNG images over HTTP, so you won't need anything fancy. Personally, I'm using a Windows Server SKU, so I'm using the built-in IIS web server but pretty much any web server will do, Apache is probably the next most common, and I've used that for projects in the past.
  4. A basic plain text editor that understands the difference between Windows and Unix line endings.
    MS Word absolutely won't do, that's a Word processor not a text editor, Windows Notepad is no good either as it can't handle Unix style line endings. Something like Notepad++ or TextPad (which I'm using) would be perfect for this, many programmer's IDEs (such as Komodo, Eclipse or Visual Studio) also include this if you already have one installed, but don't go installing an IDE just for this as it really would be overkill.
  5. A basic familiarity with Windows and basic, common Windows tools - I'm happy to walk you through the Unix/Linux tools and commands but I'll expect you to know what you're doing on the Windows side

Warning: this is going to be a long one, so I'm dividing it up into a few parts over a few days, click any of the links to skip to parts you're interested in, or read the whole series to do it all. You can do some of the following steps in different orders, but I found it easiest to start with setting up the server while I was still searching eBay for a cheap Kindle.

Next: Setting up the server.


  1. Introduction
  2. Setting up the server
    • Weather Services
    • Getting the Image Tools
    • Scheduled Tasks
    • Download the Scripts
  3. Setting up the Kindle
    • Jailbreak your Kindle
    • Setting up USB Networking (1)
    • Setting up USB Networking (2)
    • Installing Kite on the Kindle
  4. Putting it all together
    • Setting up the Weather Display on the Kindle
    • Setting up Cron on the Kindle
    • Useful References and acknowledgments
    • Todo