Saturday 24 December 2011

Beagleboard Ubuntu Backup

Overview
I use my Ubuntu 11.10 based Beagleboard as a low-power home server, which mainly stores my subversion repositories for my mobile apps development business. The server is headless and remains plugged in 24/7.
So I wanted to have some sort of cloud backup system, where my repositories and some parts of the Linux filesystem (mainly /etc) were backed-up to a remote location. At most I wanted to backup about 150MB of data.
After much searching around, I came up with the following two lists of possible solutions for backing up a home linux server:
Free:
  • Dropbox - Obvious choice given that your backups would also be accesible on various different platforms, but unfortunately they don't officially support an ARM CLI client (there have been some people who have been able to hack their packages to get them working, but I didn't want the hassle). You can login to Dropbox and vote for ARM support here.
  • UbuntuOne - This is available for ARM, but requires a GUI interface to manage, see here. There are ways around this, as described here, but I want a simple solution.
  • Using the standard backup-manager tool to backup to your own online server (unfortunately I don't have access to one). For those that do, you can use either of the following methods for transferring your archive: FTP, SSH, S3 and RSYNC. From the user guide: "The FTP, SSH and S3 methods are dedicated to upload archives, using those method depends on the use of at least one backup method. On the opposite, the RSYNC method uploads a directory to remote locations, this directory can be your repository or whatever other location of your file system".

Paid:
  • Amazon Simple Storage Service (Amazon S3), which the standard backup-manager tool can upload to. The pricing for this is flexible, and very very cheap for small amounts of data! To upload and store 1GB of data per month, the cost is less than $1.00!!! You can play around with the calculator here, be sure to select the "Amazon S3" tab along the left hand side. (Instructions for how to do this here.)
  • rsync.net - Min cost per month is $5.60 (7GBs)
  • Many other paid services, some with their own clients and others that can be used with the backup-manager package.

Which Solution?
Based on the lists above and the drawbacks of each possible solution, I decided to go with using backup-manager with Amazon S3. The backups are for my business's subversion repositories, so I don't mind spending a little each month.
You can follow the instructions I have created to setup backups from your Ubuntu based Beagleboard to Amazon S3 here.

Backing up to Amazon S3 from Ubuntu 11.10


These are the steps to setup a periodic backup directories on an Ubuntu 11.10 server to the Amazon S3 service:

WARNING: Some irony here, but make sure you backup your data before attempting this, there *shouldn't* be much risk, but I won't be responsible if you loose any data or incur any financial loss from following any of this.

1. Create Amazon S3 account & storage bucket
  1. Check you projected costs are acceptable here.
  2. Register with Amazon S3.
  3. Go to the Amazon S3 console and create a bucket for your backup.

2. Install the necessary software packages on Ubuntu
Run the following to install the packages:
apt-get install backup-manager libnet-amazon-s3-perl

3. Configure backup-manager
You configure backup manager by editing the /etc/backup-manager.conf configuration file. There are settings I have modified from the default, I haven't given the values for all of the config parameters as I recommend you think about these and use your own:
  • export BM_ARCHIVE_TTL="..." 
  • export BM_ARCHIVE_METHOD="tarball-incremental" - Note, this is recommended to reduce the amount of data transferred and stored on the S3 servers.
  • export BM_TARBALL_DIRECTORIES="..." - space separated list of the directories you would like to backup.
  • export BM_TARBALLINC_MASTERDATETYPE="..."
  • export BM_TARBALLINC_MASTERDATEVALUE="..."
  • export BM_UPLOAD_METHOD="s3"
Amazon S3 specific parameters of the config file, you will get the access and secret keys from the Security Credentials section of the Amazon S3 website:
  • export BM_UPLOAD_S3_DESTINATION="your_bucket_name"
  • export BM_UPLOAD_S3_ACCESS_KEY="..."
  • export BM_UPLOAD_S3_SECRET_KEY="..."
  • export BM_UPLOAD_S3_PURGE="false"

4. Test your settings
You can test your settings by running a manual backup (this will cost money, as you will be uploading to the Amazon S3 server):
backup-manager -v 
You can verify you backups were uploaded by looking at your Amazon S3 console.


5. Add the backup task to CRON
I decided to have my incremental backups run daily. So created a backup-manager script in /etc/cron.daily/backup_manager, containing with the following:
#!/bin/sh
/usr/sbin/backup-manager -v
Don't forget to make the script executable.

Conclusion
That's it, you should now have your backups setup and backing up to the bucket you created on the Amazon S3 service.

Disclaimer
I take no responsibility what-so-ever for any loss of data or financial costs incurred by someone following the info in this blog! Remember, each time you backup to the S3 service, it will cost you a certain amount of money.

Sunday 27 November 2011

Sleeping Arduino - Part 5 Wake Up Via The Watchdog Timer

Overview
Welcome to the fifth and final part of the "Sleeping Arduino" series, where we will cover how to wake the Arduino from sleep mode using the Watchdog Timer (WDT). When waking your Arduino from sleep, you could use one of the standard internal timers of an Arduino as I have detailed in Part 4, but if you are looking for the maximum sleep time and/or minimum sleep power consumption, you have the use the WDT;


As I have mentioned in this table, the WDT can give us a sleep time of 8 seconds, whereas the 'longest' 8/18bit timer will only give us a sleep time of ~4 seconds.


Watchdog Timer (WDT)
The Watchdog Timer on the Arduino's microprocessor only has one source to drive it: it's own separate internal 128kHz oscillator (as opposed to the 8/16bit internal timers, which can use either the 16Mhz system clock or an external clock). It is this separate oscillator that enables the WDT to function in the lowest power mode: SLEEP_MODE_PWR_DOWN (see here for more detailed info on the Arduino's power modes).

The WDT also has a prescaler, which is used to configure the timeout period. It supports timeout periods from 16ms to 8 seconds: 


Also it has three modes of operation:


  1. Interrupt - The WDT_vect interrupt sub-routing will be called when the WDT times out. Can be used to wake the micro from sleep modes, including the lowest power sleep mode (SLEEP_MODE_PWR_DOWN), where other timers are not available.
  2. System Reset - A watchdog time-out reset will occur, i.e. the micro-controller will be restarted. For use in handling code lockups.
  3. Interrupt and System Reset - First the WDT_vect interrupt sub-routing will be called, when completed a watchdog time-out reset will occur.






Please refer to my post here for the other operation modes of the WDT. The post is for the Atmega1281, but the functionality is pretty much the same.




Code


So, we will be using interrupt mode in our code to wake the Arduino from sleep every 8 seconds to toggle the state of the LED:


/*
 * Sketch for testing sleep mode with wake up on WDT.
 * Donal Morrissey - 2011.
 *
 */
#include <avr/sleep.h>
#include <avr/power.h>
#include <avr/wdt.h>

#define LED_PIN (13)

volatile int f_wdt=1;



/***************************************************
 *  Name:        ISR(WDT_vect)
 *
 *  Returns:     Nothing.
 *
 *  Parameters:  None.
 *
 *  Description: Watchdog Interrupt Service. This
 *               is executed when watchdog timed out.
 *
 ***************************************************/
ISR(WDT_vect)
{
  if(f_wdt == 0)
  {
    f_wdt=1;
  }
  else
  {
    Serial.println("WDT Overrun!!!");
  }
}


/***************************************************
 *  Name:        enterSleep
 *
 *  Returns:     Nothing.
 *
 *  Parameters:  None.
 *
 *  Description: Enters the arduino into sleep mode.
 *
 ***************************************************/
void enterSleep(void)
{
  set_sleep_mode(SLEEP_MODE_PWR_SAVE);   /* EDIT: could also use SLEEP_MODE_PWR_DOWN for lowest power consumption. */
  sleep_enable();
  
  /* Now enter sleep mode. */
  sleep_mode();
  
  /* The program will continue from here after the WDT timeout*/
  sleep_disable(); /* First thing to do is disable sleep. */
  
  /* Re-enable the peripherals. */
  power_all_enable();
}



/***************************************************
 *  Name:        setup
 *
 *  Returns:     Nothing.
 *
 *  Parameters:  None.
 *
 *  Description: Setup for the serial comms and the
 *                Watch dog timeout. 
 *
 ***************************************************/
void setup()
{
  Serial.begin(9600);
  Serial.println("Initialising...");
  delay(100); //Allow for serial print to complete.

  pinMode(LED_PIN,OUTPUT);

  /*** Setup the WDT ***/
  
  /* Clear the reset flag. */
  MCUSR &= ~(1<<WDRF);
  
  /* In order to change WDE or the prescaler, we need to
   * set WDCE (This will allow updates for 4 clock cycles).
   */
  WDTCSR |= (1<<WDCE) | (1<<WDE);

  /* set new watchdog timeout prescaler value */
  WDTCSR = 1<<WDP0 | 1<<WDP3; /* 8.0 seconds */
  
  /* Enable the WD interrupt (note no reset). */
  WDTCSR |= _BV(WDIE);
  
  Serial.println("Initialisation complete.");
  delay(100); //Allow for serial print to complete.
}



/***************************************************
 *  Name:        enterSleep
 *
 *  Returns:     Nothing.
 *
 *  Parameters:  None.
 *
 *  Description: Main application loop.
 *
 ***************************************************/
void loop()
{
  if(f_wdt == 1)
  {
    /* Toggle the LED */
    digitalWrite(LED_PIN, !digitalRead(LED_PIN));
    
    /* Don't forget to clear the flag. */
    f_wdt = 0;
    
    /* Re-enter sleep mode. */
    enterSleep();
  }
  else
  {
    /* Do nothing. */
  }
}



All parts of this series:



References

  • Martin's excellent tutorial here.
  • Darius's  excellent blog here.

Saturday 26 November 2011

Sleeping Arduino - Part 4 Wake Up Via Internal Timer

Overview
Welcome to part 4 of the Sleeping Arduino Series. In this entry we will cover how to get the Arduino to wake from sleep using one of the micro-controller's internal timers. This does not require any external hardware, all is implemented in code.

Arduino Timers
So,we need to go into a little detail on the micro controller's internal timers. The Atmega168 in the Arduino Diecimila has three internal timers:
  • Timer/Counter 0 - 8 bit (Max timer duration: 16.4ms)
  • Timer/Counter 1 - 16 bit (Max timer duration: 4.1s)
  • Timer/Counter 2 - 8 Bit (Max timer duration: 16.4ms)
Whether the timer is 8 bit or 16 bit defines how much the timer can count up to: an 8 bit timer counting to 256 and 16 bit to 65536. What drives the counter to count can be configured to be either the internal 16Mhz clock or an external clock source on T1 pin. We will be using the internal timer in this tutorial.
Once a timer's counter has reached its maximum value and increments once more, it will overflow and the counter will reset to zero. This overflow event can be configured to trigger an overflow interrupt, which we will use to wake the Arduino from sleep mode. Note you can also modify the value within a timer's counter from your code to tune the overflow period.

Figure 1. Timer source
One thing I haven't mentioned yet is the prescaler, this is another counter that is used in conjunction with the the timer counter to extend the time between timer overflows. The prescaler can be set the the following values: 1:1, 1:8, 1:64, 1:256, 1:1024, these are the ratios of prescaler overflow to clock cycles.
The following formula shows the calculation of the timeout period.


Figure 2. Timeout period formula

Take the following examples:

16 bit Timer1 no prescaler (1:1)
Overflow period =  1/16Mhz x 1 x 2^16 = 4.09 milliseconds
16 bit Timer1 with prescaler of 1:1024
Overflow period = 1/16Mhz x 1024 x 2^16 = 4.09 seconds  
So we can see that using the maximum prescaler value of 1:1024, we can get maximum Timer1 overflow period of 4.09 seconds, this is the maximum time we can put the Arduino to sleep for using Timer1. If you want a longer sleep period than this, you can use the Watch Dog Timer, which can provide a sleep time of about 8 seconds (it can also be used in a lower power/sleep mode than Timer1, see Arduino/Atmega168 Timers For Sleeping). Using the WDT for waking from sleep is described in the 5th part of this series.

Timers and Power Modes
As I have mentioned in part 1 of this series, not all hardware peripherals of the micro-controller are available in all power modes. Looking at our table here, we can see that timer1's lowest running power mode is IDLE, so when we put the micro-controller to sleep, we need to make sure we don't enter a sleep mode below this, or the timer will be disabled.

Control Registers
The following are the timer related control registers we will use to configure Timer1
  • TCCR1B - CS10, CS11 & CS12 - Timer control register B and the pre-scaler selection bits.
  • TCNT1 - 16bit counter register. This is the actual register that will count up each timer tick. When the value in this register rolls over from 65536 to 0, the overflow interrupt will fire.
  • TIMSK1 - TOIE1 - Interrupt mask register and  overflow interrupt enable bit. This register contains the control bits to enable the various interrupts available to timer1, including TOIE1 which we will set to enable the overflow interrupt.


For more information on these registers, see section 16.11 of the datasheet here.


Operation
Our code will operate as follows:
  1. Configure the serial port and LED pin.
  2. Configure Timer1's prescaler so the timer will expire every 4.09 seconds (TCCR1B).
  3. Clear Timer1's counter TCNT1
  4. Enable Timer1 overflow interrupt (TIMSK1).
  5. The main loop will then:
    1. Enter IDLE sleep mode.
    2. When the Timer1 overflow occurs, the interrupt will wake the processor
    3. The LED state will be toggled.
    4. Re-enter IDLE sleep mode.
Code
In the following code the Arduino board will go to sleep (SLEEP_MODE_IDLE) for 4.09 seconds, wake up and toggle the LED and go back to sleep again.
/*
 * Sketch for testing sleep mode with wake up on timer.
 * Donal Morrissey - 2011.
 *
 */
#include <avr/sleep.h>
#include <avr/power.h>

#define LED_PIN (13)

volatile int f_timer=0;



/***************************************************
 *  Name:        ISR(TIMER1_OVF_vect)
 *
 *  Returns:     Nothing.
 *
 *  Parameters:  None.
 *
 *  Description: Timer1 Overflow interrupt.
 *
 ***************************************************/
ISR(TIMER1_OVF_vect)
{
  /* set the flag. */
   if(f_timer == 0)
   {
     f_timer = 1;
   }
}


/***************************************************
 *  Name:        enterSleep
 *
 *  Returns:     Nothing.
 *
 *  Parameters:  None.
 *
 *  Description: Enters the arduino into sleep mode.
 *
 ***************************************************/
void enterSleep(void)
{
  set_sleep_mode(SLEEP_MODE_IDLE);
  
  sleep_enable();


  /* Disable all of the unused peripherals. This will reduce power
   * consumption further and, more importantly, some of these
   * peripherals may generate interrupts that will wake our Arduino from
   * sleep!
   */
  power_adc_disable();
  power_spi_disable();
  power_timer0_disable();
  power_timer2_disable();
  power_twi_disable();  

  /* Now enter sleep mode. */
  sleep_mode();
  
  /* The program will continue from here after the timer timeout*/
  sleep_disable(); /* First thing to do is disable sleep. */
  
  /* Re-enable the peripherals. */
  power_all_enable();
}



/***************************************************
 *  Name:        setup
 *
 *  Returns:     Nothing.
 *
 *  Parameters:  None.
 *
 *  Description: Setup for the serial comms and the
 *                timer. 
 *
 ***************************************************/
void setup()
{
  Serial.begin(9600);
  
  /* Don't forget to configure the pin! */
  pinMode(LED_PIN, OUTPUT);

  /*** Configure the timer.***/
  
  /* Normal timer operation.*/
  TCCR1A = 0x00; 
  
  /* Clear the timer counter register.
   * You can pre-load this register with a value in order to 
   * reduce the timeout period, say if you wanted to wake up
   * ever 4.0 seconds exactly.
   */
  TCNT1=0x0000; 
  
  /* Configure the prescaler for 1:1024, giving us a 
   * timeout of 4.09 seconds.
   */
  TCCR1B = 0x05;
  
  /* Enable the timer overlow interrupt. */
  TIMSK1=0x01;
}



/***************************************************
 *  Name:        enterSleep
 *
 *  Returns:     Nothing.
 *
 *  Parameters:  None.
 *
 *  Description: Main application loop.
 *
 ***************************************************/
void loop()
{
  if(f_timer==1)
  {
    f_timer = 0;
    /* Toggle the LED */
    digitalWrite(LED_PIN, !digitalRead(LED_PIN));
    
    /* Re-enter sleep mode. */
    enterSleep();
  }
}

Wednesday 23 November 2011

Arduino/Atmega168 Timers For Sleeping

The following is a table outlining the available timers of the Arduino, their max timeout period (how long you can put your Arduino to sleep for) and the lowest power saving mode that the timer can be used in. I hope it will help you in your choice of sleep timer.


Timer Size Max Timeout Period Min Power Mode
Timer0
8 bit
16.4ms
IDLE
Timer1
16 bit
4.1s
IDLE
Timer2
8 bit
16.4ms
POWER_SAVE
Watch Dog Timer
N/A
8s
PWR_DOWN


Note PWR_DOWN is the power mode with the least power consumption. Please refer to Part 1 Overview Of Arduino Sleep Modes for a description of the available power modes.


Details for the use of timers and watch dog for sleep control can be found in parts 4 and 5 of the Sleeping Arduino Series.


If you need a longer sleep period than 8 seconds, and don't want to manage a software timer that is ticked every 8 seconds, then you could consider an external real-time clock such as:
http://proto-pic.co.uk/deadon-rtc-ds3234-breakout/

Friday 18 November 2011

Beagleboard Ubuntu 11.10 Single User Mode

Oops
I recently had a problem with root access on my Beagleboard running Ubuntu 11.10 from an SD card. Basically I had accidentally removed my user account from the admin group. As it was the only user account I had set-up when installing Ubuntu: I was stuck!


Solution
The only option that I was aware of that would allow me to recover root access, was to boot Ubuntu in single user mode and add my user account back into the admin group.
I tried this by editing the UBoot bootargs parameter so I could enter Single user mode, setting it to:


OMAP3 beagleboard.org #  setenv bootargs 'console=ttyS0,115200n8 
root=1f01 rootfstype=jffs2 reboot=cold,hard S init=/bin/sh' 
OMAP3 beagleboard.org # boot
(NOTE This won't work!)


Note the addition of 'S' to the kernel parameters, which should instruct Ubuntu to enter into single user mode. AFAIK 'emergency' can also be used, but that brings you into a more basic environment.


This should have worked! But Ubuntu wasn't booting into single user mode. I didn't understand it until I looked at the complete bootcmd UBoot was using, which was the following:


setenv bootcmd 'mmc init;
fatload mmc 0 0x82000000 boot.scr;
source 0x82000000';
setenv autostart yes;
saveenv;
boot


Ahh, so the boot.scr script was being loaded from the boot partition on the SD card. I had a look at that file by mounting the SD card on my MAc and found that that script was overriding the bootargs parameter that I had manually set in UBoot.


Simple solution was to run the following commands in UBoot that bypassed the script on the boot partition of the SD card:
mmc init; mmc rescan 0; fatload mmc 0 0x82000000 
fatload mmc 0:1 0x80000000 uImage 
fatload mmc 0:1 0x81600000 uInitrd 
setenv bootargs ro vram=12M omapfb.mode=dvi:1280x720MR-16@60 
mpurate=auto root=UUID=76bfdd66-898d-4d50-be7a-4c8fd2691266 fixrtc 
console=ttyO2,115200n8 S
bootm 0x80000000 0x81600000


Once booted into the command prompt I added my user to the admin group and restarted the board. Problem solved.


Hindsight:
  • Could I have mounted the SD card on my Linux PC and edited the appropriate file to re-add my user account to the admin group?
  • Always a good idea to create a second user account that has admin access.

Tuesday 27 September 2011

Using the Watch Dog on an ATmega1281 as lock-up detector and sleep timer

Overview
In this post I will be describing how to use the Watchdog Timer of the ATmega1281 in order to detect and handle code lock-ups and at the same time, as a timer to wake the device from low power sleep mode.
Note this same approach should work with the ATmega640, ATmega1280, ATmega2560 and ATmega2561 micro-controllers.
This entry assumes the reader has some experience developing for the ATmega1281 micro-controller, understands how a typical Watchdog Timer works and understands the sleep modes of he ATmega1281.


Atmega1281 Watchdog Timer (WDT)
Functionality
The WDT has three modes of operation:

  • Interrupt - The WDT_vect interrupt sub-routing will be called when the WDT times out. Can be used to wake the micro from sleep modes, including the lowest power sleep mode (SLEEP_MODE_PWR_DOWN), where other timers are not available.
  • System Reset - A watchdog time-out reset will occur, i.e. the micro-controller will be restarted. For use in handling code lockups.
  • Interrupt and System Reset - First the WDT_vect interrupt sub-routing will be called, when completed a watchdog time-out reset will occur.
The WDT has it's own internal clock, that gives us time-out resolutions from 16ms to 8s.

Note, after a System Reset the WDT is still enabled, also the configuration will have reverted to default, i.e. 16ms time-out, If the user doesn't re-configure or disable the WDT before that time-out you can end up with the micro in a continuous System Reset loop. The following will disable the WDT before user code is executed:

/**
 * @brief Disables the watchdog at startup, otherwise after a timeout reset 
 *        it will continue to timeout with it's default value (<50mS).
 */
uint8_t mcusr_mirror __attribute__ ((section (".noinit")));
void get_mcusr(void) \
__attribute__((naked)) \
__attribute__((section(".init3")));
void get_mcusr(void)
{
    mcusr_mirror = MCUSR;
    MCUSR = 0;
    /* We always need to make sure the WDT is disabled immediately after a 
     * reset, otherwise it will continue to operate with default values.
     */
    wdt_disable();
}


Control & Registers
There are two sets of registers to be used when controlling the WDT:
  • WDTCSR - Watchdog timer Control Register, used for configuring the time-out, mode of operation, etc.
  • MCUSR - MCU Status Register, used to tell the cause of the last reset, such as brown-out reset, watchdog reset, etc. NOTE: for security reasons, there is a timed sequence for clearing the WDE and changing the time-out configuration. If you don't use this sequence properly, you'll get unexpected results.
For further detail on those registers and the timed sequence please see the ATmega1281 Reference Manual.


AVR Libraries
Atmel provide a set of macros for controlling the WDT. If you simply want to use the WDT to detect code lock-ups, that library will be fine. If you want to do anything more advanced, you will also have to manually configure the above registers.


Code
We want to use the WDT for two purposes simultaneously; detect/handle code lock-ups and as a timer to bring us out of sleep mode.
For this we will use the WDT in interrupt mode only. Depending on why the interrupt has entered, we will either force a watchdog system reset or wake the main application from it's sleep mode.
#include <stdbool.h>
#include <avr/wdt.h>
#include <avr/interrupt.h>

/**
 * @brief   Flag stating if sleep has been entered or not.
 */
volatile bool sleep_entered;


/**
 * @brief We need to enable the WDT for Interrupt mode only, so we can use it 
 *        as a sleep wakeup timer as well as for detecting lockups.
 * @note  Inline function!
 */
inline void configure_wdt(void)
{
    /* A 'timed' sequence is required for configuring the WDT, so we need to 
     * disable interrupts here.
     */
    cli(); 
    wdt_reset();
    MCUSR &= ~_BV(WDRF);
    /* Start the WDT Config change sequence. */
    WDTCSR |= _BV(WDCE) | _BV(WDE);
    /* Configure the prescaler and the WDT for interrupt mode only*/
    WDTCSR =  _BV(WDP0) | _BV(WDP3) | _BV(WDIE);
    sei();
}

/**
 * @brief Configures and enables the watchdog for the specified period. If this
 *        is not required please update the power management module.
 */
void app_sleep_init(void)
{
    /* Setup the flag. */
    sleep_entered = false;
    configure_wdt();
}


 
/**
 * @brief   Enters the module into low-power sleep mode. Execution is blocked 
 *          here until sleep is exited.
 */
void app_sleep_enter(void)
{
    set_sleep_mode(SLEEP_MODE_PWR_DOWN);
    sleep_entered = true;
    sleep_enable();
    sei();
    sleep_mode(); 
    /* Execution will resume here. */
}


/**
 * @brief   The watchdog interrupt. The watchdog has two functions: 1/ to detect 
 *          if the application code has locked up. 2/ to wake up the module
 *          from sleep.
 */
ISR(WDT_vect)
{
    /* Check if we are in sleep mode or it is a genuine WDR. */
    if(sleep_entered == false)
    {
        /* The app has locked up, force a WDR. */
        wdt_enable(WDTO_15MS);
        while(1);
    }
    else
    {
        wdt_reset();
        /* Service the timer if necessary. */
        sleep_entered = false;
        sleep_disable();
        
        /* Inline function so OK to call from here. */
        configure_wdt();
    }
}




References