Raspberry Pi Timelapse and upload to YouTube

I played with creating time lapse videos with the pi a few years ago, but recently came up with the idea of uploading a daily video to YouTube.

This has been running since late January 2019. Here’s one of the uploads:

This post describes the bash scripts to automate this process.

The goal was to create a timelapse video once every day, upload that video to YouTube and then delete the local video, this way I wouldn’t have to deal with storing the video files myself.

The following are a few BaSH scripts I put together to perform the tasks.

I had a gigabyte brix computer, so decided to use that for encoding and uploading to not overload the pi in case that interfered with taking photos while it was encoding.

The Pi and the gigabyte are on the same LAN. I set a static IP address for the gigabyte in the router configuration so there wouldn’t be a risk of it getting assigned a different IP address over time.

The high level tasks are:

On the pi:

1. Take a photo every 30 seconds and store it locally.
2. Rsync the photo to a separate machine which will encode the photos into a video once a day.

On the gigabyte:

1. A directory of photos per day (YYYY-MM-DD) is created to be copied from the Pi.
2. Cron task running once a day encodes yesterday’s photos into a video.
3. The resulting video file is uploaded to YouTube.
4. On successful upload, yesterday’s photos are deleted.
5. Once a month the previous month’s video files are deleted from the filesystem.

This assumes some first time setup for the Pi (also assumes model A or B):

• Wifi USB card.
• Camera is plugged in (blue tape facing ethernet port).

Here are the steps, based on the UK raspbian images.

US setup

Usually the problem is the ’|’ character. if you type this and get ’~’ then you should update the keyboard layout:

1sudo raspi-config

-> 4. Localization options

-> Change Keyboard Layout

-> Wait a bit to reload the keymaps.

-> other -> generic 104 key, not the International one.

It will ask you to pick from some UK options, choose other

English (US)

Then go through the next few screens

Network

Via raspi-config

1sudo raspi-config

-> 2. Network options, you must know your SSID and passphrase to connect to.

Via config files

Wireless connection info in /etc/wpa_supplicant/wpa_supplicant.conf.

This file is written when you use the raspi-config setup above.

If all else fails, this page has instructions on how to use wpa_supplicant:

https://shapeshed.com/linux-wifi/

Timezone

For list of timezones:

1ls /usr/share/zoneinfo/
2# For US timezones:
3ls /usr/share/zoneinfo/US

sudo vi /etc/timezone insert your timezone e.g.:

America/New_York

and you can also update the link

1sudo rm /etc/localtime
2sudo ln -s /usr/share/zoneinfo/US/Eastern /etc/localtime

check your time with date.

Camera

1sudo raspi-config

Interfacing options

SSH

1# This makes it start on boot:
2sudo systemctl enable ssh
3
4# This starts it now:
5sudo systemctl start ssh

Auth is password based by default. (Adjust based on your level of paranoia.)

Back to the timelapse scripts.

With all that setup done, on the Pi in the home directory:

1mkdir timelapse

and a script do_timelapse.sh in home dir:

1#!/bin/bash -
2
3mode=auto
4sleep_for=30
5# Change per your setup
6video_host=192.168.1.172
7# Change per your setup
8video_user=user
9# Change per your setup
10video_dir=timelapse/front_window
11# You can play with resolution to your liking.
12width=3280
13height=2464
14
15while true; do
16  output_dir=$(date +"%F")
17  mkdir -p $output_dir
18
19  current_hour=$(date +"%H")
20  # ${var#0} syntax removes leading zeros for integer comparison.
21  if [[ "${current_hour#0}" -gt 21 || "${current_hour#0}" -lt 9 ]]; then
22    mode=night
23  fi
24  current_datetime=$(date +"%FT%H-%M-%S%Z")
25  output_filename="$output_dir/${current_datetime}.jpg"
26  raspistill -w "$width" -h "$height" -ex "$mode" --nopreview -o "$output_filename"
27
28  ssh -i /home/pi/.ssh/gigabyte "$video_user"@"$video_host" "mkdir -p ${video_dir}/$output_dir"
29  rsync -avhz -e "ssh -i /home/pi/.ssh/gigabyte" "$output_filename" "$video_user"@"$video_host":"${video_dir}/$output_filename"
30  rm "$output_filename"
31  sleep $sleep_for
32done

I played with the ‘night’ mode and the exposure comes out better at night.

An improvement to this would be to contact a web service to request when daylight hours are based on the timezone of the OS and use that instead of hard-coding the hours.

This setup requires creating an SSH keypair on the Pi and adding the public key to the authorized_keys file on the remote machine to allow automated copying of files between machines.

To generate the keypair (run on the pi):

1ssh-keygen -t rsa -b 4096 -C comment -f ~/.ssh/key_name
2# Copy the key to authorized_keys using one of these methods:
3ssh-copy-id -i ~/.ssh/keyname user@IP_ADDRESS
4# or
5cat "$identity_file.pub" | ssh $user@$host 'cat >> ~/.ssh/authorized_keys'

Confirm SSHing after that. You should not be prompted for a password.

You could also improve on error handling: if rsyncing fails, for example, you probably don’t want to delete the image file so you can recover it later. You could also email an address to notify of the failure.

Put the following in: /etc/systemd/system/timelapse.service:

1[Unit]
2Description=Take some photos
3After=multi-user.target
4
5[Service]
6Type=simple
7ExecStart=/home/pi/timelapse/do_timelapse.sh
8
9[Install]
10WantedBy=multi-user.target

Enable the timelapse script on boot:

1sudo systemctl enable timelapse

enable right now with:

1sudo systemctl start timelapse

This unit file has the effect of starting the service on boot as well as wiring output to syslog. Any output to standard out will visible in the /var/log/syslog{*} files with output like so:

1May 17 13:14:52 raspberrypi do_timelapse.sh[20637]: sending incremental file list

On the encoding and uploading machine

1mkdir timelapse

In the timelapse directory put the following script as run_make_movie.sh:

1#!/bin/bash -
2#
3# Lists directories of current year in directory "$movie_dir", picks
4# the most recent one as sorted by date, and encodes a movie from the image stills.
5# Then uploads the movie to youtube. If uploaded successfully, then deletes the stills.
6#
7
8echo "running run_make_video script"
9home_dir="/home/user/timelapse"
10movie_dir="front_window"
11
12pushd "${home_dir}/${movie_dir}"
13
14year=$(date +%Y)
15
16# Get yesterday's directory
17pics_dir=$(ls | grep "^$year" | sort -n | tail -n 2 | sort -nr | tail -n 1)
18
19echo "pics dir: $pics_dir"
20
21if [ -z "$pics_dir" ]; then
22  echo "$pics_dir is missing."
23  exit 1
24fi
25
26echo "./make_video.sh $pics_dir"
27./make_video.sh "$pics_dir"
28
29movie_filename="timelapse_${pics_dir}.avi"
30movie_title="$pics_dir My video description timelapse"
31
32popd
33
34pushd "${home_dir}"
35
36echo "./upload_file.sh ${movie_dir}/${movie_filename} "$movie_title""
37./upload_file.sh "${home_dir}/${movie_dir}/${movie_filename}" "$movie_title"
38
39if [ $? -eq 0 ]; then
40  echo "Successfully uploaded ${movie_filename} to Youtube"
41
42  echo "rm -rf ${home_dir}/${movie_dir}/$pics_dir"
43  rm -rf "${home_dir}/${movie_dir}/$pics_dir"
44else
45  # To setup mail:
46  # https://rianjs.net/2013/08/send-email-from-linux-server-using-gmail-and-ubuntu-two-factor-authentication  
47  echo "For some reason the youtube uploader script failed." | mail -s "Your app YT-Uploader FAILED" "youremail@mail.com"
48fi
49
50# Clean up previous month's videos if they exist
51
52if command ls "${home_dir}/${movie_dir}/timelapse_$(date --date="1 month ago" +%Y-%m)"* &> /dev/null; then
53  last_month_vids=$(command ls "${home_dir}/${movie_dir}/timelapse_$(date --date="1 month ago" +%Y-%m)"*)
54  echo "removing last month's videos: $last_month_vids"
55  rm -f "$last_month_vids"
56fi
57
58popd

This is the main runner script, it relies on two other scripts: make_video.sh and upload_file.sh.

make_video.sh goes in /home/user/timelapse/$project_name make_video.sh:

1#!/bin/bash -
2#
3# Encode a directory of images into a movie file.
4# Intended use is to make a timelapse movie.
5#
6set -eu
7
8movie_width=3280
9movie_height=2464
10
11usage() {
12  echo
13  echo "Encode a directory of images into a movie."
14  echo "$0 <subdirectory>"
15  echo
16}
17
18if [ $# -eq 0 ]; then
19  echo 'Missing subdirectory argument. Exiting'
20  usage
21  exit 1
22fi
23
24if [ ! -d "$1" ]; then
25  echo
26  echo "Directory "$1" does not exist. Exiting."
27  echo
28  exit 2
29fi
30
31subdir="$1"
32# Remove trailing slash of filename
33# same effect as subdir=$(echo "$1" | sed s:/::g)
34subdir=$(sed s:/::g <(echo "$1"))
35
36if [ ! -d "$subdir" ]; then
37  echo
38  echo "Directory "$subdir" does not exist. Exiting."
39  echo
40  exit 2
41fi
42
43echo "subdir: $subdir"
44
45filelist="stills.txt"
46
47rm -f "$filelist"
48
49# awk -v is used to make a bash variable available in the awk script.
50# This creates lines in $filelist like: 2019-05-16/2019-05-16T23-25-14EDT.jpg
51# because the image files are in the subdirectory 2019-05-16.
52ls "$subdir" | awk -v dir="$subdir" '{print dir"/"$1}' > "$filelist"
53
54echo 'wrote file list:'
55head "$filelist"
56
57movie_file="timelapse_${subdir}.avi"
58
59mencoder   -nosound   -ovc lavc   -lavcopts vcodec=mpeg4:mbd=2:trell:autoaspect:vqscale=3   -vf scale="$movie_width":"$movie_height"  -mf type=jpeg:fps=20 mf://@"$filelist"   -o "$movie_file"

I had to play around with the mencoder config, copied from somewhere on the Net, -ovc is to set the output video codec. -vf is to set a video filter I’m honestly not sure what the -lavcopts options after setting the video codec to mpeg4 do… :)

After this runs we now have a video file which we can now upload to YouTube:

To actually upload the video file I am using this go package: https://github.com/porjo/youtubeuploader

You can download a binary from the releases page and follow the instructions to deal with authenticating with YouTube.

The upload script assumes the binary exists in /home/user/timelapse/youtubeuploader_linux_amd64

And for the upload script itself, add the following to /home/user/timelapse/upload_file.sh:

1#!/bin/bash -
2
3set -eu
4
5usage() {
6  echo
7  echo "Upload a movie file to youtube."
8  echo "$0 <filepath> <movie title>"
9  echo
10}
11
12if [ $# -eq 0 ]; then
13  echo 'Missing filepath and movie title argument. Exiting'
14  usage
15  exit 1
16fi
17
18if [ $# -eq 1 ]; then
19  echo 'Missing movie title argument. Exiting'
20  usage
21  exit 2
22fi
23
24if [ ! -f "$1" ]; then
25  echo
26  echo "File "$1" does not exist. Exiting."
27  echo
28  exit 3
29fi
30
31video_filename="$1"
32video_title="$2"
33
34./youtubeuploader_linux_amd64   -filename "$video_filename"   -privacy public   -title "$video_title"   -description ""   -headlessAuth

Then the last step is to add a cron job to run the upload script each day:

1crontab -e

Add the following contents:

1# This runs every day of the month, every month, on every day of the week at the
2# first minute start of second hour of the day.
30 1 * * * /home/user/timelapse/run_make_movie.sh > /home/user/timelapse_run_make_video_out.txt 2>&1

It took me a bit of manual running of the scripts and incrementally editing to get everything working so don’t be surprised if you’ll need to do the same.

• Asteroids, the game →