1.6 Development Steps this week. #7 #8

Development Step #7 USB Cameras & Raspberry Pi Camera Module

o Approach.

• Connecting USB Camera

First I want to figure out how to make a USB Camera (not the raspberry camera) work with the Raspberry Pi. For this purpose I will go through the documentation provided by the raspberry’s official website.

• Raspberry Pi Camera

I will start exploring the possibilities of the raspberry pi’s camera by going through the Readme file of the raspicam github raspiratory: https://github.com/raspberrypi/userland/blob/master/host_applications/linux/apps/raspicam/README.md

 

o Learning Experience.

• Connecting USB Camera

I found this useful page from the raspberry documentation ( https://www.raspberrypi.org/documentation/usage/webcams/ )

First, install the fswebcam package:

sudo apt-get install fswebcam

Add your user to video group

If you are not using the default pi user account, you need to add your username to the video group, otherwise you will see ‘permission denied’ errors.

sudo usermod -a -G video <username>

To check that the user has been added to the group correctly, use the groupscommand.

Basic usage

Enter the command fswebcam followed by a filename and a picture will be taken using the webcam, and saved to the filename specified:

fswebcam image.jpg

This command will show the following information:

--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
Adjusting resolution from 384x288 to 352x288.
--- Capturing frame...
Corrupt JPEG data: 2 extraneous bytes before marker 0xd4
Captured frame in 0.00 seconds.
--- Processing captured image...
Writing JPEG image to 'image.jpg'.

Note the small default resolution used, and the presence of a banner showing the timestamp.

Specify resolution

The webcam used in this example has a resolution of 1280 x 720 so to specify the resolution I want the image to be taken at, use the -r flag:

fswebcam -r 1280x720 image2.jpg

Specify no banner

Now add the --no-banner flag:

fswebcam -r 1280x720 --no-banner image3.jpg

Now the picture is taken at full resolution with no banner.

• Live Video

It was extremely hard to make the USB camera live stream footage. In the forums people are even saying that it intentional and almost feels like Raspberry is trying to make people buy their “Raspberry PI Camera”, since it requires only 1-2 lines of code to run.

After many trials and errors I finally found the way to make the live camera work by creating an IP camera. I was following this tutorial: https://pimylifeup.com/raspberry-pi-webcam-server/

• Raspberry Pi Camera

First I had to find a ribbon cable to connect the camera to the raspberry board.

To setup the camera first I need to run:

sudo apt-get update
sudo apt-get upgrade

Now I need to enable camera support using the raspi-config program

sudo raspi-config

To test that the system is installed and working, try the following command:

raspistill -v -o test.jpg

The main functions of the raspberry camera can be accessed with the following commands:

• raspistill
  • Capturing still photographs with the camera module

Basic usage of raspistill

With the camera module connected and enabled, enter the following command in the Terminal to take a picture:

raspistill -o cam.jpg

In this example the camera has been positioned upside-down. If the camera is placed in this position, the image must be flipped to appear the right way up.

Vertical Flip & Horizontal Flip

With the camera placed upside-down, the image must be rotated 180° to be displayed correctly. The way to correct for this is to apply both a vertical and a horizontal flip by passing in the -vf and -hf flags:

raspistill -vf -hf -o cam2.jpg

Now the photo has been captured correctly.

Bash script

You can create a Bash script which takes a picture with the camera. To create a script, open up your editor of choice and write the following example code:

#!/bin/bash

DATE=$(date +"%Y-%m-%d_%H%M")

raspistill -vf -hf -o /home/pi/camera/$DATE.jpg

This script will take a picture and name the file with a timestamp.

You’ll also need to make sure the path exists by creating the camera folder:

mkdir camera

Say we saved it as camera.sh, we would first make the file executable:

chmod +x camera.sh

Then run with:

./camera.sh

 

• raspivid
  • Capturing video with the camera module

Basic usage of raspivid

With the camera module connected and enabled, record a video using the following command:

raspivid -o vid.h264

Remember to use -hf and -vf to flip the image if required, like with raspistill

This will save a 5 second video file to the path given here as vid.h264 (default length of time).

Specify length of video

To specify the length of the video taken, pass in the -t flag with a number of milliseconds. For example:

raspivid -o video.h264 -t 10000

This will record 10 seconds of video.

More options

For a full list of possible options, run raspivid with no arguments, or pipe this command through less and scroll through:

raspivid 2>&1 | less

Use the arrow keys to scroll and type q to exit.

MP4 Video Format

The Pi captures video as a raw H264 video stream. Many media players will refuse to play it, or play it at an incorrect speed, unless it is “wrapped” in a suitable container format like MP4. The easiest way to obtain an MP4 file from the raspivid command is using MP4Box.

Install MP4Box with this command:

sudo apt-get install -y gpac

Capture your raw video with raspivid and wrap it in an MP4 container like this:

# Capture 30 seconds of raw video at 640x480 and 150kB/s bit rate into a pivideo.h264 file:
raspivid -t 30000 -w 640 -h 480 -fps 25 -b 1200000 -p 0,0,640,480 -o pivideo.h264 
# Wrap the raw video with an MP4 container: 
MP4Box -add pivideo.h264 pivideo.mp4
# Remove the source raw file, leaving the remaining pivideo.mp4 file to play
rm pivideo.h264

Alternatively, wrap MP4 around your existing raspivid output, like this:

MP4Box -add video.h264 video.mp4

• Time-lapse
  • Taking pictures at regular intervals and stitching them together in to a video

• raspiyuv
  • Capturing still photographs and generating raw unprocessed image files

raspiyuv has the same set of features as raspistill but instead of outputting standard image files such as .jpgs, it generates raw unprocessed image files from the camera.

o Result.

• Connecting USB Camera

Output:

• Live Video

 

• Raspberry Pi Camera

To test if the camera is working

raspistill -v -o test.jpg

The display should show a five-second preview from the camera and then take a picture, saved to the file test.jpg, whilst displaying various informational messages.

• Bash script / raspistill

• Raw video capture:

 

 

Development Step #8 – Installation of OpenCV  on Raspberry Pi and Processing for Raspberry

o Approach.

Looking online for a step-by-step guides and trying to install OpenCV.

The way to test out if I have succeeded is by executing the following command in Python IDLE:

>>> import cv2
>>> print cv2.__version__

As a response I should get displayed the installed version of the OpenCV library.

 

o Learning Experience.

I was searching around and followed several different tutorials for the Installation of OpenCV on Raspberry Pi, none of them worked. I ended up reinstalling the whole OS. And then this steps worked out:

Before installing the library I had to execute the following commands:

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo rpi-update (can be skipped, but recommended) (don’t do it if you will use the RPI cam as recommended by official RPI Website)
$ sudo reboot now

$ sudo apt-get install build-essential cmake pkg-config

$ sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev

$ sudo apt-get install libgtk2.0-dev libgstreamer0.10-0-dbg libgstreamer0.10-0 libgstreamer0.10-dev libv4l-0 libv4l-dev

$ sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
$ sudo apt-get install libxvidcore-dev libx264-dev

$ sudo apt-get install libatlas-base-dev gfortran
$ sudo apt-get install python-numpy python-scipy python-matplotlib
$ sudo apt-get install default-jdk ant
$ sudo apt-get install libgtkglext1-dev
$ sudo apt-get install v4l-utils

$ wget https://bootstrap.pypa.io/get-pip.py
$ sudo python get-pip.py

$ sudo apt-get install python2.7-dev

$ sudo pip install numpy

Next step is to download the OpenCV 3.2.0 package by typing:

$ cd ~
$ wget -O opencv.zip https://github.com/Itseez/opencv/archive/3.2.0.zip
$ unzip opencv.zip

$ wget -O opencv_contrib.zip https://github.com/Itseez/opencv_contrib/archive/3.2.0.zip
$ unzip opencv_contrib.zip

After downloading I had to prepare the build by creating the following folder structure:

$ cd ~/opencv-3.2.0/
$ mkdir build
$ cd build
$ cmake -D CMAKE_BUILD_TYPE=RELEASE \
                  -D CMAKE_INSTALL_PREFIX=/usr/local \
                  -D INSTALL_C_EXAMPLES=OFF \
                  -D INSTALL_PYTHON_EXAMPLES=ON \
                  -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-3.2.0/modules \
                  -D BUILD_EXAMPLES=ON \
                  -D ENABLE_NEON=ON ..

The following step of the installation takes several hours and at some point I lose connection with the Raspberry Pi.

$ sudo make -j3

I repeated the process several times and ended up looking for help from a student in the Electrical Engineering program. He explained to me that during this step the Raspberry Pi shuts down all unnecessary processes including the VLC server. The only thing I have to do during this part of the installation is to let it be for 4 hours and then reboot the Pi and continue with the rest of the steps.

The rest of the commands:

$ sudo make install
$ sudo ldconfig

$ sudo nano /etc/ld.so.conf.d/opencv.conf

/usr/local/lib

$ sudo ldconfig

$ sudo nano /etc/bash.bashrc

PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig
export PKG_CONFIG_PATH

$ sudo shutdown -r now

 

• Processing and Open CV

 

I installed Processing on my Raspberry by using the following command:

curl https://processing.org/download/install-arm.sh | sudo sh

after that I did :

sudo apt-get update
sudo apt-get upgrade
sudo reboot

I installed the OpenCV library through Processing and I tried several example files regarding OpenCV, but non of them worked. There were errors after errors.  So I swiftly decided to look for a solution online instead of the trial – error method.

https://pi.processing.org/tutorial/camera/

I had to install the GL Video library through Processing and after that:
GL Video library needs a special driver to be enabled on the operating system level. Add the line "bcm2835_v4l2" (without quotation marks) to the file /etc/modules. This can be accomplished by executing the following in a terminal window:

echo "bcm2835_v4l2" | sudo tee -a /etc/modules >/dev/null

After a restart you should be able to use the Pi Camera in Processing!

Let’s dig into using the GLCapture class to start capturing the video stream! The process of using GLCapture class looks like this:

• Make sure the sketch renderer is setup to be P2D or P3D
• Import the GL Video library that contains GLCapture class (import gohai.glvideo.*)
• Create a new GLCapture object that will stream and store the textures from the camera
• Initialize the GLCapture object, specifying camera framerate, width and height of the desired video stream
• Start the stream via the start() method
• Read the video stream when it is available
• Display (or otherwise) use the video

Whatever tutorial or code I was trying to run on processing I always get the same error:

Could not parse -1 for –display.

Followed by other errors:

I couldn’t find any one encountering the same error message. The only suggestion I got was to reinstall Respbian with included and pre-configured processing Image. Which took me several hours and I ended up with having the very same error message.

I decided to continue without Processing and OpenCV. I just lost way too much time on that step.

o Result.

The last thing to do is to try out if it has been installed successfully.

And finally it was!

I received several errors during the installation though. Some libraries failed to install. I consulted the Engineering student who helped me earlier and he told me to execute the following lines of code, which reinstall the troublesome library:

 

• Processing and OpenCV

My efforts regarding Processing and OpenCV lead to a endless error loop:

o Sources.

https://www.raspberrypi.org/documentation/usage/webcams/
https://www.raspberrypi.org/documentation/hardware/camera/README.md
https://www.raspberrypi.org/documentation/raspbian/applications/camera.md
https://www.raspberrypi.org/documentation/usage/camera/raspicam/
https://github.com/raspberrypi/userland/tree/master/host_applications/linux/apps/raspicam

https://pi.processing.org/download/
https://pi.processing.org/tutorial/camera/
https://forum.processing.org/two/discussion/24157/opencv-on-raspberry-pi-face-detection-sketch-errors

https://github.com/Tes3awy/OpenCV-3.2.0-Compiling-on-Raspberry-Pi

https://github.com/Itseez/opencv/archive/3.2.0.zip