1.9 Reflection (whole project)

What I have reached/achieved:

I managed to get knowledge and experience in the following directions:

• Arduino + Processing
• OpenCV
• Product design and prototyping
• Camera streaming and image processing
• Raspberry Pi
• Python
• Deeper understanding in servo motors

What I have learned:

I feel that my biggest mistake is that I had a clear goal in mind – making an animatronic eye, which can track faces. In this way I didn’t give myself enough freedom in experimenting and discovering the broader possibilities brought by the technologies I have chosen. I straight away developed the final prototype, without spending enough time to document my mistakes and struggles, which forced me to have a bit of a reverse development process – starting with the final product and figuring out ways to justify it. Despite that I still have managed to experiment and learn from those experiments.

The possibilities offered by Computer Vision are almost limitless.

Even though, this project is not related to my career path, I do believe that it thought me an important lesson about project management and work process.

Advertisement

1.8 Development Steps this week. #11 #12

Development Step #11 – Controlling Pan/Tilt servos with Raspberry Pi

 

o Approach.

I will try to calculate the Duty Cycle of the servos since they might be different from what the manufacture has claimed. I found a nice instructables tutorial to take me through the process.

o Learning Experience.

I found the manufacturer’s data sheet for the servos:

The Raspberry Pi has no analog output, but we can simulate this, using a PWM (Pulse Width Modulation) approach. What we will do is to generate a digital signal with a fixed frequency, where we will change the pulse train width, what will be “translated” as an “average” output voltage level.

Note that what matters here is not the frequency itself, but the “Duty Cycle”, that is the relation between the time that the puls is “high” divided by the wave period. For example, suppose that we will generating a 50Hz pulse frequency on one of our Raspberry Pi GPIO. The period (p) will the inverse of frequency or 20ms (1/f). If we want that our LED with a “half” bright, we must have a Duty Cycle of 50%, that means a “pulse” that will be “High” for 10ms.

This principle will be very important for us, to control our servo position, once the “Duty Cycle” will define the servo position.

I am connecting the pan/tilt servos to the following GPIO pins:

• Tilt Servo – GPIO 17
• Pan Servo – GPIO 27

Both of them are taking 5V power from Pin#2 and Pin#4, and they are also grounded to the Raspberry Pi.

In theory, the servo will be on its

• Initial Position (0 degrees) when a pulse of 1ms is applied to its data terminal
• Neutral Position (90 degrees) when a pulse of 1.5 ms is applied to its data terminal
• Final Position (180 degrees) when a pulse of 2 ms is applied to its data terminal

To program a servo position using Python will be very important to know the correspondent “Duty Cycle” for the above positions, let’s do some calculation:

• Initial Position ==> (0 degrees) Pulse width ==> 1ms ==> Duty Cycle = 1ms/20ms ==> 2.0%
• Neutral Position (90 degrees) Pulse width of 1.5 ms ==> Duty Cycle = 1.5ms/20ms ==> 7.5%
• Final Position (180 degrees) Pulse width of 2 ms ==> Duty Cycle = 2ms/20ms ==> 10%

So the Duty Cycle should vary on a range of 2 to 10 %.

Now lets find out that for each servo individually.

by typing the following commands in the Terminal

import RPi.GPIO as GPIO

GPIO.setmode(GPIO.BCM)
tiltPin = 17

Now, we must specify that this pin will be an “output”

GPIO.setup(tiltPin, GPIO.OUT)

And, what will be the frequency generated on this pin, that for our servo will be 50Hz:

tilt = GPIO.PWM(tiltPin, 50)

Now, let’s start generating a PWM signal on the pin with an initial duty cycle (we will keep it “0”):

tilt.start(0)

Now, you can enter different duty cycle values, observing the movement of your servo. Let’s start with 2% and see what happens (we expect that the servo goes to “zero position”):

tilt.ChangeDutyCycle(2)

…..

• Result:

Those are the results for both the Pan and the Tilt servos:

 

The PWM commands to be sent to our servo are in “duty cycles” as we saw on the last step. But usually, we must use “angle” in degrees as a parameter to control a servo. So, we must convert “angle” that is a more natural measurement to us in duty cycle as understandable by our Pi.

I know that duty cycle range goes from 2% to 12% and that this is equivalent to angles that will range from 0 to 180 degrees. Also, we know that those variations are linear.

So, given an angle, we can have a correspondent duty cycle:

dutycycle = angle/18 + 2

Now we can make a Python script using the formula

Servo5.py

from time import sleep
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)

def setServoAngle(servo, angle):
	pwm = GPIO.PWM(servo, 50)
	pwm.start(8)
	dutyCycle = angle / 18. + 3.
	pwm.ChangeDutyCycle(dutyCycle)
	sleep(0.3)
	pwm.stop()

if __name__ == '__main__':
	import sys
	servo = int(sys.argv[1])
	GPIO.setup(servo, GPIO.OUT)
	setServoAngle(servo, int(sys.argv[2]))
	GPIO.cleanup()

sudo python3 angleServoCtrl.py 17 45

The above command will position the servo connected on GPIO 17 with 45 degrees in “elevation”. A similar command could be used for Pan Servo control (position to 45 degrees in “azimuth”):

sudo python servo5.py 27 45

Now lets use the script for controlling both of the servos at the same time:

Servo6.py

 

from time import sleep
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)

pan = 27
tilt = 17

GPIO.setup(tilt, GPIO.OUT) # white => TILT
GPIO.setup(pan, GPIO.OUT) # gray ==> PAN

def setServoAngle(servo, angle):
	assert angle >=50 and angle <= 130
	pwm = GPIO.PWM(servo, 50)
	pwm.start(8)
	dutyCycle = angle / 18. + 2.
	pwm.ChangeDutyCycle(dutyCycle)
	sleep(0.3)
	pwm.stop()

if __name__ == '__main__':
	import sys
	if len(sys.argv) == 1:
		setServoAngle(pan, 110)
		setServoAngle(tilt, 50)
	else:
		setServoAngle(pan, int(sys.argv[1])) # 30 ==> 90 (middle point) ==> 150
		setServoAngle(tilt, int(sys.argv[2])) # 30 ==> 90 (middle point) ==> 150
	GPIO.cleanup()

When the script is executed, you must enter as parameters, Pan angle and Tilt angle. For example:

sudo python3 servo6.py 120 50

I found the proper angles for my Servos

Pan Servo:

• Min – 50
• Mid – 110
• Max – 150

Tilt Servo:

• Min – 50
• Mid – 60
• Max – 90

Now the final test is a loop cycle on the pan and tilt axis:

servo7.py

from time import sleep
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)

pan = 27
tilt = 17

GPIO.setup(tilt, GPIO.OUT) # white => TILT
GPIO.setup(pan, GPIO.OUT) # gray ==> PAN

def setServoAngle(servo, angle):
	assert angle >=50 and angle <= 130
	pwm = GPIO.PWM(servo, 50)
	pwm.start(8)
	dutyCycle = angle / 18. + 2.
	pwm.ChangeDutyCycle(dutyCycle)
	sleep(0.3)
	pwm.stop()

if __name__ == '__main__':  
    for i in range (50, 140, 15):
        setServoAngle(pan, i)
        setServoAngle(tilt, i)
    
    for i in range (130, 50, -15):
        setServoAngle(pan, i)
        setServoAngle(tilt, i)
        
    setServoAngle(pan, 110)
    setServoAngle(tilt, 50)    
    GPIO.cleanup()

o Result.

o Sources.

https://www.instructables.com/id/Pan-Tilt-Multi-Servo-Control/

Development Step #12 – Object Tracking with OpenCV and Raspberry Pi

o Approach.

First I will start with some examples using Object Tracking and OpenCV and after that I will proceed with live video and integrating the servo function.

o Learning Experience.

I already have installed the OpenCV library in a previous dev. step.

I will start by analyzing the code by Adrian posted on his page ( https://www.pyimagesearch.com/2015/09/14/ball-tracking-with-opencv/ )

I have to verify that the OpenCV library is installed correctly:

I want to test if OpenCV is working as before, by running that test python script:

import numpy as np
import cv2
cap = cv2.VideoCapture(0)
while(True):
    ret, frame = cap.read()
    frame = cv2.flip(frame, -1) # Flip camera vertically
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    
    cv2.imshow('frame', frame)
    cv2.imshow('gray', gray)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
cap.release()
cv2.destroyAllWindows()

To execute the OpenCV code I need to get into the OpenCV’s virtual environment from the terminal with the following commands:

source ~/.profile
workon cv

python simpleCamTest.py

Unfortunately I bumped into an error “workon cv is not valid”

I started backtracking the issue and reached the conclusion that there could be something wrong with the OpenCV version I am using 3.2.0 and maybe I have missed some packages during my installation, so I decided to look for them.
I headed to this installation guide : https://www.pyimagesearch.com/2017/09/04/raspbian-stretch-install-opencv-3-python-on-your-raspberry-pi/

I started installing all the mentioned libraries, but nothing fixed the issue. So after many tries I decided to completely reinstall the OpenCV library, which is a long process that I wanted to avoid.

Everything was going smooth until I started getting multiple error messages and the process stopped.

I tried again, but no effect:

This time I receive a “free space” error, even though that I use a 32GB sd card.

Going through the comments the only solution to the issue is to make a clean install of Raspbian and start installing OpenCV, VNC and all the other libraries that are needed.

Sadly because of the time scope of this project, I am unable to continue the development.

o Sources.

Ball Tracking with OpenCV

https://www.hackster.io/mjrobot/automatic-vision-object-tracking-5575c4#_=

Color Detection in Python with OpenCV

1.8 Retrospective(week 1.7) & ToDo(week 1.8)

Retrospective 1.7

• What went well?

I gained some general idea about Python. And I am quite happy that I managed to understand how to identify the pins in the different modes of the GPIO and also managed to connect servos to the Raspberry!

• What could be better?

I wish I could have experimented with a potmeter or an LED strip.

• What should be better?

I am pretty satisfied with this week’s progress.

• Time spent.

18 hours

To Do 1.8

• Calibrate servo motors according to my prototype.
• Use object tracking in combination with the servo motors.

1.7 Development Steps this week. #9 #10

Development Step #9 Python

 

o Approach.

For me personally a lynda course will help me out a lot when it comes to learning a completely new programming language:

I will follow this one:
https://www.lynda.com/Python-tutorials/Learning-Python/661773-2.html

As an editor I will be using Atom.

 

o Learning Experience.

• Atom setup.

 

 

• Lynda course.

Topics include:

• Installing Python
• Choosing an editor or IDE
• Working with variables and expressions
• Writing loops
• Using the date, time, and datetime classes
• Reading and writing files
• Fetching internet data
• Parsing and processing HTML

When I tried to run the “Helloworld!” example, the console displayed the following error message:

python : The term ‘python’ is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct

After googling it I realized that I need to add manually the python path to the Windows Environment Variables. I followed this tutorial:

 

 

 

o Result.

• Hello world example.

 

• Variables and expressions.

 

• Functions

 

• Loops

 

• Conditional structures.

• Classes

• Date and time classes

 

• Formatting time output.

• Timedelta objects

• Calendars

• Reading and writing files.

• OS path

• File system shell.

o Sources.

https://www.lynda.com/Python-tutorials/Learning-Python/661773-2.html

 

 

 

 

https://stackoverflow.com/questions/11813435/im-trying-to-use-python-in-powershell

https://docs.python.org/3/library/index.html

https://docs.python.org/3/reference/index.html

 

Development Step #10 GPIO & Servo Motors on Raspberry Pi

o Approach.

sdsds

o Learning Experience.

As a starting point I headed to the course called “Raspberry Pi: GPIO” on Lynda.com. The main topics I am interested in are:

• Locating the GPIO
• Programming with Python
• Identifying GPIO pins
• Using the WiringPi and RPi.GPIO libraries

 

Controlling an LED.

We will enable the GPIO pin through the terminal. First we need to be super users in order to do that. In stead of typing “sudo” every single line, we can simply enter superuser mode through the following command:

sudo -i

Next step is to change the directory (“cd”) to the directory where the GPIO is located with the following command:

cd /sys/class/gpio

Now we need to assign pin 4.

echo "4" > export
This command takes the value “4” and writes it into a file called export. “echo” stands for print.

Now we need to assign pin 4 as an output pin.

echo "out" > gpio4/direction

To turn the LED on we need to write the following command:

echo "1" > gpio4/value

• Result
  

  To turn off the LED we can use the reverse command:

echo "0" > gpio4/value

• Result
  

To turn off any values for pin 4, we can write:

echo "4" > unexport

As a safety measure we can also exit the “sudo” mode by typing:

exit sudo

Using a Switch / Button

First I need to assign GPIO pin 4 as input:
sudo -i
cd /sys/class/gpio
echo "4" > export
echo "in" > gpio4/direction

I can check the value of gpio4 with the following command:

cat gpio4/value

At the moment I am not having the button pressed, which means that I should receive a value of “0”.

• Result.
  

If I have the button pressed the circuit should become closed and I should receive a value of 1.

• Result
  
  

Locating Pins

There are several different numerations of the GPIO pins.

• GPIO.BOARD – is the physical location
• GPIO.BCM – is the numbering according to the default python library for Raspberry (RPi.GPIO)
• WiringPi – is related to the 3rd party WiringPi python library

Now I will be using a python code to reproduce the previous exercise. First I will import the RPi.GPIO library:

import RPi.GPIO as GPIO

Now I will select the board numbering method. In this case GPIO.BOARD

GPIO.setmode(GPIO.BOARD)

Now I need to select the input pin. In the previous experiment it was pin 4, but according to the GPIO.BCM system, now I am using the GPIO.BOARD system an the corresponding number is 7.

inputpin = 7

The next line should assign the pin as an input pin and indicate that there is a resistor in use.

GPIO.setup(inputpin,GPIO.IN,pull_up_down=GPIO.PUD_UP)

We can now add a if/else statement reflecting the state of the switch/button, if it is closed/pressed or opened/released.

while True:
if GPIO.input(inputpin):
print ("Switch Closed")
else:
print ("Switch Open")inputpin = 7

The final python script should look like that:

If we were using the GPIO.BCM numbering then it would be:

We can run the code through the terminal, navigate to and run the script that we just created:

• Result.

At the moment the button is not pressed.

As soon as I press it:

 

WiringPi Library

To install the WiringPi Library the following commands should be executed one after another:

sudo apt-get install git-core
sudo apt-get update
sudo apt-get upgrade
git clone git://git.drogon.net/wiringPi
cd wiringPi
./build

To test if the installation went smoothly type:
gpio -v

Now to see all the GPIO numbers type in the terminal:
gpio readall

• Result:

 

Servo Motors

For the following experiments I will be using the default library called “GpioZero” and HS-311 servo.

I am attaching the power cable to Pin #2, which is 5v. The ground cable to Pin#9 and the signal cable to Pin#7 (GPIO.4). And for the Raspberry I am using a 5V 4A power supply.

The first script I will use should make the servo move between its minimum, middle and maximum positions with a small delay in-between.

from gpiozero import Servo
from time import sleep

myGPIO=17

servo = Servo(myGPIO)

while True:
servo.mid()
print("mid")
sleep(0.5)
servo.min()
print("min")
sleep(1)
servo.mid()
print("mid")
sleep(0.5)
servo.max()
print("max")
sleep(1)

To execute the script I can do it from the terminal by typing:
cd experiments //change to the experiments folder I store my scripts now
python servo1.py

Or by using a Python editor called Thonny, which turned out to be more convenient.

• Result
  

first script makes use of the Gpiozero defaults. It assumes the servo uses a signal frame width of 20ms. The pulse width for the minimum and maximum rotation is assumed to be 1ms and 2ms.

I found that with the default settings my servo moves in the diapason from 30 degrees to 150 degrees. With the following script I will try to expand that diapason.

from gpiozero import Servo
from time import sleep
myGPIO=4

myCorrection=0.45
maxPW=(2.0+myCorrection)/1000
minPW=(1.0-myCorrection)/1000

servo = Servo(myGPIO,min_pulse_width=minPW,max_pulse_width=maxPW)

while True:
servo.mid()
print("mid")
sleep(0.5)
servo.min()
print("min")
sleep(1)
servo.mid()
print("mid")
sleep(0.5)
servo.max()
print("max")
sleep(1)

• Result

The last script I will experiment with generates a range of “value” numbers to sweep the servo between its maximum and minimum position :

from gpiozero import Servo
from time import sleep

myGPIO=4

myCorrection=0
maxPW=(2.0+myCorrection)/1000
minPW=(1.0-myCorrection)/1000

servo = Servo(myGPIO,min_pulse_width=minPW,max_pulse_width=maxPW)

while True:

print("Set value range -1.0 to +1.0")
for value in range(0,21):
value2=(float(value)-10)/10
servo.value=value2
print(value2)
sleep(0.5)

print("Set value range +1.0 to -1.0")
for value in range(20,-1,-1):
value2=(float(value)-10)/10
servo.value=value2
print(value2)
sleep(0.5)

The first “For” loop generates a set of integers between 0 and 20. The value has 10 subtracted from it to give a range -10 to 10. The value is then finally divided by 10 to give a range of -1 to +1. The original set of values contained 20 integers and we still have 20 steps in the sequence. To increase the number of steps to 40 you would replace 20 with 40 and subtract 20 rather than 10.

The second “For” loop does the same thing but generates a sequence from +1 back to -1.

• Result

 

o Sources.

https://www.raspberrypi-spy.co.uk/2018/02/basic-servo-use-with-the-raspberry-pi/

https://gpiozero.readthedocs.io/en/stable/api_output.html#servo

 

1.7 Retrospective(week 1.6) & ToDo(week 1.7)

Retrospective 1.6

• What went well?

I managed to take a photo, record a video and play live video from a USB Camera attached to Raspberry Pi. I experimented with the Raspberry Pi Camera module and I succeeded in capturing an image, making a simple script to automatically do it and capturing a video.

I managed to install the OpenCV library on a Raspberry Pi running Raspbian OS. I experimented with Processing for Raspberry Pi.

• What could be better?

I had major issues with the camera and Processing. I thought it is because I had too much libraries installed, which collide with one another, but even when I invested the time for a clean install of Raspbian with pre-configured Processing, the issues where still there. I wish I could re-adjust my Arduino+Processing code for Raspberry, but unfortunately it didn’t go that way.

• What should be better?

Instead of trying to figure out a solution to a problem by searching online or writing in forums (slow), I should personally contact local experts from the field, who can help me out.

• Time spent

18 hours

To Do 1.7

• Since OpenCV and Processing didn’t work out well, I am going to concentrate on OpenCV and Python. The first step is to get some understanding of the Python basics and syntax.
• I need to be able to control servo motors via the GPIO interface of the Raspberry Pi.