A Visual Recognition Security System in Java
Keiron Skillett, MEng Project 2001

Design and Methodology

4.1 User Interface

The images used will be in JPEG format, they can be of any dimensions, so long as this size is consistent with all images stored in the database. This is justified by the fact that all images received from webcam will be the same size. Due to the security nature of this project the user interface will be very limited, the user should only be able to access a button to check an image.

4.2 Data

It is necessary to limit the scope of this project due to the complex field of image recognition; unless this is done it is unlikely that anything will be achieved in the time available. Therefore this project will consider recognising simple silhouette shapes, such as letters of the alphabet; this type of function would be very useful if combined with CCTV cameras to capture car number plates at garages⁷. Each number plate could be instantly recognised and stored in a database in case the car drove away without paying.

The test cases for this project will be to recognise letters of the alphabet, all in a standard font of a standard size, on a standard size image (section 4.1). This should allow for the object to be transformed in the image, and time permitting allow for rotation or scaling (perspective), see figure 4.1).

Figure 4.1 - Image Transformations

4.3 Image Analysis Method

The following steps will take place with the image for analysis:

Convert image to greyscale. (Grey Images are as recognisable as colour images, but easier to work with, section 3.2.1).
Perform erosion on image with a symmetrical 3*3 kernel containing all 1’s so that ALL surrounding pixels are taken into account. (Kernels, section ?.?).
Threshold the image to remove any “odd” data values.
Create a polygon(s) from the white edges detected.
This creates a locally extracted feature vector (figure 3.3).

Load each image from the database, and:

Convert image to greyscale.
Perform erosion on image with a symmetrical 3*3 kernel containing all 1’s so that ALL surrounding pixels are taken into account.
Threshold the image to remove any "odd" data values.
Extend all of the edges by one pixel in each direction. This allows for some degree of error or noise whilst still obtaining quite an accurate decision. This value should be able to be set so that the edges can be extended a number of times, in unclear images.
Find an origin point that is white.
Attempt to map the polygon onto the image from this point, counting points that fall onto white pixels. A threshold value of the percentage of points which must map correctly needs to be set, default 80%.

Having processed all images in the database, display either the name of the image (from database) or “unknown image”.

4.4 Design

4.5 Class Summaries

Listed below is a brief summary of methods expected to appear in classes.

Class: ImageAnalysis

Method	Access	Returns	Description
ImageAnalysis()	Public	Void	Constructor, sets up window and menus.
main()	Public	Void	Application, needs to run from command line.
actionPerformed()	Public	Void	Action Listener, responds to commands.
getChosenFile()	Private	String	Gives the path to the file opened by a user

Class: ImageAnalysisPanel

Method	Access	Returns	Description
ImageAnalysisPanel()	Public	Void	Constructor, sets up window and menus.
setImageAnalysisPanel()	Public	Void	Set options for ImageAnalysisPanel.
paint()	Public	Void	Paint the Panel.
getImage()	Public	Image	Return image stored.
openImage()	Public	Void	Open image given by a string.
setImage()	Public	Void	Set the image being displayed.
DisplayOrig()	Public	Void	Display Original Image.
DisplayEdge()	Public	Void	Display Edge Detected Image.
DisplayDB()	Public	Void	Display Database Image.
CheckImage()	Public	Void	Perform Edge Detection and database comparison.
setImageResize()	Private	Void	Set the image and resize it.
waitForImage()	Private	Void	Load image and wait until loading is complete.
getDatabaseImage()	Private	Image	Load the image from the database.
bytes2Image()	Private	Image	Convert a stream of bytes to an Image.
PerformComparison()	Private	Integer	Map polygon to image, return accuracy.
Compare()	Private	String	Compare polygon with database images, return name of object.
PrepareImage()	Private	Image	Store Image and return edge-detected image.
getPolygons()	Private	Vector	Return polygons around objects in an image.
debug()	Private	Void	Output debugging information to console.
pad()	Private	String	Pad, strings to different lengths.

Class: ImagePrep

Method	Access	Returns	Description
EdgeDetect()	Public	Image	Perform edge detection on an image.
EdgeDetectWiden()	Public	Image	Perform edge detection on an image and widen the edges.
setDimensions()	Private	Void	Store dimensions of image.
subtract()	Private	Short[][]	Subtract one array of short[][] from another.
erode()	Private	Short[][]	Perform erosion on an array of short[][] with a kernel of short[][].
image2Short()	Private	Void	Convert image to three 2D arrays of short (r,g,b).
short2Image()	Private	Image	Convert three 2D arrays of short (r,g,b), to an image.
grey()	Private	Image	Convert image to greyscale.
threshold()	Private	Short[][]	Threshold a 2D array of short to either black(0) or white(255).
Threshold	Private	Image	Perform threshold on all three 2D arrays.
copyRGB()	Private	Void	Copy Red array to green and blue arrays.
insideContour()	Private	Image	Get inside contour of an image.
extendEdges()	Private	Short[][]	Every pixel in an image that is white, make all surrounding pixels white.
extendEdges()	Private	Image	Convert all of 2D arrays to an image with extended images.

Class: Polygons

Method	Access	Returns	Description
GeneratePolygons()	Public	Vector	Get a relative polygon of objects.
mapPolygons()	Public	Integer	Attempt to map polygons on to an image, return percentage accurate.
buildPolygonList()	Private	Void	Build a new polygon list from origin.
buildPolygonList()	Private	Void	Build polygon list onto a polygon.
filterPolys()	Private	Vector	Filter polygons to make them less.
thinPoly()	Private	Polygon	If points are on a line and next to each other, add the polygons together.
nextTo()	Private	Boolean	True if distance between two co-ordinates is less than 2.
onLine()	Private	Boolean	True if three co-ordinates are in a straight line.
max()	Private	Double	Return the maximum of two integers.
nextClosestPoly()	Private	Integer	Find the next closest polygon in polygon list.
combinePolys()	Private	Boolean	If distance between two polygons is less than 60, combine them and return true.
distance()	Private	Double	Return the distance between two polygons.
pointCount()	Private	Integer	Count points in an image, which are non-zero.
image2Short()	Private	Void	Convert image to three 2D arrays of short (r,g,b).
getRelativePolygons()	Private	Vector	Convert a polygon to be relative to point of origin (instead of origin of image).

Class: Database

Method	Access	Returns	Description
openDatabase()	Public	Void	Open the image database with username and password.
closeDatabase()	Public	Void	Clear all connections with the database.
searchDatabase()	Public	Void	Perform a query on the database.
getAllDatabase()	Public	Void	Get all rows and columns in database from search.
getNextImage()	Public	ImageRec	Get the next image record from database or return null.

Class: ImageRec

Method	Access	Returns	Description
ImageRec()	Public		Constructor, creates new record with null values.
ImageRec()	Public		Constructor creates new record with results from query.
getImageName()	Public	String	Return name of image.
getImageLocation()	Public	String	Return image location.
setImageName()	Public	Void	Set name of image.
SetImageLocation()	Public	Void	Set location of image.