Since we are currently not developing any new iOS game title we have some time to experiment with other things. Amongst others openCV on iPhone platform.
Because of the current ongoing EK soccer tournament we had our focus on football. More specifically tracking speed of a football. The concept is as follows:
Shoot a short video from somebody kicking a football (on a soccerfield), camera standing as perpendicular as possible to the assumed trajectory of the ball.
Then, based on the size of the ball in the videofootage as detected by openCV and the traveled ball distance from 2 frames in the video (recorded at a known time difference) in pixels. These pixel sizes recalculated to real life dimensions based on the known size of a typical football gives you the ballspeed. in kilometers/hour.
In theory this all sounds ok, but then we still have some practical issues to solve. First of all it turned out out that kicking the ball at fast speed (what real soccer players would do), skewed the ball enormously.
Skewed ball as caused by rolling shutter phenomenon
Here you can see some of the ball skewing. openCV "sees" the b/w filtered image
This is caused by rolling shutter artifacts see wikipedia. iPhone camera images are captured from top to bottom (landscape) line by line and not the whole frame at once. Because of this, a fast moving object will become skewed. openCV will not detect a circle when the circle is really skewed, so that’s the first problem.
Also when in low light situations, motion blurring, because of longer exposure times becomes an issue. The latest iPhone 4S support 1280×720 @60 fps which is the best bet when capping these frames. But even then motion blur can become an issue. Also ball detection is done based on the greenness of the grass. If your whitebalance is much to blue the conversion to black and white in opencv would fail. Of course you could adjust the tolerance for the detected hues but then you’re likely to run into many false positives for ball detections.
Much of these things/issues could have been fixed with some user input (let user select the correct ball, let user adjust filters for openCV based on some kind of wizard), but all in all it was a nice experiment but not good enough to make it into a proper dummy proof app. We basically didn’t want to make an app that expects all kinds of clever input from the user. We wanted the app to do some clever stuff and return you the speed of the ball without too much hassle.
opencv ball detection on the grass