Sunday, December 20, 2015

Using SIFT/SURF for Object Recognition in OpenCV Java

Hi All,

Today my post is on, how you can use SIFT/SURF algorithms for Object Recognition with OpenCV Java. I have shared this post on SURF feature detector previously. This is fully based on that post and therefore I'm just trying to show you how you can implement the same logic in OpenCV Java. It's important that you have to download previous OpenCV versions, so that you have SURF feature detector in your library. Because in the newer versions they have removed these Non-Free modules from the java wrapper. You can check this out.

I believe that you have some basic knowledge in working with OpenCV Java. If you want some beginner help you can refer the following links.
Now let's move on to our development. I'm using Eclipse for developing this. I'm going to use OpencCV 2.4.11. You can download it here

First create the user library for OpenCV as described in the previous link and add it to the build path. Then we can start developing the code for object recognition. Following is my eclipse project. I have added the OpenCV 2.4.11 library as a user library and added it to the build path. 

I'm using following images for object recognition. 

1. Object we are going to recognize. 

2. Scene that we are going to recognize the object from. 

Now following is the Java implementation of our SURF feature detector. 



Following are our outputs. 

1. Identified key-points of the object.



2. Matching the object keypoints with the scene. 


3. Object recognized image. 


You can do the same with SIFT feature detector by just changing the Feature Detector and Descriptor Extractor name to SIFT. 

So that's it. 

Hope that helps. 

Thank You. :-) 



84 comments:

  1. Great Help...& Great Post..Keep It Up..! Thank You.

    ReplyDelete
  2. Very sweet. Thanks for posting this! https://twitter.com/thefuhrmanator/status/717830067586539520

    ReplyDelete
    Replies
    1. Thank You Cris. I saw the tweet. Glad it helped. :-)

      Delete
  3. This comment has been removed by the author.

    ReplyDelete
  4. Wow! It's worked great for me.
    The only post that described Sift with java so easily and can be understood in no time.

    ReplyDelete
  5. Hi Kinath,
    I want to construct Bag of Visual Words (dictionary) using extracted sift features(which can be for set of images) using java.
    I am using opencv. But the mainclass needed for this purpose (BOWTrainer and BOWKMeansTrainer) can not be found in opencv for windows. Its not availble. Is there any alternative or still it can be possible with OpenCv Java

    ReplyDelete
    Replies
    1. Hi Vishal,I also once tried to work with BOW in OpenCV java. But I couldn't find a complete solution. If you are not dependent on the programming language, I would suggest you to use the EMGUCV wrapper for C#. They have developed a very rich wrapper for OpenCV.

      Delete
  6. Hi,
    Thank you so much for reply and suggestion. I want to do the task in java only, so I am trying to use one of the java implementations available for image processing like OpenIMAJ, LIRE etc.
    Thanks.

    ReplyDelete
    Replies
    1. That would be a great choice :-) Good Luck with your research.

      Delete
  7. I want code for SIFt methos i tried as u told but it didnot work .please send me code

    ReplyDelete
    Replies
    1. hello send me some suggestion.
      i have error
      OpenCV Error: Assertion failed (npoints >= 0 && points2.checkVector(2) == npoints && points1.type() == points2.type()) in cv::findHomography, file ..\..\..\..\opencv\modules\calib3d\src\fundam.cpp, line 1074
      Exception in thread "AWT-EventQueue-0" CvException [org.opencv.core.CvException: cv::Exception: ..\..\..\..\opencv\modules\calib3d\src\fundam.cpp:1074: error: (-215) npoints >= 0 && points2.checkVector(2) == npoints && points1.type() == points2.type() in function cv::findHomography

      Delete
    2. Hi, for the SIFT method it would be the same. You just have to change the Feature Detector from SURF to SIFT. Did you check with the images I have provided? If it works for both you will have to adjust the 'nndrRatio = 0.7f' and test your results. For a detailed description of the logic please refer to the following link which was the original post written with C++.

      'http://morf.lv/modules.php?name=tutorials&lasit=2'

      There should be an error if you are using the same code and same libraries I have specified. Please check the OpenCV version you are using. I have used 2.4.11. And if you are new to OpenCV with java, please refer to the beginner links I have provided.

      Delete
  8. Hello,Can u please send me SIFT code , i changed SURF to SIFT but not working

    ReplyDelete
  9. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. I am trying to make this work, but with this example, the object is not found. Can you help me? Thank you.

      Delete
    2. Hi bruno, Sorry for the late reply. Could you send me the stack trace?

      Delete
    3. "the object is not found"
      try original images (size 1600x1192 for bookobject and 1600x900 for bookscene). For lower resolution this code not working.

      Delete
  10. I get following error:
    trainIdx can't be resolved or is not a field
    queryIdx can't be resolved or is not a field. Pls help!

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. This comment has been removed by the author.

      Delete
    3. you can try to setup the variable goodMatchesList to LinkedList and objKeypointlist to List

      Delete
    4. Am having the same problem unluckily issues still wasn't solved, any suggestions?

      Delete
    5. hi matt,
      Could you please send me your code, so that I could check and verify it. Hopefully this code should work without any issue.

      Thanks,
      Kinath.

      Delete
    6. I am also having same error and till not solved any suggestion.

      Delete
    7. Here is exception
      Exception in thread "main" java.lang.Error: Unresolved compilation problems:
      queryIdx cannot be resolved or is not a field
      trainIdx cannot be resolved or is not a field

      at SURFDetector.main(SURFDetector.java:110)

      Delete
    8. Hi Kinath, first of all thank you very much for your swift reply. I have found the following errors when following exactly the code described in this tutorial:
      queryIdx cannot be resolved or is not a field
      trainIdx cannot be resolved or is not a field

      I have tried finding a solution but i am having quite some trouble. Hope to hear from you soon and thanks again for the help :)

      Delete
    9. Hi Matt, I tried with my code again and it worked fine with me. I have uploaded this code in GitHub and you could download it and try the code. Here's the link.

      https://github.com/kinathru/SURFDetector

      Delete
    10. Hi Kinath, it worked thanks for the help and support :)

      Delete
    11. Hi Kinath in order to train an SVM using SURF/SIFT you need to pass on the features extracted or the resultant descriptors extracted? have you ever tried it?

      Delete
    12. Hi Matt,

      I'm not really sure how to do it. I think we need to do a bit of a research on how to do that.

      Delete
  11. Hi Kinath
    I got following exception

    OpenCV Error: Bad argument (Specified feature detector type is not supported.) in create, file /home/hassan/OpenCV/modules/features2d/misc/java/src/cpp/features2d_manual.hpp, line 155
    Exception in thread "main" CvException [org.opencv.core.CvException: cv::Exception: /home/hassan/OpenCV/modules/features2d/misc/java/src/cpp/features2d_manual.hpp:155: error: (-5) Specified feature detector type is not supported. in function create
    ]
    at org.opencv.features2d.FeatureDetector.create_0(Native Method)
    at org.opencv.features2d.FeatureDetector.create(FeatureDetector.java:101)
    at SURFDetector.main(SURFDetector.java:59)

    How can i solve that ecxeption

    ReplyDelete
    Replies
    1. Hi Hassan, What is your OpenCV version? Please note that in the recent updates, they have removed SIFT/SURF detectors from the wrapper. I have used 2.4.11 for my developments. Could you please try with that?

      Delete
    2. My OpenCV version is 3.1.0.
      Is there any way to use SIFT/SURF in Latest OpenCV version or i have to move to the previous version??

      Delete
    3. Hi Hassan, Check out the following link. It will help you.

      http://stackoverflow.com/questions/27418668/nonfree-module-is-missing-in-opencv-3-0

      Delete
    4. Hi,

      What a great tutorial to start with!!

      It would be much more helpful if you let me know how to display the number of matched lines between the two Images.

      Thanks,

      Delete
    5. Thnaks Kinath its working fine.One more question as i am new in opencv could you please tell me which piece of code determine wether two images are same or not.

      Delete
    6. Hi Hassan, To my knowledge, it's not possible to point out a particular code block. Here it does not actually compare if two images are same or not. What it does is identifying a set of key points from the object image and comparing it with the identified key points of the scene image. Then those key points are matched with the KNN matching algorithm. If there are good matches, the keypoints with that match are identified as the area of the image similar to the object that we are asking to compare. You can read more about SIFT/SURF algorithms from following articles. :-)

      http://www.vision.ee.ethz.ch/~surf/eccv06.pdf
      https://en.wikipedia.org/wiki/Scale-invariant_feature_transform

      Delete
  12. Hello,

    On line 80, float nndrRatio = 0.7f;. What is the purpose of this? Why the number 0.7? I just want a basic knowledge of this to help me further on

    ReplyDelete
  13. hello ,
    we are getting error as follows:
    at java.lang.Runtime.load0(Unknown Source)
    at java.lang.System.load(Unknown Source)
    at com.dummys.learning.SURFDetector.main

    ReplyDelete
  14. Hi
    I want to ask that this coding is helpful for Object detection using SURF of static images saved in data set or drawable folder in Android Studio?

    ReplyDelete
    Replies
    1. Hi Geminie,
      Did you try to run it on Android studio? Did it work fine?

      Delete
  15. Hey plzzz answer my question its too urgent....

    ReplyDelete
    Replies
    1. Hi Ali, I'm sorry I couldn't get to you so quickly. I am not that familiar with Android Development. But to my knowledge you can use this code along with Android Studio. I don't think I have enough knowledge to tell you how to integrate this with your Android project.

      Delete
  16. Hi Kenath,
    Your post has been of great help, Thank you so much! I just wanted to know if the same code could be available in Java too?

    ReplyDelete
    Replies
    1. Hi mithila, it's already in Java :-)

      Delete
    2. I am sorry, but I mean as in complete Java, not using Opencv Java! Because due to the functions been used, I can't understand the working of the code. Thank you.

      Delete
    3. Hi Mithila,
      I guess not. Without using opencv you cant do it with java. If you are using any other language like c#, there also you will have to use opencv wrapper to access sift/surf feature detectors. :-)

      Delete
    4. Alright, thank you so much for your reply.

      Delete
  17. Hi Kenath,
    tengo este error
    OpenCV Error: Assertion failed (!outImage.empty()) in cv::drawKeypoints, file ..\..\..\..\opencv\modules\features2d\src\draw.cpp, line 115
    Error:CvException [org.opencv.core.CvException: cv::Exception: ..\..\..\..\opencv\modules\features2d\src\draw.cpp:115: error: (-215) !outImage.empty() in function cv::drawKeypoints

    ReplyDelete
    Replies
    1. Hi kevin, what is the opencv version you are using?

      Delete
    2. I have same problem,open cv version 2.4.11

      Delete
  18. hi Kinath
    I have this problem,can you help me

    OpenCV Error: Bad argument (Specified feature detector type is not supported.) in cv::javaFeatureDetector::create, file c:\builds\master_packslaveaddon-win64-vc12-static\opencv\modules\features2d\misc\java\src\cpp\features2d_manual.hpp, line 155
    Exception in thread "main" CvException [org.opencv.core.CvException: cv::Exception: c:\builds\master_packslaveaddon-win64-vc12-static\opencv\modules\features2d\misc\java\src\cpp\features2d_manual.hpp:155: error: (-5) Specified feature detector type is not supported. in function cv::javaFeatureDetector::create
    ]
    at org.opencv.features2d.FeatureDetector.create_0(Native Method)
    at org.opencv.features2d.FeatureDetector.create(FeatureDetector.java:87)
    at SURFDetector.main(SURFDetector.java:42)

    ReplyDelete
    Replies
    1. Hi Khalid,
      Can you clone the following project from github and check if it works for you?

      https://github.com/kinathru/SURFDetector

      Delete
  19. Hi Kinath, can you tell me how to set limit on SIFT/SURF detector for detecting exact number of KeyPoints, let's say 30 points..
    I am using openCv 2.4.1.3 in java

    ReplyDelete
  20. Hello Kinath, Do you have Object Recognition implementation by other algorithms (e.g. ORB / BRISK / FREAK ) or anything different which gives same good results as provided by this SURF. (As SURF is patented and need licensing to use) I need other alternative free to use algorithm implementation in java for feature detection.

    ReplyDelete
  21. Hello Kinath !
    First of all, thanks for sharing your code, there are so few examples of using SURF/SIFT using Java !

    But, I can't make your code work. I got the following error (and still don't understand why) :
    Error : impossible to find or load he main class SIFTDetector

    Usually, it means that the class I want to execute isn't a "main" class, but here, it is... So I REALLY don't get it.

    ReplyDelete
    Replies
    1. Hi, can you check my project on github?

      https://github.com/kinathru/SURFDetector

      Delete
  22. Hi Kinath,
    I get that error,

    Exception in thread "main" java.lang.NullPointerException
    at com.dummys.learning.SURFDetector.main(SURFDetector.java:31)

    Thank u

    ReplyDelete
  23. Hi, how we can recognize the image ? like the image is "book"

    ReplyDelete
    Replies
    1. If you want to do something like that then you would need to use some other AI technique along with this. I would suggest using Java Machine Learning library as a start. Here we are just matching a given object from an image that consists the object.

      Delete
  24. hello kinath
    i got an error
    java.lang.unsatisfiedlinkerror can't load ia 32-bit .dll on a amd 64-bit platform

    ReplyDelete
    Replies
    1. Hi,
      Can you checkout this from my github project and check whether you get the error?

      https://github.com/kinathru/SURFDetector

      Delete
  25. Hi,
    Your code is working fine for your example, but it quite useless for other images (it always return positive result), example:

    http://imgur.com/Z4OqDYd

    ReplyDelete
    Replies
    1. if (m1.distance <= m2.distance * nndrRatio)
      goodMatchesList.addLast(m1);
      ...
      if (goodMatchesList.size() >= 7)
      ...
      Not the best comparation images method

      Delete
    2. Hi Oleksandr, Thank you for your valuable commoments. Yeah, you are correct :-) . This is not the best implementation. But I wanted to demonstrate how we can use SIFT/SURF algorithms using java and OpenCV. So, your ideas for a better implementation is very welcome :-)

      Delete
  26. can you help to tracking the templeate

    ReplyDelete
  27. Hi Kinath.. I'm totally new to OpenCV.. I'm trying to build an app that determines one's location in an indoor environment using object/image recognition. Right now I am simply learning how to match two images using SURF. I ran your code using the exact bookimage and bookobject jpg files and the output is "Object not found".. So I'm wondering if I am doing anything wrong or if you had faced anything like this when you were testing it. Looking forward to your response.
    Thanks,
    S

    ReplyDelete
    Replies
    1. Hi,

      Can you try with the code I have uploaded to github here?
      https://github.com/kinathru/SURFDetector

      I think the issue is that there are not enough matching key points.

      Delete
  28. Hi Kinath, I was wondering if you have ever built an android app that uses SURF.. clicks a picture and returns the matches it has with a picture stored in a folder.
    Thanks,
    S

    ReplyDelete
  29. Hi Kinath,

    Can we find the rotation angle of images

    ReplyDelete
  30. hi Kinath,
    this code is not working between fingerprint image.can you please,post fingerprint detection using opencv surf

    ReplyDelete
    Replies
    1. Hi mahaveer,
      I haven't tried that with SURF. But I will do some research on that and let you know. But can you check the following link?

      https://www.packtpub.com/books/content/fingerprint-detection-using-opencv-3

      Delete
  31. This comment has been removed by the author.

    ReplyDelete
  32. I want to use this code for Object detection using SURF of static images.I integrated this code with android studio without and errors it is executing well, but it's giving the fault result(it's giving the goodmathes result less than 5 for same images also).Below are my work flow
    1)I captured tree images and stored in my drawable.
    2)By using camera i captured same image(Tree) and start compare with the drawable images.
    3)Every time it is giving the goodmathes size is less than 5.
    Could you please help me, it is too urgent .

    ReplyDelete
    Replies
    1. Hi, I think you would need to play with the nndrRatio and good matches size to fine tune your results. Through that you might be able to retrieve better results. And the following article of the original c++ implementation would give you an idea.

      https://morf.lv/complete-guide-using-surf-feature-detector#.VjRLBVN2f3s.blogger

      Delete
  33. Very good tutorial.. but are it possible if we change the clasifier to SVM (Support Vector Machine) instead of KNN ?

    ReplyDelete
  34. Very good, but can you tell me why it wont work with the images in lower resolution like mention before "the object is not found"
    try original images (size 1600x1192 for bookobject and 1600x900 for bookscene). For lower resolution this code not working."

    ReplyDelete