Wordnet Query Processing Performance - Can Multithreading Speed Up Queries

It takes too much time to identify process which takes words identification as its Part Of Speech and finds synonyms related to that. For that when try to implement threading on that it will not allow to process that function prallel in threading. Any one have idea about how to do that prallel work with use of threading which help to improve application performance. _(From http://opensource.ebswift.com/WordNet.Net/Forum/viewtopic.php?p=164#164)_

Quick Answer Multithreading may not make your program find results any faster. Putting operations in a thread can make your application's user interface "seem" more responsive, but the operation running on a background thread will probably take the same amount of time. There is also the issue of thread safety.

I think what you are asking is how to speed up the operation of discovering the part of speech and synonyms for words, and you're wondering if multi-threading will improve the speed of your application.

I have been using multi-threading to improve the query performance of an applicationt that is download Amazon.com information and building a local database. But the multithreading was to help with letting my program work while waiting on a query response from other computers. While waiting on the response from AWS, (which while often finishing in a tenth of a second or less, can sometimes take several seconds), my computer would be doing nothing if I didn't have multi-threading. I have even rewritten some of the background threads to use asynchronous calls to AWS. But the reason I did that was not to get more performance, it was to make sure the synchronous SOAP calls don't hang the thread by not returning, which happens once in every 20K-50K calls. By doing the call asynchronously, the thread can decide to abandon the call, and retry it. Setting this up considerably increased the complexity of my code (especially since the synchronous call was in a subroutine used by several threads), although it's not rocket science.

It is a different situationt than working with a Wordnet library such as the Wordnet.NET library

I've played with the desktop app a little bit, but am focusing most of my attention on the SQL Version of Wordnet Database.

Some issues you raised:

• It takes too long to identify words
  • I have only looked briefly at the code for Wordnet.NET. The "Binary Search" method appears to be a method of successive approximations - It jumps into the file in the middle, and figures out if your target word is before or after the next word it finds. If before, it jumps halfway into the section before the word, if after then it jumps halfway into the section after. It repeats this until it finds your word. This might require a lot of file IO. Each new query starts over.
  • The Wordnet ASCII based files could have an index generated for them with the offsets of words in the ASCII. This would considerably speed the lookups by allowing information to be located instantly.
• Improve application performance
  • Are you
    • trying to improve the responsiveness of a UI, or
    • are you trying to handle more batch operations in a particular period of time?
  • You can easily improve the UI responsiveness of a UI by putting long operations on a background thread. However, the work still takes the same amount of time.
  • In a case where your computer is doing all the work, it's the same amount of work to do whether it is in a single thread, or in multiple threads. In fact, having the work divided between a large number of multiple threads may slow the process down because of overhead in switching threads. In general you'll only speed things up if the threads are depending on information from some other computer. If it is your own machine that is CPU or IO bound, it will probably stay CPU or IO bound even with multiple threads.
  • Is there repetition in the words that your app is analyzing? You might write a layer between your app and WordNet.NET that would cache the answers you get from WordNet.NET.
• Thread safety - "will not allow to process that function prallel in threading"
  • I don't know if the Wordnet.NET library has been written to be thread safe. If the routines are relying on any global variables that are storing values specific to particular calls into the library, then it is probably not thread safe.

Other ideas that could increase speed:

• Placing Wordnet data files on a high speed flash drive.

Misspellings:

• sppech