Shakthydoss

Preface

In  Latent Semantic Analysis – Part 1  i have covered procedure for building Term Document Matrix (TDM) as it is a prerequisite for building LSI model . Now lets see how this TDM is supplied to SVD to obtain U , S, and V matrices.

Objective

To build a Latent Semantic Analysis (LSA) model to find statistical synonyms of a word (Term-Term Similarity) from a huge corpus.

Observation and legends:

Once we have built our  TDM matrix, we call upon a powerful but little known technique called Singular Value Decomposition or SVD to analyze the matrix for us. The reason SVD is useful, is that it  makes the best possible reconstruction of the matrix with the least possible information. To do this, it throws out noise, which does not help, and emphasizes strong patterns and trends, which do help. The trick in using SVD is in figuring out how many dimensions or “concepts” to use when approximating the matrix. In our case , once we have supplied TDM to SVD we will landed up with three matrix name U , S, V .

U matrix generally correspond to term vector coordinate , V matrix correspond to Document vector coordinate. Since our objective is to find  Term-Term Similarity , we will concentrate on U matrix. As i already mentioned that i stated build TDM using java language so it will be continent  to proceed the remain of work will same java language.

SVD is not include with basic jdk , special library files are need to process SVD . JAMA is a linear algebra package for Java , using this library files we could easily process SVD.

But before computing SVD , Let get the query term for which statistical synonym or similar term has to find. Each token in the query have to be passed to stop word list and stemming function. One done this ,find the  know token (which is already present in Count matrix ) and sum their corresponding coordinates then applied weighted function to it.

In my experiment after computing the query terms Q , i kept the Q coordinate at the last index of my TDM , this is help me to find the Cosine relation of last index with remaining indexes.

With help  JAMA library package , i passed the TDM matrix into SVD and then i retrieved U and S matrix.

<code>

Matrix A = new Matrix(Weighted_TDM);

        SingularValueDecomposition svd = A.svd();

        Matrix U = svd.getU();

        Matrix S = svd.getS();

<code>

After getting U and S matrix ,we have to multiple U and S matrix then with the resultant matrix (say T matrix ) we can find Term-Term Similarity using Cosine relation.

                             V . W
cos A  =    —————
||V|| ||w||

for Example
To find the angle between two vector coordinate say
v =  2i + 3j + k
and
       w  =  4i + j + 2k
we compute:

and

and
v ^. w  =  8 + 3 + 2 = 13
Hence

If the angle is between 0 and 90 then there exits a relation (some similarity ) between the two vector coordinated. Similarly we have compute cosine relation between query coordinated and other coordinate. lesser the angle more similarity between the terms.

It was tough for me to work from command prompt this time so i decide to build LSI model with GUI complaints.

Steps for execution.

1. Enter the corpus directory in the specified field.

2. Click Extract Keywords button

3. Enter the query keywords in the specified field.

4. Click Compute Weighted TDM button

5. Click Process Query button

6. Click Compute SVD from Weighted TDM button

Resultant of each event will display separately in differed toggle plane (Tabs).

Once the corpus directory is set and Extract keyword button is clicked the keywords from corpus which is not a stop words and stemmed will be displayed in List of words in corpus tab.

When Compute Weighted TDM button is clicked followed Process Query button is clicked after giving the query in specified field the count matrix , weighted TDM and Sum of query vector coordinate will be separately show in Count matrix tab , Weighted TDM tab ,Query Vector Coordinates tab.

Term similarity with corresponding cosine angle could be seen after clicking Compute SVD from Weighted TDM .

Test Case

To test the model different type of input (query term) has been given to find their statistical similar terms. Tabulation here show some of test case result.

 

Query	Similar terms and their corresponding angle
networking company	expertcosine A = 36.808665072949225 router cosine A = 40.57505845078519 activity cosine A = 43.89788624801361 cisco cosin A = 46.739184943348484 backbone cosin A = 48.831127830382684 bandwidth cosin A = 48.83112783038286 …….

 

Query	Similar terms and their corresponding angle
corporate in India	corporatecosin A = 0.0 mba/pgdm cosin A = 39.23152048359223 deliver cosin A = 50.76847951640772 exchange cosin A = 50.76847951640775 practice cosin A = 56.78908923910091 industrial cosin A = 58.909069642326905 potential cosin A = 58.90906964232696 opportuniti cosin A = 58.90906964232697 organiza cosin A = 59.529640534020224 …….

Result

To my understanding , construction of TDM is important role in LSI model , even though  we have removed the stop words and infection the accuracy get increased only after manually removing noise words which are neither stop word nor infection. In My experiment i encountered noise words such as numbers , page numbers , date ,  some symbols notation etc. It has also observed that no .of keywords becomes less when corpus below to same domain, if corpus consists of different domain then list of keywords increase tremendously.

Thus we have successfully found statistical similar terms for inputted query.

Download : LSI Model