[Corpora-List] Coefficients in SVM models (Summary)

Ken Litkowski ken at clres.com
Fri Aug 21 21:18:29 CEST 2015


I would like to thank the respondents who have laid out a "principled" for me to attack the problem. I'll just give a quick summary, referring back to the responses for more details.

Thomas Proisl suggested further implementation of the recursive feature elimination as suggested in the Guyon paper. This is a feasible extension to the Tratz-Hovy system. And since I have independent test sets upon which to gauge performance, I think his other concerns will not be troublesome.

Hady Elsebar suggested using SVMs with the L1 regularization penalty to reduce weights of insignificant features to zero. This is also implementable in the Tratz-Hovy system.

Maximilian Haeussler suggested trying regression or decision trees within the Weka framework. I have extended the Tratz-Hovy system to generate the ARFF files required in Weka, and have been investigating these different methods with some success. The number of features involved pushes Weka to the limits in many occasions, but application of the other techniques will enhance my "playing around."

Behrang Zadeh suggested a bag of concept method for use in projecting the vector space to smaller dimensions. In immersing myself in the data, I have clearly seen considerable redundancy in the feature space, so it will be useful to adapt the bag of contexts (from the text classification paradigm) to deal with this redundancy.

Thanks much,

Ken

On 8/20/2015 2:36 PM, Ken Litkowski wrote:
> Are there any methods in computational linguistics for interpreting
> the coefficients in SVM models? I have 120 nice preposition
> disambiguation models developed using the Tratz-Hovy parser, with an
> average of about 15,000 features for each preposition. I'd like to
> identify the significant features (hopefully lexicographically
> salient). One such method (implemented in Weka) is to square the
> coefficients and to use this as the basis for ranking the features
> (the source of this method being a classic study by Guyon et al.,
> 2002, in gene selection for cancer classification using support vector
> machines <http://link.springer.com/article/10.1023/A:1012487302797>).
> I'm extending these models (which make heavy use of WN) with other
> lexical resources, including FN, VN, and CPA. This will make the
> feature space even more hyperdimensional, so I'd like to pare them
> back in a principled way so I can see the potential contribution of
> these other resources.
>
> Thanks,
> Ken
> --
> Ken Litkowski TEL.: 301-482-0237
> CL Research EMAIL:ken at clres.com
> 9208 Gue Road Home Page:http://www.clres.com
> Damascus, MD 20872-1025 USA Blog:http://www.clres.com/blog
>
>
> _______________________________________________
> UNSUBSCRIBE from this page:http://mailman.uib.no/options/corpora
> Corpora mailing list
> Corpora at uib.no
> http://mailman.uib.no/listinfo/corpora

-- Ken Litkowski TEL.: 301-482-0237 CL Research EMAIL:ken at clres.com 9208 Gue Road Home Page:http://www.clres.com Damascus, MD 20872-1025 USA Blog:http://www.clres.com/blog

-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 4904 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20150821/1ee2b582/attachment.txt>



More information about the Corpora mailing list