|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectuk.ac.shef.dcs.oak.jate.core.npextractor.CandidateTermExtractor
public abstract class CandidateTermExtractor
Extract lexical units from texts. Also defines a number of utility methods for normalizing extracted candidate terms. A candidate lexical unit will be stored in its canonical form in lowercase, depending on the Normalizer provided. Each canonical form maps to several variants found in the document/corpus. When the frequency of terms is counted, each variant will be searched in the document/corpus and the frequency adds up to the total frequency for the canonical term form.
Field Summary | |
---|---|
protected Normalizer |
_normaliser
|
protected StopList |
_stoplist
|
Constructor Summary | |
---|---|
CandidateTermExtractor()
|
Method Summary | |
---|---|
static java.lang.String |
applyCharacterReplacement(java.lang.String string,
java.lang.String pattern)
Replaces [^a-zA-Z\-] characters with " "(white space). |
static java.lang.String[] |
applySplitList(java.lang.String string)
If the input string contains "and", "or" and "," , it is assumed to contain multiple candidate terms, and is split. |
static java.lang.String |
applyTrimStopwords(java.lang.String string,
StopList stop,
Normalizer normalizer)
If a string beings or ends with a stop word (e.g., "the"), the stop word is removed. |
static boolean |
containsDigit(java.lang.String string)
|
static boolean |
containsLetter(java.lang.String string)
|
abstract java.util.Map<java.lang.String,java.util.Set<java.lang.String>> |
extract(Corpus c)
|
abstract java.util.Map<java.lang.String,java.util.Set<java.lang.String>> |
extract(Document d)
|
abstract java.util.Map<java.lang.String,java.util.Set<java.lang.String>> |
extract(java.lang.String content)
|
static boolean |
hasReasonableNumChars(java.lang.String string)
This method is used to check if candidate term is a possible noisy symbol or non-term. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected StopList _stoplist
protected Normalizer _normaliser
Constructor Detail |
---|
public CandidateTermExtractor()
Method Detail |
---|
public abstract java.util.Map<java.lang.String,java.util.Set<java.lang.String>> extract(Corpus c) throws JATEException
c
- corpus
JATEException
public abstract java.util.Map<java.lang.String,java.util.Set<java.lang.String>> extract(Document d) throws JATEException
d
- document
JATEException
public abstract java.util.Map<java.lang.String,java.util.Set<java.lang.String>> extract(java.lang.String content) throws JATEException
content
- a string
JATEException
public static boolean containsLetter(java.lang.String string)
string
- a string
public static boolean containsDigit(java.lang.String string)
string
-
public static java.lang.String applyCharacterReplacement(java.lang.String string, java.lang.String pattern)
string
- input string to be processedpattern
- regular expression pattern defining characters to be replaced by " " (white space)
public static java.lang.String[] applySplitList(java.lang.String string)
string
- input string
public static java.lang.String applyTrimStopwords(java.lang.String string, StopList stop, Normalizer normalizer)
string
- stop
-
public static boolean hasReasonableNumChars(java.lang.String string)
string
-
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |