Results 261 to 270 of about 31,193 (295)

Application of Customized Term Frequency-Inverse Document Frequency for Vietnamese Document Classification in Place of Lemmatization

2021
Natural language processing (NLP) is a problem which attracts lots of attention from researchers. This study analyzes and compares a different method to classify text sentences or paragraphs in Vietnamese into different categories. The work utilizes a sequence of techniques for data-preprocessing, customize learning model and methods before using Term ...
Do Viet Quan, Phan Duy Hung
openaire   +1 more source

TF-SIDF: Term frequency, sketched inverse document frequency

2011 11th International Conference on Intelligent Systems Design and Applications, 2011
Exact calculation of the TF-IDF weighting function in massive streams of documents involves challenging memory space requirements. In this work, we propose TF-SIDF, a novel solution for extracting relevant words from streams of documents with a high number of terms.
Manuel Baena-García, José M. Carmona-Cejudo, Gladys Castillo, Rafael Morales Bueno   +3 more
openaire   +1 more source

Generalized inverse document frequency

Proceedings of the 17th ACM conference on Information and knowledge management, 2008
Inverse document frequency (IDF) is one of the most useful and widely used concepts in information retrieval. There have been various attempts to provide theoretical justifications for IDF. One of the most appealing derivations follows from the Robertson-Sparck Jones relevance weight. However, this derivation, and others related to it, typically make a
openaire   +1 more source

Stock Market Prediction Based on Term Frequency-Inverse Document Frequency

Journal of Economics, Business and Management, 2016
This paper presents a new method to predicting the change of stock prices by utilizing text mining news of the stock market. Term Frequency Inverse Document Frequency (TF- IDF) is one of the most useful and widely used concepts in information retrieval.
Mu’tasem Jarrah, Naomie Salim
openaire   +1 more source

A question answer chatbot using term frequency-inverse document frequency and Markov chain

International Conference on Computer Graphics, Artificial Intelligence, and Data Processing (ICCAID 2023)
Shuang Li, Yu Che, Jing Yang, Qianyu Li, Tian Tian   +4 more
exaly   +2 more sources

Subjectivity Classification of Filipino Text with Features Based on Term Frequency -- Inverse Document Frequency

2013 International Conference on Asian Language Processing, 2013
Subjectivity classification classifies a given document if it contains subjective information or not, or identifies which portions of the document are subjective. This research reports a machine learning approach on document-level and sentence-level subjectivity classification of Filipino texts using existing machine learning algorithms such as C4.5 ...
Ralph Vincent J. Regalado, Jenina L. Chua, Justin L. Co, Thomas James Z. Tiam-Lee   +3 more
openaire   +1 more source

Understanding inverse document frequency: on theoretical arguments for IDF

Journal of Documentation, 2004
The term‐weighting function known as IDF was proposed in 1972, and has since been extremely widely used, usually as part of a TF*IDF function. It is often described as a heuristic, and many papers have been written (some based on Shannon's Information Theory) seeking to establish some theoretical basis for it.
openaire   +1 more source

PU text classification enhanced by term frequency–inverse document frequency‐improved weighting

Concurrency and Computation: Practice and Experience, 2013
SUMMARYTerm frequency–inverse document frequency (TF–IDF), one of the most popular feature (also called term or word) weighting methods used to describe documents in the vector space model and the applications related to text mining and information retrieval, can effectively reflect the importance of the term in the collection of documents, in which ...
Tao Peng 0003, Lu Liu 0013, Wanli Zuo
openaire   +1 more source

Inverse Document Frequency (IDF): A Measure of Deviations from Poisson

1999
Low frequency words tend to be rich in content, and vice versa. But not all equally frequent words are equally meaningful. We will use inverse document frequency (IDF), a quantity borrowed from Information Retrieval, to distinguish words like somewhat and boycott.
Kenneth Ward Church, William A. Gale
openaire   +2 more sources

Embedding term similarity and inverse document frequency into a logical model of information retrieval

Journal of the American Society for Information Science and Technology, 2003
AbstractWe propose a novel approach to incorporate term similarity and inverse document frequency into a logical model of information retrieval. The ability of the logic to handle expressive representations along with the use of such classical notions are promising characteristics for IR systems.
David E. Losada, Alvaro Barreiro
openaire   +1 more source