TFDF, not TF-IDF in Financial Analysis

Textual analysis in the realm of business depends on text-processing techniques borrowed mainly from information retrieval. Yet, these text-processing techniques are not viable in text-based financial forecasting. In this paper, we suggest developing financial home-grown techniques for processing textual data, specifically in the course of scoring words where standard techniques are not appropriate in financial analysis. On that matter, we pursue two issues. First, we examine major information retrieval heuristics, where we find TF-IDF too facile not only in predicting trends but also in generating accurate results (in terms of errors) on large numbers in text-based financial analysis. Second, we work on a new heuristic satisfying financial concerns. We consider the relationship between the publication rate of information and its importance. The proposed heuristic provides results of unmatchable performance in both predicting trends and precision measures. In an additional analysis, we optimize our scheme using a genetic algorithm as an optimization technique and get greater precision. In comparison with TF-IDF, our proposed heuristic conduces to a 38.5 percent lower error in closeness measures which is again reduced by 16.46 percent with the help of a genetic algorithm. Our findings suggest that researchers in the field of financial textual analysis should not rely on standard information retrieval heuristics.