Lecture Notes on Text Mining and Analytics

Introduction to Text Mining and Analytics

• Definition: Text Mining and Text Analytics are roughly the same; used interchangeably.
• Reasoning for Dual Terms:
  • Mining: Emphasizes the process and provides an algorithmic view.
  • Analytics: Focuses on the results or problem-solving aspect.

Purpose of Text Mining and Analytics

• Goal: Turn text data into high-quality information or actionable knowledge.
• Challenge: Dealing with a large amount of text data to extract useful information.

Types of Results

1. High-Quality Information:
   • Concise summaries making it easier for humans to digest.
   • Example: Summary of product reviews highlighting key features like battery life.
2. Actionable Knowledge:
   • Knowledge derived for making decisions or taking actions.
   • Example: Determining the most appealing product for shopping decisions.
   • This can be termed "axiomatic knowledge" as it leads to consumer action.

Relation Between Text Mining and Text Retrieval

• Text Retrieval:
  • Essential component in text mining systems; finding relevant information from text data.
  • Covered in a separate course on text retrieval and search engines.
• Connection:
  1. Pre-Processor for Text Mining: Helps condense large text data into relevant portions.
  2. Knowledge Provision: Verifying discovered knowledge through original text data.

Text Data as a Unique Type of Data

• Concept of Human as Sensors:
  • Text data originates from humans expressing observations about the real world.
  • Comparison to physical sensors (e.g., thermometers, geosensors).
• Integration of Data Types:
  • Both text data (subjective) and non-text data (objective, generated by physical sensors).
  • Non-text data can include numerical, categorical, relational, or multimedia formats.

Importance of Text Data in Data Mining

• Rich Content: Text data contains significant semantic content, user preferences, and opinions.
• Data Mining Goal: Transform a variety of data into actionable knowledge to influence the real world positively.
• Mining Algorithms: Different algorithms needed for various data types, including specialized algorithms for text data.

Course Overview

• Focus on specialized algorithms suitable for text data mining.
• Will cover both general algorithms and those specifically tailored for text.