EMC D-DS-FN-23 - Dell Data Science Foundations

The metrics used to calculate relevance in text analysis are TF (Term Frequency) and IDF (Inverse Document Frequency). TF measures how frequently a term appears in a document, while IDF adjusts the importance of the term based on how common it is across all documents. Their combination, known as TF-IDF, helps evaluate the relevance of terms in a given document relative to a larger corpus.

MapReduce is designed to process a few large files that are split into blocks and then processed in parallel across multiple machines. This approach allows for efficient distributed processing of large datasets.

K-means clustering is best suited for numeric data, as it relies on calculating the Euclidean distance between data points, which is meaningful for numerical values.

Feature selection involves identifying the most useful input variables that contribute meaningfully to the model, improving accuracy and reducing complexity during the model building phase.

A box and whisker plot is ideal for visualizing the relationship between time spent and product category, especially when you have multiple categories. It shows the distribution of time spent in each product category, including the median, quartiles, and any potential outliers.

The dependent variable in logistic regression is categorical, typically binary (e.g., success/failure, yes/no), since the goal is to model the probability of an event occurring.

Lift measures how many times more often X and Y occur together than would be expected if they were statistically independent. It is expressed as a ratio, and a lift greater than 1 indicates a positive association between X and Y.