CertNexus AIP-210 - CertNexus Certified Artificial Intelligence Practitioner (CAIP)

Semi-supervised learning is an approach that is best if a limited portion of your training data is labeled. Semi-supervised learning is a type of machine learning that uses both labeled and unlabeled data to train a model. Semi-supervised learning can leverage the large amount of unlabeled data that is easier and cheaper to obtain and use it to improve the model’s performance. Semi-supervised learning can use various techniques, such as self-training, co-training, or generative models, to incorporate unlabeled data into the learning process.

Text vectorization is a technique that converts text into numerical vectors that can be used by machine learning models. Text vectorization can use different methods to represent text features, such as word frequency, word order, word meaning, or word context. Some of the common text vectorization methods are:

TF-IDF: TF-IDF (term frequency-inverse document frequency) is a method that assigns a weight to each word based on its frequency in a document and its rarity across a collection of documents. TF-IDF can capture the importance and relevance of words for a given topic or domain, but it does not consider the order or meaning of words.

Word2vec: Word2vec is a method that learns a vector representation for each word based on its context in a large corpus of text. Word2vec can capture the semantic and syntactic similarity and relationships among words, as well as preserve the order of words.

For an English-to-Spanish translation machine, using Word2vec would be appropriate and correctly defined, because in translation machines, we need to consider the order of the words, as well as their meaning and context.

Technical debt is a metaphor that describes the implied cost of additional work or rework caused by choosing an easy or quick solution over a better but more complex solution. Technical debt can accumulate in ML systems due to various factors, such as changing requirements, outdated code, poor documentation, or lack of testing. Some of the ways to decrease technical debt in ML systems are:

Documentation readability: Documentation readability refers to how easy it is to understand and use the documentation of an ML system. Documentation readability can help reduce technical debt by providing clear and consistent information about the system’s design, functionality, performance, and maintenance. Documentation readability can also facilitate communication and collaboration among different stakeholders, such as developers, testers, users, and managers.

Refactoring: Refactoring is the process of improving the structure and quality of code without changing its functionality. Refactoring can help reduce technical debt by eliminating code smells, such as duplication, complexity, or inconsistency. Refactoring can also enhance the readability, maintainability, and extensibility of code.

Ridge regression is a type of linear regression that adds a regularization term to the loss function to reduce overfitting and improve generalization. Ridge regression is fast and requires low memory and low processing power, as it only involves solving a system of linear equations. Ridge regression can also handle multicollinearity (high correlation among predictors) by shrinking the coefficients of correlated predictors.

Accuracy is a measure of how well a classifier can correctly predict the class of an instance. Accuracy is calculated by dividing the number of correct predictions (true positives and true negatives) by the total number of predictions. True positives are instances that are correctly predicted as positive (belonging to the target class). True negatives are instances that are correctly predicted as negative (not belonging to the target class).

Hyperparameter optimization is the process of finding the optimal values for hyperparameters that control the learning process of a given algorithm. Hyperparameters are parameters that are not learned by the algorithm but are set by the user before training. Hyperparameters can affect the performance and behavior of the algorithm, such as its speed, accuracy, complexity, or generalization. Hyperparameter optimization can help improve the efficiency and effectiveness of the algorithm by tuning its hyperparameters to achieve the best results.

k-means clustering is an algorithm that should be used by the market research team to obtain groups of patients with similar unmet needs. k-means clustering is an unsupervised learning technique that partitions the data into k clusters based on the similarity of the features. The algorithm iteratively assigns each data point to the cluster with the nearest centroid and updates the centroid until convergence. k-means clustering can help identify patterns and segments in the data that may not be obvious or intuitive. References: [K-means clustering - Wikipedia], [How to Run K-Means Clustering in Python]

Underfitting is a problem that occurs when a model learns too little from the training data and fails to capture the underlying complexity or structure of the data. Underfitting can result from using insufficient or irrelevant features, a low complexity of the model, or a lack of training data. Underfitting can reduce the accuracy and generalization of the model, as it may produce oversimplified or inaccurate predictions. Some of the ways to fix underfitting are:

Add features to training data: Adding more features or variables to the training data can help increase the information and diversity of the data, which can help the model learn more complex patterns and relationships.

Increase the complexity of the model: Increasing the complexity of the model can help increase its expressive power and flexibility, which can help it fit better to the data. For example, adding more layers or nodes to a neural network can increase its complexity.

Train the model for more epochs: Training the model for more epochs can help increase its learning ability and convergence, which can help it optimize its parameters and reduce its error.

Getting more training data will not work to fix underfitting, as it will not change the complexity or structure of the data or the model. Getting more training data may help with overfitting, which is when a model learns too much from the training data and fails to generalize well to new or unseen data.

Preexisting bias is a type of bias that originates from historical or social contexts, such as stereotypes, prejudices, or discriminations. Preexisting bias can affect the data or the algorithm used for machine learning, as well as the outcomes or decisions made by machine learning. Preexisting bias can cause unfair or harmful impacts on certain groups or individuals based on their attributes, such as gender, race, age, or disability3. In this case, the software that uses machine learning generates lower employability scores for women and predicts that women, especially with children, are less likely to get a high-paying job. This indicates that the software has preexisting bias against women, which may reflect the historical or social inequalities or expectations in the labor market.