Question

How does the Skip-gram model in Word2Vec differ fundamentally from the CBOW model?

Accepted Answer

The Skip-gram and CBOW (Continuous Bag of Words) models are both word embedding techniques in Word2Vec, but they differ in their approach to predicting words. The CBOW model predicts a target word based on the surrounding context words. It takes multiple context words as input and tries to predict the single target word. In contrast, the Skip-gram model does the opposite: it predicts the surrounding context words based on a single target word. It takes a target word as input and tries to predict the surrounding context words. For example, given the sentence &#x27;The cat sat on the mat&#x27;, the CBOW model might try to predict &#x27;sat&#x27; given the context words &#x27;The&#x27;, &#x27;cat&#x27;, &#x27;on&#x27;, and &#x27;the&#x27;. The Skip-gram model, on the other hand, might try to predict &#x27;The&#x27;, &#x27;cat&#x27;, &#x27;on&#x27;, and &#x27;the&#x27; given the target word &#x27;sat&#x27;. Because Skip-gram is predicting multiple context words from a single target word, it tends to perform better than CBOW, especially with small datasets, and is better at capturing the semantics of less frequent words. CBOW, because it averages the embeddings of multiple context words, is faster to train and performs better with frequent words. In summary, CBOW predicts the target word given the context, while Skip-gram predicts the context given the target.

Home → All Courses → Engineering and Technology Courses → Google AI Certification → Flashcard

How does the Skip-gram model in Word2Vec differ fundamentally from the CBOW model?

Community Answers