Advantages of CBOW (Continuous Bag of Words)
1️⃣ Faster Training
CBOW uses multiple context words together to predict the target word.
Example:
KRISH NAME RELATED TO → IS
Since several words are used at once, the model learns faster.
So CBOW is computationally efficient and fast to train.
2️⃣ Works Well for Frequent Words
Common words appear many times in the dataset.
Example:
the, is, data, science
Since CBOW sees these words many times, it learns good embeddings for frequent words.
3️⃣ Simple Architecture
CBOW has a simple neural network structure:
Context Words → Hidden Layer → Target Word
Because of this simple structure, it is easy to train and requires less computation.
Disadvantages of CBOW
- Word Order is Ignored
CBOW treats words like a bag of words.
Example:
dog bites man
man bites dog
Both sentences contain the same words.
CBOW may treat them similarly, even though their meanings are different.
So word order information is lost.
2. Not Good for Rare Words
If a word appears very few times in the dataset, CBOW cannot learn its meaning properly.
Example rare word:
astrophysics
Since it appears rarely, the model may not learn a good embedding for it.
3. Loss of Context Information
In CBOW, all context word vectors are averaged into a single vector.
(v1 + v2 + v3 + v4) / 4
This averaging can lose some important semantic information.