Create a python program that takes in a text file and stores in every unique wor

Question

Create a python program that takes in a text file and stores in every unique word in bigram dictionary({}{}).

. Select an appropriate data structure to store bigrams. You'll increment counts for a combination of word and previous word. This means you'll need to keep track of what the previous word was. You will compute the probability of the current word based on the previous word count. Prob(word, Iword..)Count (word.. word,) /Count (word..) Consider we observed the following word sequences: finger remarked finger on finger on finger in finger . Notice that "finger on " was observed twice. Also notice that the period is treated as a separate word. Given the information in this data structure, we can compute the probability p(onlfinger)as 2/50.4. Similarly, we can compute the probability p(.Ifinger) as 1/5 0.2 When complete, add code to write 100 random (you can select a word and bigram term randomly) probabilities to bigram_probs.txt, one per line p(onfinger) 0.4 p(.Ifinger) 0.2

Explanation / Answer

from collections import Counter, defaultdict

li=[w.lower().strip('.,') for w in s.split()]

dd=defaultdict(Counter)

for a,b in zip(li, li[1:]):

    dd[a][b]+=1

>>> dict(dd)

{'the': Counter({'dog': 1, 'cat': 1}), 'chased': Counter({'the': 1}), 'cat': Counter({'chased': 1})}

Related Questions

Navigate

• Browse (All)
• Browse C
• Subjects

---

---

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.