Given an article, design an algorithm in pseudo code to find the top 150 most fr

Question

Given an article, design an algorithm in pseudo code to find the top 150 most frequently co-occurring word-pairs in the article. Two words are said to co-occur if they appear in the same sentence. For example, the sentence, “It’s really a milestone in Chinese science fiction.” contain the following word pairs:

('It's', 'really')

('It's', 'a')

('It's', 'milestone')

('It's', 'in')

('It's', 'Chinese')

('It's', 'science')

('It's', 'fiction')

('really', 'a')

('really', 'milestone')

('really', 'in')

('really', 'Chinese')

('really', 'science')

('really', 'fiction')

('a', 'milestone')

('a', 'in')

('a', 'Chinese')

('a', 'science')

('a', 'fiction')

('milestone', 'in')

('milestone', 'Chinese')

('milestone', 'science')

('milestone', 'fiction')

('in', 'Chinese')

('in', 'science')

('in', 'fiction')

('Chinese', 'science')

('Chinese', 'fiction')

('science', 'fiction')

you can assume you have access to a subroutine, sentenceSplitter(article), that can accurately segment an article into separate sentences and return these sentences in an array-like structure.

You can also assume that you have access to another routine tokenizer( sentence), that can accurately identify the individual words contained in the input sentence and return these words in another array-like data structure.

Please describe rhe algorithm unambiguously using pseudo code with necessary comments in English.

Don't implement the algorithm, only the pseudo code is required.

Explanation / Answer

#read the file into the program

file = read_csv("file path")

# use sentence splitter to split the sentences and store in sentences data structure may be list

sentences = sentenceSplitter(file)

# take each sentence and tokenize each word

for each sentences

     words = tokenizer( sentences[i])

     # for each word pair with other word except the same word.

      for each word:

            list_pairs = list of word pair

print list of word pairs

Related Questions

Navigate

• Browse (All)
• Browse G
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.