Problem: The attached corpus is a story of "Alice in wonderland". Read the croups and perform the required text processing and text cleaning tasks. Then, segment the corpus into a list of sentences. Build Trigram Language model with Laplace smoothing technique to generate 5 sentence of size 10 tokens each, given that you entered the first two tokens for each sentence. The program must allow user to enter the first two input tokens. Finally, evaluate your language model using perplexity.

Problem: The attached corpus is a story of "Alice in wonderland". Read the croups and perform the required text processing and text cleaning tasks. Then, segment the corpus into a list of sentences. Build Trigram Language model with Laplace smoothing technique to generate 5 sentence of size 10 tokens each, given that you entered the first two tokens for each sentence. The program must allow user to enter the first two input tokens. Finally, evaluate your language model using perplexity.

Database System Concepts

7th Edition

ISBN:9780078022159

Author:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Publisher:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Chapter1: Introduction

Section: Chapter Questions

Problem 1PE

See similar textbooks

Similar questions

In Python, with use of function and input parsing/ Make a method is_anagram_of(a,b) that tests if a is an anagram of b. A string a is an anagram of a string b, if it uses exactly the same letters, but the order can be different. Spaces are ignored, as well as capitalization. Examples of anagrams: "eleven plus two" - "twelve plus one" "William Shakespeare" - "I am a weakish speller" "Tom Marvolo Riddle" - "I am Lord Voldemort" "Anagrams" - "Ars manga" "television ads" - "enslave idiots" Counter examples: "bla" - "aalb" "cat" - "tact" Hint: Make a dictionary that holds how often a letter occurs in a word.
please code in python MCQ are a very popular form of assessment because they can be automatically graded…and the students can choose an answer at random if they don’t know.The goal of this exercise is to write a function scoreMCQ(attempt, correct), where attempt is a string that contains the answers of the student and correct is a string with the correct answers.The function should return the number of correct answers.Check first that both strings have the same length, otherwise raise an exception.If both strings are empty the score should be 0 of course.To test your code, you can copy paste the following assert statements as your main function:assert(scoreMCQ("","")==0), "First test failed"assert(scoreMCQ("TFTF","TTTT")==2), "Second test failed"assert(scoreMCQ("TTTT","TTTT")==4), "Third test failed"assert(scoreMCQ("CCCC","ABCD")==1), "Fourth test failed"print("All tests succesfull")
2. Java considers the String data type as non-primitive, because it contains a sequence of charactersand hence there is a predefined class called String dedicated to String processing. The String classhas a lot of functions and continues to grow. Seehttps://www.tutorialspoint.com/java/java_strings.htm for more on strings. Your task is to then createa java program that when give a sentence will replace the first occurrence of NUST with NamibiaUniversity of Science and Technology.Sample run 1:Enter a sentence: Welcome to my NUST.Output: Welcome to my Namibia University of Science and Technology.Sample run 2:Enter a sentence: The nust is a great institution of high learning.Output: The Namibia University of Science and Technology is a great institution of high learning NB: IN JAVA PROGRAMMING
Java considers the String data type as non-primitive, because it contains a sequence of characters and hence there is a predefined class called String dedicated to String processing. The String class has a lot of functions and continues to grow. See https://www.tutorialspoint.com/java/java_strings.htm for more on strings. Your task is to then create a java program that when give a sentence will replace the first occurrence of NUST with Namibia University of Science and Technology. Sample run 1: --- Input ---Welcome to my NUST --- Expected output (text)---Welcome to my Namibia University of Science and Technology. Sample run 2: --- Input ---The nust is a great institution of high learning. --- Expected output (text)---The Namibia University of Science and Technology is a great institution of high learning. Sample run 3: --- Input ---The NuSt was renamed from Polytechnic Of Namibia under Act No. 7 of 2015. --- Expected output (text)---The Namibia University of Science and…
Java considers the String data type as non-primitive, because it contains a sequence of characters and hence there is a predefined class called String dedicated to String processing. The String class has a lot of functions and continues to grow. See https://www.tutorialspoint.com/java/java_strings.htm for more on strings. Your task is to then create a java program that when give a sentence will replace the first occurrence of NUST with Namibia University of Science and Technology. Sample run 1: Enter a sentence: Welcome to my NUST. Output: Welcome to my Namibia University of Science and Technology. Sample run 2: Enter a sentence: The nust is a great institution of high learning. Output: The Namibia University of Science and Technology is a great institution of high learning.
A spell checker in a word processing program makes suggestions when it finds a word not in the dictionary. To determine what words to suggest, it tries to find similar words. One measure of word similarity is the Levenshtein distance, which measures the number of substitutions, additions, or deletions that are required to change one word into another. For example, the words spit and spot are a distance of 1 apart; changing spit to spot requires one substitution (i for o). Likewise, spit is distance 1 from pit since the change requires one deletion (the s). The word spite is also distance 1 from spit since it requires one addition (the e). The word soot is distance 2 from spit since two substitutions would be required. a. Create a graph using words as vertices, and edges connecting words with a Levenshtein distance of 1. Use the misspelled word “moke” as the center, and try to find at least 10 connected dictionary words. How might a spell checker use this graph? b. Improve the method…
Artificial Intelligence- Java program Write code that accomplishes the following tasks. Consider two bags that can hold strings. One bag is named letters and contain seeral one-letter strings. The other bag is empty and is namedvowels. One at a time removed a string from letters. If the string contain a vowel place it into bag vowels, otherwise, discard the string. After you have checked all of the strings in letters, report the number of vowels and the number of times each voewl appears in the bag.
Python Question QUESTION 1 Recursively count the occurrences of a substring. Implement a function that counts the number of times a target string occurs in the main string. In particular, provide the implementation for count(s, target), where 's' is the main string and 'target' is the substring you wish to count. Note that 'target' is not just a single character, but a substring of any length greater than zero (length guaranteed to be positive). Some of the sample cases include, but are not limited to, the following. Input: s= 'aaabbb', target = 'aa' Output: 2 Input: s= 'bbaaa', target = 'c' Output: 0 Input: s= 'abcbc', target='bcb' Output: 1 Input: s = 'aab', target='bbbb' Output: 0 Here are the constraints: • You mustn't use any of the string methods provided by Python. (Indexing is okay) • You must use recursion. • Write a paragraph detailing the base and recursive cases. Your answer should also include the *why* and *how*.
The union of two sets A and B is a set that contains all the elements that are in A together with all the elements that are in B. The intersection of A and B is the set that contains elements that are in both A and B. The difference of A and B is the set that contains all the elements of A except for those elements that are also in B.Suppose that A and B are variables of type set in Java. The mathematical operations on A and B can be computed using methods from the Set interface. In particular: A.addAll(B) computes the union of A and B; A.retainAll(B) computes the intersection of A and B; and A.removeAll(B) computes the difference of A and B. Write a program that can be used as a "set calculator" for simple operations on sets of non-negative integers. (Negative integers are not allowed.) A set of such integers will be represented as a list of integers, separated by commas and, optionally, spaces and enclosed in square brackets. For example: [1,2,3] or [17, 42, 9, 53,108]. The…
In python, Problem Description:Sheldon and Leonard are physicists who are fixated on the BIG BANG theory. In order to exchange secret insights they have devised a code that encodes UPPERCASE words by shifting their letters forward. Shifting a letter by S positions means to go forward S letters in the alphabet. For example, shifting B by S = 3 positions gives E. However, sometimes this makes us go past Z, the last letter of the alphabet. Whenever this happens we wrap around, treating A as the letter that follows Z. For example, shifting Z by S = 2 positions gives B. Sheldon and Leonard’s code depends on a parameter K and also varies depending on the position of each letter in the word. For the letter at position P, they use the shift value of S = 3P + K. For example, here is how ZOOM is encoded when K = 3. The first letter Z has a shift valueof S = 3 × 1 + 3 = 6; it wraps around and becomes the letter F. The second letter, O, hasS = 3 × 2 + 3 = 9 and becomes X. The last two letters…
Assignment 4 In this assignment, you will implement your own interpreter pattern to check a given sentence complies with the sentence requirements given below. Your implementation make sure to do necessary corrections on the sentence. Sentence Requirements: • Every sentence starts with a capital letter. • No sentence can start with a number. • Every sentence ends with a period “.”. • No consecutive words can be same. An exemplary sentence and its corrected version: “ 1 this is is a a sample sentence” Corrected sentence: “This is a sample sentence.” Deliverable: Your Java source code with appropriate design and classes/interface(s).
In this work, we ask you to design a deterministic Finite Automaton (FA) to recognise arithmetic expressions. When we say arithmetic expression, we mean a kind of expression you can type in a very basic desk calculator. That is, neither variables nor brackets are allowed. It contains only numbers and four arithmetic operators, +, -, *, and /. The following are some examples of arithmetic expression: 1. Any number string with or without signs - e.g. 3, -1, +10, 3.14, -0.70 and 099. 2. Number strings mixed with arithmetic operators e.g. 3+5, -1+2*3, 7/10-0.7, and -1.4-+8.2. An example of FA diagram for recognising a non- negative integer is given below. You may use it as a start point. You need to add a few more states and transitions to handle numbers with decimal point and signs (e.g. -5, +2, 0.21, -32.6, +99.05, but not the form 1.0E-3). Most importantly, you also need to add a few more things to deal with the arithmetic operators +, -, *, /. →start 0-9 digit 0-9