Write the function GetToken: bool GetToken(char Block[], char Token[], int &Toke

Question

Write the function GetToken: bool GetToken(char Block[], char Token[], int &TokenType;, &pos;); This function takes a block of characters stored in Block[] and copies the next 'token' into the Token[] array. TokenType will store the type of token this is: 1: Words, 2: Numbers, 3: Punctuation, etc. In order to test this function, write functions to extract a block of text from a text file storing it in the Block[] array. Then, call on GetToken in a loop to extract successive tokens form the Block array and printing the returned token and its type in each line. Block[] : "This is the 3rd time I am typing this. lol!!" output: This [1] is [1] the [1] 3 [2] rd [1] time [1] I [1] am [1] typing [1] this [1] . [3] lol [1] ! [3] ! [3] Design: As you design this function, keep in mind that the requirements for what constitutes a Token could change easily over time and you should not have to redesign your function: You cannot be looking at a single character and decide if this character is part of a given token. Take the character '2'. As of now, '2' is part of a Numerical token, like "423" but what if we decided that our 'words' could contain numbers: "R2D2"? Think about this for a second... Here is what I propose: The pre-conditions for the GetToken function is that Block[] contains a block of text you are trying to tokenize. and pos is the index of the first character of the next token to be extracted. The post-conditions are that Token[] contains the next token, TokenType contains a code corresponding to the type of token extracted. ie 1 for words, 2 for numbers, etc. pos contains (points to) the first character of the next token to be extracted. and Block[] Stays the same. The return value of the function is a bool. It is true if pos has reached to the end of Block[] and false otherwise. Now... Your GetToken function performs 5 steps: Step 0: Push pos to the beginning of the next token if it is not already there. This is useful if the Bock contains extra spaces: " I have blank spaces " Step1: Examine the character pointed to by pos (the first character in the token) and determine its type. This will need its own function, of course. Step2: a switch statement that will call different functions such as GetWord, GetNumber, GetPunc that will be passed the Block[], the Token[], and the pos. Each function is specialized in extracting a specific type of token. By the end of this step, Token[] will contain the next token and the TokenType is set to the appropriate value. Step 3: Push pos to the start of the next token so that it is ready for the next time the function is called. Step4: Decide if you have reached the end of the Block[] and return true if you have and false otherwise. PS: please show me detail codes that could be complied.

Explanation / Answer

/* pupose: picking the character from the chunk of text, then analyst what type of that token is */ #include #include #include #include using namespace std; const int max = 1000; int type(char block[],int pos); // type of the token void getword(char block[], char token[], int &pos); void getnumber(char block[], char token[], int &pos); void print(char token[],int type); void getpunc(char block[], char token[], int &pos); bool gettoken(char block[],char token[], int &type,int &pos); int main() { int type; int pos = 0; char block[1000]= "Writing code is extremely hard!"; system("PAUSE"); return 0; } int type(char block[],int pos) { if(isalpha(block[pos])) return 1; if(isalnum(block[pos])) return 2; if(ispunct(block[pos])) return 3; } //Realize the word is the type void getword(char block[], char token[], int &pos) { int i, type; while(isalpha(block[pos])) { token[i]=block[pos]; i++; pos++; token[i] = '\0'; } } //Realize the number is the type void getnumber(char block[], char token[], int &pos) { int i,type; while(isalnum(block[pos])) { block[pos]= token [i]; i++; pos++; token[i]='\0'; } } //Realize the punctuation is the type void getpunc(char block[], char token[], int &pos) { int i, type; while(ispunct(block[pos])) { block[pos] = token[i]; i++; pos++; token[i]='\0'; } } bool gettoken(char block[],char token[], int &type,int &pos) { token[0] = block[pos]; token[1] = '\0'; type = 1; pos++; if ( pos >strlen(block)) return true; else return false; } void print(char token[],int type) { for(int i = 0; i

Related Questions

Navigate

• Browse (All)
• Browse W
• Subjects

---

---

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.