You are to develop and test a TokenizedLines class using a linked (dynamic) data
ID: 656141 • Letter: Y
Question
You are to develop and test a TokenizedLines class using a linked (dynamic) data structure for storing the tokens, line-by-line, of a given Java program. FOR THIS ASSIGNMENT, simply implement and test the TokenizedLines class with your own test driver program that allows you to enter arbitrary token codes and token values. Also include a print method to display the contents of a TokenizedLines object on the screen for testing purposes.
TOKEN
-----------
Object (class) that contains a token code and a token value (the actual token, e.g. if).
Token with token code 1 (keyword) and token value of if
TOKEN CODES
----------------------------------------------------------------------------------------------------------
Literal 0 e.g., 12, 12.40, A, Hello, true, false
Keyword 1 e.g., int, float, if, while
Identifier 2 e.g., num, gradesList, System, Scanner
Operator 3 e.g., +, *, >, &&, =
Unpaired Delimiter 4 e.g., blank, comma, semicolon
Paired Delimiter 5 e.g., parentheses, curly braces
Your TokenizedLines test driver should provide the following menu options for testing:
1 Display token codes
2 Enter a new token
3 Start new line
4 Display tokenized lines
5 Quit
Option 1 simply displays token codes 0-5 as shown above. Option 2 allows the user to enter a token code and associated token value. The token is linked in to the tokens of the current line. Option 3 starts a new line (and thus a newly-generated LineHeaderNode) in the tokenized lines.
There will be four classes involved in the implementation (besides the test driver):
TokenizedLines
Contains a variable of type LineHeaderNode pointing to the first node of the tokenized lines.
LineHeaderNode
A LineHeaderNode stores the line number of the particular line (as an integer). It also contains two links: one to the first TokenNode of a linked list of TokenNodes (contains the tokens of that line), and a link to another LineHeaderNode (i.e., the next tokenized line).
TokenNode
A TokenNode contains a Token object and a link to another TokenNode.
Token
Stores a token code (as an integer type) and the corresponding token value (as a String).
Token with token code 1 (keyword) and token value of if
Explanation / Answer
Here is the code for all classes specified in the question . A sample run of the program is shown. Please do rate the answer if it helped. Thank you.
Token.java
package tokenizer;
public class Token {
protected int code;
protected String value;
public Token (int code, String value)
{
this.code = code;
this.value = value;
}
public String toString(boolean showType)
{
if(showType)
return "[" + code + "] " + value ;
else
return value;
}
}
TokenNode.java
package tokenizer;
public class TokenNode {
protected Token token;
protected TokenNode next;
public TokenNode(Token tok)
{
token = tok;
next = null;
}
public TokenNode(Token tok, TokenNode nxt )
{
token = tok;
next = nxt;
}
public Token getToken() {
return token;
}
public void setToken(Token token) {
this.token = token;
}
public TokenNode getNext() {
return next;
}
public void setNext(TokenNode next) {
this.next = next;
}
}
LineHeaderNode.java
package tokenizer;
public class LineHeaderNode {
protected int lineNo;
protected TokenNode firstNode;
protected LineHeaderNode nextLine;
public LineHeaderNode(int lineNo)
{
this.lineNo = lineNo;
}
public void addToken(Token tok)
{
TokenNode n = new TokenNode(tok);
if(firstNode == null)
firstNode = n;
else
{
TokenNode p = firstNode;
while(p.next != null) //traverse to last node and add
p = p.next;
p.next = n;
}
}
public void setNextLine(LineHeaderNode next)
{
this.nextLine = next;
}
public LineHeaderNode getNextLine()
{
return nextLine;
}
public int getLineNo()
{
return lineNo;
}
public void display(boolean showType)
{
TokenNode t = firstNode;
if(firstNode != null)
System.out.print(" " + lineNo +". " );
while(t != null)
{
System.out.print(t.getToken().toString(showType) +" ");
t = t.next;
}
}
}
TokenizedLines.java
package tokenizer;
public class TokenizedLines {
protected LineHeaderNode firstLine;
protected LineHeaderNode current;
public void addToken(Token token)
{
if(firstLine == null)
{
firstLine = new LineHeaderNode(1);
current = firstLine;
}
current.addToken(token);
}
public void startNewLine()
{
current.setNextLine(new LineHeaderNode(current.getLineNo() + 1));
current = current.getNextLine();
}
public void display(boolean showType)
{
System.out.println("---------------------");
LineHeaderNode n = firstLine;
while(n != null)
{
n.display(showType);
n = n.nextLine;
}
System.out.println(" --------------------- ");
}
}
Driver.java
package tokenizer;
import java.util.Scanner;
public class Driver {
static Scanner keybd = new Scanner(System.in);
private static void displayTokenCodes()
{
System.out.println("***********************************");
System.out.println(" 0 Literal");
System.out.println(" 1 Keyword");
System.out.println(" 2 Identifier");
System.out.println(" 3 Operator");
System.out.println(" 4 Unpaired Delimiter");
System.out.println(" 5 Paired Delimiter ");
System.out.println("*********************************** ");
}
private static void mainMenu()
{
int choice;
int type;
String value;
String ans;
TokenizedLines lines = new TokenizedLines();
while(true)
{
System.out.println("1. Display token codes");
System.out.println("2. Enter a new token");
System.out.println("3. Start new line");
System.out.println("4. Display tokenized lines");
System.out.println("5. Quit");
System.out.print(" Enter your choice: ");
choice = keybd.nextInt();
switch(choice)
{
case 1: displayTokenCodes();
break;
case 2:
while(true)
{
System.out.print("Enter token type: ");
type = keybd.nextInt();
if(type < 0 || type > 5)
System.out.println("Invalid token type! ");
else
break;
}
keybd.nextLine(); //flush new line
System.out.print("Enter token value: ");
value = keybd.nextLine().trim();
lines.addToken(new Token(type, value));
break;
case 3:
lines.startNewLine();
break;
case 4:
System.out.print("Do you want to display token types ? y/ n: ");
ans = keybd.next();
if(ans.equals("y"))
lines.display(true);
else
lines.display(false);
break;
case 5:
return ;
default:
System.out.println("Invalid Menu choice!");
}
}
}
public static void main(String[] args) {
mainMenu();
}
}
output
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 1
***********************************
0 Literal
1 Keyword
2 Identifier
3 Operator
4 Unpaired Delimiter
5 Paired Delimiter
***********************************
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 1
Enter token value: int
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 2
Enter token value: a
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 3
Enter token value: =
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 0
Enter token value: 4
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 4
Enter token value: ,
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 2
Enter token value: b
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 3
Enter token value: =
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 0
Enter token value: 5
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 4
Enter token value: ;
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 4
Do you want to display token types ? y/ n: n
---------------------
1. int a = 4 , b = 5 ;
---------------------
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 3
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 2
Enter token value: String
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 2
Enter token value: name
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 3
Enter token value: =
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 0
Enter token value: "John"
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 2
Enter token type: 4
Enter token value: ;
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 4
Do you want to display token types ? y/ n: y
---------------------
1. [1] int [2] a [3] = [0] 4 [4] , [2] b [3] = [0] 5 [4] ;
2. [2] String [2] name [3] = [0] "John" [4] ;
---------------------
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 4
Do you want to display token types ? y/ n: n
---------------------
1. int a = 4 , b = 5 ;
2. String name = "John" ;
---------------------
1. Display token codes
2. Enter a new token
3. Start new line
4. Display tokenized lines
5. Quit
Enter your choice: 5
Related Questions
drjack9650@gmail.com
Navigate
Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.