Hello I just need the last part the getToken.cpp source file. Thank you It’s som
ID: 3677045 • Letter: H
Question
Hello I just need the last part the getToken.cpp source file. Thank you
It’s sometimes useful to make a little language for a simple problem. We are making a language to let us play
with strings and numbers, and for this assignment we are building a lexical analyzer for this simple language.
Here are the lexical rules for the language:
1. The language has identifiers. An identifier starts with a letter and is followed by zero or more letters.
2. The language has string constants. A string constant is a sequence of characters, all on one line, enclosed
in double quotes. You do not have to support special handling for escape characters.
3. The language has integer constants, defined as a sequence of digits.
4. The language has 4 operators: +, -, *, and =
5. The language has 3 keywords: “print”, “int” and “string”.
6. Statements in the language end in a semicolon.
7. The language supports parentheses.
8. White space is used to separate tokens and lines for readability.
9. A comment begins with two slashes (//) and ends at a newline.
The lexical analyzer is to be implemented in a C++ function. The function will be passed a pointer to an input
stream to read from. It will return a Token representing the token and lexeme that has been recognized.
The definitions for the Token class and unique values for each of the tokens (TokenType) that you must
recognize is provided in the header file p2lex.h, which is on the course website.
The lexical analyzer must ignore white space and comments, using them only to note separation between
tokens. The program should maintain an external integer named “linenum”, which should be initialized to 1 and
incremented whenever a newline is seen by the lexer.
A lexical error should cause the token ERR to be returned. An end of file should cause DONE to be returned.
You MUST use the p2lex.h header file for your assignment. You may not change it. You do not need to hand
in p2lex.h, but it doesn’t matter if you do. I will compile and test your program against the distributed p2lex.h
header file
You must produce and submit three files:
1. p2lex.cpp
This file should #include p2lex.h and should provide an implementation for the copy constructor, the
assignment operator, and the << operator to print out a Token.
The printed version of the token must be a string representation of the TokenType. We define the string
representation of a TokenType to be the symbol for the TokenType in lowercase letters. For example,
the string representation of the TokenType value PRINTKW should be “printkw”.
For the VAR, SCONST, ICONST and ERR tokens, the string representation must be followed by the
lexeme for the token. The lexeme should be printed in parentheses.
2. getToken.cpp
This file should #include p2lex.h and provide an implementation of the getToken function. The function
should read from the stream pointed to by the first argument and return the token that it recognizes.
3. project2.cpp
This file should #include p2lex.h and provide a main program to test the lexical analyzer
The specifications for the main program are as follows:
1. The program should accept at most one command line argument. If present, this one command
line argument is the name of a file to open and read for input. If not present, the program should
read from the standard input.
2. There may also, optionally, be an initial command line argument equal to the string “-v”. If it is,
then your program should be in “verbose mode” and should print each token that it recognizes in
addition to the remaining items.
3. The program should loop repeatedly, calling getToken(), until getToken returns an ERR or
DONE token
4. If the loop stopped because of an ERR token, it should print out the token
5. The program should print out a count of the number of times each type of token was seen.
6. The program should print out the lexeme for any string, integer or variable that appears more
than once
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
The header file which was provided.
/* * p2lex.h * * CS280 Spring 2016 Project 2 */ #ifndef P2LEX_H_ #define P2LEX_H_ #include #include using namespace std;; class Token { public: // defines all of the known token types enum TokenType { VAR, // an identifier SCONST, // a string enclosed in "" ICONST, // an integer constant // four operators PLUSOP, // the + operator MINUSOP, // the - operator STAROP, // the * operator EQOP, // the assignment op // three keywords PRINTKW, // print INTKW, // int STRKW, // string LPAREN, // left parenthesis RPAREN, // right parenthesis SC, // the semicolon ERR, // some error condition was reached DONE, // end of file }; private: TokenType ttype; // the token type for this instance string tlexeme; // the corresponding lexeme public: // constructor has default values Token(TokenType t = ERR, string l = "") { ttype = t; tlexeme = l; } // copy constructor Token(const Token&); // getters const TokenType getType() const { return ttype; } const string getLexeme() const { return tlexeme; } // equality/inequality test against a token type; bool operator==(const Token::TokenType tt) const { return ttype == tt; } bool operator!=(const Token::TokenType tt) const { return ttype != tt; } // assignment operator const Token& operator=(const Token& rhs); // for printing out using << friend ostream& operator<<(ostream& out, const Token& t); }; extern Token getToken(istream* br); extern bool pushbackToken(Token& t); extern int linenum; #endif /* P2LEX_H_ */
Explanation / Answer
p2lex.h
#include <iostream>
#include <string>
using namespace std;
enum Token {
ID, // an identifier
INT, // an integer constant
STRING, // a string enclosed in ""
PLUS, // the + operator
MINUS, // the - operator
STAR, // the * operator
SLASH, // the / operator
SET, // the set keyword
PRINT, // the print keyword
SC, // the semicolon
DONE, // end of file or some error condition was reached
};
extern Token getToken(istream *br, string& lexeme);
extern bool pushbackToken(Token t, string lexeme);
main.cpp
#include <fstream>
#include <map>
#include "p2lex.h"
//Parse tree is a binary tree
//list of statements represented by PTree statement list
//left child is a statement: right child is a statement list
//read token, if print get expr & ;
//if set see if id then expr & ; if not then not stmt & error
//id, int, or " "
//traverse tree and parse
class PTree {
PTree *left;
PTree *right;
public:
PTree(PTree *left, PTree *right = 0) {
this->left = left;
this->right = right;
}
};
class PTreeStmtList : public PTree { //constructs itself by constructing its parents
public:
PTreeStmtList(PTree *s1, PTree *s2 = 0) : PTree(s1, s2) {}
};
class PTreePlus : public PTree { //constructs itself by constructing its parents
public:
PTreePlus(PTree *plus) : PTree(plus) {}
};
class PTreePrint: public PTree {
PTreePrint(PTree *print): PTree(print) {}
};
class PTreeSet: public PTree {
PTreeSet (PTree *set): PTree(set) {}
};
class PTreeMinus: public PTree{
PTreeMinus (PTree *minus) : PTree (minus){}
};
class PTreeStar: public PTree {
PTreeStar(PTree *star) : PTree (star){}
};
class PTreeSlash: public PTree {
PTreeSlash(PTree *slash): PTree(slash) {}
};
class PTreeID: public PTree {
PTreeID(PTree *id): PTree(id) {}
};
class PTreeINT: public PTree {
PTreeINT(PTree *integer): PTree(integer) {}
};
class PTreeString: public PTree {
PTreeString (PTree *myString): PTree(myString) {}
};
//function for each PTree
//first 2 done for us
// the comments above each are the definition for each function
extern PTree *Program(istream *br);
extern PTree *StmtList(istream *br);
extern PTree *Stmt(istream *br);
extern PTree *Expr(istream *br);
extern PTree *Term(istream *br);
extern PTree *Primary(istream *br);
// Program ::= StmtList
PTree *Program(istream *br)
{
return StmtList(br);
}
// StmtList ::= Stmt | Stmt StmtList
PTree *StmtList(istream *br)
{
PTree *stmt;
stmt = Stmt(br); //find statement
if( stmt ) //if it's there return a statement list
return new PTreeStmtList( stmt, StmtList(br));
return new PTreeStmtList( stmt, StmtList(br) );
return 0;
}
PTree *Stmt(istream *br)
{
PTree * print;
PTree * set;
print = Expr(br);
set = Expr (br);
if (print){
return new PTreePlus(print, Stmt(br));
}
return 0;
}
PTree *Expr(istream *br)
{
return 0;
}
PTree *Term(istream *br)
{
return 0;
}
PTree *Primary(istream *br)
{
return 0;
}
void
usage(char *progname, string msg)
{
cerr << "Error: " << msg << endl;
cerr << "Usage is: " << progname << " [filename]" << endl;
cerr << " specifying filename reads from that file; no filename reads standard input" << endl;
}
int
main(int argc, char *argv[])
{
//bool debug = false; //uncomment this later
istream *br;
ifstream infile;
if( argc == 1 )
br = &cin;
else if( argc == 2 ) {
infile.open(argv[1]);
if( infile.is_open() )
br = &infile;
else {
usage(argv[0], "Cannot open " + string(argv[1]));
return 1;
}
}
else {
usage(argv[0], "More than one file name was given");
return 1;
}
return 0;
}
gettoken2.cpp
#include "p2lex.h"
#include <cctype>
enum LexState {
START,
INID,
ININT,
INSTRING,
INCOMMENT,
ONESLASH
};
inline Token
keywordCheck(const string& word)
{
if (word == "set") {
return SET;
}
else if (word == "print") {
return PRINT;
}
return ID;
}
int currLine = 1; //keeps track of what line
static bool pushedBack = false;
static Token pushedToken;
static string pushedLexeme;
bool
pushbackToken(Token t, string lexeme)
{
if( pushedBack ) return false; // cannot push more than 1
//remembers what to push back
pushedToken = t;
pushedLexeme = lexeme;
pushedBack = true;
return true;
}
Token
getToken(istream *br, string& lexeme)
{
if( pushedBack ) {
pushedBack = false; //forget pushback
lexeme = pushedLexeme; //copy it to lexeme
return pushedToken; //return it
}
LexState state = START;
lexeme.clear();
int inchar;
while ( (inchar = br->get()) ) {
if( inchar == ' ' ) currLine++; //if new line increment counter: error messege for line number
switch (state) {
case START:
if (isspace(inchar)) {
continue;
}
lexeme += inchar;
if (inchar == '+') {
return PLUS;
}
if (inchar == '-') {
return MINUS;
}
if (inchar == '*') {
return STAR;
}
if (inchar == ';') {
return SC;
}
if (inchar == '/') {
state = ONESLASH;
}
else if (inchar == '"') {
state = INSTRING;
}
else if (isdigit(inchar)) {
state = ININT;
}
else if (isalpha(inchar)) {
state = INID;
}
else
return DONE;
break;
case INID:
if (isalpha(inchar)) {
lexeme += inchar;
continue;
}
br->putback(inchar);
return keywordCheck(lexeme);
break;
case ININT:
if (isdigit(inchar)) {
lexeme += inchar;
continue;
}
br->putback(inchar);
return INT;
break;
case INSTRING:
lexeme += inchar;
if (inchar == ' ') {
return DONE;
}
if (inchar != '"') {
continue;
}
return STRING;
break;
case INCOMMENT:
if (inchar != ' ') {
continue;
}
state = START;
break;
case ONESLASH:
if (inchar != '/') {
br->putback(inchar);
return SLASH;
}
lexeme.clear();
state = INCOMMENT;
break;
default:
cerr << "This should never happen!" << endl;
exit(1);
break;
}
}
// I reached an end of file... perhaps it happened in the middle of something...
if (state == INID) {
return keywordCheck(lexeme);
}
else if (state == ININT) {
return INT;
}
else if (state == ONESLASH) {
return SLASH;
}
else {
return DONE;
}
}
Related Questions
drjack9650@gmail.com
Navigate
Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.