Tokens implemented in C were not chosen as a minimal set, rather everything that needed to be implemented during system bring up was left in C. Also some words that have a large amount of dependence on various facilities of the underlying system are implemented in C.<\/p>\n
![\"A](\"\/wp-content\/uploads\/sites\/2\/2014\/04\/XT-Format.png\")
<\/a>Execution token format in the Embeddable Forth Command Interpreter. There are two kinds of executions tokens: The first kind has the highest bit set to 1. The next 15 bits identify a token whose semantics is implemented in C. The low 16 bits form a parameter that is used by some token (e.g. branches, string literals, etc.).
The second type of XT is the indirect token. Its highest bit is zero and the rest of the bits form an index (cell displacement inside the dictionary area). See the text for more explanation.<\/p><\/div>\n
The Forth Virtual Machine<\/h2>\n
The virtual machine used by the Embeddable Forth Command Interpreter is similar to a traditional Forth virtual machine. It has two stacks — for Forth’s data stack and Forth’s return stack as one would expect. Both stacks grow downwards. The location of the stacks is in the runtime context<\/a> to allow separate stack space per thread. So the registers of the virtual machine would be sp<\/strong> for the data stack pointer, rp<\/strong> for the return stack pointer ip<\/strong> for the instruction pointer (index), w<\/strong> (a usual name in indirect threaded<\/a> systems) is used for instruction fetch and we keep referring to the current execution token, let’s call it xt<\/strong>. The internal interpreter’s main loop is shown below, this is in effect the Forth engine that executes threaded code.<\/p>\n After the operation of the inner interpreter has been decided a lot of pieces easily fall into place.<\/p>\n Nest into a definition, used by words created by a colon definition:<\/p>\n Undo what Nest did:<\/p>\n The execution semantics of variables.<\/p>\n The execution semantics of Forth constants.<\/p>\n The execution semantics of words defined by CREATE DOES><\/span>. In order to keep the implementation simple execution tokens are cell sized, which currently means always 32 bits. Certain tokens do in fact need extra data which is traditionally stored in the next cell in the dictionary. Because of the large token size I have decided to make to use of some of the bits inside them instead of using a second cell for it.<\/p>\n Branch tokens that are used for implementing forth control structures such as IF ELSE THEN BEGIN WHILE AGAIN UNTIL REPEAT CASE OF ENDOF ENDCASE<\/span> use the parameter bits as an adjustment to ip.<\/p>\n Short literals are supported that take the parameter bits and either sign extend or zero extend them and push them on the data stack, e.g.:<\/p>\n For implementing words such as 1+<\/span>, 1-<\/span> and CELL+<\/span>.<\/p>\n Words that implement the Forth loop construct (DO ?DO LOOP +LOOP<\/span>) also use the parameter in a similar fashion as branches do.<\/p>\n The token behind per execution thread variables (in Forth parlance USER variables) also keeps the index of the user variable in the parameter bits of the token. i.e. It pushes the start of the user area + parameter on the stack.<\/p>\n The runtime context is a data structure that contains all sorts of variables that need to be separate if two copies of the Embeddable Forth Interpreter are run in the same address space (i.e. in threads). In theory the entire dictionary could be cloned and have separate copies of everything but that is the heavyweight way of doing it. The lightweight multi-threading support is to just have separate stacks for each instance that is running, separate search order, terminal buffer, exception handler, etc. Different threads also cannot write to anything in the dictionary space, thus normal variables cannot be used thus USER<\/span> variable support is provided to allow per thread variables if the application chooses to run multiple threads that share the dictionary.<\/p>\n The function pointers that tell Forth how to interact with the terminal are also part of the runtime context (for example one thread might talk to a serial console while the other might talk to a TCP socket — the function pointers that implement what ACCEPT<\/span> and TYPE<\/span> and their kin does take care of dealing with that.<\/p>\n","protected":false},"excerpt":{"rendered":" I have to admit that I am not great on threading. Two highly publicized systems I wrote a while ago generated native code. At least one RPM system I have recently used in a product did not even allow compiling … Continue reading \r\nwhile (1)\r\n{\r\n if (istoken(xt))\r\n {\r\n switch(extract_token(xt) \r\n { \r\n \/\/ A gigantic switch statement implementing\r\n \/\/ the semantics of each token.\r\n case TOKEN_DROP: \/\/ This is how a token is implemented.\r\n sp++;\r\n break;\r\n }\r\n } else {\r\n w = extract_index(xt);\r\n xt = dictionary[w];\r\n continue;\r\n }\r\n\r\n w = ip++;\r\n xt = dictionary[w];\r\n}\r\n<\/pre>\nThe Usual Elements of a Threaded Interpreter<\/h2>\n
Nest<\/h3>\n
\r\n *(--rp) = ip; \/\/ Save the current ip on the return stack.\r\n ip = w + 1; \/\/ The new threaded code starts one cell after w.\r\n<\/pre>\n
Unnest<\/h3>\n
\r\n ip = *(rp++); \/\/ Pull the saved ip from the return stack.\r\n<\/pre>\n
DoVar<\/h3>\n
\r\n *(--sp) = (cell)(&dictionary[w + 1]); \/\/ The address of the cell after w.\r\n<\/pre>\n
DoConst<\/h3>\n
\r\n *(--sp) = dictionary[w + 1]; \/\/ The contents of the cell after w.\r\n<\/pre>\n
DoCreat<\/h3>\n
\nIn this model the DOES><\/span> part is a valid execution token (which might nest).<\/p>\n\r\n \/\/ The address of the 2nd cell after w.\r\n *(--sp) = (cell)(&dictionary[w + 2]);\r\n\r\n \/\/ Fetch the XT stored in the CREATEd word \r\n \/\/ that corresponds to the DOES> part of the definition.\r\n xt = dictionary[w + 1];\r\n\r\n \/\/ We are going back to the top of the while(1) loop. \r\n continue; \r\n<\/pre>\n
Using the Parameter Bits<\/h2>\n
Branch and 0Branch<\/h3>\n
\r\n ip += sign_extend(extract_parameter(xt));\r\n<\/pre>\n
Short Literals<\/h3>\n
\r\n *(--sp) = sign_extend(extract_parameter(xt));\r\n<\/pre>\n
Add Immediate<\/h3>\n
\r\n *sp += sign_extend(extract_parameter(xt));\r\n<\/pre>\n
DO LOOP<\/h3>\n
DoUser<\/h3>\n
The Forth Runtime Context<\/h2>\n