* pgenutil.pas, generate_specialization:
* return a generrordef instead of Nil when returning because of an error
* always check "genname" for a generic count string as a mode ObjFPC generic specialized in a mode Delphi unit will already contain a "$X" suffix and only symname will be set.
+ add (corrected) test
git-svn-id: trunk@22462 -
The cause of the internal error was the following:
We have a generic in an unit ("A") which uses another unit ("B") in the implementation section and this other unit uses unit A in the interface section. Now the generic is specialized in the interface section of B. This leads to the problem that in unit A when it tries to load the globalsymtable of unit B that globalsymtable will be Nil, because parsing of the interface section is not yet finished. Thus the change in pgenutil.pas, specialization_init: if the unit is still "in_interface" the localsymtable needs to be used instead of the globalsymtable.
This doesn't necessarily lead to a compiling test though, as there is the following possibility:
Unit A contains a generic class/record (with methods) and uses unit B in the implementation section. This unit B also contains a generic class/record (with methods) and uses unit A in the implementation section. Both units contain a specialization of the other unit's generic outside of it's own generics (such that generate_specialization is fully triggered). Let's assume compilation starts with unit A and we reach the uses of unit B. Now compilation switches to unit B and completes as unit A is already registered and in compilation. The problem now is that the generic in unit A still contains unresolved forward declarations as the implementation section of A was not yet parsed which will lead to "forward declaration not solved" errors (Note: Delphi compiles this).
The solution to this is the following: if a generic is specialized from another unit which is not in state ms_compiled then the unit of the specialization needs to wait for the unit of the generic. So the specialization's unit adds itself into a list of waiting units of the generic's unit. Now inside "proc_unit" we need to check whether this module is waiting for other modules and if so avoid "finishing" the unit (which means generating the methods of the specialization, generating assembler code and ultimately freeing the scanner and PPU). Now when the generic's unit finishes we need to check whether other modules are waiting for it and finish them (of course it's a bit more complicated in reality, but that pretty much sums it up).
+ globstat.pas: Added an unit which handles the saving and restoring of the global state which was originally inside "parser.pas, compile" so that Don't Repeat Yourself (DRY) is respected.
* fmodule.pas, tmodule:
+ add fields to keep track of the units the module is waiting for and which modules are waiting for the module
+ add field for the saved global state (raw pointer to avoid circles)
+ add field for the state which is needed to finish the unit (raw pointer to avoid circles)
+ move the code which was used in "parser.pas, compile" after a module was successfully compiled to the new virtual method "end_of_parsing"
+ fppu.pas, tppumodule.end_of_parsing:
free the ppufile here
* pgenutil.pas:
+ add new procedure "maybe_add_waiting_unit" which adds the specialization's unit to the waiting list of the generic if that unit is not yet compiled
* generate_specialization: call the new function when we add a new (true) specialization
* specialization_init: instead of not adding implementation units at all check whether the unit is still parsing the interface section and add the localsymtable in that case
* pmodules.pas:
* change "proc_unit" to a function which returns "true" if the unit was already finished (no need to wait for other units)
+ move the code from "proc_unit" from "generate_specialization_procs" on to a new procedure "finish_unit" which
* this procedure is either called immediately in "proc_unit" if the unit does not need to wait for other units or from "finish_unit" itself if a unit that is waiting for the given unit does no longer wait for another module (special care is taken in proc_unit to avoid circles)
* parser.pas, compile:
* correctly handle the case if an unit is not finished
* use the new global state functionality from globstat.pas
* pay special attention when calling "set_current_module" (see comment at that call)
+ add tests from 22160
+ add test for above mentioned "diamond" case
git-svn-id: trunk@22452 -
+ symconst.pas: add a new flag to symtables to mark them as containing at least one generic (will be used to decide whether an expression like "Foo<Bar" should even remotely be considered a specialization)
* utils/ppudump.pp: respect the new flag
* pdecl.pas, types_dec: add the flag for every generic we have parsed
* pgenutil.pas, generate_specialization: add the flag to the specialize symtable if a nested generic set it for the temporary symtable (does not happen currently, but will in the future when nested generics are supported)
+ keep references to all generic para symbols in the def; this way the symtable containing the type parameters does not need to be walked every time we need to do something with the parameters
git-svn-id: trunk@22379 -
pgenutil.pas, specialization_init:
* don't add implementation units of the generic's unit to the symtable stack if we are specializing in a different unit (thus the generic needs to be defined in the interface section) as there is the possibility that the globalsymtable of an implementation unit is not yet defined if the specialization unit is used in the interface section of an implementation unit
git-svn-id: trunk@21763 -
* pgenutil.pas: factor out the reading of generic specialization parameters (parse_generic_specialization_types) and the generation of a generic type name (generate_generic_name)
* pdecsub.pas, parse_proc_head:
* also allow an interface alias declaration if an identifier is followed by a "<" (which starts a specialization)
+ add a procedure "consume_generic_interface" which parses such a specialization (by using "parse_generic_specialization_types") - this is needed, because "consume_generic_type_parameter" can (and should not!) handle "ISomeIntf<Integer, T>" or (somewhen in the future) "ISomeIntf<TSomeOtherGeneric<T>>" - and finds the correct symbol for the interface (by utilizing the "generate_generic_name" function)
* generate the correct mapping entry (for the generic it's only needed for checking (if any), but for a specialization it's essential that we reference the correct specialization)
+ add tests which were included with the issue and also two additional ones
Note: In non-Delphi modes an interface alias can be done like in Delphi mode; "specialization" is not necessary and furthermore not even allowed!
git-svn-id: trunk@21656 -
+ pgenutil.pas:
add a procedure which adds a type symbol to a non-Delphi-mode generic class or record which has the same name as the unit global dummy symbol for that generic. I don't know why I had that idea earlier as this will simplify some of the conditions in the parser again (I haven't changed these yet, but I hope to do that at least when I start working on generic functions).
* pgenutil.pas, generate_specialization:
correctly handle "specialize TSomeGeneric<T>" as method parameter in a generic with the newly added rename symbol
* pdecobj.pas, object_dec & ptype.pas, record_dec:
call the procedure to add the rename symbol (the procedure checks whether the mode is correct)
* ppu.pas:
increase PPU version so that we don't use non-Delphi mode units with generics, but without the rename symbol
+ added tests:
the one in webtbs are for classes/objects and those in test are for records
git-svn-id: trunk@21603 -
* pgenutil.pas, generate_specialization: Correctly change the ownership of extracted symbols and defs by calling ChangeOwner on the symbol/def which does all the magic of changing the owner and adding it to the corresponding list of the given symtable. This fixes Mantis #21550
* Adjusted tw21550.pp so that it enforces the generation of debug information which (as this was the option which caused the above mentioned bug to show)
git-svn-id: trunk@21270 -
When parsing an inline specialization inside a generic we need to
respect the "parsedtype" parameter which tells us whether the first
generic parameter was already parsed. This fixes Mantis #20871 .
+ added test for this
git-svn-id: trunk@20251 -
- Remove unneeded check for "parse_generic" (it's already checked in the outer if-clause).
+ Check whether we are trying to specialize one of the surrounding type declarations of a
nested type (as long as nested generics are forbidden this is always the outermost generic
or specialization). This check can not rely on the symbol, because while parsing the
generic or the specialization the symbol's def is still an errordef. This fixes
Mantis #19498 .
+ Added test from bug report.
git-svn-id: trunk@20247 -
When specializing a generic the references from unitsyms to the loaded modules
needs to be reestablished, so that "unitidentifier.identifier" can be used
inside a generic without leading to an access violation.
Only global units are checked, because a generic must not use symbols from the
static symtable or from units used by the implementation section (the latter is
currently not checked)
+ added tests for the above problem for "normal" units as well as units with a namespace
git-svn-id: trunk@20245 -
* When building the typename for a generic use the full typename
including it's surrounding object- or abstractrecorddefs. This allows
that a nested non-generic type of a generic type A can be used as
type arguments for more than one specialization of another generic B
(there were some problems when B e.g. defined a pointer to the type
argument's type)
* Always CRC the constructed specialization name as otherwise it might
reach the limit of 255 characters (not yet including unit name,
method name or method arguments)
Note: Errors like "expected XYZ, but got ABC" will need to be
adjusted to use the prettyname...
* increased PPU version
+ added test for above's point 1
git-svn-id: trunk@20149 -
pgenutil.pas, generate_specialization:
* If we are parsing the result type of a function or operator that belongs to a generic (parse_generic is true) we need to accept also "_LT" and "_GT" as for this the "block_type" is NOT set to one of "bt_type", "bt_var_type", "bt_const_type" and only there "_LSHARPBRACKET" and "_RSHARPBRACKET" are returned by the scanner. This is part of the fix for webtbs\tw18567.pp.
* In non—Delphi modes if we encounter a specialization of the currently parsed class/record (using "specialization"!) the given "tt" will be an errordef (because the def of the generic's symbol was not yet set to "current_structdef"). To solve this we check in this case whether the calculated generic name is equal to that of the "current_structdef" and simply return that as specialized def. This fixes test\tgeneric11.pp.
* When searching for the generic symbol search if the generic belongs to a class or record then we need to search for it in the class or record. This fixes webtbs\tw16090.pp.
ptype.pas
* parse_nested_types: We now return the generic defs for specializations inside generics instead of an undefined def, so we must also parse nested type usages correctly, so that type checks don't fail (undefined defs circumvent them mostly). This fixes webtbs\tw10247.pp.
* single_type: We need to allow _LT as an indication for a Delphi specialization as return types are parsed with block_type not in "bt_type", "bt_var_type", "bt_const_type". This is also a fix a part of the fix for webtbs\tw18567.pp.
* read_named_type, expr_type:
Fixes for test\tgeneric17.pp and test\tgeneric22.pp:
(a) In non-Delphi modes we might encounter usages of the generic dummy symbol inside the generic class belonging to it. This is basically a specialization, but as the reference from the dummy symbol to the "current_structdef" is not yet established (this is done after "read_named_type" in "types_dec" returns) we need to use other ways to check for the valid use of the dummy symbol (basically we check whether the name of the dummy symbol and the name of the current_structdef without the type count match)
(b) For specializations we can check whether the genericdef of the currently parsed structdef (the specialization) is the same as the typedef of the encountered dummy symbol.
pexpr.pas, factor, factor_read_id:
Fixes for test\tgeneric17.pp and test\tgeneric22.pp:
To allow the mentioned fixes in ptype for the same tests to be usable at all we must not return an "identifier not found" error if we encounter such a valid use of a generic dummy symbol.
git-svn-id: branches/svenbarth/generics@19719 -
* symconst.pas: remove tf_methods_specialized
* psub.pas: remove check for/inclusion of tf_methods_specialized as this isn't needed anymore since the generic is specialized in a temporary symtable
pgenutil.pas: remove merge artifacts
pdecl.pas: fix compilation ("s" was duplicate)
pexpr.pas:
* fix calling of generate_specialization
* disable the goto in sub_expr for now; this will be enabled again once right hand sides work as well
git-svn-id: branches/svenbarth/generics@19675 -
The changes regarding pretty names for generics and token buffer endianess were integrated into my changes. Not every call to generate_specialization is fixed though, so compilation will fail.
git-svn-id: branches/svenbarth/generics@19674 -
Conflicts:
compiler/pdecl.pas
compiler/pexpr.pas
compiler/pgenutil.pas
compiler/ptype.pas
The original log messages as git was a bit forgetting here :( (newest at the top):
commit 7ef252de8023494ee6d39910e289f9e31658d47b
Author: Sven Barth <pascaldragon@minerva>
Date: Mon Nov 21 17:13:36 2011 +0100
Fix the compilation of inline specializations of which the generic is derived from another generic.
pgenutil.pas, generate_specialization:
* Set the "block_type" to "bt_type" when parsing the type parameters, so that the nodes are returned as "ttypenode" instead of e.g. "tloadvmtaddrnode" in case of classes outside of type sections.
* Set the "block_type" to "bt_type" before calling "read_name_type", so that no unexpected sideeffects happen, because types like classes normally only are declared inside type sections (e.g. for the case a generic class is derived from another generic class a classrefdef for the specialized parent class will be created inside the derived specialized class if the block type is not a type one).
commit 1041a8f7a3a41f4fdf2975ce40055c698281ce71
Author: Sven Barth <pascaldragon@minerva>
Date: Fri Nov 18 19:03:50 2011 +0100
Improve inline specializations a bit, so now expressions like "TSomeGeneric<TSomeType>.SomeClassProc OP SomeNonGeneric" is possible. Using another class function of a generic as the right side is not yet working (that still needs some thinking).
To achive this the generalization code must basically continue directly after the "factor" call, so that the operator and the right side are correctly parsed when walking up the call stack. This is done by jumping from the end of the specialization code in the "<"-case to the start of "sub_expr". The freshly generated node (in the above example a callnode) will be passed down the callstack through a new parameter "factornode". If that is set (currently only in the case of a specialization on the left side) "factor" won't be called and the right side will be parsed with the "factornode" as the left side. If it is not set (which is the case for all other calls to "sub_expr" in the unit) then the usual call to "factor" will be done and the result will be used as the left side.
commit a01ccd265f8d6cc5a2f3e88e23afbcd3d5960afb
Author: Sven Barth <pascaldragon@minerva>
Date: Fri Nov 18 18:37:04 2011 +0100
Fix compilation of ppudump.
symconst.pas:
* Remove sto_has_generic, which was the last remainer of my "overloaded type symbols" approach.
* Remove df_methods_specialized, as it isn't needed anymore with the recent "temporary symtable" solution.
psub.pas, specialize_objectdefs, process_abstractrecorddef:
Remove the checks for/inclusion of df_methods_specialized.
utils/ppudump.pp:
Add "sp_generic_dummy" to the symbol options.
commit d16deac060e65d4b53e8fe9c27fe7e1f6d00a416
Author: Sven Barth <pascaldragon@minerva>
Date: Wed Nov 16 16:34:51 2011 +0100
Fix compilation of "gset.pp" from fcl-stl.
nld.pas:
Extend ttypenode by a reference to the type symbol. Normally this is simply the typesym of the given def, but for specializations in type sections of generics this is not the case, because generate_specialization will return a reference to the generic definition and not the new one (thus the symbol will be wrong).
ppu.pas:
Increase PPU version because of the extension of ttypenode.
pexpr.pas:
* handle_factor_typenode: Extend the function by a "sym" parameter which will normally be "nil". In that case it is set to the def's typesym. The "typesym" field of the created type node is then set to this sym.
* For now pass nearly always "nil" for the above mentioned sym except inside factor_read_id when we've encountered a typesym.
ptype.pas, read_named_type, expr_type:
Exchange the "is_owned_by" check with a "sym_is_owned_by" check so that we can correctly detect that we are using a specialized type declaration inside a generic (once nested generic are allowed this condition needs to be checked).
commit 23668d2fc9070afc26b4288ed0db9a8eaf6f40e6
Author: Sven Barth <pascaldragon@minerva>
Date: Wed Nov 16 07:51:12 2011 +0100
psub.pas:
* tcgprocinfo.parse_body: Methods of generic classes need to set "parse_generic" as well, so that variables for "stacked generics" (generic array => generic record) inside the method body are handled correctly.
* specialize_objectdefs: Don't try to generate method bodies for abstract methods.
pdecvar.pas, read_property_dec:
Allow specializations for the return types of properties (should they be allowed for index types as well?).
symtable.pas:
Add a new class "tspecializesymtable" which is basically a globalsymtable but is always assuming to be the current unit. This symtable is used in "generate_specializations" (see below) and is needed to allow visibilty checks for "private", etc. to succeed.
pgenutil.pas, generate_specializations:
Instead of hackily pushing a symtable that may contain conflicting symbols onto the symtable stack for the specialization, a temporary global symtable using the above mentioned "tspecializesymtable" is created and pushed. After the specialization is done all symbols and defs that were added to the temporary symtable are moved to their final symtable (either the global- or localsymtable of the unit, depending on the current position of compilation). This way symbols are correctly added to a top level symtable, but without potential side effects like resolving the wrong symbol.
git-svn-id: branches/svenbarth/generics@19671 -
symtable.pas:
* reduce the "childdef" parameter of "is_owned_by" from "tabstractrecorddef" to "tdef", so that more primitive defs can be checked as well
* add a new function "sym_is_owned_by" which is similar to "is_owned_by", but takes a symbol and a symtable as parameter; the owner chain of the symtable is checked until a non-object- and non-record-symtable is reached
ptype.pas:
* extend "id_type", so that the symbol and the symtable that belongs to the returned def is returned as well
* this is needed to check inside "single_type" whether a def that is a generic was specialized inside another generic, because in that case the genericdef is returned by "generate_specialization" and not a new specialized def, but the corresponding type symbol (which is different from "hdef.typesym") belongs to the class itself; I need to admit that this solution isn't very clean and one could try to circumvent some of the checks, so I need to find a better detection for such a case (concrete example: the enumerator specialization inside the classes of "fgl.pas")
* in "read_named_type.expr_type" the check for "df_generic" is extended analogous to the previous change, but instead of relying on the symbol it uses the def. This is needed so that types like method pointers that are defined inside the current generic are not disallowed as they contain the "df_generic" flag as well; like the previous change this change isn't clean either and maybe it's better to remove the inclusion of the "df_generic" flag from everything except records and "objects" inside records/"objects" again. Such a solution will "only" reduce the problem to records and "objects" though...
pgenutil.pas:
* only add a new undefined def if we're not parsing the parent class or interfaces ("parse_class_parent" is true), otherwise the InternalError regarding the "equal count of defs" will trigger
* there are now two cases where we need to return a generic def instead of a undefined one when we're parsing a generic:
a) we have the previously mentioned case that "parse_class_parent" is true
b) an undefined def was added, but we need to return a generic def, so that checks can be passed
* use the correct variable when building the generic name, otherwise we get errors like "identifier '$1' not found"
* don't push the symtable if we're currently parsing the list of interfaces or the parent class, because then e.g. a generic interface will be included in the symtable of the implementing class which isn't what we want; the current solution is not clean though, so this needs to be investigated more
* Note: In the current state of "generate_specialization" the function could be simplyfied a bit more; this will be done when the implementation is satisfactory enough
git-svn-id: branches/svenbarth/generics@19430 -
* symconst.pas:
add an entry for the generic dummy symbol to the symbol options enumeration
* pgenutil.pas:
- extend "generate_specialization" by the possibility to pass a symbol name instead of a def
- if "symname" is given that is used; otherwise "genericdef" or "tt" is used
* pexpr.pas:
- in case of "<" we are trying to receive a generic dummy symbol from the left node (new function "getgenericsym")
- it's name is then passed to "generate_specialization" which in turn fills genericdef
- adjust call to "generate_specialization"
* pdecl.pas:
- we can now check for "sp_generic_dummy instead of "not sp_generic_para" to check whether we've found the dummy symbol of a previous generic declaration
- if a new dummy symbol is created we need to include "sp_generic_dummy"
- if we've found a non-generic symbol with the same name we need to include the "sp_generic_dummy" flag as well
* symtable.pas
- add a new function "searchsym_with_symoption" that more or less works the same as "searchsym", but only returns successfully if the found symbol contains the given flag
- "searchsym_with_symoption" and "searchsym" are based on the same function "maybe_searchsym_with_symoption" which is the extended implementation of "searchsym" (note: object symtables are not yet searched if a symoption is to be looked for)
- add a function "handle_generic_dummysym" which can be used to hide the undefineddef symbol in a symtable
- correctly handle generic dummy symbols in case of variables in "tstaticsymtable.checkduplicate"
git-svn-id: branches/svenbarth/generics@19429 -
pgenutils.pas/generate_specialization:
- parse hint directives of the generic if they are recorded
- output hint messages of the generic after the ">" is successfully parsed
pexpr.pas:
- factor: don't display hints of a potential generic type if the next token is a "<"
- sub_expr:
* added two inline methods which
a) checks whether a node is a typenode or a loadvmtaddrnode with a typenode
b) returns the typedef of such a node
* check hint directives for the first parsed type argument of a specialization
* in the case of parsing a non-generic type the hints of the left and right node of the resulting "<" node need to be checked (the right ones only if another "<" is following)
git-svn-id: branches/svenbarth/generics@17539 -
This fixes the compilation of test tests\test\tgeneric29.pp and the reminder in pdecobj.pas is not needed anymore.
Note: Perhaps this behavior should be enabled in general if "parse_generic" is true (and not only if parse_parent_class if true as well).
git-svn-id: branches/svenbarth/generics@17538 -
Reasons for the "unique symbol" approach:
- no special search operations for cross unit search needed (which is supported by Delphi) => less performance impact
- no special care needed to really find the correct generic => less increase of parser complexity
Currently all generic tests except tgeneric29.pp compile and inline specializations work as well.
The changes in detail:
* pdecl.pas/types_dec:
- The variables used to hold the final name of the symbol are now prefixed with "gen". In case of non-generics the prefixed ones are equal to the non-prefixed ones (e.g. orgtypename=genorgtypename). In case of a generic symbol the "gen"-variants contain the type parameter count suffix (e.g. '$1' in case of 'TTest<T>') as well.
- The unmodified pattern is used to insert and detect a dummy symbol with that name, so that type declarations and - more important - inline specializations can find that symbol.
- In non-Delphi modes this symbol is also used to detect whether we have a type redefinition which is not allowed currently; its typedef points to the generic def.
- In mode Delphi the def of that dummy symbol (which contains an undefineddef) is modified when a corresponding non-generic type is parsed, so that it contains the def of the real type.
* pdecsub.pas/parse_proc_head
- consume_generic_type_parameter now only parses the type parameters and picks the generic with the correct amount of parameters. The verification of the order and names of the parameters needs to be added again.
- it also does not use "def" anymore, but it sets "srsym"
- in parse_proc_head the symbol (srsym) is only searched if the symbol isn't assigned already; in case of a generic in mode FPC it will find the dummy symbol that points to the generic def
* pexpr.pas
- in factor_read_id there are three cases to handle:
+ the symbol is not assigned => error
+ a possible generic symbol (either an undefined def or the non-generic variant) => no error and no hints
+ a non-generic symbol => hints
Point 1 is handled correctly, point 2 and 3 aren't currently and also they might be needed to be moved somewhere else
- sub_expr:
+ a node can be a tloadvmtaddrnode as well if the non-generic variant of a generic symbol is a class
+ we can only check afterwards whether the specialization was successful
* pgenutil.pas/generate_specialization
using the count of the parsed types the correct symbol can be found easily
git-svn-id: branches/svenbarth/generics@17535 -
- moved "postfixoperators" from local declaration of "factor" to implementation declarations of the unit, so it can be used in "sub_expr"
- for this a parameter "getaddr:boolean" needed to be added, because it used the parameter that was defined by "factor"
=> adjustments inside "factor" for calls to "postfixoperators"
- extended the "_LT" ("<") case of "sub_expr" with handling of inline generic specializations. If a potential generic is detected (Delphi mode, left and right node are type nodes, next token is ">" or ",") it is tried to parse the generic declaration and generate a specialization. If this succeeds, potential postfix operators are parsed and a node <> caddnode is returned.
*pgenutil.pas:
"generate_specialization" was extended so that the first type identifer can already have been parsed (which is the case in inline specializations)
*ptype.pas
adjustments because of the extension of "generate_specialization"
git-svn-id: branches/svenbarth/generics@17405 -
Note: I've added this mostly empty, because I used SVN instead of GIT SVN, as I don't know whether it would handle the properties for this new file correctly.
git-svn-id: branches/svenbarth/generics@17397 -
