+ support for parsing docsets to add documentation to the parsed headers

* re-enabled support for ignoring certain methods, but removed all ignored
    methods because all of them are handled correctly with the current version
  - disabled adding the script version and time stamp to the generated headers,
    so not all headers change every time they are re-parsed
  + support for optional/required sections in protocols
  + documentation from Ryan

git-svn-id: trunk@16923 -
This commit is contained in:
Jonas Maebe 2011-02-16 18:46:26 +00:00
parent bbe5b792af
commit b3ee4ea4eb
9 changed files with 190 additions and 25 deletions

4
.gitattributes vendored
View File

@ -1574,6 +1574,10 @@ packages/cocoaint/utils/cocoa-skel/src/patches/NSObjCRuntime.patch svneol=native
packages/cocoaint/utils/cocoa-skel/src/quartzcore/QuartzCore.inc svneol=native#text/plain
packages/cocoaint/utils/cocoa-skel/src/webkit/UndefinedTypes.inc svneol=native#text/plain
packages/cocoaint/utils/cocoa-skel/src/webkit/WebKit.inc svneol=native#text/plain
packages/cocoaint/utils/doc/How[!!-~]to[!!-~]parse[!!-~]frameworks.rtf -text svneol=unset#application/rtf
packages/cocoaint/utils/doc/Make[!!-~]Cocoa[!!-~]Headers.txt svneol=native#text/plain
packages/cocoaint/utils/doc/Make[!!-~]Single[!!-~]Header.txt svneol=native#text/plain
packages/cocoaint/utils/doc/Make[!!-~]iPhone[!!-~]Headers.txt svneol=native#text/plain
packages/cocoaint/utils/frameworks.xml svneol=native#text/plain
packages/cocoaint/utils/install_objp.sh svneol=native#text/plain
packages/cocoaint/utils/make-cocoa-headers.sh svneol=native#text/plain

View File

@ -0,0 +1,34 @@
{\rtf1\ansi\ansicpg1252\cocoartf1038\cocoasubrtf350
{\fonttbl\f0\fnil\fcharset0 Verdana;}
{\colortbl;\red255\green255\blue255;}
\margl1440\margr1440\vieww10500\viewh9000\viewkind0
\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\ql\qnatural\pardirnatural
\f0\fs24 \cf0 1) Locate the .framework you want to parse and record the path for later use when building the command line.\
\
2) Add an entry to frameworks.xml (in the parser directory) with the appropriate tags for the framework. To make configuring the parser easier many options are specified here in XML instead of editing PHP variables.\
\
<root> tag is the relative path to the root include file (see step 4). The path is relative to the -root path (see step 5)\
<headers> tag is the path to the directory which contains the Objective-C headers (usually in /System/Library/Frameworks).\
<include_pattern> is a regular expression that captures the Pascal syntax of include files in the root .inc file. This is usually \{[$]+include (.*).inc\}.\
<header_pattern> is a regular expression that captures files in the header directory (defined in <headers>). This is usually ^(.*)\\.h.\
<ignore_lines> is a regular expression that causes to parser to ignore lines that present parsing errors you want to skip and edit by hand.\
<ignore_comments> is a regular expression that causes to parser to ignore comments. Note, if the comment is found inside a multi-line comment block it will break the entire block.\
<external_macro> In the frameworks the macro which defines NSString constants changes from framework to framework so you must specify it here or NSString constants will not get parsed. For example in UIKit.framework it is UIKIT_EXTERN and in QuartzCore.framework it is CA_EXTERN. AppKit and Foundation macros are defined internally as PHP variables so you do not need to specify them here.\
<ignore_types> and <ignore_methods> will cause the parser to ignore these types and methods when parsing. Separate multiple values by commas without spaces.\
<replace_types> causes the parser to replace types with a different name. For example: <type>CGFLOAT_DEFINED=__CGFLOAT_DEFINED</type> will replace all occurrences of CGFLOAT_DEFINED with __CGFLOAT_DEFINED.\
\
3) Make a framework directory in the root directory (see step 5 and -root).\
\
4) Make the root .inc file with all units which will be parsed and put it in the directory created in step 3.\
4a) Usually named a header with the same name as the framework for example, AppKit.h contains all the headers in the framework.\
4b) This step involves taking care to arrange the included units in the proper order. The order in the original "umbrella" header is usually pretty close but often requires some rearranging.\
\
5) Make the command line for the parser\
5a) Frameworks assigned in -frameworks and prefixed ^ will parse but not print.\
5b) The ^foundation framework is usually always included so extra class information is parsed for the other frameworks as they all use foundation classes occasionally. If you do not specify foundation then classes will be printed as pointers (NSStringPointer instead of NSString).\
\
The example below will parse the entire WebKit framework.\
php parser.php -objp -all -root="/Developer/ObjectivePascal" -frameworks="^foundation,webkit" \
\
6) Make the master unit file named after the framework, like AppKit.pas. This unit will include all the .inc files in the framework directory that was created in step 3. You can add extra code here if you like but normally it's enough to copy a template like AppKit.pas and change a few lines. For example, the $linkframework macro should be changed and possibly some interfaces added to the uses clause. Usually CocoaAll.pas will need to be added in order to get Foundation and AppKit functionality in supporting frameworks.}

View File

@ -0,0 +1,2 @@
1) cd to the directory which contains parser.php
2) php parser.php -cocoa -root="/Developer/ObjectivePascal"

View File

@ -0,0 +1,12 @@
The parser can parse single files at a time but must first parse the entire framework in order to merge all relevant information.
1) cd to the directory which contains parser.php
2) php parser.php -root="/Developer/ObjectivePascal" -objp -all -frameworks="foundation" -only="NSObject.h"
Note: -root is the path to the directory which contains the base appkit and foundation folders. These folders come in the FPC distribution with some hand parsed headers and the master include units for each framework.
-frameworks are the frameworks which the headers belong to.
You can also do a quick parse of a single header for testing purposes or acquiring other information by using the command below.
php parser.php -root="/Developer/ObjectivePascal" -objp -header="appkit/NSApplication.h"

View File

@ -0,0 +1,24 @@
It is against the Apple license to distribute even derivatives of the iPhone headers so you must build them manually from the Objective-C headers using the parser.
Please note you must download the iPhone SDK (2GB+) from Apple's website (https://developer.apple.com/iphone), which contains the headers. Once the files are installed make sure the following SDK is available at this location:
/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS2.2.1.sdk/System/Library/Frameworks/UIKit.framework/Headers
1) cd to the directory which contains parser.php
2) php parser.php -iphone -root="/Developer/ObjectivePascal/Headers"
Note: -root is the path to the directory which contains the base appkit and foundation folders. These folders come in the FPC distribution with some hand parsed headers and the master include units for each framework. In SVN they reside at packages/cocoaint/src.
3) Finally some handing coding must be done:
1) UITextInputTraits in UITextInputTraits.inc must be changed to UITextInputTraitsProtocol.
UITextField.inc(80,32) Error: Identifier not found "UITextInputTraits"
UITextField.inc(81,39) Error: Identifier not found "UITextInputTraits"
2) In UIAcceleration.inc you must comment out the follow 3 methods. Bug in FPC has been filed.
function x: UIAccelerationValue; message 'x';
function y: UIAccelerationValue; message 'y';
function z: UIAccelerationValue; message 'z';

View File

@ -8,10 +8,10 @@
<framework>
<name>foundation</name>
<root>/foundation/Foundation.inc</root>
<bridge>/bridgesupport/foundation.xml</bridge>
<headers>/System/Library/Frameworks/Foundation.framework/Headers</headers>
<include_pattern>{[$]+include (.*).inc}</include_pattern>
<header_pattern>^(.*)\.h</header_pattern>
<docset>/Developer/Documentation/DocSets/com.apple.adc.documentation.AppleSnowLeopard.CoreReference.docset</docset>
<!-- ignore_lines accepts regular expressions -->
<ignore_lines>
<line>#define NS_BLOCKS_AVAILABLE [0-9]+</line>
@ -19,7 +19,6 @@
<line>FOUNDATION_EXPORT void NSLogv(NSString *format, va_list args) NS_FORMAT_FUNCTION(1,0);</line>
<line>FOUNDATION_EXPORT void NSLog(NSString *format, ...) __attribute__((format(__NSString__, 1, 2)));</line>
<line>FOUNDATION_EXPORT void NSLogv(NSString *format, va_list args);</line>
</ignore_lines>
<ignore_comments>
<line>^[[:space:]]*NS.*\.h</line>
@ -32,10 +31,10 @@
<framework>
<name>appkit</name>
<root>/appkit/AppKit.inc</root>
<bridge>/bridgesupport/appkit.xml</bridge>
<headers>/System/Library/Frameworks/AppKit.framework/Headers</headers>
<include_pattern>{[$]+include (.*).inc}</include_pattern>
<header_pattern>^(.*)\.h</header_pattern>
<docset>/Developer/Documentation/DocSets/com.apple.adc.documentation.AppleSnowLeopard.CoreReference.docset</docset>
<ignore_comments>
<line>^[[:space:]]*NS.*\.h</line>
<line>MAC_OS_X_VERSION_MAX_ALLOWED</line>
@ -47,8 +46,8 @@
<framework>
<name>uikit</name>
<root>/uikit/UIKit.inc</root>
<bridge></bridge>
<headers>/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.2.sdk/System/Library/Frameworks/UIKit.framework/Headers</headers>
<docset>/Developer/Platforms/iPhoneOS.platform/Developer/Documentation/DocSets/com.apple.adc.documentation.AppleiOS4_2.iOSLibrary.docset</docset>
<include_pattern>{[$]+include (UI.*).inc}</include_pattern>
<header_pattern>^UI(.*)\.h</header_pattern>
<external_macro>UIKIT_EXTERN</external_macro>
@ -57,7 +56,6 @@
<framework>
<name>opengles</name>
<root>/opengles/OpenGLES.inc</root>
<bridge></bridge>
<headers>/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.2.sdk/System/Library/Frameworks/OpenGLES.framework/Headers</headers>
<include_pattern>{[$]+include (EAGL.*).inc}</include_pattern>
<header_pattern>^EAGL(.*)\.h</header_pattern>
@ -67,7 +65,6 @@
<framework>
<name>webkit</name>
<root>/webkit/WebKit.inc</root>
<bridge>/bridgesupport/webkit.xml</bridge>
<headers>/System/Library/Frameworks/WebKit.framework/Headers</headers>
<include_pattern>{[$]+include (.*).inc}</include_pattern>
<header_pattern>^(.*)\.h</header_pattern>
@ -123,7 +120,7 @@
<!-- No spaces! -->
<ignore_types>SomeQuartzPointer,AnotherQuartzPointer</ignore_types>
<!-- Ignore these methods (Pascal names, no spaces) -->
<ignore_methods>render_toIOSurface_bounds_colorSpace,reclaimResources</ignore_methods>
<ignore_methods></ignore_methods>
<replace_types>
</replace_types>
</framework>

View File

@ -1,6 +1,6 @@
<?php
$version = "2.1.5";
$version = "2.1.6";
require("source/objp.php");
@ -66,7 +66,11 @@ function HandleCommandLineOptions ($argv) {
case 'comments':
$options[$key] = true;
break;
case 'docsets':
$options[$key] = true;
break;
case 'merge':
$options[$key] = true;
break;
@ -181,6 +185,7 @@ $parser = new ObjectivePParser($root_path, "", $options["frameworks"], $options
// ??? These should be accessors
$parser->parse_comments = $options["comments"];
$parser->merge_headers = $options["merge"];
$parser->parse_docsets = $options["docsets"];
// Process single headers
if ($options["header"] && !$options["all"]) {

View File

@ -3,6 +3,7 @@
// Includes
require_once("utilities.php");
require_once("objp_base.php");
require_once("docset.php");
// Constants
define("CAST_HANDLE", true);
@ -334,6 +335,28 @@ class ObjectivePParser extends ObjectivePParserBase {
return $type;
}
/**
* DOCSETS UTILITIES
*/
function FindDocumentationForMethod ($class, $name) {
if ($this->docset) {
$doc = $this->docset[$class][$name];
if ($doc) return "{ $doc }";
}
}
function FindDocumentationForType ($name) {
if ($this->docset) {
foreach ($this->docset as $class) {
foreach ($class as $type => $text) {
if ($type == $name) return "{ $text }";
}
}
}
}
/**
* ADDING LANGUAGE STRUCTURE UTILITIES
*/
@ -342,7 +365,7 @@ class ObjectivePParser extends ObjectivePParserBase {
function AddMethodToClass (&$method, &$class) {
// ignore methods
// if (in_array($method["name"], $this->ignore_methods)) return false;
if (in_array($method["name"], $this->ignore_methods)) return false;
// add comment to the method
$method["comment"] = $this->InsertCurrentComment();
@ -1417,6 +1440,7 @@ class ObjectivePParser extends ObjectivePParserBase {
if ($class["methods"]) {
foreach ($class["methods"] as $method) {
if ($method["comment"]) $this->PrintOutput(2, $method["comment"]);
if ($method["documentation"]) $this->PrintOutput(2, $method["documentation"]);
$this->PrintOutput(2, $method["blocks_disable_comment"].$method["def"]." message '".$method["objc_method"]."';".$method["deprecated"]);
}
}
@ -1482,9 +1506,9 @@ class ObjectivePParser extends ObjectivePParserBase {
$this->PrintOutput(0, "{ Parsed from ".ucfirst($header["framework"]).".framework ".$header["name"]." }");
$date = @date("D M j G:i:s T Y");
$this->PrintOutput(0, "{ Version: $version - $date }");
$this->PrintOutput(0, "");
//$date = @date("D M j G:i:s T Y");
//$this->PrintOutput(0, "{ Version: $version - $date }");
//$this->PrintOutput(0, "");
$macro = strtoupper(substr($header["name"], 0, (strripos($header["name"], "."))));
@ -1598,7 +1622,13 @@ class ObjectivePParser extends ObjectivePParserBase {
// print methods
if ($protocol["methods"]) {
$section="";
foreach ($protocol["methods"] as $name => $method) {
// print the required/optional section
if ($method["section"] != $section) {
$section = $method["section"];
$this->PrintOutput(1, $section);
}
if ($method["comment"]) $this->PrintOutput(2, $method["comment"]);
$this->PrintOutput(2, $method["blocks_disable_comment"].$method["def"]." message '".$method["objc_method"]."';".$method["deprecated"]);
}
@ -2868,11 +2898,12 @@ class ObjectivePParser extends ObjectivePParserBase {
//print_r($this->dump[$file_name]["types"]);
}
// Parse all protocols in a header
function ParseHeaderProtocols ($file) {
$contents = file_get_contents($file);
$file_name = substr($file, (strripos($file, "/")) + 1, strlen($file));
$section = null;
// reset comments from previous parsing sections
$this->ResetComment();
@ -2901,13 +2932,22 @@ class ObjectivePParser extends ObjectivePParserBase {
// remove comments
$line = $this->RemoveComments($line);
// found @optional/@required section
if (eregi("^[[:space:]]*@(optional|required)+", $line, $captures)) {
$section = $captures[1];
}
// found property
if ($this->LineHasProperty($line, $captures)) {
$properties = $this->ParseClassProperty($current_protocol, $captures, $deprecatedmods);
foreach ($properties as $property) {
if ($property["setter"]) {
$property["setter"]["comment"] = $this->InsertCurrentComment();
$property["setter"]["section"] = $section;
$property["setter"]["documentation"] = $this->FindDocumentationForMethod($current_protocol, $property["setter"]["property"]);
$this->current_header["protocols"][$current_protocol]["methods"][$property["setter"]["objc_method"]] = $property["setter"];
// append to master list of protocols
@ -2916,6 +2956,9 @@ class ObjectivePParser extends ObjectivePParserBase {
if ($property["getter"]) {
$property["getter"]["comment"] = $this->InsertCurrentComment();
$property["getter"]["section"] = $section;
$property["getter"]["documentation"] = $this->FindDocumentationForMethod($current_protocol, $property["getter"]["property"]);
$this->current_header["protocols"][$current_protocol]["methods"][$property["getter"]["objc_method"]] = $property["getter"];
// append to master list of protocols
@ -2937,11 +2980,17 @@ class ObjectivePParser extends ObjectivePParserBase {
}
// append to classes
if (($method) /* && (!in_array($method["name"], $this->ignore_methods))*/ ) {
if (($method) && (!in_array($method["name"], $this->ignore_methods)) ) {
// add comment to the method
$method["comment"] = $this->InsertCurrentComment();
// add optional/required section to the method
$method["section"] = $section;
// add documentation for method
$method["documentation"] = $this->FindDocumentationForMethod($current_protocol, $method["objc_method"]);
$this->current_header["protocols"][$current_protocol]["methods"][$method["objc_method"]] = $method;
// append to master list of protocols
@ -2959,6 +3008,7 @@ class ObjectivePParser extends ObjectivePParserBase {
if ((eregi($this->regex_objc_protocol, $line, $captures)) && (!eregi(".*;$", $line))) {
$got_protocol = true;
$current_protocol = $captures[1];
print("+ Protocol $current_protocol\n");
if ($this->comment_terminated) $this->current_header["protocols"][$current_protocol]["comment"] = $this->InsertCurrentComment();
@ -3106,6 +3156,7 @@ class ObjectivePParser extends ObjectivePParserBase {
// property has attributes
if (count($parts) == 5) {
//$property["parameters"] = explode(",", $parts[1]);
$property["parameters"] = preg_split("/\s*,\s*/", $parts[1]);
$attributes = $parts[1];
$type = $parts[2];
@ -3139,7 +3190,10 @@ class ObjectivePParser extends ObjectivePParserBase {
foreach ($property_list as $property_name) {
// property name
if (eregi("([a-zA-Z0-9_]+)[[:space:]]*$", $property_name, $captures)) $property["name"] = ucwords($captures[1]);
if (eregi("([a-zA-Z0-9_]+)[[:space:]]*$", $property_name, $captures)) {
$property["name"] = ucwords($captures[1]);
$property["name_raw"] = $captures[1];
}
// property type
$type = $this->ConvertReturnType($type,$pointertype);
@ -3164,7 +3218,8 @@ class ObjectivePParser extends ObjectivePParserBase {
$method["setter"]["class"] = $class;
$method["setter"]["name"] = $name;
$method["setter"]["kind"] = "procedure";
$method["setter"]["deprecated"]=$deprecatedmods;
$method["setter"]["deprecated"] = $deprecatedmods;
$method["setter"]["property"] = $property["name_raw"];
//$method["setter"]["comment"] = $this->InsertCurrentComment();
}
@ -3184,7 +3239,8 @@ class ObjectivePParser extends ObjectivePParserBase {
$method["getter"]["class"] = $class;
$method["getter"]["name"] = $name;
$method["getter"]["kind"] = "function";
$method["getter"]["deprecated"]=$deprecatedmods;
$method["getter"]["deprecated"] = $deprecatedmods;
$method["getter"]["property"] = $property["name_raw"];
//$method["getter"]["comment"] = $this->InsertCurrentComment();
// append to array of methods
@ -3195,7 +3251,7 @@ class ObjectivePParser extends ObjectivePParserBase {
//print_r($methods);
return $methods;
}
// Parse header classes and methods
function ParseHeaderClasses ($file) {
$contents = file_get_contents($file);
@ -3333,6 +3389,10 @@ class ObjectivePParser extends ObjectivePParserBase {
// found method
if (preg_match($this->pregex_objc_method_params, $line, $captures)) {
$method = $this->ConvertObjcMethodToPascal($current, $line, $captures, $this->GetProtectedKeywords($this->current_class), true, $deprecatedmods);
// add documentation for method
$method["documentation"] = $this->FindDocumentationForMethod($current, $method["objc_method"]);
if ($this->AddMethodToClass($method, $this->dump[$file_name]["classes"][$current])) {
//if ($this->comment_terminated) $method["comment"] = $this->InsertCurrentComment();
$this->dump[$file_name]["classes"][$current]["methods"][] = $method;
@ -3340,6 +3400,10 @@ class ObjectivePParser extends ObjectivePParserBase {
} elseif (preg_match($this->pregex_objc_method_no_params, $line, $captures)) {
$method = $this->ConvertObjcMethodToPascal($current, $line, $captures, $this->GetProtectedKeywords($this->current_class), false, $deprecatedmods);
// add documentation for method
$method["documentation"] = $this->FindDocumentationForMethod($current, $method["objc_method"]);
if ($this->AddMethodToClass($method, $this->dump[$file_name]["classes"][$current])) {
//if ($this->comment_terminated) $method["comment"] = $this->InsertCurrentComment();
$this->dump[$file_name]["classes"][$current]["methods"][] = $method;
@ -3472,8 +3536,20 @@ class ObjectivePParser extends ObjectivePParserBase {
print("+ Parsed $file_name\n");
}
// Parses the docset at $path for the current framework
function ParseFrameworkDocset ($path) {
$name = basename($path);
$parser = new DocSetParser($path);
if ($parser->parse_directory($this->docset_paths[$name])) {
$this->docset = $parser->methods;
print("+ Parsed documentation for $name.\n");
}
}
// Parse all AppKit and Foundation framework headers
// Parse all headers assigned in $this->frameworks
function ParseAllFrameworks ($ignore_files, $parse_only) {
foreach ($this->frameworks as $framework_name => $framework_info) {
@ -3484,12 +3560,16 @@ class ObjectivePParser extends ObjectivePParserBase {
// set the current framework being parsed
$this->framework = $framework_name;
// get the root file path
if ($this->out != "/") {
$path = $this->root.$this->out."/".$framework_info["root"];
} else {
$path = $this->root.$framework_info["root"];
}
// Parse the framework docset
if ($this->parse_docsets) $this->ParseFrameworkDocset($framework_info["docset"]);
// Load the header if found
if (file_exists($path)) {
$contents = file_get_contents($path);
@ -3651,6 +3731,7 @@ class ObjectivePParser extends ObjectivePParserBase {
$this->frameworks[(string) $framework->name]["replace_types"] = $framework->replace_types;
$this->frameworks[(string) $framework->name]["ignore_lines"] = $framework->ignore_lines;
$this->frameworks[(string) $framework->name]["ignore_comments"] = $framework->ignore_comments;
$this->frameworks[(string) $framework->name]["docset"] = (string)$framework->docset;
$this->frameworks[(string) $framework->name]["enabled"] = false;
$this->frameworks[(string) $framework->name]["print"] = true;
}

View File

@ -29,7 +29,8 @@ class ObjectivePParserBase {
var $delegate_methods = array(); // Delegate methods and types from GEN_BRIDGE_METADATA XML data
var $delegate_method_names = array(); // Delegate method name array
var $type_encodings = array(); // Master listing of type encodings for each method in the frameworks
var $docset; // Current parsed docset for the framework (-all only)
// Comments Builder
var $comment_eol;
var $comment_terminated;
@ -68,7 +69,8 @@ class ObjectivePParserBase {
var $comment_padding_left = " "; // Padding applied to the left side of all comments
var $comment_padding_right = " "; // Padding applied to the right side of all comments
var $varargs_param_name = "firstKey"; // The name of the first parameter for methods using varargs
var $parse_docsets = false; // Parses docsets for symbol documentation
// array of all known framework classes (both regular and anonymous)
var $cocoa_classes = array("Protocol");
@ -175,8 +177,7 @@ class ObjectivePParserBase {
var $ignore_categories = array("NSURLLoading");
// Methods to ignore
var $ignore_methods = array( "observationInfo",
);
var $ignore_methods = array();
// methods to rename to particular alternatives
var $replace_instance_methods = array ( "class" => "_class", );
@ -237,7 +238,12 @@ class ObjectivePParserBase {
"[[:space:]]*NS_DEPRECATED[^(]*\([^;]*\)[[:space:]]*",
"[[:space:]]WEBKIT_OBJC_METHOD_ANNOTATION\([^;]*\)[[:space:]]*",
);
// Subpats to search for docsets
var $docset_paths = array( "com.apple.adc.documentation.AppleSnowLeopard.CoreReference.docset" => array("/Cocoa/Reference"),
"com.apple.adc.documentation.AppleiOS4_2.iOSLibrary.docset" => array("/UIKit/Reference"),
);
/**
* COMMON REGULAR EXPRESIONS