mirror of
https://gitlab.com/freepascal.org/fpc/source.git
synced 2025-08-27 20:11:02 +02:00
o readme.txt update by Karl-Michael Schindler:
* Put the usage paragraph to the top. + Add the now required make option TEST_FPC=path_to_your_compiler * corrected "make rundigest" to "make digest" * several small layout and wording changes git-svn-id: trunk@10791 -
This commit is contained in:
parent
0820353b71
commit
b58b0cc60f
137
tests/readme.txt
137
tests/readme.txt
@ -1,10 +1,33 @@
|
|||||||
|
Usage
|
||||||
|
-----
|
||||||
|
To start the test suite execute:
|
||||||
|
|
||||||
|
make full TEST_FPC=path_to_your_compiler
|
||||||
|
|
||||||
|
This should run all tests, scan the created log file and output some
|
||||||
|
statistics.
|
||||||
|
|
||||||
|
make digest TEST_FPC=path_to_your_compiler
|
||||||
|
|
||||||
|
scans the created log file and outputs some statistics
|
||||||
|
|
||||||
|
make digest USESQL=YES TEST_FPC=path_to_your_compiler
|
||||||
|
|
||||||
|
sends the results to an SQL database
|
||||||
|
|
||||||
|
When the tests are performed, first the units (e.g. rtl) needed by the
|
||||||
|
tests are compiled in a clean determined way and put in the units
|
||||||
|
directory. Then webtbs/webtbf/test/tbs/tbf are searched for t*.pp to be
|
||||||
|
compiled and executed as tests.
|
||||||
|
|
||||||
|
|
||||||
Directories
|
Directories
|
||||||
-----------
|
-----------
|
||||||
webtbs...........Tests for web-bug-database bugs (should compile/run)
|
webtbs...........Tests for web-bug-database bugs (should compile/run)
|
||||||
Digits in filename refer to bug database entry
|
Digits in filename refer to bug database entry
|
||||||
webtbf...........Tests for web-bug-database bugs (should not compile/run)
|
webtbf...........Tests for web-bug-database bugs (should not compile/run)
|
||||||
Digits in filename refer to bug database entry
|
Digits in filename refer to bug database entry
|
||||||
test.............Testsuites for different aspects of the compiler/rtl etc
|
test.............Test suites for different aspects of the compiler/rtl etc
|
||||||
tbs..............Tests for other bugs, added by the fpc core team
|
tbs..............Tests for other bugs, added by the fpc core team
|
||||||
(success in compilation) Digits in filename is a serial no
|
(success in compilation) Digits in filename is a serial no
|
||||||
tbf..............Tests for other bugs, added by the fpc core team
|
tbf..............Tests for other bugs, added by the fpc core team
|
||||||
@ -24,17 +47,16 @@ It should return 0 on success, any other value indicates failure.
|
|||||||
|
|
||||||
Test directives
|
Test directives
|
||||||
---------------
|
---------------
|
||||||
At the top of the test source code, some directives
|
At the top of the test source code, some directives can be used to
|
||||||
can be used to determine how the tests will be
|
determine how the tests will be processed (if processed automatically via
|
||||||
processed (if processed automatically via make),
|
make), e.g. {%CPU=i386}:
|
||||||
e. g. {%CPU=i386} :
|
|
||||||
|
|
||||||
OPT................Compiler option required to compile
|
OPT................Compiler option required to compile
|
||||||
CPU................Only for these CPU's (i386,m68k,etc). Might be a list.
|
CPU................Only for these CPU's (i386,m68k,etc). Might be a list.
|
||||||
SKIPCPU............Not for these CPU's (i386,m68k,etc). Might be a list.
|
SKIPCPU............Not for these CPU's (i386,m68k,etc). Might be a list.
|
||||||
TARGET.............Only for these OS targets (win32,macos,etc).
|
TARGET.............Only for these OS targets (win32,MacOS,etc).
|
||||||
Might be a list.
|
Might be a list.
|
||||||
SKIPTARGET.........Not for these OS targets (win32,macos,etc).
|
SKIPTARGET.........Not for these OS targets (win32,MacOS,etc).
|
||||||
Might be a list.
|
Might be a list.
|
||||||
VERSION............Compiler with at lest this version number required.
|
VERSION............Compiler with at lest this version number required.
|
||||||
MAXVERSION.........Compiler with at most this version number required.
|
MAXVERSION.........Compiler with at most this version number required.
|
||||||
@ -65,30 +87,17 @@ KNOWNCOMPILEERROR..Known bug, which manifest itself at compile time. To
|
|||||||
NOTE: A list consists of comma separated items, e. g. CPU=i386,m68k,powerpc
|
NOTE: A list consists of comma separated items, e. g. CPU=i386,m68k,powerpc
|
||||||
No space between the elements and the comma.
|
No space between the elements and the comma.
|
||||||
|
|
||||||
Usage
|
|
||||||
-----
|
|
||||||
To actually start the testsuite:
|
|
||||||
do a simple
|
|
||||||
make full This should create a log of all failed tests.
|
|
||||||
|
|
||||||
make rundigest scans the created log file and outputs some statistics
|
|
||||||
make rundigest USESQL=YES sends the results to an SQL database
|
|
||||||
|
|
||||||
When the tests are performed, first the units (e g rtl) needed by the tests
|
|
||||||
are compiled in a clean determined way and put in the units directory. Then
|
|
||||||
webtbs/webtbf/test/tbs/tbf are searched for t*.pp to be compiled
|
|
||||||
and executed as tests.
|
|
||||||
|
|
||||||
Controling testing in more detail
|
Controling testing in more detail
|
||||||
---------------------------------
|
---------------------------------
|
||||||
Calling "make full" will preform tests in a standard manner. To have
|
Calling "make full" will perform tests in a standard manner. To have
|
||||||
more control of the test process we must differentiate between:
|
more control of the test process one must distinguish between:
|
||||||
|
|
||||||
* Driver enviroment: compiler/rtl etc to be used by the tools which
|
* Driver enviroment: compiler/rtl etc. to be used by the tools which
|
||||||
runs and analyze the tests. All normal options to make, like FPC
|
runs and analyze the tests. All normal options to make, like FPC
|
||||||
OS_TARGET, OPT etc controls this.
|
OS_TARGET, OPT etc. controls this.
|
||||||
|
|
||||||
* Test environment: compiler/rtl etc to be tested, to be used
|
* Test environment: compiler/rtl etc. to be tested, to be used
|
||||||
*in* the tests. Ususal options, prepended with TEST_ , controls
|
*in* the tests. Ususal options, prepended with TEST_ , controls
|
||||||
this. If no such options are given, test and driver environment
|
this. If no such options are given, test and driver environment
|
||||||
will be the same.
|
will be the same.
|
||||||
@ -105,21 +114,23 @@ TEST_FPC_VERSION defaults to version of TEST_FPC
|
|||||||
TEST_CCOMPILER defaults to installed gcc compiler, but only
|
TEST_CCOMPILER defaults to installed gcc compiler, but only
|
||||||
if driver and test full-targets are the same.
|
if driver and test full-targets are the same.
|
||||||
TEST_VERBOSE let dotest be more verbose, only usefull for debugging
|
TEST_VERBOSE let dotest be more verbose, only usefull for debugging
|
||||||
TEST_DELTEMP delete temporary executable/object/ppu file, default is off
|
TEST_DELTEMP delete temporary executable/object/ppu file,
|
||||||
|
default is off
|
||||||
TEST_TIMEOUT use timeout wrapper for (remote) execution
|
TEST_TIMEOUT use timeout wrapper for (remote) execution
|
||||||
V print dotest commandline
|
V print dotest commandline
|
||||||
|
|
||||||
(Please add more test options if needed)
|
(Please add more test options if needed)
|
||||||
|
|
||||||
NOTE To clean after a test session, "make clean" must be given the same
|
NOTE: To clean after a test session, "make clean TEST_FPC=path_to_your_compiler"
|
||||||
options as when running the tests.
|
must be given the same options as when running the tests.
|
||||||
|
|
||||||
|
The utils directory is considerd to belong to the driver environment,
|
||||||
|
all other directories belong to the test environment.
|
||||||
|
|
||||||
The utils directory is considerd to belong to the driver environment,
|
|
||||||
all other directories belongs to the test environment.
|
|
||||||
|
|
||||||
Remote execution
|
Remote execution
|
||||||
----------------
|
----------------
|
||||||
Also remote execution of the testsuite is possible
|
Also remote execution of the test suite is possible.
|
||||||
|
|
||||||
Requirements:
|
Requirements:
|
||||||
- rsh/ssh must work without keyboard interaction or extra parameters
|
- rsh/ssh must work without keyboard interaction or extra parameters
|
||||||
@ -134,9 +145,11 @@ TEST_REMOTEPATH set remote path to use, default is /tmp
|
|||||||
TEST_DELBEFORE delete remote executable before uploading
|
TEST_DELBEFORE delete remote executable before uploading
|
||||||
TEST_DELTEMP delete executable after running, so the remote system
|
TEST_DELTEMP delete executable after running, so the remote system
|
||||||
doesn't need much free disk space
|
doesn't need much free disk space
|
||||||
TEST_REMOTEPW pass a password with -pw to remote tools, mainly usefull for putty
|
TEST_REMOTEPW pass a password with -pw to remote tools,
|
||||||
|
mainly usefull for putty
|
||||||
|
|
||||||
Example:
|
|
||||||
|
Examples:
|
||||||
-------
|
-------
|
||||||
make TEST_FPC=$HOME/fpc/compiler/ppcsparc TEST_BINUTILSPREFIX=sparc-linux- \
|
make TEST_FPC=$HOME/fpc/compiler/ppcsparc TEST_BINUTILSPREFIX=sparc-linux- \
|
||||||
TEST_RSH=sunny TEST_REMOTEPATH=/tmp/tests
|
TEST_RSH=sunny TEST_REMOTEPATH=/tmp/tests
|
||||||
@ -150,51 +163,65 @@ Example for win32/putty:
|
|||||||
TEST_PUTTY=root@192.168.42.210 TEST_REMOTEPATH=/tmp TEST_DELTEMP=1 \
|
TEST_PUTTY=root@192.168.42.210 TEST_REMOTEPATH=/tmp TEST_DELTEMP=1 \
|
||||||
"TEST_REMOTEPW=xxx" FPC=c:\fpc\compiler\ppc386
|
"TEST_REMOTEPW=xxx" FPC=c:\fpc\compiler\ppc386
|
||||||
|
|
||||||
|
|
||||||
Emulator execution
|
Emulator execution
|
||||||
------------------
|
------------------
|
||||||
|
Emulator execution is possible as well. It can't be combined with remote
|
||||||
|
execution though.
|
||||||
|
|
||||||
Emulator execution is possible as well. It can't be combined with remote execution though.
|
EMULATOR: name of the emulator to use
|
||||||
|
|
||||||
EMULATOR name of the emulator to use
|
Examples:
|
||||||
|
|
||||||
Example:
|
make TEST_FPC=~/fpc/compiler/ppcrossarm TEST_OPT=-XParm-linux- \
|
||||||
|
EMULATOR=qemu-arm
|
||||||
|
make TEST_FPC=~/fpc/compiler/ppcrossarm TEST_OPT=-XParm-linux- \
|
||||||
|
EMULATOR=qemu-arm digest DBDIGESTOPT="-C qemu-arm" USESQL=YES
|
||||||
|
|
||||||
make TEST_FPC=~/fpc/compiler/ppcrossarm TEST_OPT=-XParm-linux- EMULATOR=qemu-arm
|
|
||||||
make TEST_FPC=~/fpc/compiler/ppcrossarm TEST_OPT=-XParm-linux- EMULATOR=qemu-arm \
|
|
||||||
digest DBDIGESTOPT="-C qemu-arm" USESQL=YES
|
|
||||||
|
|
||||||
Example cross testing of target MacOS with driver Darwin
|
Example cross testing of target Mac OS with driver Darwin
|
||||||
--------------------------------------------------------
|
--------------------------------------------------------
|
||||||
NOTE Today it is possible to run the test suite MacOS native.
|
NOTE: Today, it is possible to run the test suite Mac OS native.
|
||||||
|
|
||||||
A machine with both MacOS X and classic MacOS installed. Note that make will not
|
A machine with both Mac OS X and classic Mac OS installed is required.
|
||||||
run the tests, that has to be done in MPW with the scripts in utils/macos.
|
Note that make will not run the tests. This has to be done in MPW with the
|
||||||
|
scripts in utils/MacOS.
|
||||||
|
|
||||||
make clean alltest TEST_OS_TARGET=macos TEST_OPT="-WT -st" \
|
make clean alltest TEST_OS_TARGET=MacOS TEST_OPT="-WT -st" \
|
||||||
USEUNITDIR=/Projekt/Freepascal/fpc/rtl/macos
|
USEUNITDIR=/Projekt/Freepascal/fpc/rtl/MacOS
|
||||||
|
|
||||||
To clean. Note that same options as above has to be given so that the correct
|
To clean. Note that same options as above has to be given so that the
|
||||||
files will be removed.
|
correct files will be removed.
|
||||||
|
|
||||||
|
make clean TEST_OS_TARGET=MacOS USEUNITDIR=/Projekt/Freepascal/fpc/rtl/MacOS
|
||||||
|
|
||||||
make clean TEST_OS_TARGET=macos USEUNITDIR=/Projekt/Freepascal/fpc/rtl/macos
|
|
||||||
|
|
||||||
Example cross testing of target arm-wince
|
Example cross testing of target arm-wince
|
||||||
-----------------------------------------
|
-----------------------------------------
|
||||||
//arm-wince example : see FPCTRUNK\DEMO\WINCE\TESTEMU\ for additional required tools
|
//arm-wince example:
|
||||||
|
see FPCTRUNK\DEMO\WINCE\TESTEMU\ for additionally required tools
|
||||||
Connect your device via ActiveSync and execute:
|
Connect your device via ActiveSync and execute:
|
||||||
|
|
||||||
make TEST_FPC=ppcrossarm TEST_CPU_TARGET=arm TEST_OS_TARGET=wince TEST_OPT="-XParm-wince- -WC -Xs" EMULATOR=MyDisc:\My\Path\to\wcetemu.exe
|
make TEST_FPC=ppcrossarm TEST_CPU_TARGET=arm TEST_OS_TARGET=wince \
|
||||||
|
TEST_OPT="-XParm-wince- -WC -Xs" \
|
||||||
|
EMULATOR=MyDisc:\My\Path\to\wcetemu.exe
|
||||||
|
|
||||||
|
Tests will be performed in the \fpctests folder on the device.
|
||||||
|
|
||||||
Tests will be performed in \fpctests folder on the device.
|
|
||||||
|
|
||||||
Result uploading
|
Result uploading
|
||||||
----------------
|
----------------
|
||||||
Results can be uploaded to the testsuite result DB (http://www.freepascal.org/cgi-bin/testsuite.cgi) by executing
|
Results can be uploaded to the test suite result DB
|
||||||
|
(http://www.freepascal.org/cgi-bin/testsuite.cgi) by executing
|
||||||
|
|
||||||
make uploadrun
|
make uploadrun
|
||||||
|
|
||||||
On unix:
|
On unix:
|
||||||
- The testing machine must be enabled to login on www.freepascal.org automatically as user fpc with ssh.
|
- The testing machine must be enabled to login on www.freepascal.org
|
||||||
|
automatically as user fpc with ssh.
|
||||||
|
|
||||||
On Windows:
|
On Windows:
|
||||||
- The putty utilities plink.exe and pscp.exe must be in the path
|
- The putty utilities plink.exe and pscp.exe must be in the path
|
||||||
- There must be a putty session named fpc@www.freepascal.org which is enabled to login automatically into www.freepascal.org
|
- There must be a putty session named fpc@www.freepascal.org which is
|
||||||
|
enabled to login automatically into www.freepascal.org
|
||||||
|
|
||||||
|
Loading…
Reference in New Issue
Block a user