From b2be8c3ab7baca371ae2d2f65555e0a5ca09f531 Mon Sep 17 00:00:00 2001 From: Sarah Hoffmann Date: Fri, 30 Dec 2016 22:55:57 +0100 Subject: [PATCH] move php tests in common test dir and unify READMEs --- phpunit.xml | 4 +- test/README.md | 133 ++++++++++++++++++ test/bdd/db/import/parenting.feature | 12 +- test/bdd/environment.py | 1 - test/bdd/steps/queries.py | 2 +- .../php}/Nominatim/NominatimTest.php | 2 +- {tests-php => test/php}/bootstrap.php | 0 test/testdb/README.md | 15 -- tests-php/README.txt | 14 -- tests/README.md | 102 -------------- 10 files changed, 145 insertions(+), 140 deletions(-) create mode 100644 test/README.md rename {tests-php => test/php}/Nominatim/NominatimTest.php (99%) rename {tests-php => test/php}/bootstrap.php (100%) delete mode 100644 test/testdb/README.md delete mode 100644 tests-php/README.txt delete mode 100644 tests/README.md diff --git a/phpunit.xml b/phpunit.xml index bea876d5..addce5ce 100644 --- a/phpunit.xml +++ b/phpunit.xml @@ -8,13 +8,13 @@ processIsolation="false" stopOnFailure="false" syntaxCheck="true" - bootstrap="tests-php/bootstrap.php" + bootstrap="test/php/bootstrap.php" > - ./tests-php/Nominatim + ./test/php/Nominatim diff --git a/test/README.md b/test/README.md new file mode 100644 index 00000000..49fb101d --- /dev/null +++ b/test/README.md @@ -0,0 +1,133 @@ +This directory contains functional and unit tests for the Nominatim API. + +Prerequisites +============= + + * Python 3 (https://www.python.org/) + * behave test framework >= 1.2.5 (https://github.com/behave/behave) + * nose (https://nose.readthedocs.org) + * pytidylib (http://countergram.com/open-source/pytidylib) + * psycopg2 (http://initd.org/psycopg/) + +To get the prerequisites on a a fresh Ubuntu LTS 16.04 run: + + [sudo] apt-get install python3-dev python3-pip python3-psycopg2 python3-tidylib phpunit + pip3 install --user behave nose + + +Overall structure +================= + +There are two kind of tests in this test suite. There are functional tests +which test the API interface using a BDD test framework and there are unit +tests for specific PHP functions. + +This test directory is sturctured as follows: + + -+- bdd Functional API tests + | \ + | +- steps Step implementations for test descriptions + | +- osm2pgsql Tests for data import via osm2pgsql + | +- db Tests for internal data processing on import and update + | +- api Tests for API endpoints (search, reverse, etc.) + | + +- php PHP unit tests + +- scenes Geometry test data + +- testdb Base data for generating API test database + + +PHP Unit Tests +============== + +Unit tests can be found in the php/ directory and tests selected php functions. +Very low coverage. + +To execute the test suite run + + cd test/php + phpunit ../ + +It will read phpunit.xml which points to the library, test path, bootstrap +strip and set other parameters. + + +BDD Functional Tests +==================== + +Functional tests are written as BDD instructions. For more information on +the philosophy of BDD testing, see http://pythonhosted.org/behave/philosophy.html + +Usage +----- + +To run the functional tests, do + + cd test/bdd + behave + +The tests can be configured with a set of environment variables: + + * `BUILD_DIR` - build directory of Nominatim installation to test + * `TEMPLATE_DB` - name of template database used as a skeleton for + the test databases (db tests) + * `TEST_DB` - name of test database (db tests) + * `ABI_TEST_DB` - name of the database containing the API test data (api tests) + * `TEST_SETTINGS_TEMPLATE` - file to write temporary Nominatim settings to + * `REMOVE_TEMPLATE` - if true, the template database will not be reused during + the next run. Reusing the base templates speeds up tests + considerably but might lead to outdated errors for some + changes in the database layout. + * `KEEP_TEST_DB` - if true, the test database will not be dropped after a test + is finished. Should only be used if one single scenario is + run, otherwise the result is undefined. + +Logging can be defined through command line parameters of behave itself. Check +out `behave --help` for details. Also keep an eye out for the 'work-in-progress' +feature of behave which comes in handy when writing new tests. + +Writing Tests +------------- + +The following explanation assume that the reader is familiar with the BDD +notations of features, scenarios and steps. + +All possible steps can be found in the `steps` directory and should ideally +be documented. + +### API Tests (`test/bdd/api`) + +These tests are meant to test the different API endpoints and their parameters. +They require a preimported test database, which consists of the import of a +planet extract. The polygons defining the extract can be found in the test/testdb +directory. There is also a reduced set of wikipedia data for this extract, +which you need to import as well. + +The official test dataset is derived from the 160725 planet. Newer +planets are likely to work as well but you may see isolated test +failures where the data has changed. To recreate the input data +for the test database run: + + wget http://free.nchc.org.tw/osm.planet/pbf/planet-160725.osm.pbf + osmconvert planet-160725.osm.pbf -B=test/testdb/testdb.polys -o=testdb.pbf + +Before importing make sure to add the following to your local settings: + + @define('CONST_Database_DSN', 'pgsql://@/test_api_nominatim'); + @define('CONST_Wikipedia_Data_Path', CONST_BasePath.'/test/testdb'); + +### Indexing Tests (`test/bdd/db`) + +These tests check the import and update of the Nominatim database. They do not +test the correctness of osm2pgsql. Each test will write some data into the `place` +table (and optionally `the planet_osm_*` tables if required) and then run +Nominatim's processing functions on that. + +These tests need to create their own test databases. By default they will be +called `test_template_nominatim` and `test_nominatim`. Names can be changed with +the environment variables `TEMPLATE_DB` and `TEST_DB`. The user running the tests +needs superuser rights for postgres. + +### Import Tests (`test/bdd/osm2pgsql`) + +These tests check that data is imported correctly into the place table. They +use the same template database as the Indexing tests, so the same remarks apply. diff --git a/test/bdd/db/import/parenting.feature b/test/bdd/db/import/parenting.feature index c00f4701..b0b76438 100644 --- a/test/bdd/db/import/parenting.feature +++ b/test/bdd/db/import/parenting.feature @@ -18,10 +18,14 @@ Feature: Parenting of objects | object | parent_place_id | | N1 | W1 | | N2 | W1 | - And search_name contains - | object | nameaddress_vector | - | N1 | 4, galoo, 12345 | - | N2 | 5, galoo, 99999 | + When searching for "4 galoo" + Then results contain + | ID | osm_type | osm_id | langaddress + | 0 | N | 1 | 4, galoo, 12345 + When searching for "5 galoo" + Then results contain + | ID | osm_type | osm_id | langaddress + | 0 | N | 2 | 5, galoo, 99999 Scenario: Address without tags, closest street Given the scene roads-with-pois diff --git a/test/bdd/environment.py b/test/bdd/environment.py index 69d93bd8..6411d011 100644 --- a/test/bdd/environment.py +++ b/test/bdd/environment.py @@ -10,7 +10,6 @@ from sys import version_info as python_version logger = logging.getLogger(__name__) userconfig = { - 'BASEURL' : 'http://localhost/nominatim', 'BUILDDIR' : os.path.join(os.path.split(__file__)[0], "../../build"), 'REMOVE_TEMPLATE' : False, 'KEEP_TEST_DB' : False, diff --git a/test/bdd/steps/queries.py b/test/bdd/steps/queries.py index 7d3dec69..c429f082 100644 --- a/test/bdd/steps/queries.py +++ b/test/bdd/steps/queries.py @@ -264,7 +264,7 @@ def send_api_query(endpoint, params, fmt, context): for h in context.table.headings: params[h] = context.table[0][h] - env = BASE_SERVER_ENV + env = dict(BASE_SERVER_ENV) env['QUERY_STRING'] = urlencode(params) env['SCRIPT_NAME'] = '/%s.php' % endpoint diff --git a/tests-php/Nominatim/NominatimTest.php b/test/php/Nominatim/NominatimTest.php similarity index 99% rename from tests-php/Nominatim/NominatimTest.php rename to test/php/Nominatim/NominatimTest.php index 7822c5dc..f8ba14c1 100644 --- a/tests-php/Nominatim/NominatimTest.php +++ b/test/php/Nominatim/NominatimTest.php @@ -2,7 +2,7 @@ namespace Nominatim; -require '../lib/lib.php'; +require '../../lib/lib.php'; class NominatimTest extends \PHPUnit_Framework_TestCase { diff --git a/tests-php/bootstrap.php b/test/php/bootstrap.php similarity index 100% rename from tests-php/bootstrap.php rename to test/php/bootstrap.php diff --git a/test/testdb/README.md b/test/testdb/README.md deleted file mode 100644 index a39b0258..00000000 --- a/test/testdb/README.md +++ /dev/null @@ -1,15 +0,0 @@ -Creating the test database -========================== - -The official test dataset is derived from the 160725 planet. Newer -planets are likely to work as well but you may see isolated test -failures where the data has changed. To recreate the input data -for the test database run: - - wget http://free.nchc.org.tw/osm.planet/pbf/planet-160725.osm.pbf - osmconvert planet-160725.osm.pbf -B=testdb.polys -o=testdb.pbf - -Before importing make sure to add the following to your local settings: - - @define('CONST_Database_DSN', 'pgsql://@/test_api_nominatim'); - @define('CONST_Wikipedia_Data_Path', CONST_BasePath.'/test/testdb'); diff --git a/tests-php/README.txt b/tests-php/README.txt deleted file mode 100644 index d551c1da..00000000 --- a/tests-php/README.txt +++ /dev/null @@ -1,14 +0,0 @@ -Basic unit tests of PHP code. Very low coverage. Doesn't cover interaction -with the webserver/HTTP or database (yet). - -You need to have -https://phpunit.de/manual/4.2/en/ -installed. - -To execute the test suite run -$ cd tests-php -$ phpunit ./ - -It will read phpunit.xml which points to the library, test path, bootstrap -strip and set other parameters. - diff --git a/tests/README.md b/tests/README.md deleted file mode 100644 index 1b1663c3..00000000 --- a/tests/README.md +++ /dev/null @@ -1,102 +0,0 @@ -This directory contains functional tests for the Nominatim API, -for the import/update from osm files and for indexing. - -The tests use the lettuce framework (http://lettuce.it/) and -nose (https://nose.readthedocs.org). API tests are meant to be run -against a Nominatim installation with a complete planet-wide -setup based on a fairly recent planet. If you only have an -excerpt, some of the API tests may fail. Database tests can be -run without having a database installed. - -Prerequisites -============= - - * lettuce framework (http://lettuce.it/) - * nose (https://nose.readthedocs.org) - * pytidylib (http://countergram.com/open-source/pytidylib) - * haversine (https://github.com/mapado/haversine) - * shapely (https://github.com/Toblerity/Shapely) - -Usage -===== - - * get prerequisites - - # on a fresh Ubuntu LTS 14.04 you'll also need these system-wide packages - [sudo] apt-get install python-dev python-pip python-Levenshtein tidy - - [sudo] pip install lettuce nose pytidylib haversine psycopg2 shapely - - * run the tests - - NOMINATIM_SERVER=http://your.nominatim.instance/ lettuce features - -The tests can be configured with a set of environment variables: - - * `NOMINATIM_SERVER` - URL of the nominatim instance (API tests) - * `NOMINATIM_DIR` - source directory of Nominatim (import tests) - * `TEMPLATE_DB` - name of template database used as a skeleton for - the test databases (db tests) - * `TEST_DB` - name of test database (db tests) - * `NOMINATIM_SETTINGS` - file to write temporary Nominatim settings to (db tests) - * `NOMINATIM_REUSE_TEMPLATE` - if defined, the template database will not be - deleted after the test runs and reused during - the next run. This speeds up tests considerably - but might lead to outdated errors for some - changes in the database layout. - * `NOMINATIM_KEEP_SCENARIO_DB` - if defined, the test database will not be - dropped after a test is finished. Should - only be used if one single scenario is run, - otherwise the result is undefined. - * `LOGLEVEL` - set to 'debug' to get more verbose output (only works properly - when output to a logfile is configured) - * `LOGFILE` - sends debug output to the given file - -Writing Tests -============= - -The following explanation assume that the reader is familiar with the lettuce -notations of features, scenarios and steps. - -All possible steps can be found in the `steps` directory and should ideally -be documented. - - -API Tests (`features/api`) --------------------------- - -These tests are meant to test the different API calls and their parameters. - -There are two kind of steps defined for these tests: -request setup steps (see `steps/api_setup.py`) -and steps for checking results (see `steps/api_result.py`). - -Each scenario follows this simple sequence of steps: - - 1. One or more steps to define parameters and HTTP headers of the request. - These are cumulative, so you can use multiple steps. - 2. A single step to call the API. This sends a HTTP request to the configured - server and collects the answer. The cached parameters will be deleted, - to ensure that the setup works properly with scenario outlines. - 3. As many result checks as necessary. The result remains cached, so that - multiple tests can be added here. - -Indexing Tests (`features/db`) ---------------------------------------------------- - -These tests check the import and update of the Nominatim database. They do not -test the correctness of osm2pgsql. Each test will write some data into the `place` -table (and optionally `the planet_osm_*` tables if required) and then run -Nominatim's processing functions on that. - -These tests need to create their own test databases. By default they will be -called `test_template_nominatim` and `test_nominatim`. Names can be changed with -the environment variables `TEMPLATE_DB` and `TEST_DB`. The user running the tests -needs superuser rights for postgres. - - -Import Tests (`features/osm2pgsql`) ------------------------------------ - -These tests check that data is imported correctly into the place table. They -use the same template database as the Indexing tests, so the same remarks apply. -- 2.45.1