PostgresOSM.fetch_data¶

PostgresOSM.fetch_data(subregion_name, layer_names=None, chunk_size=None, method='tempfile', max_size_spooled=1, decode_geojson=True, sort_by='id', table_named_as_subregion=False, schema_named_as_layer=False, verbose=False, raise_error=False, **kwargs)[source]¶

Fetch OSM data (of one or multiple layers) of a geographic (sub)region.

See also [ROP-1].

Parameters:

subregion_name (str) – name of a geographic (sub)region (or the corresponding table)
layer_names (list | None) – names of schemas for each layer of the PBF data, if None (default), the default layer names as schema names
chunk_size (int | None) – the number of rows in each batch to be written at a time, defaults to None
method (str | None) – method to be used for buffering temporary data, defaults to 'tempfile'
max_size_spooled (int, float) – see pyhelpers.sql.PostgreSQL.read_sql_query(), defaults to 1 (in GB)
decode_geojson (bool) – whether to decode string GeoJSON data, defaults to True
sort_by (str | list) – column name(s) by which the data (fetched from PostgreSQL) is sorted, defaults to 'id'
table_named_as_subregion (bool) – whether to use subregion name as a table name, defaults to False
schema_named_as_layer (bool) – whether a schema is named as a layer name, defaults to False
verbose (bool | int) – whether to print relevant information in console, defaults to False
raise_error (bool) – Whether to raise the provided exception; if raise_error=False (default), the error will be suppressed.

Returns:

PBF (.osm.pbf) data

Return type:

dict

Examples:

>>> from pydriosm.ios import PostgresOSM
>>> from pyhelpers.dirs import delete_dir

>>> osmdb = PostgresOSM(database_name='osmdb_test')
Password (postgres@localhost:5432): ***
Creating a database: "osmdb_test" ... Done.
Connecting postgres:***@localhost:5432/osmdb_test ... Successfully.

>>> subrgn_name = 'Rutland'  # name of a subregion
>>> dat_dir = "tests/osm_data"  # name of a data directory where the subregion data is

>>> # Import PBF data of Rutland
>>> osmdb.import_osm_pbf(subrgn_name, data_dir=dat_dir, verbose=True)
Proceed to import .osm.pbf data of the following geographic (sub)region(s):
    "Rutland"
  into postgres:***@localhost:5432/osmdb_test
? [No]|Yes: yes
Downloading "rutland-latest.osm.pbf" 100%|██████████| 1.89M/1.89M | 6.46MB/s ...
  Saving "rutland-latest.osm.pbf" to "./tests/osm_data/rutland/" ... Done.
Reading "tests/osm_data/rutland/rutland-latest.osm.pbf" ... Done.
Importing the data into the table "Rutland" ...
    "points" ... Done. (6420 features)
    "lines" ... Done. (10958 features)
    "multilinestrings" ... Done. (71 features)
    "multipolygons" ... Done. (9194 features)
    "other_relations" ... Done. (33 features)

>>> # Import shapefile data of Rutland
>>> rutland_shp = osmdb.reader.read_shp(
...     subrgn_name, data_dir=dat_dir, rm_extracts=True, verbose=True)
Downloading "rutland-latest-free.shp.zip" 100%|██████████| 2.71M/2.71M | 7.22...
  Saving "rutland-latest-free.shp.zip" to "./tests/osm_data/rutland/" ... Done.
Extracting "./tests/osm_data/rutland/rutland-latest-free.shp.zip"
    to "./tests/osm_data/rutland/rutland-latest-free-shp/" ... Done.
Reading the shapefile(s) at "./tests/osm_data/rutland/rutland-latest-free-shp/" ......
Deleting the extracts "./tests/osm_data/rutland/rutland-latest-free-shp/" ... Done.
>>> osmdb.import_osm_data(rutland_shp, table_name=subrgn_name, verbose=True)
Proceed to import data into the table "Rutland" at postgres:***@localhost:5432/osmdb_test
? [No]|Yes: yes
Importing the data ...
    "buildings" ... Done. (5552 features)
    "landuse" ... Done. (2436 features)
    "natural" ... Done. (664 features)
    "places" ... Done. (301 features)
    "pofw" ... Done. (65 features)
    "pois" ... Done. (1077 features)
    "railways" ... Done. (137 features)
    "roads" ... Done. (7294 features)
    "traffic" ... Done. (537 features)
    "transport" ... Done. (65 features)
    "water" ... Done. (223 features)
    "waterways" ... Done. (379 features)

>>> # Retrieve the data of specific layers
>>> lyr_names = ['points', 'multipolygons']
>>> rutland_data_ = osmdb.fetch_data(subrgn_name, lyr_names, verbose=True)
Fetching the data of "Rutland" ...
    "points" ... Done.
    "multipolygons" ... Done.
>>> type(rutland_data_)
dict
>>> list(rutland_data_.keys())
['points', 'multipolygons']

>>> # Data of the 'points' layer
>>> rutland_points = rutland_data_['points']
>>> rutland_points.head()
                                              points
0  {'type': 'Feature', 'geometry': {'type': 'Poin...
1  {'type': 'Feature', 'geometry': {'type': 'Poin...
2  {'type': 'Feature', 'geometry': {'type': 'Poin...
3  {'type': 'Feature', 'geometry': {'type': 'Poin...
4  {'type': 'Feature', 'geometry': {'type': 'Poin...

>>> # Retrieve the data of all the layers from the database
>>> rutland_data = osmdb.fetch_data(subrgn_name, layer_names=None, verbose=True)
Fetching the data of "Rutland" ...
    "points" ... Done.
    "lines" ... Done.
    "multilinestrings" ... Done.
    "multipolygons" ... Done.
    "other_relations" ... Done.
    "buildings" ... Done.
    "landuse" ... Done.
    "natural" ... Done.
    "places" ... Done.
    "pofw" ... Done.
    "pois" ... Done.
    "railways" ... Done.
    "roads" ... Done.
    "traffic" ... Done.
    "transport" ... Done.
    "water" ... Done.
    "waterways" ... Done.
>>> type(rutland_data)
dict
>>> list(rutland_data.keys())
['points',
 'lines',
 'multilinestrings',
 'multipolygons',
 'other_relations',
 'buildings',
 'landuse',
 'natural',
 'places',
 'pofw',
 'pois',
 'railways',
 'roads',
 'traffic',
 'transport',
 'water',
 'waterways']

>>> # Data of the 'waterways' layer
>>> rutland_waterways = rutland_data['waterways']
>>> rutland_waterways.head()
    osm_id  code  ...                                        coordinates  shape_type
0  3701346  8102  ...  [(-0.7536654, 52.6495358), (-0.7536236, 52.649...           3
1  3701347  8102  ...  [(-0.7948821, 52.6569468), (-0.7946128, 52.656...           3
2  3707149  8103  ...  [(-0.7262381, 52.6790459), (-0.7258244, 52.680...           3
3  3707303  8102  ...  [(-0.7213277, 52.6765954), (-0.7206778, 52.676...           3
4  4470795  8101  ...  [(-0.4995349, 52.6418825), (-0.4984075, 52.642...           3
[5 rows x 7 columns]

Delete the test database and downloaded data files:

>>> # Delete the database 'osmdb_test'
>>> osmdb.drop_database(verbose=True)
Proceed to drop the database "osmdb_test" from postgres:***@localhost:5432
? [No]|Yes: yes
Dropping "osmdb_test" ... Done.

>>> # Delete the downloaded data files
>>> delete_dir(dat_dir, verbose=True)
To delete the directory "./tests/osm_data/" (Not empty)
? [No]|Yes: yes
Deleting "./tests/osm_data/" ... Done.