PostgresOSM.fetch_data¶

PostgresOSM.fetch_data(subregion_name, layer_names=None, chunk_size=None, method='tempfile', max_size_spooled=1, decode_geojson=True, sort_by='id', table_named_as_subregion=False, schema_named_as_layer=False, verbose=False, raise_error=False, **kwargs)[source]¶

Fetch OSM data (of one or multiple layers) of a geographic (sub)region.

See also [ROP-1].

Parameters:

subregion_name (str) – name of a geographic (sub)region (or the corresponding table)
layer_names (list | None) – names of schemas for each layer of the PBF data, if None (default), the default layer names as schema names
chunk_size (int | None) – the number of rows in each batch to be written at a time, defaults to None
method (str | None) – method to be used for buffering temporary data, defaults to 'tempfile'
max_size_spooled (int, float) – see pyhelpers.sql.PostgreSQL.read_sql_query(), defaults to 1 (in GB)
decode_geojson (bool) – whether to decode string GeoJSON data, defaults to True
sort_by (str | list) – column name(s) by which the data (fetched from PostgreSQL) is sorted, defaults to 'id'
table_named_as_subregion (bool) – whether to use subregion name as a table name, defaults to False
schema_named_as_layer (bool) – whether a schema is named as a layer name, defaults to False
verbose (bool | int) – whether to print relevant information in console, defaults to False
raise_error (bool) – Whether to raise the provided exception; if raise_error=False (default), the error will be suppressed.

Returns:

PBF (.osm.pbf) data

Return type:

dict

Examples:

>>> from pydriosm.ios import PostgresOSM
>>> from pyhelpers.dirs import delete_dir

>>> osmdb = PostgresOSM(database_name='osmdb_test', verbose=True)
Password (postgres@localhost:5432): ***
Creating a database: "osmdb_test" ... Done.
Connecting postgres:***@localhost:5432/osmdb_test ... Successfully.

>>> subrgn_name = 'Rutland'  # name of a subregion
>>> dat_dir = "tests/osm_data"  # name of a data directory where the subregion data is

>>> # Import PBF data of Rutland
>>> osmdb.import_osm_pbf(subrgn_name, data_dir=dat_dir, verbose=True)
Proceed to import .osm.pbf data of the following geographic (sub)region(s):
    "Rutland"
  into postgres:***@localhost:5432/osmdb_test
? [No]|Yes: yes
Downloading "rutland-latest.osm.pbf" 100%|██████████| 1.89M/1.89M | 6.46MB/s ...
  Saving "rutland-latest.osm.pbf" to "./tests/osm_data/rutland/" ... Done.
Reading "tests/osm_data/rutland/rutland-latest.osm.pbf" ... Done.
Importing the data into the table "Rutland" ...
  "points" ... Done. (6473 features)
  "lines" ... Done. (11057 features)
  "multilinestrings" ... Done. (71 features)
  "multipolygons" ... Done. (9423 features)
  "other_relations" ... Done. (33 features)

>>> # Import shapefile data of Rutland
>>> rutland_shp = osmdb.reader.read_shp(
...     subrgn_name, data_dir=dat_dir, rm_extracts=True, verbose=True)
>>> rutland_shp is None
True

>>> # Import geopackage data of Rutland
>>> rutland_gpkg = osmdb.reader.read_gpkg(
...     subrgn_name, data_dir=dat_dir, download=True, verbose=True)
Downloading "rutland-latest-free.gpkg.zip" 100%|██████████| 3.30M/3.30M | 7.5...
  Saving "rutland-latest-free.gpkg.zip" to "./tests/osm_data/rutland/" ... Done.
Parsing the data ... Done.
>>> osmdb.import_osm_data(rutland_gpkg, table_name=subrgn_name, verbose=True)
Proceed to import data into the table "Rutland" at postgres:***@localhost:5432/osmd...
? [No]|Yes: yes
Importing the data ...
  "traffic" ... Done. (557 features)
  "places" ... Done. (301 features)
  "pois" ... Done. (1081 features)
  "transport" ... Done. (65 features)
  "pofw" ... Done. (65 features)
  "natural" ... Done. (665 features)
  "railways" ... Done. (141 features)
  "roads" ... Done. (7315 features)
  "waterways" ... Done. (379 features)
  "protected_areas" ... Done. (20 features)
  "water" ... Done. (233 features)
  "landuse" ... Done. (2461 features)
  "buildings" ... Done. (5706 features)
  "adminareas" ... Done. (58 features)

>>> # Retrieve the data of specific layers
>>> lyr_names = ['points', 'multipolygons']
>>> rutland_data_ = osmdb.fetch_data(subrgn_name, lyr_names, verbose=True)
Fetching the data of "Rutland" ...
  "points" ... Done.
  "multipolygons" ... Done.
>>> type(rutland_data_)
dict
>>> list(rutland_data_.keys())
['points', 'multipolygons']

>>> # Data of the 'points' layer
>>> rutland_points = rutland_data_['points']
>>> rutland_points.head()
                                              points
0  {'type': 'Feature', 'geometry': {'type': 'Poin...
1  {'type': 'Feature', 'geometry': {'type': 'Poin...
2  {'type': 'Feature', 'geometry': {'type': 'Poin...
3  {'type': 'Feature', 'geometry': {'type': 'Poin...
4  {'type': 'Feature', 'geometry': {'type': 'Poin...

>>> # Retrieve the data of all the layers from the database
>>> rutland_data = osmdb.fetch_data(subrgn_name, layer_names=None, verbose=True)
Fetching the data of "Rutland" ...
  "points" ... Done.
  "lines" ... Done.
  "multilinestrings" ... Done.
  "multipolygons" ... Done.
  "other_relations" ... Done.
  "adminareas" ... Done.
  "buildings" ... Done.
  "landuse" ... Done.
  "natural" ... Done.
  "places" ... Done.
  "pofw" ... Done.
  "pois" ... Done.
  "protected_areas" ... Done.
  "railways" ... Done.
  "roads" ... Done.
  "traffic" ... Done.
  "transport" ... Done.
  "water" ... Done.
  "waterways" ... Done.
>>> type(rutland_data)
dict
>>> list(rutland_data.keys())
['points',
 'lines',
 'multilinestrings',
 'multipolygons',
 'other_relations',
 'adminareas',
 'buildings',
 'landuse',
 'natural',
 'places',
 'pofw',
 'pois',
 'protected_areas',
 'railways',
 'roads',
 'traffic',
 'transport',
 'water',
 'waterways']

>>> # Data of the 'waterways' layer
>>> rutland_waterways = rutland_data['waterways']
>>> rutland_waterways.head()
    osm_id  ...                                           geometry
0  3701346  ...  LINESTRING (-0.7536654 52.6495358, -0.7536236 ...
1  3701347  ...  LINESTRING (-0.7948821 52.6569468, -0.7946128 ...
2  3707149  ...  LINESTRING (-0.7262381 52.6790459, -0.7258244 ...
3  3707303  ...  LINESTRING (-0.7213277 52.6765954, -0.7206778 ...
4  4470795  ...  LINESTRING (-0.4995349 52.6418825, -0.4984075 ...
[5 rows x 6 columns]

Delete the test database and downloaded data files:

>>> # Delete the database 'osmdb_test'
>>> osmdb.drop_database(verbose=True)
Drop the database "osmdb_test" from postgres:***@localhost:5432?
 [No]|Yes: yes
Dropping "osmdb_test" ... Done.

>>> # Delete the downloaded data files
>>> delete_dir(dat_dir, verbose=True)
To delete the directory "./tests/osm_data/" (Not empty)
? [No]|Yes: yes
Deleting "./tests/osm_data/" ... Done.